Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serialization Fails for u64 Fields Due to Incoming String Values in OpenTelemetry Metrics #2487

Open
nikhilsinhaparseable opened this issue Dec 30, 2024 · 6 comments · May be fixed by #2491

Comments

@nikhilsinhaparseable
Copy link

I am using the opentelemetry-proto crate to serialize incoming OpenTelemetry metrics data. The crate defines certain fields (count, bucket_counts under HistogramDataPoint) as u64. However, the OpenTelemetry exporter sends these fields as strings, which leads to serialization failures.

This mismatch between the data types in the crate and the incoming data prevents successful serialization of metrics.

Sample JSON
otel-metrics-histogram.json

Similar issue is found in the ExponentialHistogramDataPoint

@cijothomas
Copy link
Member

the OpenTelemetry exporter sends these fields as strings, which leads to serialization failures.

Which exporter are you referring to?

@nikhilsinhaparseable
Copy link
Author

I am using otel collector in opentelemetry-demo to send the otel logs, metrics and traces to Parseable.

@lalitb
Copy link
Member

lalitb commented Dec 31, 2024

As per the proto-json conversion standard, u64 should be serialized to string - https://protobuf.dev/programming-guides/json/. So i believe, opentelemetry-proto is doing it correctly?

@nikhilsinhaparseable
Copy link
Author

nikhilsinhaparseable commented Jan 1, 2025

I can see that the serialization to string is happening for the fields where below is available in the crate but this feature is not available for the mentioned fields.

#[cfg_attr( feature = "with-serde", serde( serialize_with = "crate::proto::serializers::serialize_u64_to_string", deserialize_with = "crate::proto::serializers::deserialize_string_to_u64" ) )]

@lalitb
Copy link
Member

lalitb commented Jan 1, 2025

I can see that the serialization to string is happening for the fields where below is available in the crate but this feature is not available for the mentioned fields.

#[cfg_attr( feature = "with-serde", serde( serialize_with = "crate::proto::serializers::serialize_u64_to_string", deserialize_with = "crate::proto::serializers::deserialize_string_to_u64" ) )]

Yes, I did realize it afterwards, and have done changes locally to test before raising PR here -
https://github.com/lalitb/opentelemetry-rust/pull/44/files

@nikhilsinhaparseable
Copy link
Author

@lalitb I see similar issue in SummaryDataPoint for the field quantile_values
as the fields quantile and value fields in the ValueAtQuantile struct are f64 data type but the data from the collector is somethings like this -
image

below is the sample JSON with summary metrics -
otel-metrics-summary.json

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants