Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

json: ser/de bytes as base64 strings not an array of bytes #2471

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+29 −3
Open

json: ser/de bytes as base64 strings not an array of bytes #2471

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8905a39
json: ser/de bytes as base64 strings not an array of bytes
adriangb Dec 25, 2024
0386ff2
add examples
adriangb Jan 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions opentelemetry-otlp/tests/integration_test/expected/traces.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,12 @@
"value": {
"stringValue": "yes"
}
},
{
"key": "data",
"value": {
"bytesValue": "gICA"
}
Copy link
Member

@lalitb lalitb Jan 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make it complete, can we also add this entry in the json samples for logs and metrics?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure yes!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added. But if these are the outputs, shouldn't tests be failing?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I wasn't sure how to add an example in a logs body field. Can you suggest the right way to do that?

Copy link
Member

@lalitb lalitb Jan 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added. But if these are the outputs, shouldn't tests be failing?

Yes these would indeed fail. Please run ./scripts/integration_tests.sh locally to validate that.

ps. - disable the failing tests from #2495 before that, so that you only get failures from your changes.

Copy link
Member

@lalitb lalitb Jan 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I wasn't sure how to add an example in a logs body field. Can you suggest the right way to do that?

Good question. The log appender doesn't handle bytes as attribute value for now. The good test probably would be adding binary data to resource, and update json accordingly.

}
],
"events": [
Expand Down
3 changes: 2 additions & 1 deletion opentelemetry-proto/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ testing = ["opentelemetry/testing"]

# add ons
with-schemars = ["schemars"]
with-serde = ["serde", "hex"]
with-serde = ["serde", "hex", "base64"]
populate-logs-event-name = []

[dependencies]
Expand All @@ -56,6 +56,7 @@ opentelemetry_sdk = { version = "0.27", default-features = false, path = "../ope
schemars = { version = "0.8", optional = true }
serde = { workspace = true, optional = true, features = ["serde_derive"] }
hex = { version = "0.4.3", optional = true }
base64 = { version = "0.22.1", optional = true }

[dev-dependencies]
opentelemetry = { features = ["testing"], path = "../opentelemetry" }
Expand Down
22 changes: 20 additions & 2 deletions opentelemetry-proto/src/proto.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,11 @@ pub(crate) mod serializers {
map.serialize_entry("intValue", &i.to_string());
map.end()
}
Some(Value::BytesValue(b)) => {
let mut map = serializer.serialize_map(Some(1))?;
map.serialize_entry("bytesValue", &base64::encode(b));
map.end()
}
Some(value) => value.serialize(serializer),
None => serializer.serialize_none(),
}
Expand Down Expand Up @@ -127,8 +132,10 @@ pub(crate) mod serializers {
value = Some(any_value::Value::KvlistValue(kv));
}
"bytesValue" => {
let bytes = map.next_value()?;
value = Some(any_value::Value::BytesValue(bytes));
let base64: String = map.next_value()?;
let decoded = base64::decode(base64.as_bytes())
.map_err(|e| de::Error::custom(e))?;
value = Some(any_value::Value::BytesValue(decoded));
}
_ => {
//skip unknown keys, and handle error later.
Expand Down Expand Up @@ -182,6 +189,17 @@ pub(crate) mod serializers {
let s: String = Deserialize::deserialize(deserializer)?;
s.parse::<i64>().map_err(de::Error::custom)
}

pub fn serialize_vec_u8_as_base64_string<S: Serializer>(v: &Vec<u8>, s: S) -> Result<S::Ok, S::Error> {
adriangb marked this conversation as resolved.
Show resolved Hide resolved
let base64 = base64::encode(v);
String::serialize(&base64, s)
}

pub fn deserialize_base64_string_to_vec_u8<'de, D: Deserializer<'de>>(d: D) -> Result<Vec<u8>, D::Error> {
let base64 = String::deserialize(d)?;
base64::decode(base64.as_bytes())
.map_err(|e| serde::de::Error::custom(e))
}
}

#[cfg(feature = "gen-tonic-messages")]
Expand Down
Loading