[Inference] Image content #205371
Merged
[Inference] Image content #205371
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dgieselaar
added
release_note:skip
|
/ci |
pgayvallet
approved these changes
Jan 6, 2025
| @@ -153,7 +153,24 @@ const messagesToBedrock = (messages: Message[]): BedRockMessage[] => { | |||
| case MessageRole.User: | |||
| return { | |||
| role: 'user' as const, | |||
| rawContent: [{ type: 'text' as const, text: message.content }], | |||
| rawContent: (typeof message.content === 'string' | |||
There was a problem hiding this comment.
NIT:
rawContent: typeof message.content === 'string'
? [{ text: message.content, type: 'text' } satisfies BedRockTextPart]
: message.content.map((contentPart) => {
if (contentPart.type === 'text') {
return { text: contentPart.text, type: 'text' } satisfies BedRockTextPart;
} else {
return {
type: 'image',
source: {
data: contentPart.source.data,
mediaType: contentPart.source.mimeType,
type: 'base64',
},
} satisfies BedRockImagePart;
}
}),There was a problem hiding this comment.
wdyt about castArray() instead?
Comment on lines
+199
to
+213
| parts: (typeof message.content === 'string' ? [message.content] : message.content).map( | ||
| (contentPart) => { | ||
| if (typeof contentPart === 'string') { | ||
| return { text: contentPart } satisfies Gemini.TextPart; | ||
| } else if (contentPart.type === 'text') { | ||
| return { text: contentPart.text } satisfies Gemini.TextPart; | ||
| } | ||
| return { | ||
| inlineData: { | ||
| data: contentPart.source.data, | ||
| mimeType: contentPart.source.mimeType, | ||
| }, | ||
| } satisfies Gemini.InlineDataPart; | ||
| } | ||
| ), |
There was a problem hiding this comment.
Same NIT as for claude, but that's probably a personal preference at this point.
|
@elasticmachine merge upstream |
💚 Build Succeeded
Metrics [docs]Public APIs missing comments
History
cc @dgieselaar |
|
Starting backport for target branches: 8.x |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jan 7, 2025
Adds support for image content parts in the Inference plugin. Only base64 encoded images are supported, as this capability is shared across all three LLM providers. --------- Co-authored-by: Elastic Machine <[email protected]> (cherry picked from commit 2cfc167)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Jan 7, 2025
# Backport This will backport the following commits from `main` to `8.x`: - [[Inference] Image content (#205371)](#205371) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Dario Gieselaar","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-01-07T11:17:56Z","message":"[Inference] Image content (#205371)\n\nAdds support for image content parts in the Inference plugin. Only\nbase64 encoded images are supported, as this capability is shared across\nall three LLM providers.\n\n---------\n\nCo-authored-by: Elastic Machine <[email protected]>","sha":"2cfc16709d9af1f1b9a7d094f596e0c88bba0179","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:version","v8.18.0"],"title":"[Inference] Image content","number":205371,"url":"https://github.com/elastic/kibana/pull/205371","mergeCommit":{"message":"[Inference] Image content (#205371)\n\nAdds support for image content parts in the Inference plugin. Only\nbase64 encoded images are supported, as this capability is shared across\nall three LLM providers.\n\n---------\n\nCo-authored-by: Elastic Machine <[email protected]>","sha":"2cfc16709d9af1f1b9a7d094f596e0c88bba0179"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205371","number":205371,"mergeCommit":{"message":"[Inference] Image content (#205371)\n\nAdds support for image content parts in the Inference plugin. Only\nbase64 encoded images are supported, as this capability is shared across\nall three LLM providers.\n\n---------\n\nCo-authored-by: Elastic Machine <[email protected]>","sha":"2cfc16709d9af1f1b9a7d094f596e0c88bba0179"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Dario Gieselaar <[email protected]>
kowalczyk-krzysztof
pushed a commit
to kowalczyk-krzysztof/kibana
that referenced
this pull request
Jan 7, 2025
Adds support for image content parts in the Inference plugin. Only base64 encoded images are supported, as this capability is shared across all three LLM providers. --------- Co-authored-by: Elastic Machine <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds support for image content parts in the Inference plugin. Only base64 encoded images are supported, as this capability is shared across all three LLM providers.