[Obs AI Assistant] Add knowledge base migration test to the serverless test suite #205631
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
viduni94
added
release_note:skip
|
Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant) |
viduni94
commented
Jan 6, 2025
| } | ||
|
|
||
| describe('When there are knowledge base entries (from 8.15 or earlier) that does not contain semantic_text embeddings', function () { | ||
| this.tags(['skipMKI']); |
There was a problem hiding this comment.
I added this tag because this test fails in MKI due to a security exception:
This security exception is caused by await esArchiver.load(archive);
debg Requesting url (redacted): [https://keepvuw-serverless-mki-testing-bf2788.kb.eu-west-1.aws.qa.elastic.cloud/api/status]
│ info [x-pack/test/functional/es_archives/observability/ai_assistant/knowledge_base_8_15] Loading "mappings.json"
│ info [x-pack/test/functional/es_archives/observability/ai_assistant/knowledge_base_8_15] Loading "data.json.gz"
└- ✖ fail: Observability AI Assistant API tests knowledge_base/knowledge_base_migration.spec.ts When there are knowledge base entries (from 8.15 or earlier) that does not contain semantic_text embeddings "before all" hook in "When there are knowledge base entries (from 8.15 or earlier) that does not contain semantic_text embeddings"
│ ResponseError: security_exception
│ Root causes:
│ security_exception: action [indices:admin/settings/update] is unauthorized for user [testing-internal] with effective roles [superuser] on restricted indices [.kibana_security_solution_1,.kibana_task_manager_1,.kibana_alerting_cases_1,.kibana_usage_counters_1,.kibana_1,.kibana_ingest_1,.kibana_analytics_1], this action is granted by the index privileges [manage,all]
60287e4 to
c32cb2f
Compare
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#7648[❌] x-pack/test_serverless/api_integration/test_suites/observability/ai_assistant/config.ts: 131/200 tests passed. |
Does this mean you won't merge this until the deployment agnostic testing environment is available? And then move this over to that environment? I think that's the right approach. |
viduni94
force-pushed
the
knowledge-base-migration-test-for-serverless
branch
from
c32cb2f to
aa7de68
Compare
Hey @sorenlouv |
neptunian
approved these changes
Jan 7, 2025
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]
History
cc @viduni94 |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#7655[❌] x-pack/test_serverless/api_integration/test_suites/observability/ai_assistant/config.ts: 119/150 tests passed. |
@sorenlouv or we can merge it now and have arturo move it over in his PR. That way we could remove all the serverless environment like the api client in that PR. |
sorenlouv
approved these changes
Jan 7, 2025
Sgtm. What I'm a little worried about is having duplicate tests that will eventually drift apart and thus increase the maintenance cost. Let's aim to move this to the agnostic environment asap. |
|
Starting backport for target branches: 8.x |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jan 8, 2025
…s test suite (elastic#205631) Closes elastic#205537 ## Summary The knowledge base migration test suite is missing in serverless. This PR adds it to the serverless test suite. - This has a dependancy to elastic#205194 since we are removing all serverless tests and adding them to DA tests. - If the DA tests PR gets merged first, I'll refactor this PR to add it there. ### Checklist - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) (cherry picked from commit e32ff8e)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Jan 8, 2025
…verless test suite (#205631) (#205832) # Backport This will backport the following commits from `main` to `8.x`: - [[Obs AI Assistant] Add knowledge base migration test to the serverless test suite (#205631)](#205631) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Viduni Wickramarachchi","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-01-08T01:46:51Z","message":"[Obs AI Assistant] Add knowledge base migration test to the serverless test suite (#205631)\n\nCloses https://github.com/elastic/kibana/issues/205537\r\n\r\n## Summary\r\n\r\nThe knowledge base migration test suite is missing in serverless. This\r\nPR adds it to the serverless test suite.\r\n- This has a dependancy to https://github.com/elastic/kibana/pull/205194\r\nsince we are removing all serverless tests and adding them to DA tests.\r\n- If the DA tests PR gets merged first, I'll refactor this PR to add it\r\nthere.\r\n\r\n### Checklist\r\n\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n- [x] The PR description includes the appropriate Release Notes section,\r\nand the correct `release_note:*` label is applied per the\r\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"e32ff8e9f3384b651cc2eeba88c0ac232551b7e9","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Obs AI Assistant","backport:version","v8.18.0"],"title":"[Obs AI Assistant] Add knowledge base migration test to the serverless test suite","number":205631,"url":"https://github.com/elastic/kibana/pull/205631","mergeCommit":{"message":"[Obs AI Assistant] Add knowledge base migration test to the serverless test suite (#205631)\n\nCloses https://github.com/elastic/kibana/issues/205537\r\n\r\n## Summary\r\n\r\nThe knowledge base migration test suite is missing in serverless. This\r\nPR adds it to the serverless test suite.\r\n- This has a dependancy to https://github.com/elastic/kibana/pull/205194\r\nsince we are removing all serverless tests and adding them to DA tests.\r\n- If the DA tests PR gets merged first, I'll refactor this PR to add it\r\nthere.\r\n\r\n### Checklist\r\n\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n- [x] The PR description includes the appropriate Release Notes section,\r\nand the correct `release_note:*` label is applied per the\r\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"e32ff8e9f3384b651cc2eeba88c0ac232551b7e9"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205631","number":205631,"mergeCommit":{"message":"[Obs AI Assistant] Add knowledge base migration test to the serverless test suite (#205631)\n\nCloses https://github.com/elastic/kibana/issues/205537\r\n\r\n## Summary\r\n\r\nThe knowledge base migration test suite is missing in serverless. This\r\nPR adds it to the serverless test suite.\r\n- This has a dependancy to https://github.com/elastic/kibana/pull/205194\r\nsince we are removing all serverless tests and adding them to DA tests.\r\n- If the DA tests PR gets merged first, I'll refactor this PR to add it\r\nthere.\r\n\r\n### Checklist\r\n\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n- [x] The PR description includes the appropriate Release Notes section,\r\nand the correct `release_note:*` label is applied per the\r\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"e32ff8e9f3384b651cc2eeba88c0ac232551b7e9"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Viduni Wickramarachchi <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #205537
Summary
The knowledge base migration test suite is missing in serverless. This PR adds it to the serverless test suite.
Checklist
release_note:*label is applied per the guidelines