ci(ftr): notify owners in the slack message when failures #205260

v1v · 2024-12-30T13:23:40Z

Summary

Customise Slack messages with the Slack Team owners when the FTRs fail.

As long as the PING_SLACK_TEAM is set in the Buildkite pipeline, then the slack notification will poke the Slack team set in the environment variable when the step failed only.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

See some risk examples
...

…-fix'

jbudz · 2024-12-30T14:30:12Z

.buildkite/scripts/steps/functional/apm_cypress.sh

+  # Report the error to the team using the subfolder matching the Slack Team
+  popd
+  mkdir "$REPORT_SLACK_TEAM"
+  touch "$REPORT_SLACK_TEAM"/obs-ux-infra_services-team.slack


Can you help me understand the changes here vs passing a string PING_SLACK_TEAM='obs-ux-infra_services-team?

I'm not opposed to files but it seems to add some additional complexity.

Sure thing.

I wanted to make pingTeam agnostic to the team name. But read the files under target/report-slack-team, and those files match the name of the slack team to be notified if any errors.

Regarding your question about PING_SLACK_TEAM='obs-ux-infra_services-team', I'm not sure how the post_command can read that value and whether I could use something like ts-node .buildkite/scripts/lifecycle/ping_team_ftrs.ts "$PING_SLACK_TEAM" in here

Maybe I could use the buildkite metadata set command to notify the Slack team to reduce the additional complexity with files under target/report-slack-team.

I think I've found a better approach:

d30ec5f (#205260)

Pretty much, if the env variable PING_SLACK_TEAM exists and the step fails, then it will notify the slack team.

.buildkite/scripts/steps/functional/profiling_cypress.sh

Apply suggestions from code review rename

v1v · 2024-12-30T17:21:44Z

However, I've just refactored a bit more:

2e28915 (#205260)

So I created a specialised library for the Slack notifications, so I don't mix concerns with the TestFailures, in addition, I also changed the post-command hook to run outside of the metadata called "${BUILDKITE_JOB_ID}_is_test_execution_step" - as far as I see, it's set in other places but it's not set when running .buildkite/scripts/steps/functional/profiling_cypress.sh or .buildkite/scripts/steps/functional/apm_cypress.sh

elasticmachine · 2024-12-30T19:00:57Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: d2cc67c

Failed CI Steps

FTR Configs #41

Test Failures

[job] [logs] FTR Configs #41 / InfraOps App Metrics UI Node Details #Asset Type: host Osquery Tab should show a date picker

Metrics [docs]

✅ unchanged

History

💔 Build #263385 failed d30ec5f
💚 Build #263351 succeeded 4f0069c

cc @v1v

delanni · 2025-01-06T14:08:06Z

.buildkite/scripts/lifecycle/post_command.sh

+if [[ $BUILDKITE_COMMAND_EXIT_STATUS -ne 0 ]]; then
+  # If the slack team environment variable is set, ping the team in slack
+  if [ -n "${PING_SLACK_TEAM:-}" ]; then
+    ts-node .buildkite/scripts/lifecycle/ping_slack.ts


Just a consideration: the PR is now mostly just wrapping around the typescript file, but that can be done with a simple one-liner

buildkite-agent meta-data set 'slack:ping_team:body' \ "${PING_SLACK_TEAM}, can you please take a look at the test failures?"

(also add ELASTIC_SLACK_NOTIFICATIONS_ENABLED == 'true' above)

@delanni, sure thing, I can simplify it. let me do it now

Thanks! I prefer it like this, without the extra bloat for now (while this is not doing more than adding the metadata)

kibanamachine · 2025-01-07T11:53:24Z

Starting backport for target branches: 7.17, 8.16, 8.17, 8.x

https://github.com/elastic/kibana/actions/runs/12651030780

…5260) (cherry picked from commit acc5e03)

kibanamachine · 2025-01-07T11:59:52Z

💔 Some backports could not be created

Status	Branch	Result
❌	7.17	Backport failed because of merge conflicts
✅	8.16
✅	8.17
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 205260

Questions ?

Please refer to the Backport tool documentation

…5260) (#205716) # Backport This will backport the following commits from `main` to `8.17`: - [ci(ftr): notify owners in the slack message when failures (#205260)](#205260)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Victor Martinez <[email protected]>

…5260) (#205715) # Backport This will backport the following commits from `main` to `8.16`: - [ci(ftr): notify owners in the slack message when failures (#205260)](#205260)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Victor Martinez <[email protected]>

) (#205717) # Backport This will backport the following commits from `main` to `8.x`: - [ci(ftr): notify owners in the slack message when failures (#205260)](#205260)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Victor Martinez <[email protected]>

…5260)

ci(ftr): notify owners in the slack message when failures

d470111

v1v requested a review from a team December 30, 2024 13:23

v1v self-assigned this Dec 30, 2024

v1v requested a review from a team as a code owner December 30, 2024 13:23

v1v added release_note:skip Skip the PR/issue when compiling release notes backport:all-open Backport to all branches that could still receive a release labels Dec 30, 2024

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

4f0069c

…-fix'

jbudz reviewed Dec 30, 2024

View reviewed changes

v1v commented Dec 30, 2024

View reviewed changes

.buildkite/scripts/steps/functional/profiling_cypress.sh Outdated Show resolved Hide resolved

use env variable to ping the slack team

d30ec5f

Apply suggestions from code review rename

v1v force-pushed the feature/ping-errors-eams branch from eeb53eb to d30ec5f Compare December 30, 2024 15:42

jbudz approved these changes Dec 30, 2024

View reviewed changes

refactor

2e28915

fix: unbound variable

d2cc67c

delanni reviewed Jan 6, 2025

View reviewed changes

v1v and others added 2 commits January 7, 2025 11:25

simplify

aa29b60

Merge branch 'main' into feature/ping-errors-eams

ba3d202

v1v enabled auto-merge (squash) January 7, 2025 10:27

delanni approved these changes Jan 7, 2025

View reviewed changes

v1v merged commit acc5e03 into elastic:main Jan 7, 2025
9 checks passed

kibanamachine added the v9.0.0 label Jan 7, 2025

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jan 7, 2025

ci(ftr): notify owners in the slack message when failures (elastic#20…

92f744d

…5260) (cherry picked from commit acc5e03)

kibanamachine mentioned this pull request Jan 7, 2025

[8.16] ci(ftr): notify owners in the slack message when failures (#205260) #205715

Merged

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jan 7, 2025

ci(ftr): notify owners in the slack message when failures (elastic#20…

3573cbb

…5260) (cherry picked from commit acc5e03)

kibanamachine mentioned this pull request Jan 7, 2025

[8.17] ci(ftr): notify owners in the slack message when failures (#205260) #205716

Merged

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jan 7, 2025

ci(ftr): notify owners in the slack message when failures (elastic#20…

a38a181

…5260) (cherry picked from commit acc5e03)

kibanamachine mentioned this pull request Jan 7, 2025

[8.x] ci(ftr): notify owners in the slack message when failures (#205260) #205717

Merged

kibanamachine added the v8.17.1 label Jan 7, 2025

kibanamachine added the v8.16.3 label Jan 7, 2025

kibanamachine added the v8.18.0 label Jan 7, 2025

kowalczyk-krzysztof pushed a commit to kowalczyk-krzysztof/kibana that referenced this pull request Jan 7, 2025

ci(ftr): notify owners in the slack message when failures (elastic#20…

12c32b0

…5260)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci(ftr): notify owners in the slack message when failures #205260

ci(ftr): notify owners in the slack message when failures #205260

v1v commented Dec 30, 2024 •

edited by kibanamachine

Loading

jbudz Dec 30, 2024

v1v Dec 30, 2024

v1v Dec 30, 2024

v1v commented Dec 30, 2024

elasticmachine commented Dec 30, 2024

delanni Jan 6, 2025

v1v Jan 7, 2025

delanni Jan 7, 2025

kibanamachine commented Jan 7, 2025

kibanamachine commented Jan 7, 2025

ci(ftr): notify owners in the slack message when failures #205260

ci(ftr): notify owners in the slack message when failures #205260

Conversation

v1v commented Dec 30, 2024 • edited by kibanamachine Loading

Summary

Checklist

Identify risks

jbudz Dec 30, 2024

Choose a reason for hiding this comment

v1v Dec 30, 2024

Choose a reason for hiding this comment

v1v Dec 30, 2024

Choose a reason for hiding this comment

v1v commented Dec 30, 2024

elasticmachine commented Dec 30, 2024

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

History

delanni Jan 6, 2025

Choose a reason for hiding this comment

v1v Jan 7, 2025

Choose a reason for hiding this comment

delanni Jan 7, 2025

Choose a reason for hiding this comment

kibanamachine commented Jan 7, 2025

kibanamachine commented Jan 7, 2025

💔 Some backports could not be created

Manual backport

Questions ?

v1v commented Dec 30, 2024 •

edited by kibanamachine

Loading