Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate onboarding Cypress tests to Scout #205482

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

mykolaharmash
Copy link
Contributor

@mykolaharmash mykolaharmash commented Jan 3, 2025

This change converts Cypress tests for the custom logs flow into Playwright using the Scout wrapper.

Note

As Scout package is still being developed, the PR pipeline configured to runs Playwright tests only when code in certain plugins have been changed and not on every PR.

How to run tests locally

Start the Scout server

node scripts/scout.js start-server --stateful

In a separate terminal run the tests

npx playwright test --config x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/playwright.config.ts

Playwright runs browsers in a headless mode by default, user --headed option if needed

@mykolaharmash mykolaharmash requested a review from a team as a code owner January 3, 2025 12:12
@mykolaharmash mykolaharmash added release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting labels Jan 3, 2025
@mykolaharmash mykolaharmash force-pushed the migrate-onboarding-cypress-tests branch from 740b0d5 to f11351b Compare January 3, 2025 13:54
@mykolaharmash mykolaharmash requested a review from a team as a code owner January 3, 2025 14:06
Copy link
Member

@jbudz jbudz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.buildkite/scripts/pipelines/pull_request/pipeline.ts LGTM

@mykolaharmash mykolaharmash added the ci:project-deploy-observability Create an Observability project label Jan 3, 2025
Copy link
Contributor

github-actions bot commented Jan 3, 2025

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

Copy link
Contributor

@awahab07 awahab07 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to see the tests migration to Scout.

test('Users should expand and collapse the Advanced settings section', async ({
pageObjects: { customLogsPage },
}) => {
await expect(customLogsPage.namespaceInput()).not.toBeInViewport();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is using toBeInViewport here intentional or is it a better use case for toBeVisible ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, it was a workaround for the way EUI accordion works, but not a very good one. I've changed this test to check the visibility of the whole enclosing container instead of individual input fields, this was I can use toBeVisible()

page,
}) => {
await customLogsPage.integrationNameInput().fill('H3llowOrld');
await expect(page.getByText('An integration name should be lowercase.')).toBeVisible();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: messages could be extracted out as in

assertionMessages.QUERY_BAR_VALIDATION.SUGGESTIONS_COUNT

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've put those messages into a static property of the page object, seems like a mo contextual place for them

});

test.describe('when user is missing privileges', () => {
test.beforeEach(async ({ browserAuth, page }) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: beforeAll and afterAll here seem more suitable compared to beforeEach and afterEach.

Copy link
Contributor Author

@mykolaharmash mykolaharmash Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, but the thing it, Playwright does not allow using page and context (which browserAuth uses) fixtures in beforeAll/afterAll. We'd need to create both fixtures manually, which does not seem to worth it in this case. Playwright doc for reference, just in case.

Comment on lines 25 to 27
logFilePathList() {
return this.page.locator(`[data-test-subj^=obltOnboardingLogFilePath-]`);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, we shouldn't wrap the locators in functions which are not performing any action or are not parametrized. Keeping them as public instance variables would result in cleaner Page Objects, as shown in the example in docs.

@dmlemeshko what are your thoughts on this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point, I'll convert those without any parameters or actions into properties 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Migrated to use properties

@mykolaharmash mykolaharmash force-pushed the migrate-onboarding-cypress-tests branch from f3f1c8b to 53b9f23 Compare January 6, 2025 09:09
@mykolaharmash mykolaharmash force-pushed the migrate-onboarding-cypress-tests branch from 4a2ea2c to c188f73 Compare January 6, 2025 13:00
@mykolaharmash
Copy link
Contributor Author

/oblt-deploy

@mykolaharmash mykolaharmash requested a review from awahab07 January 6, 2025 13:34
@mykolaharmash
Copy link
Contributor Author

@awahab07 thank you for a thorough review, I've addressed the points you've mentioned, please take another look

Copy link
Contributor

@awahab07 awahab07 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

The reporter error in the pipeline step "Run Failed Test Reporter" is linked to TEST_CONFIG: unbound variable error or does it happen if a test fails too? And do we need to do anything about it in this PR?

@mykolaharmash
Copy link
Contributor Author

image

The reporter error in the pipeline step "Run Failed Test Reporter" is linked to TEST_CONFIG: unbound variable error or does it happen if a test fails too? And do we need to do anything about it in this PR?

Sorry about this, that was just an error in the script, I'm working on the fix. Did not notice that script initially, I though that modifying the pipeline.yml would be enough to run tests for another plugin. I'll ping you again once the script is updated and the CI is green 🙌

@elasticmachine
Copy link
Contributor

elasticmachine commented Jan 8, 2025

💔 Build Failed

  • Buildkite Build
  • Commit: 1b7cc5f
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-205482-1b7cc5f343e4

Failed CI Steps

Metrics [docs]

‼️ ERROR: no builds found for mergeBase sha [e04b200]

History

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants