feat: test examples in real browsers

[olizilla]

Pulls example tests out to own dir so we can run the same test for each ui library!

fixes: #222

You can see it run in CI at https://github.com/web3-storage/w3ui/actions/runs/3901558574/jobs/6663550379#step:6:13 and you can try it out locally from this branch by running:

$ pnpm install

# one time download of test browsers
$ pnpx playwright install

# fails for vanilla currently, but is ok, proceed.
$ pnpm build:examples 

$ pnpm test:examples

Scope: 20 of 33 workspace projects
examples/test/playwright test$ playwright test
[30 lines collapsed]
│ [30/33] [webkit] › sign-up-in.spec.ts:4:3 › vue: sign in
│ [31/33] [webkit] › uploads-list.spec.ts:4:3 › react: uploads list
│ [32/33] [webkit] › uploads-list.spec.ts:4:3 › solid: uploads list
│ [33/33] [webkit] › uploads-list.spec.ts:4:3 › vue: uploads list
│ 
│   33 passed (7s)
│ To open last HTML report run:
│ 
│   npx playwright show-report
│ 
└─ Done in 7.6s

note: the build fails currently as our vanilla examples fail to build currently, which is why we need these tests!

Adds and updates run scripts to make it easy to run package or example tests. Most of the time we'd want to run package tests locally, so this is set as the default.

# runs package tests
$ pnpm test

# builkd examples. must be run before test so the dist dir exits for each
$ pnpm build:examples

# run example test
$ pnpm test:examples

# serve examples, so you can poke around as a dev. 
$ pnpm serve:examples

Updates examples to produce relative URLs by setting base: '' in vite config. This allows us to serve all the examples from a single static server in the examples dir, and serve is added with a serve.json config to rewrite the paths so the dist dir is served for each example.

Adds playwright config to fire up the example server

Adds github workflow to set up and run the browser tests only if we make a change to the examples. We may change this to run for every change in the future, as we should probably check if the examples still work if we change a package!

Reduces the scope of our local github action to just handle setting up node and pnpm which is needed by every workflow. lint and test are now specified explicitly in workflows that should run them. We don't need to run those steps when we're doing browser testing.

• Vanilla JS examples are not currently tested as they fail to build vanilla js examples fail to build #243 - we can fix in a separate PR.
• We need a mechanism to be able to register a space from browser, or add pre-registered test account fixture - Add example test fixture for pre-registered account #244
• We are missing a multi-file-upload example - Add multi-file-upload Vue example #245

whoo!

License: MIT
Signed-off-by: Oli Evans [email protected]

[codesandbox-ci]

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

[olizilla]

I explored using vitest to drive playwright tests. It's ok, but a little clunky and I think we want as little friction as possible for browser testing. i'm now inclined to use @playwright/test instead after trying it in #238 and now i see it has support for what we need to do here https://playwright.dev/docs/test-advanced#projects and https://playwright.dev/docs/test-parameterize and if we stick with vanilla @playwright/test we can use a service to run our tests for us if we want to.

[olizilla]

Here's a good example of the kind of info we get from this... Our vue multi-file-upload example is having a bad day. Passes in all browsers for other ui libraries

EDIT: oh ho ho... we haven't written that ui + example combo yet! but you get the point, we will get a heads up when one of the 20 existing examples breaks.

[travis]

I love it! I'm going to start a discussion around which examples we want to keep longer-term so I wouldn't go too much deeper on this for now, but I'm super excited about the approach!

[travis]

this is dope af

[olizilla]

@travis thanks! i'm gonna land this but i'm more than happy to iterate on it to work with #256