Skip to content

Commit a6c43aa

Browse files
giamirdancormierb-kelly
authored
test(code): introduce component and visual testing to stacks (#1194)
Co-authored-by: Dan Cormier <dcormier@stackoverflow.com> Co-authored-by: Ben Kelly <b-kelly@users.noreply.github.com>
1 parent cd0ec76 commit a6c43aa

32 files changed

+5797
-2591
lines changed

.gitattributes

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
screenshots/** filter=lfs diff=lfs merge=lfs -text

.github/workflows/main.yml

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [develop]
6+
pull_request:
7+
branches: [develop]
8+
9+
jobs:
10+
build-and-test:
11+
name: Build and Test
12+
runs-on: ubuntu-latest
13+
steps:
14+
- name: ⬇️ Checkout
15+
uses: actions/checkout@v3
16+
with:
17+
lfs: true
18+
19+
- name: ⎔ Setup node
20+
uses: actions/setup-node@v3
21+
with:
22+
node-version: 'lts/*'
23+
24+
- name: 🏗 Install and Build
25+
run: |
26+
npm ci
27+
npx playwright install --with-deps
28+
npm run build
29+
30+
- name: ▶️ Lint and Test
31+
run: |
32+
npm run lint
33+
npm run test -- --config web-test-runner.config.ci.mjs
34+
35+
- name: ⬆️ Upload Visual Regression Test Results
36+
uses: actions/upload-artifact@v3
37+
if: failure()
38+
with:
39+
name: visual-regression-test-results
40+
path: screenshots
41+
42+
43+
# cancel the jobs if another workflow is kicked off for the same branch
44+
concurrency:
45+
group: ${{ github.workflow }}-${{ github.ref }}
46+
cancel-in-progress: true

.gitignore

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -78,11 +78,13 @@ dist/
7878

7979
# Testing Assets #
8080
###############
81-
/backstop_data/bitmaps_test/
82-
/backstop_data/bitmaps_reference/
83-
/backstop_data/html_report/
81+
/screenshots/**/failed/**
8482

8583
# GitHub Actions #
8684
###############
8785
/.github/main.workflow
86+
87+
# Editors #
88+
###########
8889
.vscode/
90+
/.idea

README.md

Lines changed: 40 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,9 +55,47 @@ npm run lint:format
5555
```
5656

5757
## Testing Stacks
58-
Stacks has implemented visual regression testing with [Backstop](https://github.com/garris/BackstopJS). To test if your new feature introduces visual regressions, run `npm run test` in a new Terminal window while Stacks is running. After the tests have run, a new browser window with any regressions will show. If the regressions are desired, you can run `npm run approve` to establish the new baseline.
5958

60-
Individual routes to test are found in [backstop.json](/backstop.json)
59+
Run all test suites by running:
60+
```sh
61+
npm test
62+
```
63+
### Unit/Component Tests
64+
65+
Unit/Component tests are written with [DOM Testing Library](https://testing-library.com/docs/dom-testing-library/intro).
66+
Please follow the library's principles and documentation to write tests.
67+
68+
Stacks uses [Web Test Runner](https://modern-web.dev/docs/test-runner/overview/) and [Playwright](https://modern-web.dev/docs/test-runner/browser-launchers/playwright/) to run tests in a real browser context.
69+
70+
Execute the unit/component tests suite by running:
71+
```sh
72+
npm run test:unit
73+
```
74+
or if you prefer watch mode run:
75+
```sh
76+
npm run test:unit:watch
77+
```
78+
79+
### Visual Regression Tests
80+
81+
**Prerequisite:**
82+
In order to pull and upload baseline images you need to have `git lfs` installed in your local machine. Please follow [this guide](https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage) to install that.
83+
84+
85+
This [Web Test Runner plugin](https://www.npmjs.com/package/@web/test-runner-visual-regression) is used to run visual regression tests. [DOM Testing Library](https://testing-library.com/docs/dom-testing-library/intro).
86+
Visual regression tests end with this suffix `*.visual.test.ts`.
87+
88+
Execute the visual regression tests suite by running:
89+
```sh
90+
npm run test:visual
91+
```
92+
93+
Update the visual baseline via:
94+
```sh
95+
npm run test:visual:update
96+
```
97+
98+
Failing tests (including diffs) can be found under `screenshots/[browser]/failed/` folders.
6199

62100
## Bugs and feature requests
63101
Have a bug or feature request? First search existing or closed issues to make sure the issue hasn’t been noted yet. If not, review our [issue guidelines](/CONTRIBUTING.md#open-an-issue) for submitting [a bug report](/CONTRIBUTING.md#reporting-bugs) or [feature request](/CONTRIBUTING.md#feature-requests).

0 commit comments

Comments
 (0)