Skip to content

Introduce Testing to Stacks #1194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 38 commits into from
Dec 15, 2022
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
f07da4d
test(setup): add a poc for component testing
giamir Oct 7, 2022
d1e430f
match orginial indentation for diff purposes
dancormier Oct 18, 2022
414fb28
test(tooltip): add wtr visual regression example
giamir Oct 27, 2022
de78eb0
Merge branch 'develop' into gbuoncristiani/introduce-testing-setup
giamir Nov 25, 2022
b3c1ba1
refactor(tooltip): remove old tooltip version
giamir Nov 25, 2022
3df1318
test(backstop): mark backstop tests as legacy
giamir Nov 25, 2022
7f075c3
ci(workflow): add basic pipeline to build and run tests
giamir Nov 28, 2022
664bef5
fix(styles): fix stylelint rules violations
giamir Nov 28, 2022
53937b3
fix(ts): fix eslint rules violations
giamir Nov 28, 2022
b5d1aea
fix(format): fix all prettier violations
giamir Nov 28, 2022
7559641
test(visual): remove baseline images from regular git tracking
giamir Nov 28, 2022
4793b1a
test(visual): track baseline images with git lfs
giamir Nov 28, 2022
688c071
fix(lint-ts): adjust rule exception after formatting
giamir Nov 28, 2022
d03dea1
ci(test): trigger another netlify deployment
giamir Nov 28, 2022
0d0b6dd
Merge branch 'develop' into gbuoncristiani/testing
giamir Nov 28, 2022
55d0c85
ci(test): update github workflow
giamir Nov 29, 2022
440ecc1
test(tooltip): fix width for visual regression test
giamir Nov 29, 2022
b7ab801
test(tooltip): adjust width for visual regression test
giamir Nov 29, 2022
1a850b9
ci(test): upload visual regression diffs on failure
giamir Nov 29, 2022
4e1fecf
ci(test): fix workflow syntax
giamir Nov 29, 2022
5f7a2f1
test(visual): settle on a default font family for tests
giamir Nov 29, 2022
56d88db
ci(test): fix small typo
giamir Nov 29, 2022
3613a6d
test(visual): set a failure threshold
giamir Nov 29, 2022
bf8999a
ci(test): adjust playwright install command
giamir Nov 29, 2022
9cc7f65
ci(test): add firefox launch options
giamir Nov 29, 2022
a4e6d71
ci(test): disable firefox temporarily
giamir Nov 29, 2022
b472f08
test(tooltip): add prefix s- for consistency
giamir Nov 29, 2022
f795b07
ci(test): fix and add back testing against firefox in CI
giamir Nov 29, 2022
690a8fe
Update README.md
giamir Dec 1, 2022
6055c67
Merge branch 'develop' into gbuoncristiani/testing
giamir Dec 1, 2022
c46b5b9
Merge branch 'develop' into gbuoncristiani/testing
giamir Dec 2, 2022
170ba48
fix(dev-server-rollup): apply temp patch to run tests in windows
giamir Dec 4, 2022
b4e1b0b
test(visual): relax failure threshold to 5 percent
giamir Dec 4, 2022
166236f
test(cleanup): remove backstop legacy tests
giamir Dec 4, 2022
91a56eb
test(visual): update default fonts
giamir Dec 6, 2022
b1ebfac
Merge branch 'develop' into gbuoncristiani/testing
giamir Dec 6, 2022
0caa564
Update .gitignore
giamir Dec 15, 2022
483c0cc
Merge branch 'develop' into gbuoncristiani/testing
giamir Dec 15, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
screenshots/** filter=lfs diff=lfs merge=lfs -text
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 Nice, this should keep us from accidentally committing images to the git history

46 changes: 46 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: CI

on:
push:
branches: [develop]
pull_request:
branches: [develop]

jobs:
build-and-test:
name: Build and Test
runs-on: ubuntu-latest
steps:
- name: ⬇️ Checkout
uses: actions/checkout@v3
with:
lfs: true

- name: ⎔ Setup node
uses: actions/setup-node@v3
with:
node-version: 'lts/*'

- name: 🏗 Install and Build
run: |
npm ci
npx playwright install --with-deps
npm run build

- name: ▶️ Lint and Test
run: |
npm run lint
npm run test -- --config web-test-runner.config.ci.mjs

- name: ⬆️ Upload Visual Regression Test Results
uses: actions/upload-artifact@v3
if: failure()
with:
name: visual-regression-test-results
path: screenshots


# cancel the jobs if another workflow is kicked off for the same branch
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,13 @@ dist/
/backstop_data/bitmaps_test/
/backstop_data/bitmaps_reference/
/backstop_data/html_report/
/screenshots/**/failed/**

# GitHub Actions #
###############
/.github/main.workflow
.vscode/

# JetBrains Editors #
###############
/.idea
4 changes: 3 additions & 1 deletion .prettierignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
*.md
.vscode/
dist/
dist/
# TODO: prettier is breaking less files while formatting
*.less
3 changes: 2 additions & 1 deletion .prettierrc
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,6 @@
"singleQuote": false,
"quoteProps": "consistent",
"trailingComma": "es5",
"printWidth": 80
"printWidth": 80,
"endOfLine": "auto"
}
66 changes: 65 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ Stacks documentation can be found at https://stackoverflow.design/

- [Using Stacks](#using-stacks)
- [Building Stacks](#building-stacks)
- [Linting Stacks](#linting-stacks)
- [Testing Stacks](#testing-stacks)
- [Bugs and feature requests](#bugs-and-feature-requests)
- [Contributing](#contributing)
- [License](#license)
Expand All @@ -32,8 +34,70 @@ To contribute to Stacks documentation or its CSS library, you’ll need to build

Having trouble getting these steps to work? Open [an issue](https://github.com/StackExchange/Stacks/issues/new) with a `setup` label.

## Linting Stacks

Run all lint suites by running:
```sh
npm run lint
```

Lint the styles (stylelint) by running:
```sh
npm run lint:css
```
Lint the typescript source code (eslint) via running:
```sh
npm run lint:ts
```
Lint the source code format (prettier) via running:
```sh
npm run lint:format
```
## Testing Stacks
Stacks has implemented visual regression testing with [Backstop](https://github.com/garris/BackstopJS). To test if your new feature introduces visual regressions, run `npm run test` in a new Terminal window while Stacks is running. After the tests have run, a new browser window with any regressions will show. If the regressions are desired, you can run `npm run approve` to establish the new baseline.

Run all test suites by running:
```sh
npm test
```
### Unit/Component Tests

Unit/Component tests are written with [DOM Testing Library](https://testing-library.com/docs/dom-testing-library/intro).
Please follow the library's principles and documentation to write tests.

Stacks uses [Web Test Runner](https://modern-web.dev/docs/test-runner/overview/) and [Playwright](https://modern-web.dev/docs/test-runner/browser-launchers/playwright/) to run tests in a real browser context.

Execute the unit/component tests suite by running:
```sh
npm run test:unit
```
or if you prefer watch mode run:
```sh
npm run test:unit:watch
```

### Visual Regression Tests

**Prerequisit:**
In order to pull and upload baseline images you need to have `git lfs` installed in your local machine. Please follow [this guide](https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage) to install that.


This [Web Test Runner plugin](https://www.npmjs.com/package/@web/test-runner-visual-regression) is used to run visual regression tests. [DOM Testing Library](https://testing-library.com/docs/dom-testing-library/intro).
Visual regression tests end with this suffix `*.visual.test.ts`.

Execute the visual regression tests suite by running:
```sh
npm run test:visual
```

Update the visual baseline via:
```sh
npm run test:visual:update
```

Failing tests (including diffs) can be found under `screenshots/[browser]/failed/` folders.

### Visual Regression Tests (Legacy)
Stacks has implemented visual regression testing with [Backstop](https://github.com/garris/BackstopJS). To test if your new feature introduces visual regressions, run `npm run test:legacy:regression` in a new Terminal window while Stacks is running. After the tests have run, a new browser window with any regressions will show. If the regressions are desired, you can run `npm run test:legacy:regression:update` to establish the new baseline.

Individual routes to test are found in [backstop.json](/backstop.json)

Expand Down
2 changes: 1 addition & 1 deletion lib/css/atomic/gap.less
Original file line number Diff line number Diff line change
Expand Up @@ -40,5 +40,5 @@
.gx0, .gx1, .gx2, .gx4, .gx6, .gx8, .gx12, .gx16, .gx24, .gx32, .gx48, .gx64,
.gy0, .gy1, .gy2, .gy4, .gy6, .gy8, .gy12, .gy16, .gy24, .gy32, .gy48, .gy64,
.g0, .g1, .g2, .g4, .g6, .g8, .g12, .g16, .g24, .g32, .g48, .g64 {
gap: var(--_gap-y, 0) var(--_gap-x, 0);
gap: var(--_gap-y, 0) var(--_gap-x, 0);
}
2 changes: 1 addition & 1 deletion lib/css/components/cards.less
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
--_ca-bc: var(--bc-light);

// Dim only the first child card content
> * {
> * {
opacity: 0.65;
}
}
Expand Down
2 changes: 1 addition & 1 deletion lib/css/components/expandable.less
Original file line number Diff line number Diff line change
Expand Up @@ -111,4 +111,4 @@
-webkit-clip-path: var(--_ex-clip-path);
clip-path: var(--_ex-clip-path);
transition: clip-path 0s var(--_ex-transition-duration), -webkit-clip-path 0s var(--_ex-transition-duration);
}
}
22 changes: 4 additions & 18 deletions lib/css/components/inputs.less
Original file line number Diff line number Diff line change
Expand Up @@ -609,34 +609,18 @@ fieldset {
font-size: var(--fs-caption);
}

// $$ SIZES
// $$ PADDING ADJUSTMENTS AND SIZES
// ----------------------------------------------------------------------------
.s-input__sm,
.s-textarea__sm,
.s-select__sm > select {
font-size: var(--fs-caption);
}
.s-input__md,
.s-textarea__md,
.s-select__md > select {
font-size: var(--fs-body3);
}
.s-input__lg,
.s-textarea__lg,
.s-select__lg > select {
font-size: var(--fs-title);
}
.s-input__xl,
.s-textarea__xl,
.s-select__xl > select {
font-size: var(--fs-headline1);
}

// $$ PADDING ADJUSTMENTS WITHIN SIZES
// ----------------------------------------------------------------------------
.s-input__md,
.s-textarea__md,
.s-select__md > select {
font-size: var(--fs-body3);
padding-top: 0.5em;
padding-bottom: 0.5em;
border-radius: calc(var(--br-sm) + 1px);
Expand All @@ -650,13 +634,15 @@ fieldset {
.s-input__lg,
.s-textarea__lg,
.s-select__lg > select {
font-size: var(--fs-title);
padding: 0.45em 0.6em;
border-radius: calc(var(--br-sm) + 1px);
}

.s-input__xl,
.s-textarea__xl,
.s-select__xl > select {
font-size: var(--fs-headline1);
padding: 0.4em 0.5em;
border-radius: var(--br-md);
}
Expand Down
2 changes: 1 addition & 1 deletion lib/css/components/pagination.less
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
.s-pagination {
& &--item {
& &--item {
--_pa-item-bg: transparent;
--_pa-item-bc: var(--bc-medium);
--_pa-item-fc: var(--fc-medium);
Expand Down
13 changes: 6 additions & 7 deletions lib/css/components/popovers.less
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,15 @@
--_po-arrow-after-r: unset;
--_po-arrow-after-t: unset;
--_po-arrow-after-bs: unset;

// CONTEXTUAL STYLES
.dark-mode({
--_po-bg: var(--black-075);
--_po-bc: var(--bc-light);
--_po-bs: var(--bs-lg);
--_po-arrow-fc: var(--black-075);
});

// MODIFIERS
&.is-visible {
--_po-d: block;
Expand All @@ -34,7 +34,7 @@
--_po-wmn: unset;
--_po-w: auto;
}

// CHILD ELEMENTS
// Arrow
&[data-popper-placement^="top"] > &--arrow,
Expand Down Expand Up @@ -128,14 +128,14 @@
right: calc(var(--su8) * -1); // Compensate for s-popover's padding
padding: var(--su8) !important;
}

background-color: var(--_po-bg);
border: 1px solid var(--_po-bc);
box-shadow: var(--_po-bs);
display: var(--_po-d);
min-width: var(--_po-wmn);
width: var(--_po-w);

border-radius: var(--br-md);
color: var(--fc-dark);
font-size: var(--fs-body1);
Expand All @@ -144,5 +144,4 @@
position: absolute;
white-space: normal; // Guard against popovers being in a container with white-space: nowrap. Without this, the content pops *out* of the popover.
z-index: var(--zi-popovers);
}

}
62 changes: 62 additions & 0 deletions lib/test/s-tooltip.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
import { html, fixture, expect } from "@open-wc/testing";
import { screen, waitForElementToBeRemoved } from "@testing-library/dom";
import userEvent from "@testing-library/user-event";
import "../ts/index";

const user = userEvent.setup();

describe("s-tooltip", () => {
it("should make the tooltip element visible on hover (after a delay)", async () => {
const trigger = await fixture(html`
<button
class="s-btn s-btn__filled"
role="button"
data-controller="s-tooltip"
title="tooltip content"
data-s-tooltip-placement="bottom-start"
>
Hover tooltip popover
</button>
`);

expect(screen.queryByRole("tooltip")).to.be.null;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooooh, I like this! Forcing the tests to use the accessibility tree is also a way to force a11y testing in general.


await user.hover(trigger);
const tooltip = await screen.findByRole("tooltip");
expect(tooltip).to.be.visible;

await user.unhover(trigger);
await waitForElementToBeRemoved(() => screen.queryByRole("tooltip"));
});

it("should not flicker when the host contains an SVG and the user hover on it", async () => {
await fixture(html`
<button
class="s-btn"
role="button"
title="tooltip content"
data-controller="s-tooltip"
data-s-tooltip-placement="bottom-start"
>
<svg
data-testid="svg"
aria-hidden="true"
class="bg-red-500"
width="18"
height="18"
viewBox="0 0 18 18"
></svg>
👈 Shouldn't flicker 🤷‍
</button>
`);
const button = screen.getByRole("button");
const svg = screen.getByTestId("svg");

await user.hover(button);
const tooltip = await screen.findByRole("tooltip");
expect(tooltip).to.be.visible;

await user.hover(svg);
expect(tooltip).to.be.visible;
});
});
31 changes: 31 additions & 0 deletions lib/test/s-tooltip.visual.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import { html, fixture } from "@open-wc/testing";
import { screen } from "@testing-library/dom";
import userEvent from "@testing-library/user-event";
import { visualDiff } from "@web/test-runner-visual-regression";
import "../ts/index";

const user = userEvent.setup();

describe("s-tooltip", () => {
it("should not introduce visual regressions", async () => {
const wrapper = await fixture(html`
<div style="height: 100px; width: 160px; display: inline-block;">
<button
class="s-btn s-btn__filled"
role="button"
data-controller="s-tooltip"
title="tooltip content"
data-s-tooltip-placement="bottom-start"
>
Hover tooltip popover
</button>
</div>
`);

const trigger = screen.getByRole("button");
await user.hover(trigger);
await screen.findByRole("tooltip");

await visualDiff(wrapper, "s-tooltip");
});
});
Loading