Skip to content

Commit 3949e8b

Browse files
committed
Adding 2.0 unit testing updates from @Vaivaswat2244
1 parent 479e246 commit 3949e8b

File tree

3 files changed

+73
-0
lines changed

3 files changed

+73
-0
lines changed
5.43 KB
Loading
10.6 KB
Loading

contributor_docs/unit_testing.md

+73
Original file line numberDiff line numberDiff line change
@@ -249,3 +249,76 @@ visualSuite('3D Model rendering', function() {
249249
});
250250
```
251251

252+
253+
Different operating systems and browsers render graphics with subtle variations. These differences are normal and shouldn't cause tests to fail.
254+
Common acceptable differences include:
255+
256+
* Single-pixel shifts in line positions
257+
* Slight variations in anti-aliasing
258+
* Text rendering differences (especially font weight and kerning)
259+
* Minor differences in curve smoothness
260+
261+
For example, text rendered on macOS might appear slightly different from text rendered in a Linux CI environment. The same applies to thin lines, curves, and other graphical elements with anti-aliasing.
262+
An example of this can be the below image which earlier caused tests to fail in CI because of different rendering environments.
263+
264+
![Example](./images/pixelmatch2.png)
265+
266+
The p5.js visual testing system uses a sophisticated algorithm to distinguish between acceptable rendering variations and actual bugs:
267+
268+
* Initial comparison - Compares pixels with a moderate threshold (0.5) to identify differences using [pixelmatch](https://github.com/mapbox/pixelmatch) library for pixel to pixel comparison.
269+
* Cluster identification - Groups connected difference pixels using a Breadth-First Search (BFS) algorithm
270+
* Pattern recognition - The algorithm specifically identifies:
271+
272+
- "Line shift" clusters - differences that likely represent the same visual element shifted by 1px
273+
- Isolated pixel differences (noise)
274+
275+
276+
* Smart failure criteria - Applies different thresholds:
277+
278+
- Ignores clusters smaller than 4 pixels
279+
- Allows up to 40 total significant difference pixels
280+
- Permits minor line shifts that are typical across platforms
281+
282+
The below is the example of the tests that should fail:
283+
284+
![Example](./images/pixelmatch.png)
285+
286+
287+
288+
This approach balances sensitivity to real bugs while tolerating platform-specific rendering variations. The algorithm uses these key parameters:
289+
```js
290+
const MIN_CLUSTER_SIZE = 4; // Minimum significant cluster size
291+
const MAX_TOTAL_DIFF_PIXELS = 40; // Maximum allowed significant differences
292+
```
293+
The algorithm identifies line shifts by analyzing the neighborhood of each difference pixel. If more than 80% of pixels in a cluster have ≤2 neighbors, it's classified as a line shift rather than a structural difference.
294+
This intelligent comparison ensures tests don't fail due to minor rendering differences while still catching actual visual bugs.
295+
296+
It's important to note that the improved algorithm described above allows tests with acceptable platform-specific variations to pass correctly. Tests that previously failed due to minor rendering differences (like anti-aliasing variations or subtle text rendering differences) will now pass as they should, while still detecting actual rendering bugs.
297+
For example, a test showing text rendering that previously failed on CI (despite looking correct visually) will now pass with the improved algorithm, as it can distinguish between meaningful differences and acceptable platform-specific rendering variations. This makes the test suite more reliable and reduces false failures that require manual investigation.
298+
299+
### Some best practices for writing visual tests
300+
301+
When creating visual tests for p5.js, following these practices will help ensure reliable and efficient tests:
302+
303+
* Keep canvas sizes small - Use dimensions close to 50x50 pixels whenever possible. The test system resizes images for efficiency before comparison, and smaller canvases result in faster tests, especially on CI environments.
304+
* Focus on visible details - At small canvas sizes, intricate details may be hard to distinguish. Design your test sketches to clearly demonstrate the feature being tested with elements that are visible at the reduced size.
305+
* Use multiple screenshots per test - Instead of cramming many variants into a single screenshot, call screenshot() multiple times within a test:
306+
```js
307+
visualTest('stroke weight variations', function(p5, screenshot) {
308+
p5.createCanvas(50, 50);
309+
310+
// Test thin stroke
311+
p5.background(200);
312+
p5.stroke(0);
313+
p5.strokeWeight(1);
314+
p5.line(10, 25, 40, 25);
315+
screenshot(); // Screenshot with thin lines
316+
317+
// Test thick stroke
318+
p5.background(200);
319+
p5.strokeWeight(5);
320+
p5.line(10, 25, 40, 25);
321+
screenshot(); // Screenshot with thick lines
322+
});
323+
```
324+

0 commit comments

Comments
 (0)