← 返回首页
simplify screenshot testing by kushalkolar · Pull Request #447 · fastplotlib/fastplotlib · GitHub
Skip to content

Navigation Menu

Toggle navigation
Sign in
Appearance settings
Search or jump to...

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Resetting focus

simplify screenshot testing#447

Merged
kushalkolar merged 15 commits into
mainfrom
simplify-screenshot-testing
Mar 31, 2024
Merged

simplify screenshot testing#447
kushalkolar merged 15 commits into
mainfrom
simplify-screenshot-testing

Conversation

kushalkolar commented Mar 23, 2024
edited
Loading

Copy link
Copy Markdown
Member

use ideas from pygfx/pygfx#684

closes #446
A threshold of 0.2 seemed like it was too lenient, I was able to change the color of one of the circles in the line collection examples and the RMSE was 0.09

kushalkolar marked this pull request as ready for review March 23, 2024 04:24
kushalkolar requested a review from clewis7 as a code owner March 23, 2024 04:24

kushalkolar commented Mar 23, 2024
edited
Loading

Copy link
Copy Markdown
Member Author

This seems to work quite well. The ground truths were generated on my RX 570 at home, the largest diff was with the scatter_size example, due to the text. The diff on github actions was slightly over 0.02. I also tested on the GPUs on the Flatiron cluster, the diff for that example is 0.020826.

There seems to be no difference between the diffs on Nvidia Tesla V100 and Nvidia A100 on the cluster, at least Nvidia is consistent about one think I guess 😂

EDIT: Just tested on the Nvidia H100 and I get a max diff of 0.02077 on the scatter_size example. The diffs are a bit difference than on the V100 and A100 so I guess Nvidia actually isn't consistent.

diffs for all screenshots on a Tesla V100 on the Flatiron cluster for reference:
This is the diff between the Rx 570 and the overpowered Nvidia GPUs

examples/tests/test_examples.py::test_that_we_are_on_lavapipe DiscreteGPU Vulkan examples/tests/test_examples.py::test_example_screenshots[gridplot] 0.0025418061938043534 examples/tests/test_examples.py::test_example_screenshots[gridplot_non_square] 0.0017253794384335269 examples/tests/test_examples.py::test_example_screenshots[heatmap] 0.0026005766798322047 examples/tests/test_examples.py::test_example_screenshots[heatmap_cmap] 0.002662877302239744 examples/tests/test_examples.py::test_example_screenshots[heatmap_data] 0.0020797810741957933 examples/tests/test_examples.py::test_example_screenshots[heatmap_vmin_vmax] 0.002914066889570422 examples/tests/test_examples.py::test_example_screenshots[image_cmap] 0.0019563093643064503 examples/tests/test_examples.py::test_example_screenshots[image_rgb] 0.0014489276911573798 examples/tests/test_examples.py::test_example_screenshots[image_rgbvminvmax] 0.0005612692156754262 examples/tests/test_examples.py::test_example_screenshots[image_simple] 0.0016309417498936698 examples/tests/test_examples.py::test_example_screenshots[image_vminvmax] 0.0005612692156754262 examples/tests/test_examples.py::test_example_screenshots[line] 0.0029302534153462386 examples/tests/test_examples.py::test_example_screenshots[line_cmap] 0.003085553070868819 examples/tests/test_examples.py::test_example_screenshots[line_colorslice] 0.0031247686572509034 examples/tests/test_examples.py::test_example_screenshots[line_dataslice] 0.0023580007672577286 examples/tests/test_examples.py::test_example_screenshots[line_present_scaling] 0.0027367853208208497 examples/tests/test_examples.py::test_example_screenshots[line_collection] 0.003779845416274792 examples/tests/test_examples.py::test_example_screenshots[line_collection_cmap_values] 0.0023366316256708897 examples/tests/test_examples.py::test_example_screenshots[line_collection_cmap_values_qualitative] 0.002891149865417338 examples/tests/test_examples.py::test_example_screenshots[line_collection_colors] 0.0`022398405285301453 examples/tests/test_examples.py::test_example_screenshots[line_stack] 0.005911900712291164 examples/tests/test_examples.py::test_example_screenshots[scatter] 0.012353324462660456 examples/tests/test_examples.py::test_example_screenshots[scatter_cmap] 0.0030825225024386587 examples/tests/test_examples.py::test_example_screenshots[scatter_colorslice] 0.004020251842124368 examples/tests/test_examples.py::test_example_screenshots[scatter_dataslice] 0.013067512988075274 examples/tests/test_examples.py::test_example_screenshots[scatter_present] 0.013810446602977897 examples/tests/test_examples.py::test_example_screenshots[scatter_size] 0.020826465412837777

Comment thread examples/tests/test_examples.py Outdated Show resolved Hide resolved

Copy link
Copy Markdown
Member Author

So on Mac there seems to be large diffs with the heatmaps, no sure why, will figure out later.




kushalkolar commented Mar 23, 2024
edited
Loading

Copy link
Copy Markdown
Member Author

I think that we should figure out how to make the heatmap test screenshot have more of the actual heatmap, not sure why it's small in the top left corner.

Or maybe think about getting rid of heatmap since the Texture dimension limit seems to no longer apply with WGPU for the past while, need to investigate this 🤔

kushalkolar requested a review from almarklein March 23, 2024 05:17
kushalkolar merged commit 9a1bfbe into main Mar 31, 2024
kushalkolar deleted the simplify-screenshot-testing branch March 31, 2024 09:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

better image comparison for tests

2 participants

Footer

© 2026 GitHub, Inc.