I've used the subTest method of unittest.TestCase in test_env_vars_for_windows_tests. This pytest plugin gives pytest full support for that mechanism. It lets pytest continue with the other subtests even after one fails, as the unittest test runner would, and show separate passes and failures for the individual subtests. Separate results are shown only when at least one fails, so when everything passes the report remains uncluttered. (It also provides a subtests pytest fixture, though since this is not an autouse fixture it is not usable from within a method of a class that inherits from unittest.TestCase.)

I think this is useful to have going forward, since we have many test cases that are large with many separate assertions of separate facts about the system under test, and as they are updated, some of them could be improved by having their separate claims divided into subtests so they can be individually described and so failures don't unnecessarily block later subtests.

However, if you'd rather this plugin be used, it can be removed. test_env_vars_for_windows_tests could retain subtests--they will still each run if none fails, just like multiple assertions in a test case without subtests. Or I could replace the subtests with more @ddt parameterization, or manually, etc.

I all sounds reasonable and in particular, since you are the one doing the work it seems fair that you can use the tooling you see as best fit. I also have no experience here and no preferences, and believe that anything that improves the tests in any way is very welcome. Thank you!

Sounds good. For the conceptually unrelated reason that it would facilitate more fine-grained xfail marking--so we don't habitually carry XPASSing cases that aren't actually expected to have failed--I think the test class involved here, test.test_util.TestUtils, should be split, so that the cygpath-related tests can be pure pytest tests. The reason to split it is:

• @pytest.mark.parametrize supports fine-grained marking of individual test cases, including as skip or xfail, but that form of parameterization cannot be applied to test methods in unittest.TestCase subclasses. So at least those tests would currently (i.e., until the underlying causes of some cases failing are addressed) benefit from being pure pytest tests.
• Some of the test methods, not for cygpath, use the rorepo fixture provided by test.lib.helper.TestBase, which inherits from unittest.TestCase. Although this could be converted to a pytest fixture, I'd rather wait to do that until after more operating systems, at least Windows, are tested on CI, and also until I have more insight into whether it makes sense to do that at all, rather than replacing rorepo and other fixtures with new corresponding fixtures that use isolated repositories (#914, #1693 (review)). So at least those tests should currently remain in a unittest.TestCase subclass.

So long as it's acceptable to have multiple test classes in the same test module, this could be done at any time, and it may facilitate some other simplifications. I mention it here because I think it might lead to the elimination of subtests in this particular module, either by using @pytest.mark.parametrize for this too or for other reasons.

If that happens, I might remove the pytest-subtests test dependency, even though it might be re-added later, because the alternative of pytest-check may be preferable in some of the large test methods if they can also be converted to be pure pytest tests (because pytest-check supports a more compact syntax).

I all sounds reasonable and in particular, since you are the one doing the work it seems fair that you can use the tooling you see as best fit. I also have no experience here and no preferences, and believe that anything that improves the tests in any way is very welcome. Thank you!