autogenerate ducktyping test suites

[keewis]

I'm the co-maintainer of a library providing a few types which wrap duck types. To make the adoption easier I would like to provide a function which generates a test suite to check their compatibility with my library.

I came up with something like this:

def generate_tests(create, extra_asserts=None, marks=None):
    class TestDuckType:
        class TestClassA:
            @pytest.mark.parametrize("x", ("a", "b", "c"))
            def test_func(self, x):
                data = func(x)
                instance = create(data)
                # ...
                extra_asserts(actual, expected)

            @pytest.mark.parametrize("y", (0, 2, 4, 6))
            def test_func2(self, y):
                # ...
        class TestClassB:
            # ...
        # ...
    # attach marks
    return TestDuckType

where marks can be used to skip / xfail specific tests / test items.

However, attaching marks to specific test items seems to require a pytest_collection_modifyitems hook, which would require me to write and distribute a plugin if I want to use generate_tests in other projects. Additionally, since the creation and possibly the test itself might need more parameters depending on the duck type I would either need to figure out a way to dynamically add more parameters, or switch to using fixtures and their params kwarg instead of creation functions.

I did consider using subclasses and staticmethods for the callbacks (create, etc), but rejected this because the parametrize marks get stripped for subclasses (I think there was a issue for that but I can't find it right now).

All of this makes me wonder if I'm missing something obvious. Are there any established patterns or libraries which handle this sort of testing? If not, is there a way to make this a bit easier to maintain?

[RonnyPfannschmidt]

pytest currently has no utilities/patterns to inherit/reconfigure partial testsuites

so currently anything you want to ship with your lib to enable plugin/extension implementers to build those things is currently to be selfmade

some people use inheritance/mixins to work around that