feat: Introduce `testing.constructors` module and a pytest plugin by FBruzzesi · Pull Request #3552 · narwhals-dev/narwhals

FBruzzesi · 2026-04-16T13:26:49Z

Description

This PR introduces two new features:

A rework of tests/conftest.py into narwhals/testing/constructors
- TBD what should be publicly exposed in here
A pytest plugin that exposes the fixtures for dataframe/lazyframe constructors.
- I wonder if we should already return them as narwhals DataFrame/LazyFrame. If we find a way to achieve the following points, then I personally think we should do so, as we almost always run nw.from_native right after.
  - Distinguish between native frame and nw.from_native kwargs -> For this it's enough to provide it as different proper arguments in __call__, e.g. nw_kwargs, backend_kwargs.
  - How do we know which version a user would use? Should we require to pass either the Version.X or the namespace {nw, nw_v1, nw_v2} ? If we do so, I would argue that we must enforce it and never assume for the user. In my opinion nw_frame_constructor(data, version=nw) would still look better than nw.from_native(nw_frame_constructor(data))
- In light of the CI failures for downstream projects, I also realized that having fixtures called constructor and constructor_eager is not really intuitive. I am happy to brainstorm better names for downstream users. And similarly the flags --all-cpu-constructors, --constructors. (*)
  Update: refactor: Rename exposed fixtures and pytest options #3556
- We could provide a nw_lazy_constructor for "free"
- I believe it might be useful to expose a series constructor (under the name nw_series_constructor). I know it's possible to make a dataframe and then get a column. But it would definitely improve users ergonomics. However, that's feature request/improvement and out of scope of this PR

(*) Renaming proposal

fixtures: nw_frame_constructor, nw_eager_frame_constructor
pytest options: --nw-all-backends (misleading that has no CPU in it?), --nw-backends=.... I think the term constructor might be be too vague, and there is no other case in the codebase. While we have a bunch of cases in which we allow to pass a backend

Now implemented in #3556, targeting this branch

pyest-cov changes

The [project.entry-points.pytest11] registers the pytest plugin which we dogfood in the testsuite here. When pytest starts, it loads all entry-point plugins before pytest-cov installs its coverage tracer.

Importing narwhals.testing.pytest_plugin forces Python to load narwhals/__init__.py (to resolve the package), which eagerly imports the entire public API.

On main, no entry point existed, so no narwhals code loaded before coverage started.

The fix:

Remove pytest-cov dependency
Add coverage options in pyproject.toml to track subprocess, execv and fork (the latter two are not available on windows, hence multiple hacks are performed in GithubAction's to dynamically populate the fields in pyproject.toml)
Manually run coverage run -m pytest ..., coverage combine and coverage report, so that coverage run starts the tracer at Python startup before any imports

Thanks to @EdAbati and @dangotbanned - I mostly adjusted from your work, not much more than that!

What type of PR is this? (check all applicable)

Related issues

Related issue [Enh]: Provide testing module to enable downstream libraries testing #739 (not marking as closing, since we should fully dogfood in the test suite)

Checklist

Code follows style guide (ruff)
Tests added
Documented the changes

dangotbanned · 2026-04-16T14:12:41Z

Just a quick one from me RE:

Don't panic due to the number of file changes - I will explain

We have 257 file changes because all tests change the imports as:
- from tests.utils import Constructor, ConstructorEager 
+ from narwhals.testing.typing import Constructor, ConstructorEager

Is it possible to add something like this, and keep the imports in place (temporarily)?:

# tests.utils.py
from narwhals.testing.typing import Constructor as Constructor , ConstructorEager as ConstructorEager

I really REALLY want this PR, but the diff 😳 (yes yes, I know I'm guilty of this too 😉)

FBruzzesi · 2026-04-16T14:50:32Z

@dangotbanned reverted such commit 😉

Coverage is not picked up in CI though 🥹

FBruzzesi · 2026-04-18T21:23:31Z

@MarcoGorelli @dangotbanned @camriddell I would say this PR is ready to review as is - yet not ready to be merged because all the open points + conflicts with plotly & fairlearn pytest fixtures (#3556 addresses this last issue)

camriddler

As a first pass I focused on the testing.constructors.py since that is where the implementation details are.

The current registration mechanism has implicit conditions (e.g. name in cls.__dict__) that can cause subclasses to fail to register without error (I believe this is used to implicitly differentiate leaf classes). Combined with metadata split between class-body attributes and subclass keyword arguments, this increases contributor error surface and makes registration behavior harder to reason about.

The above observations increase the chance that downstream users will need to read the source code implementation if they are cut by either of above details. (e.g. "why is my class not registering? Oh I forgot to add a name=...)

Open for discussion on this, but I'm still a fan of the decorator + function based registration for a few reasons:

Constructing instances is easier to validate/invalidate than initializing subclasses which may have some documentation-only fields (e.g. anything you can put into the class body is fair game)
The registration mechanism is an explicit "opt-in" (either the decorator is there or it is not) instead of being tucked behind a class definition + specific field name availablility
The functions can keep their same name as they originally had and we don't need concern with the legacy_name field at all while we transition away from the str(constructor) stuff.

from dataclasses import dataclass, field

@dataclass(frozen=True)
class Constructor:
    name: str
    is_eager: bool
    # omitting fields for brevity

    func: Callable[[...], ...]

    _registry = {} # could be a global or class var

    def __call__(self, *args, **kwargs):
        return self.func(*args, **kwargs)

    @classmethod
    def from_func(cls, name, is_eager):
        def decorator(func):
            inst = cls(name=name, is_eager=is_eager, func=func)
            cls._registry[name] = inst
            return inst
        return decorator

@Constructor.from_func("pandas", is_eager=True)
def pandas_constructor(data): # can keep the original function calls, str(...) behavior does not change
    import pandas as pd
    return pd.DataFrame(data)

print(f'{Constructor._registry = }')

# constructors are the same as before, but have metadata attached directly to them
print(f'{pandas_constructor = }')
print(f'{pandas_constructor.is_eager = }')

outputs

Constructor._registry = {'pandas': Constructor(name='pandas', is_eager=True)}
pandas_constructor = Constructor(name='pandas', is_eager=True)
pandas_constructor.is_eager = True

The caveat of course being that the function pandas_constructor is no longer a function, it is an instance of Constructor that has an identical __call__ behavior to the function it decorated.

camriddler · 2026-04-20T20:49:02Z

+        cls.needs_gpu = needs_gpu
+
+        if "name" not in cls.__dict__:
+            return


Does this if check this only exist to catch classes that don't have a name to them? Thus meaning "if you don't have a name specified, you don't get inserted in to the registration mechanism? Seems surprising that if I forget to add a name= to my class that I get neither the registration behavior nor an error telling me I forgot something.

camriddler · 2026-04-20T20:55:24Z

+        implementation: Implementation | None = None,
+        requirements: tuple[str, ...] = (),
+        legacy_name: str = "",
+        is_non_nullable: bool = False,


I'd personally prefer is_nullable instead of is_non_nullable (if we don't have a backwards compat to worry about)? I find the boolean logic to be easier to understand when making assertions about the positive case as something like not is_non_nullable is a touch trickier to reason about than not is_nullable.

Absolutely I fully agree

camriddler · 2026-04-20T20:57:34Z

+            needs_gpu: Whether the backend requires GPU hardware.
+            **kwargs: Forwarded to `super().__init_subclass__`.
+        """
+        super().__init_subclass__(**kwargs)


Is there a suspected case for this super? I can't imagine nesting classes that override this logic.

Fair enough - I kept it as "you never know what users want to do"

That's fair, and this won't hurt us so I'm +1 to keep it in. Was curious if you had something in mind :)

camriddler · 2026-04-20T20:59:30Z

+        raise NotImplementedError
+
+
+class LazyFrameConstructor(FrameConstructor):


Do these Lazy|Eager... sub-classes help anything? They set a default class variable for their children, but it feels like boilerplate outside of that.

Useful for type checking (especially constructor_eager) + setting one class variable.

camriddler · 2026-04-20T21:00:17Z

+):
+    """Constructor backed by `pandas.DataFrame` with default NumPy dtypes."""
+
+    name = "pandas"


Feels odd that this argument needs to be attached to the class body instead of passing into the class construction mechanism.

It was an hack to allow to create the two intermediate EagerFrameConstructor and LazyFrameConstructor without passing a name (hence without registering the two)

camriddler · 2026-04-20T21:01:57Z

+
+ALL_CPU_CONSTRUCTORS: frozenset[str] = frozenset(
+    name for name, c in FrameConstructor._registry.items() if not c.needs_gpu
+)


Can ALL_CPU_CONSTRUCTORS be a module property or function so that it is kept it in sync in-case the registry is updated after this variable is created?

Exactly as what was done with available_constructors below.

Makes sense, thanks for suggesting it

camriddler · 2026-04-20T21:08:40Z

+def prepare_constructors(
+    *, include: Iterable[str] | None = None, exclude: Iterable[str] | None = None
+) -> list[FrameConstructor]:
+    """Return available constructors, optionally filtered.


We should work more with the underlying types when possible instead of strings. A tricky bug will probably be prepare_constructors(include=["pndas"]) and you don't get anything back (not even an error). Whereas prepare_constructors(include=testing.PndasConstructor) provides an easy to understand error about what was mistyped.

Needs a comment that the exclude is given precedence here since it is the last conditions that are checked (e.g. it is currently allowed to do prepare_constructors(include=["pandas"], exclude=["pandas"]) and becomes ambiguous as to what one should expect to return when perhaps this should be an errant case?

I think I am not even using it anymore now - if we are going to expose something like this, then I am definitely going to add better type checking. Sure will add better docstrings if it stays

camriddler · 2026-04-20T21:16:41Z

+)
+
+
+class FrameConstructor(Protocol):


What is gained from making this a typing.Protocol?

I think it has to do with the two EagerFrameConstructor and LazyFrameConstructor but it was a bunch of commits ago... I swear I already forgot 😭

If it isn't needed can we remove it? At least in my mind (and someone please update my way of thinking), but a Protocol shouldn't have any logic embedded into its methods just scaffolding for attribute & method names/signatures and types. But that's more of a conceptual framework rather than something enforced strongly at runtime.

FBruzzesi

Thanks for the review @camriddell - there are a lot of great points made there! I replied to single comments

RE: #3552 (review)

My TL;DR is that I don't have a strong preference.

The current registration mechanism has implicit conditions (e.g. name in cls.__dict__) that can cause subclasses to fail to register without error (I believe this is used to implicitly differentiate leaf classes). Combined with metadata split between class-body attributes and subclass keyword arguments, this increases contributor error surface and makes registration behavior harder to reason about.

That's totally valid, and yes we should pick one way of declaring what we need, and avoid having to declare both a class variable and metadata, it should be either one or the other. Also we should not fail silently.

The functions can keep their same name as they originally had and we don't need concern with the legacy_name field at all while we transition away from the str(constructor) stuff.

Oh man, that's a killer feature!
I will sleep on how to do it a bit longer, thanks for providing example code as well!

The caveat of course being that the function pandas_constructor is no longer a function, it is an instance of Constructor that has an identical __call__ behavior to the function it decorated.

I would not worry too much about this to be honest

FBruzzesi · 2026-04-20T22:05:22Z

+def prepare_constructors(
+    *, include: Iterable[str] | None = None, exclude: Iterable[str] | None = None
+) -> list[FrameConstructor]:
+    """Return available constructors, optionally filtered.


I think I am not even using it anymore now - if we are going to expose something like this, then I am definitely going to add better type checking. Sure will add better docstrings if it stays

FBruzzesi · 2026-04-20T22:06:06Z

+
+ALL_CPU_CONSTRUCTORS: frozenset[str] = frozenset(
+    name for name, c in FrameConstructor._registry.items() if not c.needs_gpu
+)


Makes sense, thanks for suggesting it

FBruzzesi · 2026-04-20T22:07:41Z

+):
+    """Constructor backed by `pandas.DataFrame` with default NumPy dtypes."""
+
+    name = "pandas"


It was an hack to allow to create the two intermediate EagerFrameConstructor and LazyFrameConstructor without passing a name (hence without registering the two)

FBruzzesi · 2026-04-20T22:09:48Z

+        raise NotImplementedError
+
+
+class LazyFrameConstructor(FrameConstructor):


Useful for type checking (especially constructor_eager) + setting one class variable.

FBruzzesi · 2026-04-20T22:12:47Z

+            needs_gpu: Whether the backend requires GPU hardware.
+            **kwargs: Forwarded to `super().__init_subclass__`.
+        """
+        super().__init_subclass__(**kwargs)


Fair enough - I kept it as "you never know what users want to do"

FBruzzesi · 2026-04-20T22:13:17Z

+        implementation: Implementation | None = None,
+        requirements: tuple[str, ...] = (),
+        legacy_name: str = "",
+        is_non_nullable: bool = False,


Absolutely I fully agree

FBruzzesi · 2026-04-20T22:14:15Z

+)
+
+
+class FrameConstructor(Protocol):


I think it has to do with the two EagerFrameConstructor and LazyFrameConstructor but it was a bunch of commits ago... I swear I already forgot 😭

camriddell · 2026-04-21T13:53:24Z

@FBruzzesi want to correct an earlier statement, (yeah this one woke me up in the middle of the night)

The functions can keep their same name as they originally had and we don't need concern with the legacy_name field at all while we transition away from the str(constructor) stuff.

This wasn't actually true, we would need to override the class __str__ method as you had done previously. The only benefit would be that the name isn't a floating string, it can just be captured from the function name.

print(f'{str(pandas_constructor) = }') # Constructor(name='pandas', is_eager=True, ...)
print(f'{str(pandas_constructor.func) = }') # '<function pandas_constructor at 0x7f6f96213b60>'

The modifications:

class Constructor:
    ....
    def __str__(self):
        return self.func.__name__ # or str(self.func) for a more literal backwards compat

FBruzzesi · 2026-04-21T21:22:54Z

@camriddell I moved to the registry pattern in e1d09e5, looking forward to your feedback!

FBruzzesi added 14 commits April 15, 2026 20:40

feat: pytest plugin

100ac3b

simplify a bit

1fd776d

Simplify narwhals codebase

6ba28de

bulk tests change with minimal impact

7e2c50d

Adjust TPCH

a6361f9

split testing plugin and constructors into multiple test files

8c0f27f

mix with Dan implementation

24743ce

fixup and allow kwargs

1e05a49

simplify pytest_plugin

c55beaa

merge main

879fb3e

defer imports

7e05734

adjust GHA accordingly

db4784e

constructors module coverage

1ad6ffb

add makefile rule

c833b87

FBruzzesi added enhancement New feature or request tests labels Apr 16, 2026

FBruzzesi added 2 commits April 16, 2026 16:00

fix test deps

380b7be

try with coverage run ... && coverage report

612b470

FBruzzesi added 7 commits April 16, 2026 16:27

fix(typing): WIP

af99861

revert 7e2c50d

6e01b9b

forgot one file :(

81f41c9

alias imports

92a2ab1

fixup tests

9d9d0d2

try inline &&

af31a60

c.__name__ -> str(c)

06be300

FBruzzesi added 3 commits April 16, 2026 17:13

try with fail-under flag under report

e9b94bb

try no pytest-cov

2ac3da9

fixup command

0e313de

FBruzzesi added 17 commits April 18, 2026 00:11

add patch=subprocess option

b0fbd1b

pragma: no cover constructors as current main

edb0263

add patches to coveragepy

06a6961

remove mapping hint in tests

024640a

rename base classes

7d127f7

merge main

e5c192e

rm execv patch flag

76b5851

addressing typing issues part 1

e1749ac

add NativeIbis to v1 IntoDataFrame

df1afc3

match polars signature in assert_frame_equal to fix typing

21befee

try hack to define coverage patch flags dynamically

96b6a44

port fix from #3555

57a5f18

fix: pytest on py3.14 ported wrong min coverage

ffc2166

Run join_test with eager frames as well

ffbf61a

do not collect too early

8042564

fixup join test part 2

25126ea

test_join_duplicate_column_names fix

024135a

FBruzzesi mentioned this pull request Apr 18, 2026

refactor: Rename exposed fixtures and pytest options #3556

Open

8 tasks

camriddler reviewed Apr 20, 2026

View reviewed changes

FBruzzesi commented Apr 20, 2026

View reviewed changes

FBruzzesi added 7 commits April 21, 2026 21:57

Adopt registry pattern, tackle a few other feedbacks

e1d09e5

fixup typing and prepare_constructors regression

e97f331

fix __eq__

e76e50d

match is_nullable default

b1da6a7

fixup typevar default

bb6b042

Fixup default backends parsing

32018b3

no cover + GHA

37c1edb

		raise NotImplementedError


		class LazyFrameConstructor(FrameConstructor):

Conversation

FBruzzesi commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

(*) Renaming proposal

pyest-cov changes

What type of PR is this? (check all applicable)

Related issues

Checklist

Uh oh!

dangotbanned commented Apr 16, 2026

Uh oh!

FBruzzesi commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FBruzzesi commented Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

camriddler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FBruzzesi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

camriddell commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FBruzzesi commented Apr 21, 2026

Uh oh!

Reviewers

Assignees

Labels

FBruzzesi commented Apr 16, 2026 •

edited

Loading

FBruzzesi commented Apr 16, 2026 •

edited

Loading

FBruzzesi commented Apr 18, 2026 •

edited

Loading

camriddell commented Apr 21, 2026 •

edited

Loading