Summary
The GTFN imperative backend (GTFNTranslationStep(use_imperative_backend=True)) fails symbol-table validation on stencils that produce many common-subexpression temporaries. test_hdiff reproduces it:
gt4py.eve.exceptions.EveValueError: Symbols {SymbolRef('_cs_14'), SymbolRef('_cs_15'),
SymbolRef('_cs_16'), SymbolRef('_cs_17'), SymbolRef('_cs_18'), SymbolRef('_cs_19'),
SymbolRef('_cs_20'), SymbolRef('_cs_29'), SymbolRef('_cs_30'), SymbolRef('_cs_31'),
SymbolRef('_cs_32'), SymbolRef('_cs_33'), SymbolRef('_cs_34')} not found.
src/gt4py/eve/traits.py:106: EveValueError
The _cs_* symbols introduced by common-subexpression elimination are referenced but never declared in the IR the imperative code path builds, so SymbolTableTrait validation rejects it.
Why this has not been seen before
run_gtfn_imperative was not actually imperative. It is declared as
run_gtfn_imperative = GTFNBackendFactory(
name_postfix="_imperative",
otf_workflow__translation__use_imperative_backend=True,
)
but the cached_translation trait replaces translation with a CachedStep, so the
factory-boy __-path override never reached the wrapped GTFNTranslationStep. The
resulting backend has use_imperative_backend=False — it is the declarative backend under
another name.
Since ProgramBackendId.GTFN_CPU_IMPERATIVE is a full entry in the backend test matrix,
the imperative code path has effectively never been exercised by CI.
Reproduce on current main
Change the override so it reaches the translation step:
run_gtfn_imperative = GTFNBackendFactory(
name_postfix="_imperative",
- otf_workflow__translation__use_imperative_backend=True,
+ otf_workflow__bare_translation__use_imperative_backend=True,
)
then:
uv run pytest tests/next_tests/integration_tests/multi_feature_tests/iterator_tests/test_hdiff.py -k imperative
Verified failing on main at b1cdb47 with the error above.
Scope
Narrow. With a genuinely-imperative backend, the whole tests/next_tests/integration_tests
suite is 520 passed, 1 failed (test_hdiff) on CPU. Only stencils whose CSE output the
imperative path mishandles are affected, so an uses_lift-wide exclusion would be far too
broad — other uses_lift tests pass.
Context
The silent-no-op is fixed in #2808, which replaces the factory-boy factories with plain
builders; run_gtfn_imperative becomes genuinely imperative there. That PR adds a narrowly
scoped pytest.xfail for test_hdiff on this backend, referencing this issue, so the real
bug is recorded rather than re-hidden.
Two possible resolutions:
- Fix the imperative code path so CSE symbols are declared (removes the xfail).
- If the imperative backend is not intended to be supported, retire
use_imperative_backend and the GTFN_CPU_IMPERATIVE matrix entry rather than keeping a
parametrization that silently tests the declarative path.
Summary
The GTFN imperative backend (
GTFNTranslationStep(use_imperative_backend=True)) fails symbol-table validation on stencils that produce many common-subexpression temporaries.test_hdiffreproduces it:The
_cs_*symbols introduced by common-subexpression elimination are referenced but never declared in the IR the imperative code path builds, soSymbolTableTraitvalidation rejects it.Why this has not been seen before
run_gtfn_imperativewas not actually imperative. It is declared asbut the
cached_translationtrait replacestranslationwith aCachedStep, so thefactory-boy__-path override never reached the wrappedGTFNTranslationStep. Theresulting backend has
use_imperative_backend=False— it is the declarative backend underanother name.
Since
ProgramBackendId.GTFN_CPU_IMPERATIVEis a full entry in the backend test matrix,the imperative code path has effectively never been exercised by CI.
Reproduce on current
mainChange the override so it reaches the translation step:
run_gtfn_imperative = GTFNBackendFactory( name_postfix="_imperative", - otf_workflow__translation__use_imperative_backend=True, + otf_workflow__bare_translation__use_imperative_backend=True, )then:
Verified failing on
mainat b1cdb47 with the error above.Scope
Narrow. With a genuinely-imperative backend, the whole
tests/next_tests/integration_testssuite is 520 passed, 1 failed (
test_hdiff) on CPU. Only stencils whose CSE output theimperative path mishandles are affected, so an
uses_lift-wide exclusion would be far toobroad — other
uses_lifttests pass.Context
The silent-no-op is fixed in #2808, which replaces the
factory-boyfactories with plainbuilders;
run_gtfn_imperativebecomes genuinely imperative there. That PR adds a narrowlyscoped
pytest.xfailfortest_hdiffon this backend, referencing this issue, so the realbug is recorded rather than re-hidden.
Two possible resolutions:
use_imperative_backendand theGTFN_CPU_IMPERATIVEmatrix entry rather than keeping aparametrization that silently tests the declarative path.