fix: apply declared nullability from AnyType return type to resolved concrete type#205
Merged
benbellick merged 2 commits intosubstrait-io:mainfrom Mar 4, 2026
Conversation
…concrete type When a function declares a nullable return type like `any1?` (e.g., `nullif` with `nullability: DECLARED_OUTPUT`), the `?` nullability was parsed and stored in the `AnyType.Nullability` field but never applied to the concrete type resolved by `ReturnType()`. `unwrapAnyTypeWithName` returns the matching argument type as-is, discarding the declared nullability. For example, `nullif(string, string)` with return type `any1?` would produce a non-nullable `string` output instead of the expected nullable `string?`. This caused downstream consumers (e.g., Calcite-based optimizers) to incorrectly treat NULLIF results as non-nullable, leading to the elimination of enclosing COALESCE calls that depended on NULLIF's potential NULL output. The fix applies the AnyType's declared nullability to the resolved concrete type in `ReturnType()` when the AnyType is nullable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
miretskiy
approved these changes
Mar 4, 2026
benbellick
reviewed
Mar 4, 2026
Member
benbellick
left a comment
There was a problem hiding this comment.
Just the one comment about how DECLARED_OUTPUT interacts with the nullability-required case. Thanks!
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #205 +/- ##
=======================================
Coverage 68.23% 68.24%
=======================================
Files 47 47
Lines 10711 10713 +2
=======================================
+ Hits 7309 7311 +2
Misses 3046 3046
Partials 356 356 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
benbellick
reviewed
Mar 4, 2026
DELCARED_OUTPUT case
DELCARED_OUTPUT case
Member
|
For those coming back to this in the future, this did not close substrait-io/substrait#943. That was a mistake. That issues is still being resolved upstream. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix
AnyType.ReturnType()to apply the declared nullability (?suffix) from the return type definition to the resolved concrete type.Fixes substrait-io/substrait#943
Problem
When a function declares a nullable return type like
any1?(e.g.,nullifwithnullability: DECLARED_OUTPUTas introduced in substrait spec v0.81.0), the?nullability is correctly parsed and stored inAnyType.Nullability, butReturnType()never applies it to the resolved concrete type.unwrapAnyTypeWithName()returns the matching argument type as-is, discarding the declared nullability. For example:Calling
nullif(string NOT NULL, string NOT NULL)producesstring NOT NULLas the return type, instead of the expectedstring NULL— becauseNULLIFcan always return NULL when both arguments are equal.Fix
In
AnyType.ReturnType(), after resolving the concrete type viaunwrapAnyTypeWithName, apply theAnyType's declared nullability when it is nullable:Tests
extensions/variants_test.go:nullif(any1, any1) -> any1? [DECLARED_OUTPUT]— verifies non-nullable arguments still produce nullable output when the function declaresreturn: any1?types/any_type_test.go(any,anyOtherName) whose expected results now correctly reflect nullable output when theAnyTypedeclares?go test ./...