Skip to content

[Bug] make_timestamp does not throw under spark.sql.ansi.enabled=true #4679

@andygrove

Description

@andygrove

Describe the bug

MakeTimestamp honours spark.sql.ansi.enabled in Spark 3.4 / 3.5 / 4.0: under ANSI mode it throws on an invalid (year, month, day, hour, min, sec[, timezone]) argument (ansiDateTimeArgumentOutOfRange / invalidFractionOfSecondError), and returns NULL only when ANSI is disabled.

The Comet wiring (CometMakeTimestamp, spark/src/main/scala/org/apache/comet/serde/datetime.scala) does not check failOnError. CometMakeTimestamp is a bare CometCodegenDispatch[MakeTimestamp] with no ANSI gating, so under ANSI mode Comet returns NULL where Spark would throw.

This is the same shape as the make_date ANSI divergence captured in #4451 (make_date_ansi.sql).

Steps to reproduce

With spark.sql.ansi.enabled=true, evaluate make_timestamp with an out-of-range argument (for example an invalid month or a seconds value outside the valid fraction range). Spark throws; Comet returns NULL.

Expected behavior

Under spark.sql.ansi.enabled=true, Comet should match Spark and throw on invalid arguments. The support level should be gated appropriately until a fix lands, and a make_timestamp_ansi.sql regression test should be added once the fix is in place (analogous to make_date_ansi.sql).

Additional context

Split out from #4502 (item 4), surfaced by the audit-comet-expression skill run in #4448. Cross-references #4451.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions