Skip to content

feat(extensions): add fp16, decimal256, and duration extension types #978

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
kadinrabo wants to merge 17 commits into
base: main
Choose a base branch
from
Draft

feat(extensions): add fp16, decimal256, and duration extension types #978

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
d19c87a
feat(extensions): add unsigned integer extension types (u8, u16, u32,…
kadinrabo Jan 29, 2026
a413f3e
feat(extensions): add arithmetic function impls for unsigned types
kadinrabo Jan 30, 2026
b4bd79b
feat(tests): add UDT argument support in test framework
kadinrabo Jan 30, 2026
719ddb5
chore: regenerate ANTLR parsers
kadinrabo Jan 30, 2026
37a1a40
feat(tests): add unsigned integer test cases
kadinrabo Jan 30, 2026
41cee40
chore: update test counts and baseline
kadinrabo Jan 30, 2026
10baf81
chore: add dependency on extension_types_numeric
kadinrabo Jan 30, 2026
078ddbc
qualify UDT references with dependency alias
kadinrabo Feb 11, 2026
0d4aa26
rename type file to unsigned_integers, update URN
kadinrabo Feb 11, 2026
8d37e9e
add string structure encoding for unsigned types
kadinrabo Feb 11, 2026
ec638bf
move unsigned functions into self-contained extension file
kadinrabo Feb 11, 2026
b7990a8
remove redundant hardcoded UDT type mappings
kadinrabo Feb 11, 2026
a14d6d7
remove unused dependency alias grammar changes
kadinrabo Feb 11, 2026
856e63d
split unsigned tests into separate files, fix test framework
kadinrabo Feb 11, 2026
f55cc39
add float16 extension type and tests
kadinrabo Feb 11, 2026
0403b34
add decimal256 extension type
kadinrabo Feb 11, 2026
f47bbbc
add duration extension type
kadinrabo Feb 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 149 additions & 0 deletions extensions/decimal256.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
%YAML 1.2
---
urn: "extension:io.substrait:decimal256"

types:
- name: decimal256
description: 256-bit decimal with up to 76 digits of precision
parameters:
- name: precision
type: integer
min: 0
max: 76
- name: scale
type: integer
min: 0
max: 76
structure:
value: str

scalar_functions:
-
name: "add"
description: "Add two decimal256 values."
impls:
- args:
- name: x
value: u!decimal256
- name: y
value: u!decimal256
options:
overflow:
values: [ SILENT, SATURATE, ERROR ]
return: u!decimal256
-
name: "subtract"
description: "Subtract one decimal256 value from another."
impls:
- args:
- name: x
value: u!decimal256
- name: y
value: u!decimal256
options:
overflow:
values: [ SILENT, SATURATE, ERROR ]
return: u!decimal256
-
name: "multiply"
description: "Multiply two decimal256 values."
impls:
- args:
- name: x
value: u!decimal256
- name: y
value: u!decimal256
options:
overflow:
values: [ SILENT, SATURATE, ERROR ]
return: u!decimal256
-
name: "divide"
description: >
Divide x by y.
impls:
- args:
- name: x
value: u!decimal256
- name: y
value: u!decimal256
options:
overflow:
values: [ SILENT, SATURATE, ERROR ]
on_domain_error:
values: [ "NULL", ERROR ]
on_division_by_zero:
values: [ "NULL", ERROR ]
return: u!decimal256
-
name: "modulus"
description: >
Calculate the remainder when dividing dividend (x) by divisor (y).
impls:
- args:
- name: x
value: u!decimal256
- name: y
value: u!decimal256
options:
overflow:
values: [ SILENT, SATURATE, ERROR ]
on_domain_error:
values: [ "NULL", ERROR ]
return: u!decimal256
-
name: "abs"
description: "Calculate the absolute value of the argument."
impls:
- args:
- name: x
value: u!decimal256
return: u!decimal256

aggregate_functions:
- name: "sum"
description: Sum a set of decimal256 values. The sum of zero elements yields null.
impls:
- args:
- name: x
value: u!decimal256
options:
overflow:
values: [ SILENT, SATURATE, ERROR ]
nullability: DECLARED_OUTPUT
decomposable: MANY
intermediate: u!decimal256?
return: u!decimal256?
- name: "min"
description: Min of a set of decimal256 values.
impls:
- args:
- name: x
value: u!decimal256
nullability: DECLARED_OUTPUT
decomposable: MANY
intermediate: u!decimal256?
return: u!decimal256?
- name: "max"
description: Max of a set of decimal256 values.
impls:
- args:
- name: x
value: u!decimal256
nullability: DECLARED_OUTPUT
decomposable: MANY
intermediate: u!decimal256?
return: u!decimal256?
- name: "avg"
description: Average a set of decimal256 values.
impls:
- args:
- name: x
value: u!decimal256
options:
overflow:
values: [ SILENT, SATURATE, ERROR ]
nullability: DECLARED_OUTPUT
decomposable: MANY
intermediate: "STRUCT<u!decimal256,i64>"
return: u!decimal256
80 changes: 80 additions & 0 deletions extensions/duration.yaml
View file
Open in desktop
Copy link
Copy Markdown
Member

@nielspardon nielspardon Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the difference between duration and the interval_year / interval_day data types?

Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
%YAML 1.2
---
urn: "extension:io.substrait:duration"

types:
- name: duration
description: >-
An absolute elapsed time span. The unit parameter specifies the
precision of the stored value.
parameters:
- name: unit
type: string
description: >-
The time unit of the duration value. Must be one of
SECOND, MILLISECOND, MICROSECOND, or NANOSECOND.
structure:
value: str

scalar_functions:
-
name: "add"
description: "Add two duration values."
impls:
- args:
- name: x
value: u!duration
- name: y
value: u!duration
options:
overflow:
values: [ SILENT, SATURATE, ERROR ]
return: u!duration
-
name: "subtract"
description: "Subtract one duration value from another."
impls:
- args:
- name: x
value: u!duration
- name: y
value: u!duration
options:
overflow:
values: [ SILENT, SATURATE, ERROR ]
return: u!duration

aggregate_functions:
- name: "sum"
description: Sum a set of duration values. The sum of zero elements yields null.
impls:
- args:
- name: x
value: u!duration
options:
overflow:
values: [ SILENT, SATURATE, ERROR ]
nullability: DECLARED_OUTPUT
decomposable: MANY
intermediate: u!duration?
return: u!duration?
- name: "min"
description: Min of a set of duration values.
impls:
- args:
- name: x
value: u!duration
nullability: DECLARED_OUTPUT
decomposable: MANY
intermediate: u!duration?
return: u!duration?
- name: "max"
description: Max of a set of duration values.
impls:
- args:
- name: x
value: u!duration
nullability: DECLARED_OUTPUT
decomposable: MANY
intermediate: u!duration?
return: u!duration?
99 changes: 99 additions & 0 deletions extensions/float16.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
%YAML 1.2
---
urn: "extension:io.substrait:float16"

types:
- name: fp16
description: "Half-precision 16-bit floating point (IEEE 754)"
structure:
value: str

scalar_functions:
-
name: "add"
description: "Add two fp16 values."
impls:
- args:
- name: x
value: u!fp16
- name: y
value: u!fp16
options:
rounding:
values: [ TIE_TO_EVEN, TIE_AWAY_FROM_ZERO, TRUNCATE, CEILING, FLOOR ]
return: u!fp16
-
name: "subtract"
description: "Subtract one fp16 value from another."
impls:
- args:
- name: x
value: u!fp16
- name: y
value: u!fp16
options:
rounding:
values: [ TIE_TO_EVEN, TIE_AWAY_FROM_ZERO, TRUNCATE, CEILING, FLOOR ]
return: u!fp16
-
name: "multiply"
description: "Multiply two fp16 values."
impls:
- args:
- name: x
value: u!fp16
- name: y
value: u!fp16
options:
rounding:
values: [ TIE_TO_EVEN, TIE_AWAY_FROM_ZERO, TRUNCATE, CEILING, FLOOR ]
return: u!fp16
-
name: "divide"
description: "Divide one fp16 value by another."
impls:
- args:
- name: x
value: u!fp16
- name: y
value: u!fp16
options:
rounding:
values: [ TIE_TO_EVEN, TIE_AWAY_FROM_ZERO, TRUNCATE, CEILING, FLOOR ]
on_domain_error:
values: [ NAN, "NULL", ERROR ]
on_division_by_zero:
values: [ IEEE, LIMIT, "NULL", ERROR ]
return: u!fp16

aggregate_functions:
- name: "sum"
description: Sum a set of fp16 values. The sum of zero elements yields null.
impls:
- args:
- name: x
value: u!fp16
nullability: DECLARED_OUTPUT
decomposable: MANY
intermediate: u!fp16?
return: u!fp16?
- name: "min"
description: Min of a set of fp16 values.
impls:
- args:
- name: x
value: u!fp16
nullability: DECLARED_OUTPUT
decomposable: MANY
intermediate: u!fp16?
return: u!fp16?
- name: "max"
description: Max of a set of fp16 values.
impls:
- args:
- name: x
value: u!fp16
nullability: DECLARED_OUTPUT
decomposable: MANY
intermediate: u!fp16?
return: u!fp16?
Loading