[3.0-dev] Backport function/window compatibility (Fixes #24185)#24194
Open
jiangxinmeng1 wants to merge 6 commits intomatrixorigin:3.0-devfrom
Open
[3.0-dev] Backport function/window compatibility (Fixes #24185)#24194jiangxinmeng1 wants to merge 6 commits intomatrixorigin:3.0-devfrom
jiangxinmeng1 wants to merge 6 commits intomatrixorigin:3.0-devfrom
Conversation
Fix the `ELT()` function to be fully compatible with MySQL.
**Root Cause:**
`eltCheck` skipped casting for integer types other than `int64` (e.g. `int32`, `int16`, `bool`), but the `Elt` executor hardcodes `GenerateFunctionFixedTypeParameter[int64]`, causing a type mismatch panic when using `INT`/`SMALLINT`/`TINYINT` columns or `BOOL` values as the index argument.
**Fix:**
In `eltCheck`, explicitly handle all numeric types (`bool`, `integer`, `float`, `decimal`, `bit`) by always casting the first argument to `int64`. For other types (e.g. `varchar`), fall back to `tryToMatch`.
**Test Coverage:**
Updated BVT tests (52 SQL statements) covering:
- INT/SMALLINT/TINYINT/UNSIGNED column as index (was panic)
- BOOL index: `ELT(TRUE, ...)` / `ELT(FALSE, ...)`
- Float index with rounding: `ELT(1.9, ...)` → 2
- String index: `ELT('2', ...)`
- Negative/zero float index
- Table with NULL index rows
- Table with out-of-range index rows
- BINARY return type
- Expression as index
- ELT + FIELD complement
- Removed all `@bvt:issue#22992` skip tags
Approved by: @heni02, @ouyuanning, @aunjgr
This PR implements a minimal MySQL-compatible AES_ENCRYPT/AES_DECRYPT enhancement aligned with block_encryption_mode. What changed: Added 3-argument overloads (AES_ENCRYPT(str, key, iv) / AES_DECRYPT(crypt, key, iv)) for varchar/char/text/blob inputs, returning blob/varchar as appropriate. Implemented MySQL-style XOR key folding and switched key derivation away from SHA1. Added AES-CBC support alongside the existing ECB path, gated by block_encryption_mode (aes-128-ecb default, aes-256-cbc supported). Enforced IV requirements for CBC (missing/short IV returns NULL, matching MySQL’s error-to-NULL behavior). Added focused tests for ECB/CBC roundtrip and CBC missing-IV behavior. Files touched: [func_binary.go](https://file+.vscode-resource.vscode-cdn.net/Users/moukeyu/.vscode/extensions/openai.chatgpt-0.4.67-darwin-x64/webview/#) [list_builtIn.go](https://file+.vscode-resource.vscode-cdn.net/Users/moukeyu/.vscode/extensions/openai.chatgpt-0.4.67-darwin-x64/webview/#) [func_binary_aes_test.go](https://file+.vscode-resource.vscode-cdn.net/Users/moukeyu/.vscode/extensions/openai.chatgpt-0.4.67-darwin-x64/webview/#) Test: go test ./pkg/sql/plan/function -count=1 Approved by: @ouyuanning, @heni02, @aunjgr
Backport of matrixorigin#23762 and matrixorigin#23764 to 3.0-dev, adapted to the older aggexec API on this branch (no SaveIntermediateResult / streaming serialization helpers, [][]int64 instead of []i64Slice, 3-return marshalToBytes / 2-arg unmarshalFromBytes / 4-arg initAggResultWithFixedTypeResult). CUME_DIST() returns the cumulative distribution of a value within a partition: (rows with values <= current row) / (total rows in partition). PERCENT_RANK() returns the relative rank of a row within a partition as a percentage: (rank - 1) / (total rows - 1). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The earlier ELT cherry-pick (b130341) brought in a copy of func_binary_test.go from main that contains tests for TimeFormat, TimestampDiffDateString, TimestampDiffStringDate, etc.—functions that exist on main but not on 3.0-dev. It also forgot to register the ELT function id and overload, so 'elt(...)' resolution would fail. This commit: * Adds the ELT function id (348) and 'elt' name mapping. * Registers the ELT overload in list_builtIn.go bound to the existing Elt / eltCheck implementations from func_binary.go. * Updates function_id_test.go's predefined map. * Removes the main-only TimeFormat / TIMESTAMPDIFF / DATE_*_ADD test blocks that don't compile on 3.0-dev, while preserving the ELT-specific tests at the end of the file. Adds the 'vector' import needed by those preserved tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jiangxinmeng1
requested review from
aunjgr,
heni02,
iamlinjunhong and
ouyuanning
as code owners
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Which issue(s) this PR fixes:
fixes #24185
What this PR does / why we need it:
ELT()MySQL compatibilityAES_ENCRYPT()/AES_DECRYPT()CUME_DIST()window functionPERCENT_RANK()window functionNotes on adaptation
CUME_DISTandPERCENT_RANKdid not cherry-pick cleanly because main'spkg/sql/colexec/aggexeclayer has been refactored substantially relativeto 3.0-dev (different signatures for
initAggResultWithFixedTypeResult,marshalToBytes,unmarshalFromBytes, plus newSaveIntermediateResult*/UnmarshalFromReaderinterface methods and ani64Slicetype).These two executors were rewritten to follow 3.0-dev's existing
singleWindowExecpattern ([][]int64groups, 3-returnmarshalToBytes,2-arg
unmarshalFromBytes, no streaming serialization). Algorithms matchthe originals exactly.
The ELT cherry-pick brought in a copy of
func_binary_test.gofrom maincontaining tests for
TimeFormat/TimestampDiffDateString/ etc. thatdon't exist on 3.0-dev. Those test blocks were removed; the ELT-specific
tests at the end of the file are preserved. Also added the
ELTfunctionid and overload registration that the cherry-pick didn't include.
Verification
all pass.
Fixes #24185