Add a registration mechanism to Aggregators - #30

Draft

devavret wants to merge 949 commits into

rapidsai:velox-cudffrom

devavret:register-aggs

This pull request is big! We're only showing the most recent 250 commits

Commits on Mar 7, 2025

update cudf to fix kvikio build issue
karthikeyann
committed
49c1d9e
enable ccache for cuda
karthikeyann
committed
2419703
Merge branch 'main' of github.com:facebookincubator/velox into fix-update_cudf
karthikeyann
committed
bf4c9f4
Merge pull request #82 from karthikeyann/fix-update_cudf
bdice
authored
e88206d
Fix use after move in other types of tables
devavret
committed
d3d43ee
made some progress
devavret
committed
fcfdf61
Make right semi join work with filter
devavret
committed
5d0dcfd
Merge pull request #83 from karthikeyann/update-2025-03-05
karthikeyann
authored
c7ae92a
Merge branch 'rapidsai:velox-cudf' into fea-addFilter
karthikeyann
authored
d48b5ee
Merge branch 'rapidsai:velox-cudf' into fea-Project_more_ops2
karthikeyann
authored
fba518f

Commits on Mar 8, 2025

Add test for right semi join without filter
devavret
committed
767a6d6
Add right join filer test
devavret
committed
66c0a1b

Commits on Mar 10, 2025

Add antijoin with filter + anti join tests
devavret
committed
80f39a6
Add left semi join filter + tests
devavret
committed
b66c0ab
Inner join filter + test
devavret
committed
1b4765e
Avoid constucting hash build object when not needed
devavret
committed
d8dd56b
Remove code related to pre-compute columns in expressions for now
devavret
committed
d782ef8
Merge Single table and two table ast context
devavret
committed
4b3fade
Move static function `can_be_evaluated` out of AstContext
devavret
committed
ea94ee6
Remove unused functions
devavret
committed
b8d9dd0
Merge branch 'velox-cudf' into gpu-local-partition
devavret
committed
676b68d
Merge pull request #70 from rapidsai/gpu-local-partition
devavret
authored
5aa4f22
remove commented code
devavret
committed
cc737ad
Merge branch 'velox-cudf' into pr/karthikeyann/71
devavret
committed
b6a15ce
remove mallochost
devavret
committed
b5ac229
Merge branch 'velox-cudf' of github.com:rapidsai/velox-private into fea-Project_more_ops2
karthikeyann
committed
7c4faa8
replace balanced tree with chain of binaryops in ast
karthikeyann
committed
ae08312
rename ast variable
karthikeyann
committed
e863aba
style fix
karthikeyann
committed
88dbbd0
Merge branch 'velox-cudf' into cudf-limit
karthikeyann
authored
1ef3c50
Merge branch 'pr-78' into local-all_PRs_merged1
karthikeyann
committed
3093390
Merge branch 'pr-79' into local-all_PRs_merged1
karthikeyann
committed
e5d4ea3
Merge branch 'pr-80' into local-all_PRs_merged1
karthikeyann
committed
2ef90c6

Commits on Mar 11, 2025

fix merge issue
karthikeyann
committed
3dcab6e
fix merge issue
devavret
committed
0244670
Review fix
devavret
committed
ef540c8
Add filter to right join
devavret
committed
d670cd5
won't let me force push
devavret
committed
87469a7
Add minimal code that demonstrates cudf integration
devavret
committed
cf55f95
Remove all other operators from ToCudf
devavret
committed
2c03cbf
Make it compile
devavret
committed
e846132
remove unused interop code
devavret
committed
5ee96b3
Merge remote-tracking branch 'upstream/main' into update-velox-2025-03-11
bdice
committed
967baef
Fix style.
bdice
committed
1fb7eb1
Remove duplicate function declaration.
bdice
committed
8e40630

Commits on Mar 12, 2025

Merge pull request #86 from bdice/update-velox-2025-03-11
karthikeyann
authored
4954d8e
Update velox/experimental/cudf/exec/ExpressionEvaluator.cpp
karthikeyann
authored
dbeb17a
Merge branch 'pr-71' into local-all_PRs_merged1
karthikeyann
committed
d5e0718
Merge branch 'main-rapids' into cmake-upstreaming
devavret
committed
d8feb54
Use adapters-cuda images from upstream.
bdice
committed
5aba514
Merge pull request #79 from karthikeyann/fea-Project_more_ops2
karthikeyann
authored
42540b6
Merge branch 'velox-cudf' into fea-addFilter
karthikeyann
authored
46b25cb
Pin rmm and kvikio
devavret
committed
46f0a94
Remove manually adding fmt
devavret
committed
47913ae
Remove debug prints
devavret
committed
5effbf2
Re-enable some warnings
devavret
committed
0e43fc1
Update CudfVector.h
karthikeyann
authored
4e9295a
Merge pull request #89 from rapidsai/fix-merge1
karthikeyann
authored
c8567c3
Merge pull request #78 from rapidsai/cudf-limit
karthikeyann
authored
2eb238d
Merge branch 'rapidsai:velox-cudf' into fea-addFilter
karthikeyann
authored
1ac0c46
Merge branch 'velox-cudf' of github.com:rapidsai/velox-private into local-all_PRs_merged1
karthikeyann
committed
a3ed711
remove debug print, style fix
karthikeyann
committed
4bc810a
stylefix
karthikeyann
committed
affc476
fix defaults in benchmarks
karthikeyann
committed
cf3da81

Commits on Mar 13, 2025

Update ToCudf.cpp
karthikeyann
authored
84effce
Merge pull request #80 from karthikeyann/fea-addFilter
karthikeyann
authored
e0ab0aa
Merge branch 'velox-cudf' into local-all_PRs_merged1
karthikeyann
authored
7279af1
update cmake on centos
devavret
committed
9dc09e1
Add our team to codeowners
devavret
committed
e0ccfbd
Check off some todos
devavret
committed
d00db4f
remove commented tests
devavret
committed
5accb50
Merge pull request #88 from bdice/use-adapters-cuda
karthikeyann
authored
4bc9e28

Commits on Mar 17, 2025

Ignore known warnings just for cudf_exec target
devavret
committed
5fe2575
Misc review fixes:
devavret
committed
c9b2c1a
Revert erroneous hardcoding of `numSplitsPerFile`
mhaseeb123
committed
ebaa994
Revert erroneous hardcoding of `numSplitsPerFile`
mhaseeb123
committed
ef95aa8
Merge branch 'velox-cudf' into pr/karthikeyann/71
devavret
committed
ddb5fb6
Merge pull request #71 from karthikeyann/fea-more_hashjoins
devavret
authored
5529c0a

Commits on Mar 18, 2025

Commits on Mar 19, 2025

add estimateFlatSize to CudfVector
karthikeyann
committed
9bc7263
Misc review changes requested by @bdice
devavret
committed
5a75b9e
Remove only cudf adapter
devavret
committed
3f4ca09
Add clang format to our subdir
devavret
committed
6f7d72e

Commits on Mar 20, 2025

Fix style
devavret
committed
2d679a4
use stream_c
karthikeyann
committed
2fd784e
Merge branch 'enh-memory_cudfvector' of github.com:karthikeyann/velox-private into local-all_PRs_merged1
karthikeyann
committed
9f63a02

Commits on Mar 21, 2025

Fix naming
devavret
committed
757bce9
Fix more style
devavret
committed
436d721
Error out when cuda architecture is less than 70
devavret
committed
5538888

Commits on Mar 23, 2025

Use streaming aggregation in partial step and disable in final step for testing with q13
devavret
committed
5d6e583

Commits on Mar 24, 2025

Commits on Mar 25, 2025

rename cudfIsRegistered to isCudfRegistered
karthikeyann
committed
5979144
Misc. review changes
devavret
committed
71624fa
Misc review changes
devavret
committed
6ccdb57
Prevent merging vectors whose total size exceeds vector_size_t max
devavret
committed
ab399f2
Misc review changes
devavret
committed
327f718
address review comments
karthikeyann
committed
3c1554d
Merge pull request #57 from karthikeyann/fea-cudfTableScan_TPCH
karthikeyann
authored
08e2d68
remove unused use_arrow_schema
karthikeyann
committed
ea70e4b
Merge branch 'velox-cudf' of github.com:rapidsai/velox-private into local-all_PRs_merged1
karthikeyann
committed
6d67a56
remove unused include
karthikeyann
committed
3d43de8
cleanup parquet filter to separate PR
karthikeyann
committed
fa2ebea
cleanup Utilities.h
karthikeyann
committed
d4d119a
remove unused include
karthikeyann
committed
201c800

Commits on Mar 26, 2025

include cleanup
karthikeyann
committed
d28820b
style fix
karthikeyann
committed
1038579
cleanup cmakelists.txt
karthikeyann
committed
5216246
replace env variable with gflags and CudfOptions
karthikeyann
committed
5754b3c
replace gpu batch size env variable with a QueryConfig entry
karthikeyann
committed
538398e
Add back optional debug printing of plans
devavret
committed
7570d56
Add a table sizing method that does not need stream
devavret
committed
702cebe
Misc review changes
devavret
committed
e633371

Commits on Mar 31, 2025

replace env variables with gflags
karthikeyann
committed
9ef1dff
cleanup
karthikeyann
committed
05c4465
Merge pull request #92 from karthikeyann/enh-memory_cudfvector
devavret
authored
b5c9901
rename to cudfTableScanEnabled
karthikeyann
committed
ca349b9
Add clang tidy
devavret
committed
86738ed
remove aacidental flags added to all of velox
devavret
committed
7b18528

Commits on Apr 1, 2025

Merge remote-tracking branch 'rapids/cmake-upstreaming' into cleanup-utilities
karthikeyann
committed
b9e56ae
cleanup more includes
karthikeyann
committed
bcf1d88
Merge branch 'velox-cudf' of github.com:rapidsai/velox-private into cleanup-utilities
karthikeyann
committed
a1cc5fa
revert local changes
karthikeyann
committed
7483674
remove re2
karthikeyann
committed
fcf6afc
cleanup
karthikeyann
committed
6d48542
style fix
karthikeyann
committed
5114e07

Commits on Apr 2, 2025

Trim extra items from nvtx range name
devavret
committed
8192488
fix bug of sticky payload
devavret
committed
5ce0269
Add plan node id to range string
devavret
committed
934e8fd
Move parquet connector range to nvtx domain and display table name
devavret
committed
074aa5e
README updates
bdice
committed
737464c
Fix sentence
bdice
committed
aa750f8
Fix typo in comment.
bdice
authored
c0c5851
Merge pull request #95 from karthikeyann/cleanup-utilities
karthikeyann
authored
780da70
add support to use non-output column names in filter
karthikeyann
committed
fd34c6b
Merge branch 'velox-cudf' into TableScanFilter
karthikeyann
authored
b43b6f9
fix merge issue
karthikeyann
committed
dd7d1fb

Commits on Apr 3, 2025

Merge branch 'velox-cudf' into enh-nvtx-cleanup
karthikeyann
authored
199ec92
Merge pull request #97 from rapidsai/enh-nvtx-cleanup
karthikeyann
authored
850d7ae
Merge pull request #96 from bdice/velox-cudf-readme
karthikeyann
authored
e00a541
Merge branch 'velox-cudf' of github.com:rapidsai/velox-private into TableScanFilter
karthikeyann
committed
d8dbbf2
change args to benchmark
karthikeyann
committed
76d313f
update cudf upto PR #18395
karthikeyann
committed
5f063ae
fix merge issue
karthikeyann
committed
1584c38
add comments
karthikeyann
committed
1b8654b
revert add support to use non-output column names in filter
karthikeyann
committed
e2c3a8b
style fix
karthikeyann
committed
d0b9f4f
disable related unit test
karthikeyann
committed
531c71c
update kvikio, rmm, rapids-cmake
karthikeyann
committed
61f5368
Add ARM support.
bdice
committed
5358a1e
revert cmake
karthikeyann
authored
41e32c6

Commits on Apr 4, 2025

Merge pull request #98 from karthikeyann/update_cudf_host_decompress_Fix
karthikeyann
authored
59cf040

Commits on Apr 7, 2025

Commits on Apr 8, 2025

Cudf driver adapter without storing plan nodes
devavret
committed
5575205
re-fix conversion to RowVector in sink
devavret
committed
01f1aaf

Commits on Apr 9, 2025

add link scope to velox_cudf_exec
devavret
committed
a6c99ff
Merge branch 'cmake-upstreaming' into compilestate-fix-gluten
devavret
committed
8b7992c
Cleanup CudfVector.cpp and the most obvious bad comment I had
devavret
committed
5fce74a
Cleanup comments in LocalPartition
devavret
committed
f2075f3
Cleanup comments in CudfHashAggregation
devavret
committed
d6d6029
Add comments in ExpressionEvaluator
devavret
committed
aac3984
Missed tostring change in CudfLocalPartition.
devavret
committed
b7fcf6d
Merge pull request #101 from bdice/velox-cudf-codeowners
bdice
authored
46e2db7
fix compile
devavret
committed
e5b33ae
fix cmake dependency
karthikeyann
committed
68aac81

Commits on Apr 10, 2025

Commits on Apr 14, 2025

Add missed stream usage in scalar accessors causing use of default stream
devavret
committed
cdc6944

Commits on Apr 15, 2025

Merge pull request #2 from rapidsai/scalar-stream
devavret
authored
4892a81

Commits on Apr 18, 2025

Commits on Apr 19, 2025

Merge branch 'velox-cudf' into eager-groupby
devavret
committed
8a1a1d7

Commits on Apr 20, 2025

Style
bdice
committed
da0be93
Temporarily disable restore action.
bdice
committed
903ccdf

Commits on Apr 21, 2025

Merge branch 'update-velox-2025-04-17' into eager-groupby
devavret
committed
aba48bc
Run clang tidy on all files in experimental/cudf
devavret
committed
c74cdf0
Merge branch 'clangtidy-cleanups-0' into eager-groupby
devavret
committed
35a6fd8
Re-add stash restore
bdice
authored
7ed1fc5
Revert "Re-add stash restore"
bdice
committed
2926e7e
add nvtx range
devavret
committed
92ce4ef
Merge pull request #3 from bdice/update-velox-2025-04-17
bdice
authored
5ec5b0b

Commits on Apr 22, 2025

enable avg agg in streaming partial groupby
devavret
committed
859c8ff

Commits on Apr 23, 2025

Commits on Apr 24, 2025

Fix issue with reordering of results. Issue exposed with Q1 which has a lot of aggs
devavret
committed
05e8830
Use stream again instead of default
devavret
committed
1e7700b
Make partial output a CudfVectorPtr so it can store its stream
devavret
committed
0c6684c
cleanup and reorganize addInput code
devavret
committed
4fe7aa8
cleanup final groupby handling
devavret
committed
6cf35b8
Fix for when grouping keys are moved around
devavret
committed
a8ba91f

Commits on Apr 25, 2025

Commits on Apr 29, 2025

Support spark operator and companion function agg name
jinchengchenghh
committed
18ec27d
fix avg name
jinchengchenghh
committed
30c71f8
address comments for map
jinchengchenghh
committed
eab0108

Commits on Apr 30, 2025

Merge pull request #5 from jinchengchenghh/rapidvelox
karthikeyann
authored
134426c
Merge remote-tracking branch 'upstream/main' into update-velox-2025-04-30
bdice
committed
6b829cc
Merge remote-tracking branch 'upstream/main' into update-velox-2025-04-30
bdice
committed
2f53b21
Update target name
bdice
committed
16196a6
add batching in CudfToVelox
karthikeyann
committed
4ad6fdc
copy and enable OrderByTest.outputBatchRows
karthikeyann
committed
8814526
style fix
karthikeyann
committed
b71dc62
update the deprecated includes
karthikeyann
committed
b3ff196

Commits on May 1, 2025

Commits on May 2, 2025

Commits on May 5, 2025

Allow setting partial agg memory limit in query benchmarks
devavret
committed
ed34ab4
Comment cleanup
devavret
committed
01828f6
Fix Q13
devavret
committed
38dfbb3

Commits on May 6, 2025

Add debug utility function to print the cudf table
jinchengchenghh
committed
48372d1
address comments
jinchengchenghh
committed
38177fc
fix
jinchengchenghh
committed
e65941f

Commits on May 7, 2025

Merge pull request #12 from jinchengchenghh/rapidvelox
bdice
authored
4b29b5b

Commits on May 8, 2025

Merge pull request #6 from rapidsai/eager-groupby
devavret
authored
d7af6c9

Commits on May 12, 2025

add recordOutputStats to CudfLocalPartition
Ubuntu
committed
9409ee9

Commits on May 21, 2025

Fix the companion function changes step behavior (#13)
jinchengchenghh
authored
773208d

Commits on May 27, 2025

Commits on May 28, 2025

Merge branch 'main-meta' into merge-from-upstream-28-05-2025
devavret
committed
917dafa
Restore ccache
bdice
committed
5fede81
Use non-hidden directory for ccache artifacts.
bdice
committed
437d0da
Revert "Restore ccache"
bdice
committed
344f9e7
Reapply "Restore ccache"
bdice
committed
312e17e
Enable actions: read permission.
bdice
committed
d0acbb8
Revert "Provide token."
bdice
committed
55963e8
Revert actions: read permissions.
bdice
authored
313a93d
Merge pull request #22 from bdice/restore-ccache
bdice
authored
1ce567e
Merge branch 'velox-cudf' into merge-from-upstream-28-05-2025
bdice
authored
17e97db
Merge pull request #25 from devavret/merge-from-upstream-28-05-2025
bdice
authored
a8f645d

Commits on Jun 9, 2025

Add aggregator registration mechanism and function to register ones with presto prefix
devavret
committed
1aa8b76

Commits on Jun 10, 2025

register companion aggs separately instead of wrangling with their names
devavret
committed
32464ba