Bug description
Currently there isn't a unite precision, some numeric timestamps could be in seconds, milliseconds, microseconds or nanoseconds in the original dataset; even for DATESTRING, it could be either milliseconds or nanoseconds. This uncertainess causes that the pushdown cannot work correctly for all cases.
There is a ongoing refactoring in CLP-S which unites the precesion stored in the split to be nanoseconds. That way, we can always force the pushdown converts whatever precision to nanoseconds, then the pushdown can work for the dataset with timestamps with different formats and different precisions.
System information
Velox System Info v0.0.2
Commit: 97907e9
CMake Version: 3.28.3
System: Linux-4.15.0-213-generic
Arch: x86_64
C++ Compiler: /usr/bin/c++
C++ Compiler Version: 11.4.0
C Compiler: /usr/bin/cc
C Compiler Version: 11.4.0
CMake Prefix Path: /usr/local;/usr;/;/usr/local/lib/python3.10/dist-packages/cmake/data;/usr/local;/usr/X11R6;/usr/pkg;/opt
Relevant logs
Bug description
Currently there isn't a unite precision, some numeric timestamps could be in seconds, milliseconds, microseconds or nanoseconds in the original dataset; even for
DATESTRING, it could be either milliseconds or nanoseconds. This uncertainess causes that the pushdown cannot work correctly for all cases.There is a ongoing refactoring in CLP-S which unites the precesion stored in the split to be nanoseconds. That way, we can always force the pushdown converts whatever precision to nanoseconds, then the pushdown can work for the dataset with timestamps with different formats and different precisions.
System information
Velox System Info v0.0.2
Commit: 97907e9
CMake Version: 3.28.3
System: Linux-4.15.0-213-generic
Arch: x86_64
C++ Compiler: /usr/bin/c++
C++ Compiler Version: 11.4.0
C Compiler: /usr/bin/cc
C Compiler Version: 11.4.0
CMake Prefix Path: /usr/local;/usr;/;/usr/local/lib/python3.10/dist-packages/cmake/data;/usr/local;/usr/X11R6;/usr/pkg;/opt
Relevant logs