What feature or improvement would you like to see?
Currently to read data from a partition, using Java as an example:
- a user must use AdbcStatement.executePartitioned() to get a PartitionedResult containing a list of PartitionDescriptors.
- For each PartitionDescriptor the user must call AdbcConnection.readPartitioned().
If the user intends to distribute the work across separate processes (or nodes in a distributed system), they must have each process go through the work of creating an AdbcDatabase, building up connection options, then connecting to a node to create a full-fledged AdbcConnection. This could be costly -- for example this may requiring re-running the auth workflow when an auth token was generated already or creating a session object instead of re-using the session already established.
An idea would be to have AdbcDatabase let the caller construct an AdbcPartitionedReader from a PartitionDescriptor instead of requiring the full connection process. The driver implementation can bake all the details on how to connect to and use the node holding the partition (including stateful information such as auth access tokens or session identifiers) so it can skip the potentially heavy-weight connection process.
What feature or improvement would you like to see?
Currently to read data from a partition, using Java as an example:
If the user intends to distribute the work across separate processes (or nodes in a distributed system), they must have each process go through the work of creating an AdbcDatabase, building up connection options, then connecting to a node to create a full-fledged AdbcConnection. This could be costly -- for example this may requiring re-running the auth workflow when an auth token was generated already or creating a session object instead of re-using the session already established.
An idea would be to have AdbcDatabase let the caller construct an AdbcPartitionedReader from a PartitionDescriptor instead of requiring the full connection process. The driver implementation can bake all the details on how to connect to and use the node holding the partition (including stateful information such as auth access tokens or session identifiers) so it can skip the potentially heavy-weight connection process.