Conversation
* update_row reuses the cached per-index dup-check iterator instead of building and tearing down an o(num_sstables) merge heap on every row, the same caching write_row already does.matters for a bulk update that shifts a unique-indexed column * restore statement-level atomicity inside begin...commit.a per-statement savepoint is armed in external_lock and, on statement error, we roll back to it and revert the plugin-side state to the same boundary -- txn_key_state through an undo journal, fts meta through a snapshot, and the statement's row locks to a marker -- then bump the txn generation so cached iterators rebuild.the old path did a full transaction rollback and shifted the snapshot, losing every earlier statement's work.its cited savepoint deep-copy cost was wrong, tidesdb savepoints are o(1).falls back to full rollback when a bulk mid-commit or a user rollback-to-savepoint voids it * skip compression on an encrypted table's data column family.rows are encrypted before they reach the library and ciphertext does not compress, so running lz4 over it on every flush and compaction spent cpu for nothing. index cfs hold unencrypted keys and keep their compression * advertise ha_keyread_only per key part.the covering bitmap is built by index_flags(idx, part), so a part is keyread-only only when its field can be rebuilt from its sort key.undecodable types (varchar, decimal, float, multi-byte char) no longer get priced as a row-fetch-free covering scan that then pays a hidden pk lookup per row.reading only the decodable columns of a mixed index still covers * seek to the upper bound for where seccol > x on a secondary index instead of stepping over every duplicate-prefix entry, with a single step past the one row whose pk encodes to all-0xff * probe pk uniqueness with tidesdb_txn_contains on the write txn and drop the per-row dedicated read-committed txn and its commit+reset churn.the non-tracking check keeps the probe out of the reservation base, and the insert's own reservation still enforces uniqueness at commit * take the row count from tidesdb_cf_estimate_cardinality, which dedupes the mvcc versions of a key within an sstable that total_keys counts separately and includes the shared unified memtable.fixes table_rows and index cardinality reading a floored minimum for unflushed tables * drop the misleading tdb_max_txn_ops line from the bulk-commit batching comments, the real bound is txn memory not the op cap * new mtr tests for statement atomicity, encrypted-cf compression, keyread decodability, secondary greater-than range, and row cardinality.re-record index_stats, info_schema, mrr, online_ddl and per_index_btree whose baselines had captured the old floored stats
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
update_row reuses the cached per-index dup-check iterator instead of building and tearing down an o(num_sstables) merge heap on every row, the same caching write_row already does.matters for a bulk update that shifts a unique-indexed column
restore statement-level atomicity inside begin...commit.a per-statement savepoint is armed in external_lock and, on statement error, we roll back to it and revert the plugin-side state to the same boundary -- txn_key_state through an undo journal, fts meta through a snapshot, and the statement's row locks to a marker -- then bump the txn generation so cached iterators rebuild.the old path did a full transaction rollback and shifted the snapshot, losing every earlier statement's work.its cited savepoint deep-copy cost was wrong, tidesdb savepoints are o(1).falls back to full rollback when a bulk mid-commit or a user rollback-to-savepoint voids it
skip compression on an encrypted table's data column family.rows are encrypted before they reach the library and ciphertext does not compress, so running lz4 over it on every flush and compaction spent cpu for nothing. index cfs hold unencrypted keys and keep their compression
advertise ha_keyread_only per key part.the covering bitmap is built by index_flags(idx, part), so a part is keyread-only only when its field can be rebuilt from its sort key.undecodable types (varchar, decimal, float, multi-byte char) no longer get priced as a row-fetch-free covering scan that then pays a hidden pk lookup per row.reading only the decodable columns of a mixed index still covers
seek to the upper bound for where seccol > x on a secondary index instead of stepping over every duplicate-prefix entry, with a single step past the one row whose pk encodes to all-0xff
probe pk uniqueness with tidesdb_txn_contains on the write txn and drop the per-row dedicated read-committed txn and its commit+reset churn.the non-tracking check keeps the probe out of the reservation base, and the insert's own reservation still enforces uniqueness at commit
take the row count from tidesdb_cf_estimate_cardinality, which dedupes the mvcc versions of a key within an sstable that total_keys counts separately and includes the shared unified memtable.fixes table_rows and index cardinality reading a floored minimum for unflushed tables
drop the misleading tdb_max_txn_ops line from the bulk-commit batching comments, the real bound is txn memory not the op cap
new mtr tests for statement atomicity, encrypted-cf compression, keyread decodability, secondary greater-than range, and row cardinality.re-record index_stats, info_schema, mrr, online_ddl and per_index_btree whose baselines had captured the old floored stats