Antalya 26.3: apassos-1405: combined port of 5 PRs

[zvonand]

Changelog category (leave one):

• New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Forward port of list objects cache #1040 (#1405 by @arthurpassos, #1388 by @arthurpassos, #1593 by @arthurpassos, #1517 by @arthurpassos, #1631 by @arthurpassos).

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Performance tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All with Aarch64
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• Tiered Storage (2h)

Combined port of 5 PR(s) (group apassos-1405). Cherry-picked from #1405, #1388, #1593, #1517, #1631.

• Antalya 26.1 - Forward port of list objects cache #1040 #1405 — Antalya 26.1 - Forward port of list objects cache Antalya 25.8 - Forward port of #805 List objects cache #1040
• Antalya 26.1 - Forward port of export part and partition #1388 — Antalya 26.1 - Forward port of export part and partition
• Export Partition - release the part lock when the query is cancelled #1593 — Export Partition - release the part lock when the query is cancelled
• Fix IPartitionStrategy race condition #1517 — Fix IPartitionStrategy race condition
• Fix condition for using parquet metadata cache #1631 — Fix condition for using parquet metadata cache

---

#1405: Antalya 26.1 - Forward port of list objects cache #1040

Documentation entry for user-facing changes

Cache for listobjects calls

---

#1388: Antalya 26.1 - Forward port of export part and partition

Documentation entry for user-facing changes

Export merge tree part and partition (we still need to rebase #1177 afterwards)

---

#1593: Export Partition - release the part lock when the query is cancelled

During export partition, parts are locked by replicas for exports. This PR introduces a change that releases these locks when an export task is cancelled. Previously, it would not release the lock. We did not catch this error before because the only cases an export task was cancelled we tested were KILL EXPORT PARTITION and DROP TABLE. In those cases, the entire task is cancelled, so it does not matter if a replica does not release its lock.

But a query can also be cancelled with 'SYSTEM STOP MOVES', and in that case, it is a local operation. The lock must be released so other replicas can continue.

Documentation entry for user-facing changes

...

---

#1517: Fix IPartitionStrategy race condition

IPartitionStrategy::computePartitionKey might be called from different threads, and it writes to cached_result concurrently without any sort of protection. It would be easier to add a mutex around it, but we can actually make it lock-free by moving the cache write to the constructor.

Documentation entry for user-facing changes

...

---

#1631: Fix condition for using parquet metadata cache

Apache Iceberg queries were not htiting the parquet metadata cache because object_info->getFileFormat() resolves to IcebergDataObjectInfo::getFileFormat, which gets its return value from IcebergObjectSerializableInfo. This field is filled with the value from Apache Iceberg manifest file, and it is upper case by default, which then fails clickhouse check for parquet metadata cache usage.

Documentation entry for user-facing changes

...

[github-actions]

Workflow [PR], commit [7c8db01]