Split persistent state on parts by shards

[SmaGMan]

Introduces the split of the persistent state on to 2^persistent_state_split_depth files by shards. It supports write/import from splitted files, exchange via RPC, backward compatibility with a single-file persistent state.

It writes persistent state metadata file indicating the split config even when no split required.

<block_id>.meta.json
---
{
  "split_depth": 2,
  "parts": ["0xa000000000000000", "0x2000000000000000"]
}

Final persistent state bundle will be:

<block_id>_part_0xa000000000000000.boc
<block_id>_part_0x2000000000000000.boc
<block_id>.boc
<block_id>.meta.json

If state does not contain accounts in some shard, no part file is generated for that shard.

Crash recovery:

• write sequence: parts, metadata, main file
• if main file does not exist, bundle is not finished
• already written persistent state parts will be reused on the next attempt

Pull Request Checklist

NODE CONFIGURATION MODEL CHANGES

Added optional param. Default = 0

{
  "core_storage": {
    "persistent_state_split_depth": 2
  }
}

BLOCKCHAIN CONFIGURATION MODEL CHANGES

[None]

---

COMPATIBILITY

Fully compatible. Existing single-file persistent state treated as created with persistent_state_split_depth = 0.

SPECIAL DEPLOYMENT ACTIONS

[Not Required]

---

PERFORMANCE IMPACT

State 10kk.

• persistent_state_split_depth = 2

write start: Jun 19 11:31
write stop: Jun 19 11:49
elapsed: 18 min
size: ~1.4G (each part by 350M)

• persistent_state_split_depth = 0

write start: Jun 19 11:30
write stop: Jun 19 12:42
elapsed: 72 min
size: ~1.4G

---

TESTS

Unit Tests

Covered by:

• split_persistent_bundle_paths_are_rejected_for_zerostate_import
• persistent_state_meta_roundtrip
• shard_state_part_file_name_parser_recognizes_only_parts
• test_check_can_reuse_shard_state_part_files
• cache_rejects_split_sidecars_without_metadata
• persistent_state_writer_rejects_missing_absent_cell
• split_persistent_shard_state_import_from_dump
• overlay_server_split_persistent_state

Network Tests

[No coverage]

Manual Tests

Manual tests used
devnet5

• deploy 10kk
• init hack to write persistent on every key block on validator 2 and 3
• make a small change in blockchain config to force key block
• check persistent state creation

[github-actions]

🧪 Network Tests

To run network tests for this PR, use:

gh workflow run network-tests.yml -f pr_number=1100

Available test options:

• Run all tests: gh workflow run network-tests.yml -f pr_number=1100
• Run specific test: gh workflow run network-tests.yml -f pr_number=1100 -f test_selection=ping-pong

Test types: destroyable, ping-pong, one-to-many-internal-messages, fq-deploy, nft-index, persistent-sync

Results will be posted as workflow runs in the Actions tab.

[codecov]

Codecov Report

❌ Patch coverage is 73.29873% with 463 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.04%. Comparing base (7235431) to head (bd512d3).

Files with missing lines	Patch %	Lines
core/src/block_strider/starter/cold_boot.rs	0.00%	165 Missing ⚠️
core/src/storage/persistent_state/tests.rs	77.81%	15 Missing and 56 partials ⚠️
core/src/storage/persistent_state/mod.rs	67.12%	30 Missing and 18 partials ⚠️
core/src/storage/shard_state/cell_storage/raw.rs	89.37%	19 Missing and 15 partials ⚠️
core/src/storage/shard_state/mod.rs	73.64%	20 Missing and 14 partials ⚠️
core/src/storage/shard_state/store_state_raw.rs	80.57%	15 Missing and 19 partials ⚠️
...src/storage/persistent_state/shard_state/writer.rs	89.47%	11 Missing and 11 partials ⚠️
core/src/block_strider/starter/starter_client.rs	0.00%	13 Missing ⚠️
...src/storage/persistent_state/shard_state/reader.rs	72.41%	3 Missing and 5 partials ⚠️
core/src/blockchain_rpc/service.rs	84.44%	7 Missing ⚠️
... and 7 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1100      +/-   ##
==========================================
+ Coverage   59.86%   60.04%   +0.18%     
==========================================
  Files         484      485       +1     
  Lines       82721    84269    +1548     
  Branches    82721    84269    +1548     
==========================================
+ Hits        49522    50602    +1080     
- Misses      30850    31205     +355     
- Partials     2349     2462     +113     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[SmaGMan]

Follow-up to improve an interrupted raw import handling #1102

[Mododo]

lgtm

[SmaGMan]

All ongoing fixes are in #1111

[SmaGMan]

Replaced by #1111