Skip to content

Optimize trie builder#15977

Open
gf2121 wants to merge 11 commits intoapache:mainfrom
gf2121:speed_up_TrieBuilder
Open

Optimize trie builder#15977
gf2121 wants to merge 11 commits intoapache:mainfrom
gf2121:speed_up_TrieBuilder

Conversation

@gf2121
Copy link
Copy Markdown
Contributor

@gf2121 gf2121 commented Apr 22, 2026

This PR speeds up TrieBuilder and reduces its memory footprint by replacing the in-memory object tree with a compact prefix-coded byte buffer during the building phase, and using a frontier-based approach during the saving phase.

Previously, TrieBuilder constructed a large in-memory tree using Node objects. This approach was memory-intensive (O(total nodes * ~120 bytes per node)) and caused massive object allocations, which made it incredibly slow when dealing with large terms.

Main Changes

  • Compact Byte Buffer (Building Phase): Replaced the object-heavy Node and SaveFrame classes with a sequential ByteBuffersDataOutput buffer. Entries are now prefix-encoded (storing prefix length, suffix length, and suffix bytes) and appended sequentially.
  • Zero-Overhead Appends: The first non-empty key (minKey) is stored separately. This allows the append() method to re-encode only the first entry and bulk-copy the remaining bytes with zero per-entry overhead.
  • Frontier-Based Save (Saving Phase): Reconstructs the trie structure on-the-fly during saveNodes() using a FrontierNode array bounded by maxKeyDepth, rather than requiring the whole tree to exist in memory.
  • Removed Status Management: Remove status management because nothing got destroyed after append or save.
  • No File Format Changes: The on-disk file format remains completely unchanged, ensuring backward compatibility.

Memory & Performance Impact

  • Memory Usage: Drops from O(total nodes) to O(total encoded bytes) during building, and down to O(max key depth) during the save operation.
  • Speed: Drastically reduces GC pressure and object allocations, bringing a massive speedup especially for very long terms.

@gf2121 gf2121 requested review from mikemccand and romseygeek April 22, 2026 16:50
@github-actions github-actions Bot modified the milestones: 11.0.0, 10.5.0 Apr 22, 2026
@gf2121
Copy link
Copy Markdown
Contributor Author

gf2121 commented Apr 23, 2026

I get some number on building index of 1M uuids.

FINAL STATS (1 Million 16-byte UUIDs)

Metric Old Trie New Trie FST FST (No Suffix Sharing)
Memory Used (Before Saving) 673.06 MB 20.26 MB 57.57 MB 32.06 MB
Append Time 298.82 ms 166.27 ms 4587.17 ms 741.30 ms
Save Time 217.39 ms 218.97 ms 49.28 ms 46.27 ms
Total Time 516.21 ms 385.23 ms 4636.45 ms 787.57 ms

@romseygeek
Copy link
Copy Markdown
Contributor

I get some number on building index of 1M uuids.

Can you explain the difference between New Trie, FST and FST (no suffix sharing)? I'm not sure what they are referring to.

@gf2121
Copy link
Copy Markdown
Contributor Author

gf2121 commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

@romseygeek romseygeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is better on every measure? Nice! Thanks for picking this up. LGTM.

Comment thread lucene/CHANGES.txt Outdated

* GITHUB#15970: Reduce memory usage of fields with long terms during segment merges. (Alan Woodward)

* GITHUB#15977: Speed up TrieBuilder and reduce its memory footprint by replacing the in-memory object tree with a compact prefix-coded byte buffer. (Guo Feng)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge this with the previous entry? My one is superseded now!

@romseygeek
Copy link
Copy Markdown
Contributor

@gf2121 is this ready to be merged?

@mikemccand
Copy link
Copy Markdown
Member

This sounds awesome -- I'm not sure I'll have time for a close review so please don't wait for me.

Does merging also use this same path (we don't have a merge-optimized Trie merging path or so)? This should be needlemoving in luceneutil -- we can watch nightly benchy after it goes in (hopefully no other massive change lands at the same time -- I swear it's better than chance how often this happens ;) ).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants