Fix Unigram trainer prune loss to use per-piece alternative count#2070

Open

hunter-heidenreich wants to merge 2 commits into

huggingface:mainfrom

hunter-heidenreich:fix/unigram-prune-per-piece-alternatives

Commits on May 24, 2026

Fix Unigram trainer prune loss to use per-piece alternative count

hunter-heidenreich
and
claude
committed
Document prune loss parity with SentencePiece

hunter-heidenreich
and
claude
committed