Skip to content

fix(bpe): widen pair_counts from i32 to i64 to prevent overflow on large corpora#2059

Closed
xodn348 wants to merge 1 commit into
huggingface:mainfrom
xodn348:fix/bpe-trainer-i32-overflow
Closed

fix(bpe): widen pair_counts from i32 to i64 to prevent overflow on large corpora#2059
xodn348 wants to merge 1 commit into
huggingface:mainfrom
xodn348:fix/bpe-trainer-i32-overflow

Commits

Commits on May 17, 2026