[RFC] Replace LEB128 with Huffman

[mstembera]

I asked Opus 5 to replace the LEB128 compression with Huffman and this is what it produced. The net goes from 95,144,073 bytes to 70,836,201 saving about 25MB in the binary. If people think this is a good change I will "attempt" a matching PR on the trainer side although I may need lots of help there.

[Edit]
I pushed an improved version.
The current net goes from
95,144,073 Bytes to 68,640,242 Bytes so a 26.5MB or 28% size reduction.

My stripped universal Windows binary built w/ GCC 16.1 goes from:
99,842,204 to 73,460,380
and gzipped w/ default compression they become:
72,786,006 and 70,032,562 respectively.

Note the new non gzipped binary is only 674,374 Bytes bigger than the current gzipped binary.

No functional change
bench: 2829394

[github-actions]

clang-format 20 needs to be run on this PR.
If you do not have clang-format installed, the maintainer will run it when merging.
For the exact version please see https://packages.ubuntu.com/questing/clang-format-20.
An easier way to install it might be through https://apt.llvm.org/#llvmsh.

(execution 31238927343 / attempt 1)

[sscg13]

I think better would be to first run a pass of @anematode idea of compressing unused rows, and then run compression on the resulting net.

[anematode]

also keep in mind that we distribute the binary under gzip... what is the savings there?

[mstembera]

My gziped binary goes from 71,556,261 to 69,743,794 (Default compression level). Maybe we would no longer have to distribute under gzip. AFAIK gzip was only added after NNUE was introduced due to the large binaries but was considered a compromise because people now have to unzip after downloading.

[ddobbelaere]

Maybe we would no longer have to distribute under gzip.

Why not both? There are still other artifacts in the archive that benefit from compression. Also, the decompression argument is moot because a tarball needs to untarred also, and decompression happens at the same time. Same number of clicks for user in UI, or same command in terminal.

[niklasf]

And with transport compression that could be used transparently in a browser (like on Lichess):

Before:

95144073	nn-ab28990d4ea3.nnue
66217407	nn-ab28990d4ea3.nnue.br # wins overall by a small margin :(
70745942	nn-ab28990d4ea3.nnue.gz
70205610	nn-ab28990d4ea3.nnue.zst # -19

After:

70836201	nn-8742bfb9865c.nnue
67516470	nn-8742bfb9865c.nnue.br
68923085	nn-8742bfb9865c.nnue.gz
68190056	nn-8742bfb9865c.nnue.zst # -19

So doesn't stack significantly worse, if we even still want to stack transport compression on top.

[anematode]

(base) cowpox@Cowpox src % time ./stockfish compiler
Stockfish dev-20260803-df99402a by the Stockfish developers (see AUTHORS file)

Compiled by                : clang++ 17.0.0 on Apple
Compilation architecture   : apple-silicon
Compilation settings       : 64bit NEON_DOTPROD POPCNT
Compiler __VERSION__ macro : Apple LLVM 17.0.0 (clang-1700.0.13.5)

./stockfish compiler  0.74s user 0.05s system 99% cpu 0.800 total
(base) cowpox@Cowpox src % time ./stockfish.master compiler
Stockfish dev-20260720-f4bcd404 by the Stockfish developers (see AUTHORS file)

Compiled by                : clang++ 17.0.0 on Apple
Compilation architecture   : apple-silicon
Compilation settings       : 64bit NEON_DOTPROD POPCNT
Compiler __VERSION__ macro : Apple LLVM 17.0.0 (clang-1700.0.13.5)

./stockfish.master compiler  0.12s user 0.05s system 95% cpu 0.176 total

Startup time is a lot slower... so we'd need to optimize the decompression fist

[mstembera]

The main reason it takes longer is that we now also compress the 8 bit weights(which there are many more of) rather than just the 16 bit weights. (LEB128 doesn't work on 8 bit weights.) That of course is how we get the extra 25MB size reduction. I pushed a new version that hopefully should get the startup time penalty to under 0.5s. I don't know if 0.5s is a big deal or not.

[anematode]

We could probably get a lot better performance, close to the original, by encoding/decoding multiple Huffman streams at once

Latest version:

(base) cowpox@Cowpox src % time ./stockfish compiler
Stockfish dev-20260803-12e05a1f by the Stockfish developers (see AUTHORS file)

Compiled by                : clang++ 17.0.0 on Apple
Compilation architecture   : apple-silicon
Compilation settings       : 64bit NEON_DOTPROD POPCNT
Compiler __VERSION__ macro : Apple LLVM 17.0.0 (clang-1700.0.13.5)

./stockfish compiler  0.50s user 0.06s system 94% cpu 0.592 total

[Sopel97]

At this point I'd rather consider using an already established compressor like zstd to make it easier for other tools to process networks.

https://github.com/welcome-to-the-sunny-side/misa77 could also be considered if we value decompression speed so much

[mstembera]

I have no strong preference. I did this because while LEB128 was a good choice when we had all 16-bit weights it's not now for mostly 8-bit weights. I wanted to avoid any external dependencies just like the current LEB128 does. Especially because we need C++ here and presumably Python on the trainer side. Finally, I don't really know to what degree we care about decompression speed or if an extra 0.416s matters. (I can update this w/ the @anematode suggestion to decode multiple streams in parallel if deemed worth while.) Requesting maintainer guidance.

[anematode]

My concern about decompression speed is more about perf on lower-end devices, including on wasm... idk. Maybe unimportant

[mstembera]

I pushed an improved version.
The current net goes from
95,144,073 Bytes to 68,640,242 Bytes so a 26.5MB or 28% size reduction.

My stripped universal Windows binary built w/ GCC 16.1 goes from:
99,842,204 to 73,460,380
and gzipped w/ default compression they become:
72,786,006 and 70,032,562 respectively.

The new non gzipped binary is only 674,374 Bytes bigger than the current gzipped binary.