Compress threat indexing, update Full Threats version and hash

[sscg13]

Merge X -> X threats and X -> ~X threats into a single X -> X plane, by taking advantage of previously unused indices. Also remove unused white pawn -> black pawn indices.

All code implementation and net processing by @Viren6

Since this is the last major change to the threat features planned for now, we would like to mark this as a new version of Full Threats.

No functional change

[github-actions]

clang-format 20 needs to be run on this PR.
If you do not have clang-format installed, the maintainer will run it when merging.
For the exact version please see https://packages.ubuntu.com/plucky/clang-format-20.

(execution 22475973553 / attempt 1)

[sscg13]

As a milestone, the current NNUE compresses to approximately the same size as the old SFNNv9 master net.

[vondele]

can you run non-regression on fishtest for this.

Ideally, we can generate a net directly with the trainer using this new layout, just to ensure we can.

[sscg13]

STC nonregression running at
https://tests.stockfishchess.org/tests/view/699d533bbfc3f89338de3f61

The corresponding nnue-pytorch PR should also be given a test run, though I have already checked the indexing matches by manually inputting test cases through LLDB.

[Disservin]

The corresponding nnue-pytorch PR should also be given a test run, though I have already checked the indexing matches by manually inputting test cases through LLDB.

for nnue-pytorch you could also add some static_asserts for expected values if there are good cases only requires constexpr to be added to threat_index, for quick verification

static_assert(Full_Threats::threat_index(...) == 1);

[xu-shawn]

small nitpick: maybe rename full_threats.cpp to full_threatsv2.cpp? it's more consistent with halfkav2

[sscg13]

The nonregression STC seems to have failed after 380k games, which may be a fluke considering elo is still neutral within error bars, so it is up to maintainer decision.

[vondele]

I think this can be merged, as the size savings is significant, as soon as the corresponding trainer has been tested to be consistent with it.

[snicolet]

I have pushed another non-regression run on fishtest, to double-check if the previous failed non-regression test was a fluke. If it was, this verification run should pass quickly.

https://tests.stockfishchess.org/tests/view/69a2f38d9fb1257c48a2c67c

Edit : second non-regression test failed, posting an -1.65 Elo result.

[anematode]

As I mentioned on Discord, I measure a consistent slowdown locally, probably because of the std::swap, so I'm not surprised it fails on fishtest (especially at STC). But I don't think a small slowdown is a big deal vs. the considerable size savings

[mstembera]

If the swap() is the issue we may be ok? I started https://tests.stockfishchess.org/tests/view/69a6ac76fdc0405ac16afbbe If it's the branch then we may still be slower.

[sscg13]

It looks like branching causes a noticeable slowdown no matter what. There is an alternate compression scheme that shouldn't affect indexing speed at all but is slightly uglier, I'll see if I can get that working.

(I would advise holding off on this PR for a bit longer in the meanwhile)