Honor explicit BONSAI_VARIANT during M5 metallib swap gating in setup.sh#23
Conversation
BONSAI_VARIANT during M5 metallib swap gating in setup.sh
There was a problem hiding this comment.
Pull request overview
This PR fixes setup.sh variant resolution so an explicitly set BONSAI_VARIANT is not overridden by local models/ auto-detection, ensuring the intended metallib swap gating behavior on affected M5 systems.
Changes:
- Preserve
BONSAI_VARIANTas the primary source of truth for_variant. - Run binary-model auto-detection only when
BONSAI_VARIANTis unset.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # to swap whenever a 1-bit model is already present under models/. Binary users | ||
| # who run setup before downloading a model should export BONSAI_VARIANT=binary. | ||
| _variant="${BONSAI_VARIANT:-ternary}" | ||
| for _bin_model in "$SCRIPT_DIR"/models/bonsai-image-4B-binary-*; do | ||
| [ -d "$_bin_model" ] && _variant="binary" && break | ||
| done | ||
| if [ -z "${BONSAI_VARIANT:-}" ]; then | ||
| for _bin_model in "$SCRIPT_DIR"/models/bonsai-image-4B-binary-*; do |
bri-prism
left a comment
There was a problem hiding this comment.
The fix is correct and nicely scoped. Gating the binary auto-detection behind [ -z "${BONSAI_VARIANT:-}" ] is the right call, since an explicit BONSAI_VARIANT=ternary now reaches the metallib swap branch instead of being clobbered when a binary model happens to be sitting in models/.
I walked through the three cases and they all hold:
- Unset: defaults to ternary, then auto-detect can promote it to binary, same as before.
ternary: stays ternary, detection skipped. That is the case we wanted to fix.binary: stays binary, detection skipped.
One minor note, not a blocker: when no binary model is present the glob still falls through cleanly because the directory test rejects the literal pattern, so that path is unchanged.
Looks good to me.
On M5 systems with the NAX miscompile,
setup.shcould detect the issue but still skip the ternary metallib swap when both model variants existed locally. The script unconditionally re-inferredbinaryfrommodels/and overrode an explicitBONSAI_VARIANT=ternary.Variant resolution behavior
_variant="${BONSAI_VARIANT:-ternary}"as the primary source of truth.BONSAI_VARIANTis unset.Impact on swap path
BONSAI_VARIANT=ternaryto reach the ternary metallib swap branch instead of being silently forced tobinary.