Skip to content

Signed Quantized Weights - #649

Merged
CC-Yeh merged 45 commits into
mainfrom
signed-quantized-weights
Jul 31, 2026
Merged

Signed Quantized Weights#649
CC-Yeh merged 45 commits into
mainfrom
signed-quantized-weights

Conversation

@eugenebokhan

@eugenebokhan eugenebokhan commented Jul 27, 2026

Copy link
Copy Markdown
Contributor
  • load signed quantized weights in rht linear layers on mxu enabled devices
  • merged rht with activations quantization
  • update the benches code a bit
  • split mxu bulky code into separate files

the local iPhone benches show up to 15 % perf improvement on the a8w4 benches over the previous iter

@eugenebokhan eugenebokhan self-assigned this Jul 27, 2026
@eugenebokhan
eugenebokhan force-pushed the signed-quantized-weights branch from aaf76f7 to ac85838 Compare July 27, 2026 14:16
@eugenebokhan
eugenebokhan marked this pull request as ready for review July 28, 2026 19:14

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ec9c484885

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/backend-uzu/src/encodable_block/linear/qlora_wrapper.rs Outdated
Comment thread crates/backend-uzu/src/encodable_block/linear/rht_wrapper.rs Outdated
@CC-Yeh

CC-Yeh commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Any new benchmark results?
Can you also provide a short bullet points in PR body?

@CC-Yeh CC-Yeh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice cleanup 🚀

Have some concerns about performance regression, please take a look

Comment thread crates/backend-uzu/src/encodable_block/linear/matmul.rs Outdated
Comment thread crates/backend-uzu/src/encodable_block/linear/qlora_wrapper.rs Outdated
Comment thread crates/backend-uzu/src/backends/metal/kernel/matmul/common/mxu_fragment/ops.h Outdated
Comment thread crates/backend-uzu/src/backends/metal/kernel/matmul/common/mxu_fragment/ops.h Outdated
Comment thread crates/backend-uzu/src/backends/common/gpu_types/activation_transform.rs Outdated
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@eugenebokhan
eugenebokhan requested a review from CC-Yeh July 30, 2026 18:28

@CC-Yeh CC-Yeh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@CC-Yeh
CC-Yeh merged commit 4978ea2 into main Jul 31, 2026
7 checks passed
@CC-Yeh
CC-Yeh deleted the signed-quantized-weights branch July 31, 2026 01:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants