4 bit models on hopper support #48

Open

opened

We would like to run the GLM 5.2 model in int4/mxfp4 on a single node with 8×H20 GPUs. Could you let us know whether Tilert currently has plans to support 4-bit inference on the Hopper architecture?

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests