GitHub - warpfront/hipfire: RDNA-native LLM inference engine in Rust.
RDNA-native LLM inference engine in Rust. . Contribute to warpfront/hipfire development by creating an account on GitHub.
ML Model Repository
Summary
Quantized Qwen3.8-27B model artifacts (MQ3V2-MQ6V2, Lloyd-Max MQ4L tiers) for hipfire, a Rust-native LLM inference engine targeting AMD GPUs. Three quality tiers per bit-width with DFlash drafters for speculative decoding.
Highlights
- Complete quantization ladder from 3.5 to 6.6 bits-per-weight with three tiers per bit-width (xt, base, pro) offering a quality/size tradeoff spectrum
- MQ4L tiers use per-tensor 16-level Lloyd-Max codebooks instead of uniform 4-bit grids, reducing teacher KLD by roughly 15% at identical bits per weight
- Targets AMD GPUs specifically: RDNA4 (gfx1201) supports both prefill and decode; RDNA3 (gfx1100/gfx1151) supports decode only
- MQ4V2 supersedes MQ4V1 as the canonical product; files are self-describing via HFQM quant-type IDs for backward compatibility
- Defaults include Q8 KV cache, 262K-token context window, and 81K-token max output length
auto-generated
Context
Audience
Developers and ML engineers running LLM inference on AMD GPUs who need quantized model artifacts with varying quality and size tradeoffs
DomainLLM Inference and Quantization
Formatmodel artifact repository
Accessfree on Hugging Face (open source)
Topics
Related
hipfire inference engineQwen 3.8 model familyAMD RDNA4 and RDNA3 GPUsDFlash speculative decodingLloyd-Max quantizationAWQ activation-aware quantization
