GitHub - warpfront/hipfire: RDNA-native LLM inference engine in Rust.
RDNA-native LLM inference engine in Rust. . Contribute to warpfront/hipfire development by creating an account on GitHub.
ML Model Repository
Summary
Quantized Qwen3.8-27B model artifacts (MQ3V2-MQ6V2, Lloyd-Max MQ4L tiers) for hipfire, a Rust-native LLM inference engine targeting AMD GPUs. Three quality tiers per bit-width with DFlash drafters for speculative decoding.
Highlights
- Complete quantization ladder from 3.5 to 6.6 bits-per-weight with three tiers per bit-width (xt, base, pro) offering a quality/size tradeoff spectrum
- MQ4L tiers use per-tensor 16-level Lloyd-Max codebooks instead of uniform 4-bit grids, reducing teacher KLD by roughly 15% at identical bits per weight
- Targets AMD GPUs specifically: RDNA4 (gfx1201) supports both prefill and decode; RDNA3 (gfx1100/gfx1151) supports decode only
- MQ4V2 supersedes MQ4V1 as the canonical product; files are self-describing via HFQM quant-type IDs for backward compatibility
- Defaults include Q8 KV cache, 262K-token context window, and 81K-token max output length
auto-generated
Context
Audience
Developers and ML engineers running LLM inference on AMD GPUs who need quantized model artifacts with varying quality and size tradeoffs
DomainLLM Inference and Quantization
Formatmodel artifact repository
Accessfree on Hugging Face (open source)
Source Overview
This repository provides quantized Qwen3.8-27B model files (MQ3V2 through MQ6V2, plus MQ4L Lloyd-Max tiers) for use with hipfire, a Rust-native LLM inference engine targeting AMD GPUs.
Each bit-width ships three tiers (xt, base, pro) with DFlash drafters for speculative decoding; MQ4V2 is the canonical MQ4 product. Runtime support covers RDNA4 prefill and decode plus RDNA3 decode, with defaults of Q8 KV cache, 262K-token context, and 81K-token max output.
Topics
Related
hipfire inference engineQwen 3.8 model familyAMD RDNA4 and RDNA3 GPUsDFlash speculative decodingLloyd-Max quantizationAWQ activation-aware quantization
