hipfire-models/qwen3.8-27b · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
GPU Inference Framework
Summary
hipfire is a Rust and HIP-based LLM inference engine targeting AMD RDNA and CDNA GPUs. It provides an Ollama-style CLI and OpenAI-compatible API with no Python in the hot path. v0.4.0 supports quantized Qwen models, speculative decoding, and image generation via a kernel dispatch layer called Redline.
Highlights
- Rust + HIP with no Python in the hot path; Ollama-style CLI and OpenAI-compatible API on loopback port 11435
- v0.4.0 achieves ~5,120 tok/s prefill on R9700 with native fp8 KV; speculative decode boosts codeedit from 39.3 to 68.1 tok/s
- Supports Qwen3.8-27B with MQ4V2 XT quantization, Qwen3.8 Flash-Next MoE at 262K context, and Flux Schnell image generation
- Redline kernel dispatch layer covers RDNA1 through RDNA4; weights distributed via Hugging Face hipfire-models org
- MQ4R format combines uniform MQ4 attention with graded routed experts and fused gate path for MoE models
auto-generated
Context
Audience
AMD GPU users seeking local LLM inference without Python dependencies, particularly those on RDNA3 or RDNA4 hardware
DomainGPU-accelerated LLM inference
Formatopen-source software project
Accessopen source on GitHub; weights on Hugging Face
Source Overview
hipfire is a Rust and HIP inference engine for AMD RDNA and CDNA GPUs, providing an Ollama-style CLI and an OpenAI-compatible API. v0.4.0 runs quantized Qwen models with speculative decode and image generation on RDNA1 through RDNA4 via a kernel dispatch layer called Redline.
Weights ship through Hugging Face; the daemon binds to loopback port 11435 by default.
Topics
Related
ROCmOllamaQwen3.8Flux SchnellHugging Face hipfire-modelsAMD RDNA architecture
