hipfire-models/qwen3.8-27b · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
AI Model Quantization
Summary
Custom ROCmFP4 and ROCmFP8 quantized GGUF builds of the Muse-Glimmer-30B multimodal model, tested on AMD Strix Halo. Requires a patched ROCmFPX runtime; not compatible with stock llama.cpp. Includes main models, BF16 vision projector, and DFlash drafters.
Highlights
- First Muse-Glimmer-30B ROCmFPX repository on Hugging Face at publication time; no other ROCmFPX builds existed for this model
- Requires patched charlie12345/ROCmFPX runtime with a specific patch applied; stock llama.cpp cannot load these tensor layouts
- Prompt throughput ranges from 39 to 113.7 tokens/s; output rates from 7.8 to 28.3 tokens/s depending on quantization and drafter
- DFlash speculative decoding drafters boost output to ~28 tokens/s with a six-token draft window on the FP4 main model
- BF16 vision projector (3.59 GiB) is compatible with all three text model variants
auto-generated
Context
Audience
AI engineers and hobbyists running local multimodal models on AMD Strix Halo hardware who want optimized quantized builds
DomainAI Model Quantization
FormatModel repository with documentation
AccessFree download from Hugging Face Hub
Source Overview
Custom ROCmFP4 and ROCmFP8 quantized GGUF builds of the Muse-Glimmer-30B multimodal model, tested on AMD Strix Halo (gfx1151).
Files include main-model quantizations (14.17–26.77 GiB), a BF16 vision projector, and DFlash drafters; they require a patched ROCmFPX runtime and will not load in stock llama.cpp. Measured prompt throughput ranges from 39 to 113.7 tokens/s with output rates of 7.8 to 28.3 tokens/s depending on quantization and drafter configuration.
Topics
Related
ROCmFPX runtimeMuse-Glimmer-30BAMD Strix Halo (gfx1151)DFlash speculative decodingllama.cppGGUF format
