outsourc-e/Qwen3.8-27B-Unleashed-GGUF · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
LLM Quantization Tool
Summary
Unsloth Dynamic 3.0 GGUFs are post-training quantizations of Qwen3.8-27B using refined imatrix calibration and per-layer quantization selection. They claim over 10% better top-1% accuracy at equal file size versus other providers, compatible with llama.cpp and Unsloth Desktop.
Highlights
- Post-training quantization (no QAT/QAD) using refined imatrix calibration dataset and per-layer quantization selection
- Over 10% better top-1% accuracy at equal file size vs other providers on Qwen3.8-27B
- File sizes range from UD-IQ1_S at 6.2GB to UD-Q2_K_XL at 9.83GB; quants under 8.37GB omit MTP module to save ~500MB
- Evaluated via Divergence-300 @32 and KL Divergence on 300 held-out prompts from Terminal-Bench, DeepSWE, Harbor, MathArena
- Over 5.1 million Qwen3.8 downloads in 5 days; imatrix file shared publicly for community use and fine-tuning
auto-generated
Context
Audience
LLM practitioners, local inference developers, and researchers interested in model quantization and efficient deployment
DomainLarge Language Model Quantization
FormatDocumentation / Technical Release
AccessFree online documentation; quants available for download
Source Overview
Unsloth Dynamic 3.0 GGUFs are post-training quantizations (no QAT/QAD) using a refined imatrix calibration dataset and per-layer quantization selection, compatible with llama.cpp and Unsloth Desktop.
Benchmarks for Qwen3.8-27B report over 10% better top-1% accuracy at equal file size versus other providers, evaluated via Divergence-300 @32 and KL Divergence on held-out prompts. File sizes range from UD-IQ1_S at 6.2GB to UD-Q2_K_XL at 9.83GB, with quants under 8.37GB omitting the MTP module to save roughly 500MB.
Topics
Related
llama.cppUnsloth DesktopQwen3.8-27Bimatrix calibrationGGUF formatDivergence-300 benchmark
