Unsloth Dynamic 3.0 GGUFs | Unsloth Documentation
Unsloth Dynamic 3.0 GGUFs are post-training quantizations (no QAT/QAD) using a refined imatrix calibration dataset and per-layer quantization selectio...
Machine Learning Model
Summary
This repository provides quantized GGUF versions of the uncensored Qwen 3.8 27B model using per-tensor dynamic bit allocation. It features nine quantization tiers designed to optimize the balance between model size and perplexity, supporting massive context windows and high-speed inference.
Highlights
- Uses per-tensor dynamic bit allocation (Unsloth Dynamic 3.0 recipe) to maintain precision in sensitive tensors while maximizing compression in others.
- Offers nine distinct quantization tiers, including specialized architectures for extremely low-bit models like IQ1_M and IQ2_S.
- Supports an extensive 250,000-token context window with high-accuracy exact-string needle retrieval.
- Provides technical documentation for manual quantization, including specific layer pruning and metadata override requirements.
auto-generated
Context
Audience
Machine Learning Engineers and Local LLM Enthusiasts
DomainMachine Learning
Formatmodel weights
Accessopen source
Source Overview
GGUF quantization of Qwen3.8-27B Uncensored using per-tensor dynamic bit allocation derived from the Unsloth Dynamic 3.0 recipe.
Nine tiers (IQ1_M through IQ4_XS) are published, with the 13.2 GB Q3_K_XL matching Q4 perplexity and sustaining 112.8 tok/s on an RTX 4090. Exact-string needle retrieval is confirmed at 250,806-token context depth (98% window occupancy).
Topics
Related
QwenGGUFLlama.cppUnslothQuantizationHugging Face
