Welcome! Type "help" for available commands.
$
Machine Learning Model
Summary
This repository provides quantized GGUF versions of the uncensored Qwen 3.8 27B model using per-tensor dynamic bit allocation. It features nine quantization tiers designed to optimize the balance between model size and perplexity, supporting massive context windows and high-speed inference.
Highlights
- Uses per-tensor dynamic bit allocation (Unsloth Dynamic 3.0 recipe) to maintain precision in sensitive tensors while maximizing compression in others.
- Offers nine distinct quantization tiers, including specialized architectures for extremely low-bit models like IQ1_M and IQ2_S.
- Supports an extensive 250,000-token context window with high-accuracy exact-string needle retrieval.
- Provides technical documentation for manual quantization, including specific layer pruning and metadata override requirements.
auto-generated
Context
Audience
Machine Learning Engineers and Local LLM Enthusiasts
DomainMachine Learning
Formatmodel weights
Accessopen source
Topics
Related
QwenGGUFLlama.cppUnslothQuantizationHugging Face