AI Model Adapter
Summary
This Hugging Face repository provides a QLoRA adapter for Google Gemma 4 31B-it, fine-tuned on a curated subset of the Opus-4.6 dataset focusing on math and code reasoning. The model uses 4-bit NF4 quantization and BF16 precision, achieving an eval loss of 3.6018.
Highlights
- Fine-tuned on a cleaned 2025-row subset of the Opus-4.6 dataset, focusing on 1899 math and 126 code rows.
- Uses QLoRA with 4-bit NF4 quantization and targets specific linear projection modules (q_proj, k_proj, etc.).
- Achieved a final eval loss of 3.6018 and perplexity of 36.66 after 2 epochs.
- Requires loading via PEFT alongside the base google/gemma-4-31B-it model.
- Excludes unrelated instruction corpora to maintain pure reasoning-style training.
auto-generated
Context
Audience
AI Researchers and Machine Learning Engineers
DomainMachine Learning
Formatmodel repository
Accessopen source
Topics
Related
Gemma 4 31BQLoRAOpus-4.6 ReasoningPEFTHugging Face