
Train your own Decision Model with Unsloth | Unsloth Documentation
Fine-tune LLMs like Qwen and Gemma to make decisions with calibrated probabilities.
A household RAG chatbot uses a 0.6B Qwen model fine-tuned with Unsloth/QLoRA on ~850 labeled entries to classify user questions into metadata categories before querying a vector database.
Baseline prompting of the unmodified model scored 10% accuracy on 131 test cases; fine-tuning raised accuracy to 79%. Remaining errors involve partial category names (e.g., "ac" instead of "hvac") and confusion between semantically overlapping categories such as pool, fountain, and water heater.