Ships with a Multi-Token Prediction drafter for llama.cpp speculative decoding that shares the target KV cache. Supports text and image input, 256K context, Apache 2.0 license.

Topics

Quantization-Aware Training Speculative Decoding Gemma 4 Local LLM Inference GGUF

Visit Site All Bookmarks

Discover Similar Content

William's Bookmark Library

/**/

unsloth/gemma-4-26B-A4B-it-qat-GGUF · Hugging Face

huggingface.coSaved August 1, 202618 min

Summary

QAT-quantized GGUF formats (Q4_0, Q4_K_XL) of Google's Gemma 4 26B A4B MoE model (25.2B total, 3.8B active params) by unsloth.

Ships with a Multi-Token Prediction drafter for llama.cpp speculative decoding that shares the target KV cache. Supports text and image input, 256K context, Apache 2.0 license.

Topics

Quantization-Aware Training Speculative Decoding Gemma 4 Local LLM Inference GGUF

Visit Site All Bookmarks