~/bookmarks

William's Bookmark Library

/**/

GitHub - AtomicBot-ai/atomic-llama-cpp-turboquant: llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% throughput).

github.comSaved May 21, 202624 min

Preview of GitHub - AtomicBot-ai/atomic-llama-cpp-turboquant: llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% throughput).

AtomicBot-ai · via GitHub

Summary

Fork of llama.cpp for C/C++ LLM inference with TurboQuant low-bit KV cache and weight compression.

Supports Gemma 4 MTP and Qwen 3.6 NextN speculative decoding, with claimed throughput gains of about 30–50%. Includes install, Docker, binary release, and source build instructions.

Topics

LLM Inference Optimization Llama.cpp Forks Speculative Decoding Techniques LLM Quantization & Compression On-Device AI Deployment

View on GitHub All Bookmarks

Discover Similar Content