GitHub - ubergarm/llama.cpp at ug/port-sweep-bench
LLM inference in C/C++. Contribute to ubergarm/llama.cpp development by creating an account on GitHub.
Welcome! Type "help" for available commands.
$
Summary
cafe-llama.cpp is a llama.cpp fork adding CLI flags to offload Mixture of Experts weights to pinned host or CPU RAM for VRAM savings.
It provides flags to mmap or disable the Qwen 3.8 Flash Next internal N-gram PLE table (~51B params) and supports MTP speculative decoding with pre-built GGUF draft models. The project builds on CUDA, Vulkan, ROCm/HIP, Metal, and CPU backends.
