Welcome! Type "help" for available commands.

LaurentZuijdwijk/llama.cpp: llama.cpp with adaptive speculative...

~/bookmarks

/**/

GitHub - LaurentZuijdwijk/llama.cpp: llama.cpp with adaptive speculative decoding (--spec-draft-adaptive) and a Vulkan backend tuned for AMD Strix Halo. 4.7x on structured output, 1.9x mainline prefill on MoE.

github.comSaved August 29, 202621 min

Machine Learning Software

Summary

This GitHub repository is a specialized fork of llama.cpp featuring adaptive speculative decoding and a Vulkan backend optimized for AMD Strix Halo hardware. It provides significant performance enhancements for LLM inference, specifically targeting structured output generation and MoE prefill speeds.

Highlights

Implements adaptive speculative decoding that adjusts draft length based on measured acceptance rates rather than a fixed value.
Features a heavily optimized Vulkan backend tuned for AMD Strix Halo (Radeon 8060S) using stock Mesa RADV drivers.
Delivers massive performance gains, including a 4.7x increase in structured output generation and a 1.9x increase in MoE prefill.
Includes support for ROCmFPx/FP4 quantization and optimized handling for specific batch widths (3-8).

auto-generated

Preview of GitHub - LaurentZuijdwijk/llama.cpp: llama.cpp with adaptive speculative decoding (--spec-draft-adaptive) and a Vulkan backend tuned for AMD Strix Halo. 4.7x on structured output, 1.9x mainline prefill on MoE.

LaurentZuijdwijk · via GitHub

Context

Audience

Machine Learning Engineers and Developers optimizing LLM performance on AMD hardware

DomainMachine Learning Inference

FormatOpen source repository

AccessOpen source

Topics

LLM Inference Vulkan Speculative Decoding AMD Strix Halo Llama.cpp

View on GitHub All Bookmarks

llama.cppVulkan APIAMD Strix HaloSpeculative DecodingLLM Quantization

~/bookmarks

GitHub - LaurentZuijdwijk/llama.cpp: llama.cpp with adaptive speculative decoding (--spec-draft-adaptive) and a Vulkan backend tuned for AMD Strix Halo. 4.7x on structured output, 1.9x mainline prefill on MoE.

Summary

Highlights

Context

Topics

Related

GitHub - LaurentZuijdwijk/llama.cpp: llama.cpp with adaptive speculative decoding (--spec-draft-adaptive) and a Vulkan backend tuned for AMD Strix Halo. 4.7x on structured output, 1.9x mainline prefill on MoE.

Summary

Highlights

Context

Topics

Related

~/bookmarks

GitHub - LaurentZuijdwijk/llama.cpp: llama.cpp with adaptive speculative decoding (--spec-draft-adaptive) and a Vulkan backend tuned for AMD Strix Halo. 4.7x on structured output, 1.9x mainline prefill on MoE.

Summary

Highlights

Context

Topics

Related

Discover Similar Content

GitHub - julianmb/q38rocm: Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.

GitHub - quimmedes/cafe-llama.cpp: A experimental fork of llama.cpp with LLM inference in C/C++

GitHub - ubergarm/llama.cpp at ug/port-sweep-bench

Discover Similar Content

GitHub - LaurentZuijdwijk/llama.cpp: llama.cpp with adaptive speculative decoding (--spec-draft-adaptive) and a Vulkan backend tuned for AMD Strix Halo. 4.7x on structured output, 1.9x mainline prefill on MoE.

Summary

Highlights

Context

Topics

Related

Discover Similar Content

GitHub - julianmb/q38rocm: Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.

GitHub - quimmedes/cafe-llama.cpp: A experimental fork of llama.cpp with LLM inference in C/C++

GitHub - ubergarm/llama.cpp at ug/port-sweep-bench