z-lab/Qwen3.6-27B-DFlash · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Machine Learning Model
Summary
This content provides a specialized DSpark speculator model designed to optimize the inference speed of the Qwen3.6-35B-A3B large language model through speculative decoding. It includes technical details on the model's architecture, training datasets, and specific implementation commands for deployment using vLLM.
Highlights
- Utilizes DSpark architecture which extends DFlash with a Markov head and a confidence head for improved token dependency and acceptance prediction.
- Trained using the Speculators library on Magpie and ultrachat datasets to enhance speculative decoding efficiency.
- Optimized for deployment via vLLM using 8-token speculative blocks to increase inference throughput.
- Provides comprehensive technical workflows for data preparation, training, and vLLM integration.
auto-generated
Context
Audience
Machine Learning Engineers and AI Researchers
DomainMachine Learning
FormatModel Repository
AccessOpen source
Source Overview
DSpark speculative decoding model for Qwen3.6-35B-A3B, trained via the Speculators library on Magpie-Llama-3.1-Pro-300K-Filtered and ultrachat_200k data.
Extends DFlash with a Markov head for intra-block token dependency and a confidence head for per-position acceptance prediction. Deployed through vLLM with 8-token speculative blocks; average accepted length ranges from 3.4 (QA) to 5.03 (math_reasoning) tokens.
Topics
Related
QwenvLLMSpeculative DecodingSpeculators libraryDSparkHugging Face
