GitHub - bilikaz/qwen38-flash-next-recipe: qwen38-flash-next recipe for solo GB10
qwen38-flash-next recipe for solo GB10. Contribute to bilikaz/qwen38-flash-next-recipe development by creating an account on GitHub.
Machine Learning Deployment Guide
Summary
A technical deployment guide for serving the 99 GiB Qwen3.8-Flash-Next vision-language model on a single DGX Spark system using vLLM. It provides a self-contained setup process, specific memory management configurations via PLE table mapping, and detailed performance benchmarks for multimodal inference.
Highlights
- Enables serving a large 99 GiB vision-language model (text, image, video) on a single DGX Spark's 121 GiB unified memory.
- Utilizes vLLM with PLE table offloading and memory-mapping for efficient resource utilization.
- Delivers high-performance throughput, reaching 162.9 tok/s aggregate across 8 concurrent streams.
- Includes a streamlined workflow with dedicated scripts for downloading, starting, and gracefully stopping the service.
auto-generated
Context
Audience
Machine Learning Engineers and AI Infrastructure Specialists
DomainMachine Learning
FormatTechnical Documentation
Accessopen source
Source Overview
Serves a 99 GiB Qwen3.8-Flash-Next-NVFP4 vision-language model (text, image, video) from a single DGX Spark's 121 GiB unified memory via vLLM with the PLE table memory-mapped.
Ships with 262k context, FP8 KV cache, MTP 3, and up to 8 concurrent sequences; measured at 48.7 tok/s single-stream and 162.9 tok/s aggregate at 8 streams. Requires ~130 GiB free disk, reaches health in 10-12 minutes, and serves an API on port 8888.
Topics
Related
vLLMQwen3.8-Flash-NextDGX SparkVision-Language ModelsFP8 Quantization
