GitHub - bilikaz/qwen38-flash-next-recipe: qwen38-flash-next recipe for solo GB10
qwen38-flash-next recipe for solo GB10. Contribute to bilikaz/qwen38-flash-next-recipe development by creating an account on GitHub.
Technical Documentation
Summary
This documentation provides technical instructions for deploying the Qwen3.8-27B vision-language model using the SGLang framework. It covers hardware compatibility, quantization options, and specialized memory management configurations for hybrid GDN architectures.
Highlights
- Supports deployment on high-end hardware including H200, RTX 5090, and DGX Spark
- Offers multiple quantization precisions: BF16, FP8, and NVFP4 W4A4
- Includes a Mamba SSM memory ratio calculator to optimize GDN state pool sizing
- Supports speculative decoding via EAGLE or DSPARK to improve performance
auto-generated
Context
Audience
Machine Learning Engineers and AI Infrastructure Developers
DomainMachine Learning
FormatTechnical Documentation
AccessFree online
Source Overview
SGLang serves Qwen3.8-27B, a 27B hybrid Gated Delta Networks vision-language model, on a single H200, RTX PRO 6000, RTX 5090, or DGX Spark.
Checkpoints come in BF16, FP8, and NVFP4 W4A4 with an in-checkpoint MTP head and 262,144-token native context. Deployment supports EAGLE or DSPARK speculative decoding and includes a Mamba SSM memory ratio calculator for GDN state pool sizing.
Topics
Related
SGLangQwen 3Mamba ArchitectureSpeculative DecodingFP8 Quantization
