GitHub - MiaAI-Lab/Qwen3.8-27B-DGX-Spark-RTX-6000: Qwen3.8 27B for DGX Spark / RTX 6000 PRO
Qwen3.8 27B for DGX Spark / RTX 6000 PRO. Contribute to MiaAI-Lab/Qwen3.8-27B-DGX-Spark-RTX-6000 development by creating an account on GitHub.
Open Source Software
Summary
This GitHub repo offers Docker configs for running LLMs on RTX 3090s using vLLM or llama.cpp. It focuses on Qwen3.6-27B, supporting single or dual GPU setups with OpenAI-compatible APIs and detailed benchmarks.
Highlights
- Multi-engine support for vLLM, llama.cpp, and SGLang.
- Optimized for RTX 3090 hardware in 1x, 2x, or cluster setups.
- Production-ready configs for Qwen3.6-27B model.
- Includes OpenAI-compatible API via Docker Compose.
- Provides scaling docs and performance benchmarks.
auto-generated
Context
Audience
AI developers and homelab enthusiasts deploying local LLMs
DomainMachine Learning Infrastructure
Formatopen source repository
Accessopen source
Source Overview
Repository providing Docker-based configurations for running large language models locally on RTX 3090 GPUs using multiple inference engines (vLLM, llama.cpp, SGLang).
Currently supports Qwen3.6-27B with options for single or dual GPU setups, offering throughput up to 127 tokens/second or robust 262K context handling depending on engine choice. Includes OpenAI-compatible API, benchmarking tools, and scaling guidance for 3+ GPU clusters.
Topics
Related
vLLMllama.cppSGLangQwen3.6-27BRTX 3090Docker Compose
