Welcome! Type "help" for available commands.

Niko1221/Strata: Qwen3.8-Flash-Next (125B MoE) on a 8GB+ NVIDIA...

~/bookmarks

/**/

GitHub - Niko1221/Strata: Qwen3.8-Flash-Next (125B MoE) on a 8GB+ NVIDIA GPU: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

github.comSaved October 5, 202610 min

Local LLM Inference Tool

Summary

Strata runs the 125B Qwen3.8-Flash-Next MoE model on a single 12-24 GB NVIDIA GPU with 64 GB RAM at 52-90 tokens per second. A one-click installer for Windows and Linux exposes OpenAI and Anthropic-compatible APIs on localhost with optional image input.

Highlights

Runs a 125B-parameter MoE model on a single consumer GPU (12-24 GB VRAM) with 64 GB system RAM, eliminating the need for server-grade hardware
Achieves 52-90 tokens/s generation speed depending on quantization level, with prompt reading at 1,070-1,310 tokens/s
One-click installer for both Windows and Linux; exposes OpenAI and Anthropic-compatible REST APIs on localhost
Offers multiple quantization tiers (Q2_0 through IQ3_S) trading speed for quality, plus a Coder variant with half the experts removed for coding and tool-use tasks
Supports 128K context windows and optional image input

auto-generated

Preview of GitHub - Niko1221/Strata: Qwen3.8-Flash-Next (125B MoE) on a 8GB+ NVIDIA GPU: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

Niko1221 · via GitHub

Context

Audience

Developers and AI enthusiasts who want to run frontier-scale language models locally on a gaming PC without cloud dependencies

DomainLarge Language Model Inference

Formatopen-source software project with documentation

Accessfree and open source on GitHub

Topics

NVIDIA GPU Mixture-of-Experts Qwen3.8-Flash-Next Model Quantization Local LLM Inference

View on GitHub All Bookmarks

Qwen3.8-Flash-NextMixture-of-Experts architectureModel quantization IQ formatsOllamallama.cppLocal LLM serving

~/bookmarks

GitHub - Niko1221/Strata: Qwen3.8-Flash-Next (125B MoE) on a 8GB+ NVIDIA GPU: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

Summary

Highlights

Context

Topics

Related

GitHub - Niko1221/Strata: Qwen3.8-Flash-Next (125B MoE) on a 8GB+ NVIDIA GPU: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

Summary

Highlights

Context

Topics

Related

~/bookmarks

GitHub - Niko1221/Strata: Qwen3.8-Flash-Next (125B MoE) on a 8GB+ NVIDIA GPU: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

Summary

Highlights

Context

Topics

Related

Discover Similar Content

GitHub - claudejaune/Qwen3.8-Halogen-Flash-Strix-Halo: Run Qwen3.8-Flash-Next (180B, 6B active) on a single AMD Strix Halo via the halogen-flash-server container

GitHub - julianmb/q38rocm: Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.

GitHub - gufo-org/gufo: Strix Halo inference engine. Qwen Flash Next Q4_K_XL: 1,628.52pp, 59.41tg single user, 157.22 tok/s 8 users; Qwen27B Q4_K_XL: 656.33pp, 70.56tg tok/s single user with DFlash2

Discover Similar Content

GitHub - Niko1221/Strata: Qwen3.8-Flash-Next (125B MoE) on a 8GB+ NVIDIA GPU: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

Summary

Highlights

Context

Topics

Related

Discover Similar Content

GitHub - claudejaune/Qwen3.8-Halogen-Flash-Strix-Halo: Run Qwen3.8-Flash-Next (180B, 6B active) on a single AMD Strix Halo via the halogen-flash-server container

GitHub - julianmb/q38rocm: Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.

GitHub - gufo-org/gufo: Strix Halo inference engine. Qwen Flash Next Q4_K_XL: 1,628.52pp, 59.41tg single user, 157.22 tok/s 8 users; Qwen27B Q4_K_XL: 656.33pp, 70.56tg tok/s single user with DFlash2