Lucebox is a $6,499 open-source local AI inference box pairing 128 GB unified memory with a 32 GB AMD R9700 GPU, pre-loaded with tuned models and an OpenAI/Anthropic-compatible endpoint.
It delivers 51.1 tok/s median on DeepSeek V4 Flash and runs Qwen3.6 and GLM-4.6 entirely on-prem. The 6 kg aluminium unit ships with Ubuntu and the inference engine pre-installed for immediate local deployment.
Lucebox optimization hub: hand-tuned LLM inference, built for specific consumer hardware. - Luce-Org/lucebox-hub
LM Studio 0.4.0 introduced llmster and the lms CLI. Here is how I set up Gemma 4 26B for local inference on macOS that can be used with Claude Code.
Community recipes for serving LLMs on RTX 3090. Multi-engine (vLLM, llama.cpp, SGLang) and model-agnostic. Currently shipping Qwen3.6-27B configs for ...
