unsloth/gemma-4-26B-A4B-it-qat-GGUF · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Local AI Inference Tutorial
Summary
A practical guide to running Google Gemma 4 26B-A4B locally on macOS using LM Studio 0.4.0's new lms CLI, achieving 51 tokens/sec on a 48GB M4 Pro MacBook and integrating with Claude Code for zero-cost local inference.
Highlights
- LM Studio 0.4.0 introduces llmster daemon and lms CLI for headless local model serving
- Gemma 4 26B-A4B uses MoE architecture activating only 3.8B params per token, achieving 51 tok/s on 48GB M4 Pro
- Scores 82.6% MMLU Pro and 88.3% AIME 2026, close to the 31B dense variant while running dramatically faster
- Setup involves installing lms CLI, starting the daemon, loading Q4_K_M quantized model, and aliasing with Claude Code
- MoE rule of thumb: effective quality ~ sqrt(total x active params), placing this model around 10B effective
auto-generated
Context
Audience
Developers and AI practitioners who want to run capable LLMs locally on Apple Silicon Macs without API costs or data leaving their machine
DomainLocal LLM Inference
Formatlong-form technical article
Accessfree online
Source Overview
LM Studio 0.4.0 introduces llmster and lms CLI for headless local inference of Google Gemma 4 26B-A4B on macOS.
The MoE model activates 3.8B parameters per token, achieving 51 tokens/second on a 48GB MacBook Pro M4 Pro with 82.6% MMLU Pro score. Setup involves installing lms CLI, starting the daemon, and loading the Q4_K_M quantized model for use with Claude Code.
Topics
Related
LM StudioClaude CodeGoogle Gemma 4Mixture-of-Experts architectureQ4_K_M quantizationMMLU Pro benchmark
