Welcome! Type "help" for available commands.
$
Summary
Lily is a Rust runtime with custom Metal kernels that runs Qwen3.6-35B-A3B locally on Apple silicon without PyTorch or MLX in the execution path.
On an M5 Max MacBook Pro, it averages 1.23x prefill and 1.35x decode throughput over MLX-LM across 256–128K token contexts. Phase-specific optimizations include Metal 4 tensor ops for prefill GEMM, vector ALUs for batch-1 decode GEMV, and tailored kernels for Qwen's MoE routing, Gated DeltaNet recurrence, and grouped-query attention layers.