Demo repository for running PrismML's Bonsai 2 27B (ternary), Bonsai (1-bit), and Ternary-Bonsai language models locally on Mac, Linux, Windows, or CPU.
Bonsai 2 27B supports vision input, OpenAI-style tool calling, and a 262K-token context at 5.9 GB in ternary weights, requiring the PrismML llama.cpp fork. Earlier Bonsai families are available in 27B, 8B, 4B, and 1.7B sizes; setup is a two-command process via setup.sh or setup.ps1.
PrismML's Bonsai 27B, based on Qwen3.6 27B, is a multimodal model with 262K-token context and tool-calling support, shipped in ternary (5.9 GB) and 1-...
Community recipes for serving LLMs on RTX 3090. Multi-engine (vLLM, llama.cpp, SGLang) and model-agnostic. Currently shipping Qwen3.6-27B configs for ...
