unsloth/Kimi-K2-Instruct-GGUF · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Open Source LLM
Summary
Kimi Linear is an open-source 48B-parameter LLM from Moonshot AI featuring a hybrid linear attention architecture (KDA) that achieves 1M token context with up to 6x faster decoding and 75% KV cache reduction compared to full attention.
Highlights
- Hybrid 3:1 KDA-to-global MLA ratio reduces KV cache by up to 75% while maintaining or surpassing full attention quality
- Achieves 6.3x faster TPOT vs MLA and 3.98x speedup at 128k context on RULER benchmark
- 48B total params with only 3B activated, trained on 5.7T tokens, supporting 1M context length
- Kimi Delta Attention refines Gated DeltaNet with fine-grained gating for efficient finite-state RNN memory
- Open-sourced KDA kernel in FLA library with Hugging Face Transformers integration
auto-generated
Context
Audience
ML researchers, AI engineers, and developers interested in efficient long-context LLMs and linear attention mechanisms
DomainLarge Language Models / Attention Mechanisms
FormatModel card / technical report
AccessFree, open source on Hugging Face
Source Overview
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Topics
Related
Gated DeltaNetFLA (Flash Linear Attention) libraryMoonshot AIMMLU-Pro benchmarkRULER benchmark
