RedHatAI/gemma-4-31B-it-NVFP4 · Hugging Face
NVFP4 variant of gemma-4-31B-it.
Quantized Language Model
Summary
A quantized 8-bit MLX-format version of the Jan-nano model (1.13B params), converted from Menlo/Jan-nano, designed for efficient local inference on Apple Silicon using the mlx-lm library.
Highlights
- 1.13B parameter model quantized to 8-bit for efficient local inference
- Converted to MLX format for native Apple Silicon (M-series) optimization
- Uses BF16 and U32 tensor types for balanced precision and memory efficiency
- Part of a model tree with 18 quantized variants available
- 452 downloads last month, indicating active community interest
auto-generated
Context
Audience
Developers and ML practitioners working on Apple Silicon who want to run small language models locally with reduced memory footprint
DomainMachine Learning
Formatmodel repository
Accessfree online, open source
Source Overview
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Topics
Related
mlx-lmMenlo/Jan-nanoApple Silicon MLX ecosystem8-bit quantizationHugging Face model hub
