GitHub - kyuz0/amd-strix-halo-toolboxes
Contribute to kyuz0/amd-strix-halo-toolboxes development by creating an account on GitHub.
AI Infrastructure Setup
Summary
Strix Halo Toolboxes offer containerized environments for local AI on AMD Ryzen AI MAX+ processors with 128GB unified memory. They provide setups for inference, generation, serving, and fine-tuning using Toolbx on Fedora 43.
Highlights
- Supports AMD Ryzen AI MAX+ with Zen 5 and RDNA 3.5 architecture.
- Provides containers for LLAMA.CPP, ComfyUI, vLLM, and fine-tuning.
- Uses custom kernel parameters for dynamic memory allocation.
- Includes RCCL patches for high-speed clustering.
- Maintained by Donato Capitella for Framework Desktop users.
auto-generated
Context
Audience
System Administrators and AI Developers
DomainLocal AI Inference
Formatsoftware repository
Accessopen source
Source Overview
Strix Halo Toolboxes provide containerized environments using Toolbx for running AI workloads on AMD's Ryzen AI MAX+ (Strix Halo) processors with up to 128GB unified memory.
They include repositories for LLAMA.CPP (LLM inference with RDMA/Vulkan/ROCm), ComfyUI (image/video generation), VLLM (serving with RCCL patches), and LLM fine-tuning (QLoRA/full for Gemma 3/Qwen 3). Setup on Fedora 43 with Framework Desktop uses custom kernel parameters for dynamic memory allocation, GPU device mapping, and accelerator-performance tuning.
Topics
Related
AMD ROCmToolbxLLAMA.CPPComfyUIvLLMFramework Desktop
