Does the Desktop have ROCM support - Framework Desktop - Framework Community
If you are a beginner like me, you may find those container based solutions helpful. If you are on Fedora42, just follow the explanations on his GitHu...
Summary
The post discusses experiments with hybrid AI inferencing using both integrated and discrete GPUs, specifically with AMD Ryzen AI Max and various dGPUs, to accelerate large language model deployments.
The key advantage comes from storing KV cache on the dGPU to improve inference speed, while avoiding excessive model quantization to maintain output quality. The user notes hardware limitations with some GPUs and seeks support for verifying proper dGPU compatibility and performance on the Framework Desktop.
Plans include building an affordable cluster capable of running large models like Llama4-400b and DeepSeek-671b for small teams or personal use.
