AMD’s Radeon RX 7900 XTX is proving to be a powerhouse when it comes to running the DeepSeek R1 AI model, outperforming even NVIDIA’s formidable GeForce RTX 4090 in inference benchmarks.
—
DeepSeek’s latest AI model is making waves across the tech industry. While it’s intriguing to consider the sheer computing resources that fuel its training, everyday users now have the opportunity to achieve impressive performance with AMD’s “RDNA 3” Radeon RX 7900 XTX GPU. AMD has showcased DeepSeek’s R1 inference benchmark results, revealing that their flagship RX 7000 series GPU surpasses NVIDIA’s counterpart in various models.
In a tweet, David McAfee highlighted how well DeepSeek performs on the AMD Radeon 7900 XTX. He also provided a link to more detailed information on running these AI models on Radeon GPUs and Ryzen AI APUs.
For consumers looking to utilize their GPUs for AI tasks, this is an exciting development. Traditionally, these tasks have demanded expensive AI accelerators, but consumer GPUs have now become viable alternatives, thanks to their cost efficiency. Plus, running models locally means your privacy is better protected—a significant concern with AI data models like DeepSeek. AMD has thoughtfully provided a comprehensive guide for running DeepSeek R1 distillations on their GPUs:
1. Ensure you have the 25.1.1 Optional or newer Adrenalin driver installed.
2. Visit lmstudio.ai/ryzenai to download LM Studio version 0.3.8 or later.
3. Install LM Studio and skip through the onboarding steps.
4. Navigate to the ‘discover’ tab.
5. Select your preferred DeepSeek R1 distill. For beginners, smaller distills like Qwen 1.5B are recommended for their rapid performance; however, larger distills deliver enhanced reasoning abilities.
6. On the right, ensure the “Q4 K M” quantization is selected before downloading.
7. Once downloaded, switch back to the chat tab, pick the DeepSeek R1 distill from the drop-down, and tick “manually select parameters.”
8. Maximize the GPU offload layers by sliding the control all the way to the right.
9. Load the model.
10. Engage with a reasoning model running entirely on your AMD setup!
If these steps seem daunting, don’t worry—AMD has released a YouTube tutorial that guides users through each phase. It’s worth exploring to gain the confidence needed to use DeepSeek’s LLMs on your AMD hardware, securing your data from potential misuse. As NVIDIA and AMD continue to roll out new GPUs, we expect the efficiency and power of inferencing to soar, particularly with the integration of dedicated AI engines designed to handle such workloads.
