Executive Summary

• Old Xeon processors can still run modern AI models with decent performance
• Forum users share their experiences with running Gemma 4 models on 10 year old Xeons
• Local AI processing is becoming increasingly powerful and accessible

The Buzz Score

The Internet’s Verdict: 70% Hyped, 30% Skeptical

Forum Voices

Users are sharing their experiences with running AI models on old Xeon processors.

Glad to see other people realizing this. I’ve been running Gemma 26B-A4B Q4 on a 2012 Xeon. It’s getting around 8 to 12 tokens per second.

Others have also managed to get modern AI models running on older hardware.

Hi HN. I wrote this post after getting frustrated by the lack of ways to run the new Gemma 4 Drafter models, and mainstream tools not prioritizing this, and hiding all the performance levers. I ended up getting a modern 26B MoE model (Gemma 4) running at reading speed on an old recycled server with a single Xeon E5-2620 v4 and 128GB of DDR3 RAM (and no GPU).

Implications and Future

The ability to run AI models locally on older hardware has significant implications for the future of AI development and accessibility.

Focus Keyword: Xeon Performance