Executive Summary
- Experts discuss the feasibility of running GLM 5.2 on slow computers
- Forum users share their experiences with locally hosted LLMs and potential solutions
- Developers are working on similar projects targeting various hardware and software platforms
The Buzz Score
The Internet’s Verdict: 70% Hyped, 30% Skeptical
Forum Voices
Experts and developers are weighing in on the topic, sharing their thoughts and experiences.
My main question is whether when put into practical use, this can be measured in tokens/second, or more like 1 token per minute…
Some users have seen locally hosted LLMs that are as slow as 1 token/second still be very useful for certain tasks.
I think this is a fantastic project in general concept, and look forward to seeing more efforts towards the general idea of being able to run a 350B to 900B size model locally, even if as slow as 1 tok/s, on hardware that ordinary people can afford.
Developer Perspectives
Developers are working on similar projects, targeting various hardware and software platforms.
One developer is working on a project targeting macOS on Apple Silicon, using compressed partial residency in unified memory and native Metal kernels.
This is exactly the kind of technology that I expect Apple to ship anytime soon given the RAM prices and their HW/SW integration skills…
Focus Keyword: GLM 5.2
