I upgraded the RAM on my HP desktop on August 9th. 8GB -> 32GB.
The reason was gemma4:26b-a4b-it-qat. Twenty-six billion parameters, quantized, but it still won’t fit in 8GB without constant swapping. The box had been running smaller background jobs with a RAM threshold gate, a floor of 900MB free to ensure Ollama had room to work. A 26B model changes the math.
With 32GB, the model loads and stays resident. About 10.6GB free with it warm and my Home Assistant VM running alongside. The upgrade also surfaced something I hadn’t thought about. The old idle gate floor was set to 900MB, which had made sense when the box had 8GB but was now stale by a factor of four. I reset it to 4GB and updated the scripts. The background jobs that use HP’s local Ollama picked up from there. The log digest runs daily at 6:10 AM; the board grooming script calls over Tailscale from EC2 every couple hours.
It works. And then I ran a benchmark.
Generation speed on gemma4:26b-a4b-it-qat on this box is about 10 tokens per second. Prompt eval runs around 108 tok/s. But generation is the number that matters when something is waiting on a response. Ten tok/s means a 200-token answer takes twenty seconds. A 500-token response is nearly a minute.
The box has an Intel i7-8700 and Intel UHD 630 integrated graphics. No discrete GPU. Inference is CPU-bound. That’s what 10 tok/s looks like on a 2017 i7 with no accelerator.
For batch jobs, fine. The log digest doesn’t need to summarize an anomaly log quickly. The board groomer isn’t time-sensitive. The speed works for the workload the setup was built for.
Where it doesn’t work is anything interactive. An agentic loop where the next call waits on the current response needs faster generation than this. Ten tok/s is too slow to think out loud. For that kind of work, the box needs a GPU. That’s a different upgrade.
Not yet where I want to be with my local LLM setup, but I’m closer :)