Skip to content

Comparison of Local LLMs

English | Japanese

« Home

OpenRheo assumes two tiers: a high-performance cloud LLM as teacher, a local LLM on site as executor. This page compares local LLMs as measured on a GH200. How the agent asks the LLM and checks its answers is on The local-LLM agent.

Five local models: accuracy by kind of question, and speed

Where the local LLM runs

One GH200 node has 120,857 MB of memory on the Grace CPU and 97,280 MB on the Hopper GPU (measured 2026-07-27). The local LLM is served by ollama on the GPU; every measurement below was taken that way. The model files take 4.7 GB (qwen2.5 7b) and 9.0 GB (qwen2.5 14b), and a 14b and a 72b model (about 9 + 45 GB) stay resident together on the 96 GB GPU. Running the LLM on the same GPU as an LBM run at the same time has not been measured.

Measurements (2026-07-15, ollama on a GH200, Q4 quantisation)

model easy questions (24) diagnose a failed run (6) paraphrased requests (27) generation, median tok/s latency, median s
qwen2.5 7b 24 4 21 227.5 0.3
qwen2.5 14b 24 4 24 132.0 0.6
gemma3 27b 24 4 21 75.8 1.2
llama4 scout 24 6 27 97.7 1.2
qwen2.5 72b 24 6 24 40.8 1.7

Every model answers the easy calls. The models differ only when they must diagnose a failed run from its log, and on paraphrased requests. So the calls are routed per call site (2026-08-21): a 7b or 14b model for most calls, the 72b model only for diagnosis.

Teacher → executor, two tiers

A cloud teacher prepares, a local LLM fills in

  • Teacher (cloud): prepares, once and beforehand, the case catalogue and schemas, the playbook of known failures and the golden tests of each module, plus a retrieval index for people.
  • Executor (local): fills the bounded answers of each request. The rules of the agent choose the tier, submit the runs and decide the verdict.

Prepared knowledge carries a small model a long way: teaching the 7b model through retrieval raised its score on the driver battery from 5/10 to 9/10, and on the knowledge battery from 1/8 to 8/8 (2026-07-15).

From a request to a result: who does what

Status

  • The duct-system ladder (1D → OpenFOAM → LBM) ran end to end, automated, on 2026-07-16. The record does not say whether a live LLM or the stand-in (mock) drove that run.
  • The station's default model is still qwen3:14b; the measurements above used qwen2.5.
  • The numbers are as of 2026-07-15. Models advance quickly, so they need to be re-measured from time to time.