HilbertRaum
DE EN
GitHub ↗ Join the waitlist
Benchmark

How good are the answers of AI models that run on your own computer?

We ran 29 model variants through the same test, in German and English. All of them run on a computer with 8 to 32 GB of memory or on a single graphics card with up to 24 GB. We measure the quality of the answers, not the speed.

What
Answer quality, 0 to 100 points
Which models
Open models, files from 0.6 to 22 GB
Not included
Speed, models over 24 GB
At a glance

What the results say

Best in the test

Gemma 4 31B: 94.6

The top score in the test. Its file is 17.7 GB, so it needs a machine with 24 GB of memory or more.

Best for 16 GB RAM

Gemma 4 12B: 89.5

A 7 GB file that runs on a 16 GB computer and lands about 5 points behind the best model in the test.

Compression

Small steps cost little

Qwen3.8 27B scores 89.9, 90.0 and 90.1 at Q4, Q5 and Q6. Squeezed down to Q2 it drops to 86.4.

All results

29 model variants, ranked by answer quality

Quality score from 0 to 100. The German and English columns cover the tasks that exist in both languages. Choose your computer to see which models run on it.

Your computer

#ModelVariantFileRAM fromVRAM fromQualityGermanEnglish
1 Gemma 4 31B too large Q4_0 QAT 17.7 GB 24 GB 22 GB 94.6 95.4 95.8
2 Qwen3.6 27B too large Q4_K_M 16.8 GB 20 GB 19 GB 92.8 92.7 94.0
3 Qwen3.5 27B too large UD-Q4_K_XL 17.6 GB 24 GB 19 GB 92.7 93.2 94.5
4 Qwen3.6 27B too large Q5_K_M 19.5 GB 24 GB 21 GB 92.6 92.2 94.0
5 Gemma 4 26B-A4B too large Q4_0 QAT 14.4 GB 20 GB 17 GB 90.5 90.6 91.5
6 Qwen3.8 27B too large UD-Q6_K 22.0 GB 26 GB 23 GB 90.1 88.9 91.7
7 Qwen3.8 27B too large UD-Q5_K_M 19.8 GB 23 GB 22 GB 90.0 88.2 92.3
8 Qwen3.6 35B-A3B too large UD-Q4_K_XL 22.4 GB 24 GB 23 GB 90.0 90.2 90.4
9 Qwen3.8 27B too large UD-Q4_K_M 16.5 GB 21 GB 19 GB 89.9 88.8 91.2
10 Gemma 4 12B 16 GB RAM, tight Q4_0 QAT 7.0 GB 14 GB 10 GB 89.5 87.5 92.0
11 Qwen3.5 35B-A3B too large UD-Q4_K_XL 22.2 GB 24 GB 23 GB 89.2 88.6 90.9
12 Qwen3 14B 16 GB RAM, tight Q4_K_M 9.0 GB 14 GB 12 GB 87.4 86.9 89.5
13 Qwen3.8 27B 16 GB RAM, tight UD-Q2_K_XL 9.8 GB 14 GB 12 GB 86.4 86.6 86.2
14 Qwen3.5 9B 16 GB RAM UD-Q4_K_XL 6.0 GB 12 GB 8 GB 86.1 84.6 89.0
15 Qwen3.5 9B 16 GB RAM, tight Q8_0 9.5 GB 14 GB 11 GB 85.9 85.0 87.8
16 Qwen3.5 9B 16 GB RAM UD-Q3_K_XL 5.1 GB 12 GB 7 GB 83.2 82.0 86.5
17 Gemma 4 E4B 16 GB RAM Q4_0 QAT 5.2 GB 12 GB 6 GB 82.9 80.9 85.1
18 Qwen3 4B 2507 16 GB RAM Q4_K_M 2.5 GB 8 GB 5 GB 82.0 81.6 87.2
19 Qwen3 30B-A3B too large Q4_K_M 18.6 GB 24 GB 20 GB 79.5 78.3 81.3
20 Granite 4.1 8B 16 GB RAM Q4_K_M 5.3 GB 12 GB 9 GB 79.0 79.3 82.8
21 Qwen3 8B 16 GB RAM Q4_K_M 5.0 GB 12 GB 8 GB 77.2 71.6 82.8
22 Qwen3.5 4B 16 GB RAM UD-Q4_K_XL 2.9 GB 8 GB 5 GB 76.9 75.3 77.3
23 Qwen3 4B 16 GB RAM Q4_K_M 2.5 GB 8 GB 5 GB 76.2 74.5 78.7
24 Granite 4.2 8B 16 GB RAM Q4_K_M 5.3 GB 12 GB 9 GB 74.5 71.3 78.0
25 Gemma 4 E2B 16 GB RAM Q4_0 QAT 3.3 GB 8 GB 4 GB 73.7 70.2 75.6
26 Ministral 3 14B 16 GB RAM, tight Q4_K_M 8.2 GB 14 GB 11 GB 72.5 67.4 75.0
27 Ministral 3 8B 16 GB RAM Q4_K_M 5.2 GB 12 GB 8 GB 64.7 60.9 65.0
28 Qwen3.5 2B 16 GB RAM UD-Q4_K_XL 1.3 GB 8 GB 4 GB 56.7 48.2 62.8
29 Qwen3.5 0.8B 16 GB RAM Q6_K 0.6 GB 8 GB 3 GB 39.4 36.2 43.3
Method

How we measure

What is tested

Seven tasks: reading comprehension (Belebele), general knowledge (Global-MMLU-Lite), logical inference (XNLI), recognising paraphrases (PAWS-X), answering questions about German texts (GermanQuAD), maths word problems (MGSM) and following precise instructions (BenchMAX). Six of the seven run in German and English.

How much

3,500 tasks per model. The 2,200 multiple-choice questions among them are scored a second time from the probability the model gives to each answer letter.

How it is scored

Every task gives a score from 0 to 100, with German and English averaged first. The overall score is the plain average of the seven tasks.

Which computer runs what

The RAM figures are the requirements the HilbertRaum app uses: the minimum to run a model and the amount recommended for comfortable use. Tight means the computer has the minimum but not the recommended amount. The VRAM figure is measured: for every model we read what the runtime itself plans to use on the graphics card under the start settings of the HilbertRaum app, and added 1.75 GB: 1 GB that the runtime keeps free on the card and about 0.75 GB that the system itself held on the three computers we checked. A model close to the limit may still not fit completely on your card. A model that does not fit the card still runs from RAM, more slowly. For five variants that are not in the app, the RAM figures come from the app model of the nearest file size.

Setup

One machine with an RTX 3090 (24 GB), llama.cpp build b12110, a context of 32,768 tokens, GGUF model files, one run per model. Study finished on 15 November 2026.

What the numbers do not say

Want local AI without the setup?

The HilbertRaum Kit is a portable USB drive with the app and models preloaded. Join the waitlist to hear when it ships.