Device 01
Xiaomi TV Box S (3rd Gen)
4× Cortex-A55 · 1.9 GB RAM, about 1 GB usable · Android 14 · no root
- Everyday modelQwen3.5-2B
- 7.7 tok/s
- Biggest modelLing-mini-2.0, 16B
- 5.1 tok/s
The biggest model. The smallest hardware.
TV boxes, old tablets, single-board computers and routers sit idle most of the day. smolmaxxing finds the biggest language model each of them can really run, then pushes it further with custom kernels, careful memory use and weights streamed from flash.
On the front page of Hacker News. Read the discussion
260K to 35B parameters on hardware you already own.
The TV box and the iPad run 8B and 16B models on about 1 GB of RAM, and the phone runs a 35B one with about 3 GB free. They all keep only part of each model in memory and read the rest from flash as it’s needed.
Device 02
iPad Air (2013)
Apple A7, 2 cores · 1 GB RAM, about 700 MB usable · iOS 12.5.7 · jailbroken
- Everyday modelBonsai-1.7B, 1-bit
- 24–26 tok/s
- Biggest modelLing-mini-2.0, 16B
- 2.6–4.5 tok/s
Device 03
OnePlus Nord CE 2 (2022)
Dimensity 900, 8 cores · 7.4 GB RAM, about 3 GB free · Android 13 · no root
- Everyday modelLing-mini-2.0, 16B
- 11–15 tok/s
- Biggest modelQwen3.6-35B-A3B
- 4.9–5.7 tok/s
TV box: the best model of each size, first run to now
| Size class | Best model | Tok/s, first run → now | Gain | Bits/byte |
|---|---|---|---|---|
| 0.8B | Qwen3.5-0.8B | 7.5 → 16.8 bench | ×2.2 | ≈1.00* |
| 2B | Qwen3.5-2BThe default: best overall | 0.1 → 7.7 chat | ×77 | 0.909 |
| 4B | Ternary-Bonsai-4B | 2.2 → 4.3 chat | ×2.0 | ≈0.965 |
| 8B dense | Bonsai-8B, 1-bit | 1.21 → 5.2 chat | ×4.3 | 0.974 |
| 8B mixture of experts | Moon-Beam-8B-A1BThe fastest 8B | 1.87 → 7.2 chat | ×3.9 | 0.996 |
| 9B dense | Qwen3.5-9BBest quality; limited by flash | 0.16 → 0.27 chat | ×1.7 | 0.885 |
| 16B mixture of experts | Ling-mini-2.0The biggest at a usable speed | 4.1 → 5.1 chat | ×1.2 | 1.054 |
iPad Air: what a tablet from 2013 runs
| Model | File | Tok/s | Notes |
|---|---|---|---|
| Bonsai-1.7B, 1-bit | 237 MiB | 23.7–25.8 | The one to use: answers well and writes correct code |
| Bonsai-4B, 1-bit | 546 MiB | 9.4–11.4 | With a 512-token context |
| Moon-Beam-8B-A1B | 2.6 GiB, 474 MB mapped | 3.5–3.6 | 8B mixture of experts, paged from flash |
| Ling-mini-2.0 | 5.2 GiB, 466 MB mapped | 2.6–4.5 | 16B mixture of experts; faster settings cost quality |
| SmolLM2-360M | 258 MiB | 10.9–12.5 | The lightest good option |
Phone: from 16B to a 35B model
| Model | Settings | Tok/s | Notes |
|---|---|---|---|
| Ling-mini-2.0, 16B | Exact routing | 11.1 | Three times the TV box |
| Ling-mini-2.0, 16B | The TV box’s speed settings | 15.3 | Faster, at a cost in quality |
| Qwen3.6-35B-A3B | Exact routing | 4.9 | 35B mixture of experts, 12.3 GB file |
| Qwen3.6-35B-A3B | Slightly biased to cached experts | 5.7 | 0.7% worse perplexity |
“Chat” is real chat requests through llama-server; “bench” is llama-bench. Bits per byte measures quality, and lower is better. * Measured on the pruned head’s positions, so not exactly comparable. All 163 experiments, failures included, are in the experiment log.
Past RAM, flash sets the speed.
A dense model that streams its weights can never beat the flash. A mixture-of-experts model touches only a few experts per token, so most reads are skipped. That is how an 8B model runs about 27 times faster than a dense 9B on the same box.
On the TV box
flash reads 278 MB/s dense 9B, streamed ~1,600 MB per token ceiling 278 / 1,600 = 0.17 tok/s with MTP 0.27 tok/s 8B mixture of experts 7.2 tok/s 16B mixture of experts 5.1 tok/s
How it gets there.
All of it is a patch on llama.cpp. Every technique, with its measured gain.
Kernels built for the chip.
A 1-bit lookup-table kernel and repacked 2-bit and ternary formats for the Cortex-A55, with versions for the A7, which has no dot-product instructions.
Less RAM per model.
Most of a small model’s vocabulary is rarely used, so those output rows are dropped. On the TV, a headless mode also stops the launcher while a model runs.
Experts paged from flash.
A mixture-of-experts model uses a few experts per token. The popular ones stay in RAM and the rest are read on demand. This is how 8B and 16B models run on 1 GB, and a 35B model on a phone.
Streaming for dense models.
A dense model bigger than RAM keeps part of itself in memory and streams the rest from flash at the flash’s full speed.
Nine model files, ready to download.
The best file of each size class, exactly as each device runs it, is on Hugging Face. They need the patched llama.cpp.
| Model | Device | Tok/s | Hugging Face |
|---|---|---|---|
| Qwen3.5-0.8B | TV box | 16.8 | Qwen3.5-0.8B-tvbox-GGUF |
| Qwen3.5-2BThe default | TV box | 7.7 | Qwen3.5-2B-tvbox-GGUF |
| Ternary-Bonsai-4B | TV box | 4.3 | Ternary-Bonsai-4B-tvbox-GGUF |
| Bonsai-8B, 1-bit | TV box | 5.2 | Bonsai-8B-tvbox-GGUF |
| Moon-Beam-8B-A1B | TV box | 7.2 | Moon-Beam-8B-A1B-tvbox-GGUF |
| Ling-mini-2.0, 16B | TV box | 5.1 | Ling-mini-2.0-tvbox-GGUF |
| Moon-Beam-8B-A1B | iPad Air | 3.5–3.6 | Moon-Beam-8B-A1B-ipad-GGUF |
| Moon-Beam-8B-A1B, XLWithout the lookup-table repack | iPad Air | about 4% slower | Moon-Beam-8B-A1B-ipad-xl-GGUF |
| Ling-mini-2.0, 16B | iPad Air | 2.6–4.5 | Ling-mini-2.0-ipad-GGUF |
All models on Hugging Face. Model weights keep their own licences. The phone’s 35B file isn’t published; a script in the repo builds it on the phone. On the iPad, the smaller Bonsai models are used exactly as their authors publish them.
What it can’t do yet.
Three devices are done.
The TV box, the iPad Air and one phone are the only devices with results so far. Single-board computers, routers and more phones are coming, and you can submit numbers from your own device.
Bigger isn’t automatically better.
Ling-mini-2.0 is twice Moon-Beam’s size, but it runs slower and scores worse per byte.
Dense models past RAM are slow.
The 9B’s 0.27 tok/s on the TV, and the dense 8B’s 0.08 on the iPad, show what can run, not what you’d want to chat with.
It’s an engineering project, not an app.
The released model files need the patched llama.cpp; the stock build can’t load them. The TV needs network ADB and the iPad needs a jailbreak.
Try it.
The quickest start is a ready-made model file from Hugging Face, on the TV box, another ARMv8.2 Android device, or a jailbroken iPad on iOS 12.
TV box setupBuild the patched llama.cpp and run your first model.iPad Air setupThe iOS 12 build, the chat app and the scripts.Phone resultsThe OnePlus phone, from 16B to a 35B model.ModelsDownloads for the TV box and the iPad, rebuilds and checksums.Submit your deviceRun a model on your own small device and send the numbers.
The default model on the TV box
hf download hyperionaidev/Qwen3.5-2B-tvbox-GGUF --local-dir models/ adb push models/w-q4k-h65p8.gguf /data/local/tmp/llm/ tv-llm-ui/start.sh # then open http://<tv-ip>:8090 on your network
On the TV, everything lives in /data/local/tmp/llm and changes no system settings, so one rm -rf removes it all.