Measured on our rig, 2026-10
How to read a local-AI speed number on Radeon, then the handful of figures we actually ran on wtpai1. Nothing here is a scraped leaderboard. If a cell is empty we write not measured.
pp vs tg
llama.cpp’s llama-bench prints pp(prompt processing / prefill) and tg(token generation / decode). Prefill is how fast the prompt is ingested. Decode is how fast new tokens appear — the number people mean by “tok/s” in chat. Every first-party figure on this page is tgunless a column says otherwise.
Short vs long
Short context is a small prompt — the happy path. At 32K and 256K the KV cache is larger, the card works harder, and decode drops. Comparing a 4K blog number to a 262K daily driver is a category error.
Where the weights live
Tokens stay fast when the whole file and its KV sit in one GPU. Spill into system RAM and decode falls off a cliff. Our 35B daily driver is 0% RAM offload on one 32 GB card. Flash Q6 is a much larger file — expect RAM to hold most of it. Fit check:calculator.
The runtime
The same GGUF on llama.cpp and Strata is two different programs. Consumer default is llama.cpp(Vulkan on this 35B). Strata is the Flash try we measured. A score without engine, quant, context, KV type, and gfx is not a score.
Measured on our rig, 2026-10
Green numbers on this site come from wtpai1. Source note:wtpai1 strata-test report / Local AI measurements. If we did not run it, we do not print a tok/s.
Cited from elsewhere
We would rather omit a third-party table than paste an unverified leaderboard. Official docs below are for words (pp/tg, model cards, the SKU) — not for someone else’s tok/s.
wtpai1
Two AMD Radeon AI PRO R9700 (32 GB GDDR6, gfx1201). 256 GB DDR5. Daily-driver and 35B rows use ONE card. The second card is there; do not assume a split.
Unsloth GGUF, f16 KV, one R9700, 0% offload. Full 262144 context is available for the daily driver. Long-context decode is only published for UD-Q5_K_XL.
| Quant | Context | tg (decode) | Fit |
|---|---|---|---|
| UD-Q5_K_XL | short | ~100.8 tok/s | Calculator |
| UD-Q5_K_XL | 32K | ~88 tok/s | Calculator |
| UD-Q5_K_XL | 256K | ~50 tok/s | Calculator |
| Q5_K_M | short | ~108.3 tok/s | Calculator |
| Q4_K_M | short | ~116.5 tok/s | Calculator |
Same file, two engines, 262K. Strata v0.1.42 via strata-flash-q6versus llama.cpp Q6 with 8 threads.
| Engine | Context | tg (decode) | Fit |
|---|---|---|---|
| Strata · strata-flash-q6 | 262K | ~32.6 tok/s | Calculator |
| llama.cpp · 8 threads | 262K | ~16 tok/s | Calculator |
Built only from the runs above. Share links open the matchingcalculatorpreset — VRAM totals there stay estimates.
| Engine | Quant | Context | Metric | tok/s | Calculator |
|---|---|---|---|---|---|
| llama.cpp | UD-Q5_K_XL | short | tg (decode) | ~100.8 tok/s | Open |
| llama.cpp | UD-Q5_K_XL | 32K | tg (decode) | ~88 tok/s | Open |
| llama.cpp | UD-Q5_K_XL | 256K | tg (decode) | ~50 tok/s | Open |
| llama.cpp | Q5_K_M | short | tg (decode) | ~108.3 tok/s | Open |
| llama.cpp | Q4_K_M | short | tg (decode) | ~116.5 tok/s | Open |
| Strata | UD-Q6_K_XL | 262K | tg (decode) | ~32.6 tok/s | Open |
| llama.cpp (8 threads) | UD-Q6_K_XL | 262K | tg (decode) | ~16 tok/s | Open |
Measured on our rig, 2026-10. Source: wtpai1 strata-test report / Local AI measurements.
strata-flash-q6, no MTP, 262K.Strata. Do not treat this as a HIP-versus-Vulkan verdict.Official pages for terminology and SKUs. They are not tok/s claims. Third-party speed tables are omitted.