local desktop · 2026-10-13
Two R9700s, one 35B
Unsloth UD-Q5_K_XL on llama.cpp Vulkan, DDR5-5200. two AMD Radeon AI PRO R9700 32 GB (gfx1201). Every tok/s, VRAM, and soak figure below is first-party unless a citation says otherwise.
Daily driver · 35B llama.cpp · 262K
Minimum system RAM needed
42.9 GB
Measured · 42.9 GiB raw · 256K context
local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16
RAM needed at full context
42.9 GB
Measured · 42.9 GiB raw · 256K full context
local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16
Measured peak system RAM on the local desktop: 42.9 GB (Linux GiB labeled GB). Measured, pending re-verify after RAM fix.
Breakdown
- Offloaded weights ·
- 0 GiB
- Separable embed in RAM ·
- 0 GiB
- KV in RAM at selected ·
- 0 GiB
- KV in RAM at full context ·
- 0 GiB
- Runtime / backend ·
- 0.85 GiB
- OS / desktop baseline ·
- 8.00 GiB
- Headroom ·
- 16.0 GiB
RAM is Linux GiB labeled GB. The kit is 256 GB (about 249 usable). Linux GiB labeled GB on a 256 GB kit.
Measured on our rig, 2026-10
Two copies (one per card) behind a least-connections balancer
About 100 tok/s per request. Peak VRAM is Linux GiB labeled GB. DDR5 about 13 GB.
Per request
100 tok/s
About 100 tok/s · one copy per card
Total · 2 concurrent
172 tok/s
Total · 4–8 concurrent
~177 tok/s
Soak
0 errors
423 requests · 20.6 minutes
- Peak VRAM
- 27.7 / 29.7 GB
- Linux GiB labeled GB · card A / card B
- DDR5
- about 13 GB
- Against the 256 GB kit · 249 usable
- Balancer
- Least connections
- One llama-server per card
Measured on our rig, 2026-10
One server split across both cards
Official llama-bench 2026-10-12 (c060ca974 Vulkan). 42.03 tok/s at 256K; 82.78 at 4K. The older ~57 / 79 pair is retired.
Single request · 256K
42.03 tok/s
4K decode · split
82.78 tok/s
| Context | Decode | Prefill | VRAM / card | VRAM total | System RAM | Card |
|---|---|---|---|---|---|---|
| short | 82.78 tok/s | 2637.3 tok/s | 13.28 / 13.11 GB | 26.39 GB | 37.6 GB | local-desktop-20261012-2319-35b-udq5kxl-2gpu-4k-llamacpp-f16 |
| 32K | 74.8 tok/s | 1916 tok/s | 13.77 / 13.64 GB | 27.41 GB | 38.16 GB | local-desktop-20261012-2319-35b-udq5kxl-2gpu-32k-llamacpp-f16 |
| 128K | 57.94 tok/s | 1071.1 tok/s | 15.47 / 15.48 GB | 30.94 GB | 41.02 GB | local-desktop-20261012-2320-35b-udq5kxl-2gpu-128k-llamacpp-f16 |
| 256K | 42.03 tok/s | 671.7 tok/s | 17.8 / 17.99 GB | 35.74 GB | 44.71 GB | local-desktop-20261012-2321-35b-udq5kxl-2gpu-256k-llamacpp-f16 |
Official llama-bench split (c060ca974 Vulkan, UD-Q5_K_XL, two R9700s). A split 35B is slower than one card and much slower than two copies. House rule still stands: one LLM per GPU unless you deliberately combine.
Reference · one card
Single-card 35B
Same rig, one R9700, llama.cpp Vulkan, f16 KV. Measured on our rig, 2026-10.
| Quant | Context | Decode | Note |
|---|---|---|---|
| UD-Q5_K_XL | 4K | 102.39 tok/s | Official llama.cpp c060ca974 |
| UD-Q5_K_XL | 32K | 90.52 tok/s | Official 2026-10-12 |
| UD-Q5_K_XL | 128K | 66.43 tok/s | Official 2026-10-12 |
| UD-Q5_K_XL | 256K | 44.4 tok/s | Official 2026-10-12 |
| Q5_K_M | 4K | 109.73 tok/s | Official 2026-10-12 |
| Q4_K_M | 4K | 117.69 tok/s | Official 2026-10-12 |
| Q6_K | 4K | 103.23 tok/s | Official 2026-10-12 Q6_K single-GPU 4K cell |
Community image · first-party band
vLLM on gfx1201
Image stilldeadcode/vllm-radiance:0.5.8, INT4-W4A16, 1 card.
About 50–61 tok/s on one card.
Better for concurrency and prefill, worse for single-user decode than llama.cpp.
vLLM stays unused on the engine catalog — this is a cited try, not the daily driver.
Overnight results.json
Space for overnight results.json rows. Empty until that file lands.
Empty
No overnight results.json rows yet. This slot stays reserved so a drop-in file can fill the table without a layout rewrite.
Sources
- stilldeadcode/vllm-radiance on Docker Hub · retrieved 2026-10-13
- StillDeadcode/vllm-radiance on Codeberg · retrieved 2026-10-13
- Qwen3.6 27B + 35B on vLLM, single R9700 (gfx1201) · retrieved 2026-10-13
- Qwen3.6-35B-A3B GGUF · retrieved 2026-10-13