UD-Q5_K_XL · short
UD-Q5_K_XL
- 1× decode
- 102.39 tok/s
- Split decode
- 82.78 tok/s
- 1× prefill
- 2761.6 tok/s
- Split prefill
- 2637.3 tok/s
- 1× VRAM total
- 25.97 GB
- Split VRAM / card
- 13.28 / 13.11 GB
- 1× system RAM
- 37.32 GB
- Split system RAM
- 37.6 GB
GPU directory · Radeon AI PRO · RDNA 4 · gfx1201
Radeon AI PRO R9700
The AMD Radeon AI PRO R9700 is a 32 GB workstation / local-AI card on RDNA 4 (gfx1201). AMDai measures local Qwen on this SKU. There is no consumer RX 9700 — the gaming sibling is the 16 GB RX 9070 series.
Published VRAM is 32 GB. First-class on ROCm 10.0 / 10.1. Our test-rig daily driver. Official on the current ROCm matrix.
First-run setup
Measured recipe
Stack Lab
Compare GPUs
Radeon vs NVIDIA
Open in calculator
Out of VRAM
MoE on Radeon
35B model page
Flash model page
gpt-oss 20B
Gemma 3 27B
Llama 3.3 70B
GLM 5.3 Flash
Qwen3.6 27B
Qwen3.8 27B
Fine-tuning
Open in Stack Lab
If the app is not installed, packages are not published yet. There is no public download URL. Read the Stack Lab install notes . If Stack Lab is already running on this machine, open the local app .
What fits on this card (256K context)
Shopper verdict at 4K, then follow the pair page for 32K and full context. Q4 and up. No printed prices.
| Model | Verdict |
|---|---|
| Qwen3.6-35B on R9700 | Fits fully on GPU |
| Qwen 3.8 Flash on R9700 | Fits with RAM offload |
| gpt-oss 20B on R9700 | Fits fully on GPU |
| Gemma 3 27B on R9700 | Needs 2 GPUs |
| Llama 3.3 70B on R9700 | Fits with RAM offload |
| GLM 5.3 Flash on R9700 | Fits with RAM offload |
| Qwen3.6 27B on R9700 | Needs 2 GPUs |
| Qwen3.8 27B on R9700 | Needs 2 GPUs |
What this card is for
Buy this card when you want a 35B daily driver that stays in VRAM at full 262K on llama.cpp, or a second card for two copies / Flash offload. It is not Instinct, and it is not an automatic 2× decode when you seat two.
Published VRAM is 32 GB. That is the hardware label, the same units AMD prints. Measured peak VRAM on the overnight cards is lower than 32 GB for the daily-driver Q5 — read the cells, do not invent headroom.
AMD product page: Radeon AI PRO R9700. Directory of other Radeon cards: /gpus.
Start → setup → recipe → Stack Lab
One path. Do not skip to a harness. llama.cpp first.
-
Step 1
Start
Pick the card, then ROCm, then llama.cpp. Getting started is the map; this page is the measured R9700 sheet.
-
Step 2
First-run setup
Ubuntu 24.04+ is the measured path to a first token. Windows first-run lives on /setup/windows — cited, not measured on the local desktop.
-
Step 3
Measured recipe
Daily driver: Qwen3.6-35B-A3B Unsloth UD-Q5_K_XL on one R9700, llama.cpp Vulkan, 262144 context, f16 KV. Open the recipe, then copy the launch.
-
Step 4
Stack Lab
Ubuntu companion that applies that recipe and records an amdai-bench/run@1 card. Packages are not published yet.
Measured 35B on this card
Qwen3.6-35B-A3B on llama.cpp Vulkan, f16 KV. UD-Q5_K_XL, Q5_K_M, Q4_K_M, and Q6_K
at 4K / 32K / 128K / 256K. One Radeon AI PRO R9700 versus a split across two.
Every tok/s and measured GB cites an official amdai-bench/run@1 card.
Empty stays not measured.
How we test.
Daily driver — UD-Q5_K_XL
One card is the house daily driver. A split of the same file is slower at short context. Two full copies are a different layout — 35B on two cards. When the second card helps or hurts: Dual / multi-GPU. Hands-on llama.cpp how-to: /setup/multi-gpu (the old /two-r9700 path still redirects).
| Context | 1× decode | 1× prefill | 1× VRAM total | 1× system RAM | Split decode | Split VRAM / card |
|---|---|---|---|---|---|---|
| short | 102.39 tok/s | 2761.6 tok/s | 25.97 GB | 37.32 GB | 82.78 tok/s | 13.28 / 13.11 GB |
| 32K | 90.52 tok/s | 1937.9 tok/s | 26.7 GB | 37.83 GB | 74.8 tok/s | 13.77 / 13.64 GB |
| 128K | 66.43 tok/s | 942.8 tok/s | 28.82 GB | 40.18 GB | 57.94 tok/s | 15.47 / 15.48 GB |
| 256K | 44.4 tok/s | 519.2 tok/s | 29.34 GB | 42.9 GB | 42.03 tok/s | 17.8 / 17.99 GB |
One GPU vs two — all measured quants
Full filterable table lives on one vs two GPU. The 33-card matrix is /benchmarks.
UD-Q5_K_XL · 32K
UD-Q5_K_XL
- 1× decode
- 90.52 tok/s
- Split decode
- 74.8 tok/s
- 1× prefill
- 1937.9 tok/s
- Split prefill
- 1916 tok/s
- 1× VRAM total
- 26.7 GB
- Split VRAM / card
- 13.77 / 13.64 GB
- 1× system RAM
- 37.83 GB
- Split system RAM
- 38.16 GB
UD-Q5_K_XL · 128K
UD-Q5_K_XL
- 1× decode
- 66.43 tok/s
- Split decode
- 57.94 tok/s
- 1× prefill
- 942.8 tok/s
- Split prefill
- 1071.1 tok/s
- 1× VRAM total
- 28.82 GB
- Split VRAM / card
- 15.47 / 15.48 GB
- 1× system RAM
- 40.18 GB
- Split system RAM
- 41.02 GB
UD-Q5_K_XL · 256K
UD-Q5_K_XL
- 1× decode
- 44.4 tok/s
- Split decode
- 42.03 tok/s
- 1× prefill
- 519.2 tok/s
- Split prefill
- 671.7 tok/s
- 1× VRAM total
- 29.34 GB
- Split VRAM / card
- 17.8 / 17.99 GB
- 1× system RAM
- 42.9 GB
- Split system RAM
- 44.71 GB
Q5_K_M · short
Q5_K_M
- 1× decode
- 109.73 tok/s
- Split decode
- 88.19 tok/s
- 1× prefill
- 2710.8 tok/s
- Split prefill
- 2647.2 tok/s
- 1× VRAM total
- 24.68 GB
- Split VRAM / card
- 12.71 / 12.41 GB
- 1× system RAM
- 35.83 GB
- Split system RAM
- 37.17 GB
Q5_K_M · 32K
Q5_K_M
- 1× decode
- 96.21 tok/s
- Split decode
- 78.91 tok/s
- 1× prefill
- 1902.6 tok/s
- Split prefill
- 1920.9 tok/s
- 1× VRAM total
- 25.39 GB
- Split VRAM / card
- 13.2 / 12.95 GB
- 1× system RAM
- 36.46 GB
- Split system RAM
- 37.8 GB
Q5_K_M · 128K
Q5_K_M
- 1× decode
- 69.64 tok/s
- Split decode
- 59.86 tok/s
- 1× prefill
- 940.3 tok/s
- Split prefill
- 1067.3 tok/s
- 1× VRAM total
- 27.43 GB
- Split VRAM / card
- 14.98 / 14.79 GB
- 1× system RAM
- 38.65 GB
- Split system RAM
- 40.73 GB
Q5_K_M · 256K
Q5_K_M
- 1× decode
- 40.54 tok/s
- Split decode
- 42.33 tok/s
- 1× prefill
- 541.7 tok/s
- Split prefill
- 664.6 tok/s
- 1× VRAM total
- 29.15 GB
- Split VRAM / card
- 17.45 / 17.3 GB
- 1× system RAM
- 42.79 GB
- Split system RAM
- 46.21 GB
Q4_K_M · short
Q4_K_M
- 1× decode
- 117.69 tok/s
- Split decode
- 93.28 tok/s
- 1× prefill
- 2733.9 tok/s
- Split prefill
- 2660.4 tok/s
- 1× VRAM total
- 21.43 GB
- Split VRAM / card
- 10.83 / 10.85 GB
- 1× system RAM
- 33.74 GB
- Split system RAM
- 33.74 GB
Q4_K_M · 32K
Q4_K_M
- 1× decode
- 101.88 tok/s
- Split decode
- 82.57 tok/s
- 1× prefill
- 1902.1 tok/s
- Split prefill
- 1933.1 tok/s
- 1× VRAM total
- 22.17 GB
- Split VRAM / card
- 11.34 / 11.39 GB
- 1× system RAM
- 34.58 GB
- Split system RAM
- 34.39 GB
Q4_K_M · 128K
Q4_K_M
- 1× decode
- 72.48 tok/s
- Split decode
- 62.12 tok/s
- 1× prefill
- 942.1 tok/s
- Split prefill
- 1078.4 tok/s
- 1× VRAM total
- 24.27 GB
- Split VRAM / card
- 13.03 / 13.23 GB
- 1× system RAM
- 36.5 GB
- Split system RAM
- 38.04 GB
Q4_K_M · 256K
Q4_K_M
- 1× decode
- 47.77 tok/s
- Split decode
- 42.88 tok/s
- 1× prefill
- 558.5 tok/s
- Split prefill
- 673 tok/s
- 1× VRAM total
- 27.03 GB
- Split VRAM / card
- 15.28 / 15.74 GB
- 1× system RAM
- 39.05 GB
- Split system RAM
- 40.84 GB
Q6_K · short
Q6_K
- 1× decode
- 103.23 tok/s
- Split decode
- 83.5 tok/s
- 1× prefill
- 2623.6 tok/s
- Split prefill
- 2572.1 tok/s
- 1× VRAM total
- 29.26 GB
- Split VRAM / card
- 15.01 / 14.63 GB
- 1× system RAM
- 40.15 GB
- Split system RAM
- 38.22 GB
Q6_K · 32K
Q6_K
Partly offloaded to DDR5 on 1 GPU
- 1× decode
- 71.19 tok/s
- Split decode
- 75.43 tok/s
- 1× prefill
- 1683 tok/s
- Split prefill
- 1890.7 tok/s
- 1× VRAM total
- 29.17 GB
- Split VRAM / card
- 15.5 / 15.16 GB
- 1× system RAM
- 44.01 GB
- Split system RAM
- 38.98 GB
Q6_K · 128K
Q6_K
Partly offloaded to DDR5 on 1 GPU
- 1× decode
- 54.51 tok/s
- Split decode
- 58.04 tok/s
- 1× prefill
- 800.3 tok/s
- Split prefill
- 1059.7 tok/s
- 1× VRAM total
- 29.25 GB
- Split VRAM / card
- 17.21 / 17 GB
- 1× system RAM
- 45.43 GB
- Split system RAM
- 41.41 GB
Q6_K · 256K
Q6_K
Partly offloaded to DDR5 on 1 GPU
- 1× decode
- 37.78 tok/s
- Split decode
- 42.2 tok/s
- 1× prefill
- 471.6 tok/s
- Split prefill
- 666.2 tok/s
- 1× VRAM total
- 29.3 GB
- Split VRAM / card
- 19.46 / 19.51 GB
- 1× system RAM
- 43.69 GB
- Split system RAM
- 45.15 GB
System RAM for this card
Two lines everywhere RAM offload or full context shows up: Minimum system RAM needed at the context you picked, and RAM needed at full context (262144 for the house 35B). We round up to a real DIMM kit (48 / 64 / 96 / 128 / 192 / 256 GB). The house box ships 256 GB DDR5. A measured peak is labeled measured; a calculator line is labeled estimate until a card records RSS.
Daily driver · UD-Q5_K_XL · 262K · one card
Minimum system RAM needed
42.9 GB
Measured · 42.9 GiB raw · 256K context
local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16
RAM needed at full context
42.9 GB
Measured · 42.9 GiB raw · 256K full context
local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16
Measured peak system RAM on the local desktop: 42.9 GB (Linux GiB labeled GB). Measured, pending re-verify after RAM fix.
Breakdown
- Offloaded weights ·
- 0 GiB
- Separable embed in RAM ·
- 0 GiB
- KV in RAM at selected ·
- 0 GiB
- KV in RAM at full context ·
- 0 GiB
- Runtime / backend ·
- 0.85 GiB
- OS / desktop baseline ·
- 8.00 GiB
- Headroom ·
- 16.0 GiB
One R9700 · short
| Quant | Peak system RAM | Minimum system RAM needed | RAM needed at full context |
|---|---|---|---|
| UD-Q5_K_XL measured | 37.32 GB | 48 GB kit (measured · 37.32 GB peak) | 48 GB kit (measured · 42.9 GB peak) |
| Q5_K_M measured | 35.83 GB | 48 GB kit (measured · 37.32 GB peak) | 48 GB kit (measured · 42.9 GB peak) |
| Q4_K_M measured | 33.74 GB | 48 GB kit (measured · 33.74 GB peak) | 48 GB kit (measured · 39.05 GB peak) |
| Q6_K measured | 40.15 GB | 48 GB kit (measured · 40.91 / 40.15 GB peak) | 48 GB kit (measured · 43.69 GB peak) |
| UD-Q3_K_XL estimate | 32 GB kit (estimate) | 32 GB kit (estimate) | 32 GB kit (estimate) |
One R9700 · 256K
| Quant | Peak system RAM | Minimum system RAM needed | RAM needed at full context |
|---|---|---|---|
| UD-Q5_K_XL measured | 42.9 GB | 48 GB kit (measured · 42.9 GB peak) | 48 GB kit (measured · 42.9 GB peak) |
| Q5_K_M measured | 42.79 GB | 48 GB kit (measured · 42.9 GB peak) | 48 GB kit (measured · 42.9 GB peak) |
| Q4_K_M measured | 39.05 GB | 48 GB kit (measured · 39.05 GB peak) | 48 GB kit (measured · 39.05 GB peak) |
| Q6_K measured | 43.69 GB | 48 GB kit (measured · 43.69 GB peak) | 48 GB kit (measured · 43.69 GB peak) |
| UD-Q3_K_XL estimate | 32 GB kit (estimate) | 32 GB kit (estimate) | 32 GB kit (estimate) |
Split · short
| Quant | Peak system RAM | Minimum system RAM needed | RAM needed at full context |
|---|---|---|---|
| UD-Q5_K_XL measured | 37.6 GB | 48 GB kit (measured · 37.6 GB peak) | 48 GB kit (measured · 44.71 GB peak) |
| Q5_K_M measured | 37.17 GB | 48 GB kit (measured · 37.6 GB peak) | 48 GB kit (measured · 44.71 GB peak) |
| Q4_K_M measured | 33.74 GB | 48 GB kit (measured · 33.74 GB peak) | 48 GB kit (measured · 40.84 GB peak) |
| Q6_K measured | 38.22 GB | 48 GB kit (measured · 38.22 GB peak) | 48 GB kit (measured · 45.15 GB peak) |
| UD-Q3_K_XL estimate | 32 GB kit (estimate) | 32 GB kit (estimate) | 32 GB kit (estimate) |
Split · 256K
| Quant | Peak system RAM | Minimum system RAM needed | RAM needed at full context |
|---|---|---|---|
| UD-Q5_K_XL measured | 44.71 GB | 48 GB kit (measured · 44.71 GB peak) | 48 GB kit (measured · 44.71 GB peak) |
| Q5_K_M measured | 46.21 GB | 48 GB kit (measured · 44.71 GB peak) | 48 GB kit (measured · 44.71 GB peak) |
| Q4_K_M measured | 40.84 GB | 48 GB kit (measured · 40.84 GB peak) | 48 GB kit (measured · 40.84 GB peak) |
| Q6_K measured | 45.15 GB | 48 GB kit (measured · 45.15 GB peak) | 48 GB kit (measured · 45.15 GB peak) |
| UD-Q3_K_XL estimate | 32 GB kit (estimate) | 32 GB kit (estimate) | 32 GB kit (estimate) |
Q3 is a labeled estimate
Quant picker: Q6 / Q5 / Q4 + Q3 estimate · Quants · Calculator.
Recipes we have
A row is one launch that passed enough gates to publish. llama.cpp first. Spec-only cards stay listed. Empty cells stay empty.
| Model | Setup | Backend | Quant | Ctx | Decode | Evidence |
|---|---|---|---|---|---|---|
| Qwen3.6-35B-A3B Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf | 1× r9700 | llama.cpp vulkan | UD-Q5_K_XL | 256K | 44.4 | Tested on our rig |
| Qwen 3.8 Flash UD-Q6_K_XL (folder) | 2× r9700 | llama.cpp vulkan | UD-Q6_K_XL | 262K | 16 | Tested on our rig |
| Qwen3.6-35B-A3B Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf | 1× r9700 | llama.cpp vulkan | UD-Q5_K_XL | 32K | 90.52 | Tested on our rig |
| Qwen3.6-35B-A3B Qwen3.6-35B-A3B-Q4_K_M.gguf | 1× r9700 | llama.cpp vulkan | Q4_K_M | short | 117.69 | Tested on our rig |
| Qwen3.6-35B-A3B Qwen3.6-35B-A3B-Q5_K_M.gguf | 1× r9700 | llama.cpp vulkan | Q5_K_M | short | 109.73 | Tested on our rig |
| Qwen3.6-35B-A3B Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf | 1× r9700 | llama.cpp vulkan | UD-Q5_K_XL | short | 102.39 | Tested on our rig |
| Qwen 3.8 Flash UD-Q6_K_XL (folder) | 2× r9700 | Strata hip | UD-Q6_K_XL | 262K | 32.6 | Tested on our rig |
Official run cards
Measured cells on this page come from verified amdai-bench/run@1 cards credited to the submitter.
The catalog is
/results.
amdai-bench/run@1 · single · short
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 4,096 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:11:50-05:00
- Prefill
- 2761.6 tok/s
- Decode
- 102.39 tok/s
- Aggregate
- not measured
- Peak VRAM
- 25.45 GB/GPU · 25.97 GB total
- Peak system RAM
- 37.32 GB
- Errors · duration
- 0 · 9 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · single · 32K
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 32,768 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:12:00-05:00
- Prefill
- 1937.9 tok/s
- Decode
- 90.52 tok/s
- Aggregate
- not measured
- Peak VRAM
- 26.16 GB/GPU · 26.7 GB total
- Peak system RAM
- 37.83 GB
- Errors · duration
- 0 · 23 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · single · 128K
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 131,072 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:12:23-05:00
- Prefill
- 942.8 tok/s
- Decode
- 66.43 tok/s
- Aggregate
- not measured
- Peak VRAM
- 28.22 GB/GPU · 28.82 GB total
- Peak system RAM
- 40.18 GB
- Errors · duration
- 0 · 103 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · single · 256K
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 262,144 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:14:07-05:00
- Prefill
- 519.2 tok/s
- Decode
- 44.4 tok/s
- Aggregate
- not measured
- Peak VRAM
- 28.73 GB/GPU · 29.34 GB total
- Peak system RAM
- 42.9 GB
- Errors · duration
- 0 · 330 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · short
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 4,096 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:19:38-05:00
- Prefill
- 2637.3 tok/s
- Decode
- 82.78 tok/s
- Aggregate
- not measured
- Peak VRAM
- 13.28 / 13.11 GB/GPU · 26.39 GB total
- Peak system RAM
- 37.6 GB
- Errors · duration
- 0 · 9 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · 32K
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 32,768 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:19:48-05:00
- Prefill
- 1916 tok/s
- Decode
- 74.8 tok/s
- Aggregate
- not measured
- Peak VRAM
- 13.77 / 13.64 GB/GPU · 27.41 GB total
- Peak system RAM
- 38.16 GB
- Errors · duration
- 0 · 24 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · 128K
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 131,072 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:20:12-05:00
- Prefill
- 1071.1 tok/s
- Decode
- 57.94 tok/s
- Aggregate
- not measured
- Peak VRAM
- 15.47 / 15.48 GB/GPU · 30.94 GB total
- Peak system RAM
- 41.02 GB
- Errors · duration
- 0 · 97 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · 256K
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 262,144 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:21:50-05:00
- Prefill
- 671.7 tok/s
- Decode
- 42.03 tok/s
- Aggregate
- not measured
- Peak VRAM
- 17.8 / 17.99 GB/GPU · 35.74 GB total
- Peak system RAM
- 44.71 GB
- Errors · duration
- 0 · 259 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · single · short
Qwen3.6-35B-A3B · Q6_K
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 4,096 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:26:34-05:00
- Prefill
- 2621.7 tok/s
- Decode
- 82.62 tok/s
- Aggregate
- not measured
- Peak VRAM
- 28.78 GB/GPU · 29.32 GB total
- Peak system RAM
- 40.91 GB
- Errors · duration
- 0 · 18 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes. · Q6_K single-GPU 4K table cell is 103.23 tok/s from the 23:55 card (the 4K point in the complete Q6_K single-GPU series). The earlier 23:26 card at 82.62 tok/s stays on /results and is not averaged into the cell.
amdai-bench/run@1 · single · short
Qwen3.6-35B-A3B · Q5_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 4,096 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:27:26-05:00
- Prefill
- 2710.8 tok/s
- Decode
- 109.73 tok/s
- Aggregate
- not measured
- Peak VRAM
- 24.16 GB/GPU · 24.68 GB total
- Peak system RAM
- 35.83 GB
- Errors · duration
- 0 · 13 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · single · 32K
Qwen3.6-35B-A3B · Q5_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 32,768 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:27:39-05:00
- Prefill
- 1902.6 tok/s
- Decode
- 96.21 tok/s
- Aggregate
- not measured
- Peak VRAM
- 24.87 GB/GPU · 25.39 GB total
- Peak system RAM
- 36.46 GB
- Errors · duration
- 0 · 23 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · single · 128K
Qwen3.6-35B-A3B · Q5_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 131,072 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:28:02-05:00
- Prefill
- 940.3 tok/s
- Decode
- 69.64 tok/s
- Aggregate
- not measured
- Peak VRAM
- 26.93 GB/GPU · 27.43 GB total
- Peak system RAM
- 38.65 GB
- Errors · duration
- 0 · 104 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · single · 256K
Qwen3.6-35B-A3B · Q5_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 262,144 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:29:46-05:00
- Prefill
- 541.7 tok/s
- Decode
- 40.54 tok/s
- Aggregate
- not measured
- Peak VRAM
- 28.56 GB/GPU · 29.15 GB total
- Peak system RAM
- 42.79 GB
- Errors · duration
- 0 · 310 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · short
Qwen3.6-35B-A3B · Q5_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 4,096 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:34:56-05:00
- Prefill
- 2647.2 tok/s
- Decode
- 88.19 tok/s
- Aggregate
- not measured
- Peak VRAM
- 12.71 / 12.41 GB/GPU · 25.13 GB total
- Peak system RAM
- 37.17 GB
- Errors · duration
- 0 · 10 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · 32K
Qwen3.6-35B-A3B · Q5_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 32,768 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:35:07-05:00
- Prefill
- 1920.9 tok/s
- Decode
- 78.91 tok/s
- Aggregate
- not measured
- Peak VRAM
- 13.2 / 12.95 GB/GPU · 26.13 GB total
- Peak system RAM
- 37.8 GB
- Errors · duration
- 0 · 23 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · 128K
Qwen3.6-35B-A3B · Q5_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 131,072 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:35:31-05:00
- Prefill
- 1067.3 tok/s
- Decode
- 59.86 tok/s
- Aggregate
- not measured
- Peak VRAM
- 14.98 / 14.79 GB/GPU · 29.76 GB total
- Peak system RAM
- 40.73 GB
- Errors · duration
- 0 · 96 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · 256K
Qwen3.6-35B-A3B · Q5_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 262,144 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:37:08-05:00
- Prefill
- 664.6 tok/s
- Decode
- 42.33 tok/s
- Aggregate
- not measured
- Peak VRAM
- 17.45 / 17.3 GB/GPU · 34.54 GB total
- Peak system RAM
- 46.21 GB
- Errors · duration
- 0 · 258 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · single · short
Qwen3.6-35B-A3B · Q4_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 4,096 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:41:50-05:00
- Prefill
- 2733.9 tok/s
- Decode
- 117.69 tok/s
- Aggregate
- not measured
- Peak VRAM
- 20.84 GB/GPU · 21.43 GB total
- Peak system RAM
- 33.74 GB
- Errors · duration
- 0 · 12 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · single · 32K
Qwen3.6-35B-A3B · Q4_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 32,768 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:42:03-05:00
- Prefill
- 1902.1 tok/s
- Decode
- 101.88 tok/s
- Aggregate
- not measured
- Peak VRAM
- 21.55 GB/GPU · 22.17 GB total
- Peak system RAM
- 34.58 GB
- Errors · duration
- 0 · 22 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · single · 128K
Qwen3.6-35B-A3B · Q4_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 131,072 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:42:26-05:00
- Prefill
- 942.1 tok/s
- Decode
- 72.48 tok/s
- Aggregate
- not measured
- Peak VRAM
- 23.61 GB/GPU · 24.27 GB total
- Peak system RAM
- 36.5 GB
- Errors · duration
- 0 · 102 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · single · 256K
Qwen3.6-35B-A3B · Q4_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 262,144 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:44:09-05:00
- Prefill
- 558.5 tok/s
- Decode
- 47.77 tok/s
- Aggregate
- not measured
- Peak VRAM
- 26.36 GB/GPU · 27.03 GB total
- Peak system RAM
- 39.05 GB
- Errors · duration
- 0 · 292 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · short
Qwen3.6-35B-A3B · Q4_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 4,096 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:49:02-05:00
- Prefill
- 2660.4 tok/s
- Decode
- 93.28 tok/s
- Aggregate
- not measured
- Peak VRAM
- 10.83 / 10.85 GB/GPU · 21.69 GB total
- Peak system RAM
- 33.74 GB
- Errors · duration
- 0 · 9 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · 32K
Qwen3.6-35B-A3B · Q4_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 32,768 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:49:11-05:00
- Prefill
- 1933.1 tok/s
- Decode
- 82.57 tok/s
- Aggregate
- not measured
- Peak VRAM
- 11.34 / 11.39 GB/GPU · 22.72 GB total
- Peak system RAM
- 34.39 GB
- Errors · duration
- 0 · 22 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · 128K
Qwen3.6-35B-A3B · Q4_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 131,072 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:49:35-05:00
- Prefill
- 1078.4 tok/s
- Decode
- 62.12 tok/s
- Aggregate
- not measured
- Peak VRAM
- 13.03 / 13.23 GB/GPU · 26.26 GB total
- Peak system RAM
- 38.04 GB
- Errors · duration
- 0 · 95 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · 256K
Qwen3.6-35B-A3B · Q4_K_M
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 262,144 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:51:11-05:00
- Prefill
- 673 tok/s
- Decode
- 42.88 tok/s
- Aggregate
- not measured
- Peak VRAM
- 15.28 / 15.74 GB/GPU · 31.02 GB total
- Peak system RAM
- 40.84 GB
- Errors · duration
- 0 · 257 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · single · short
Qwen3.6-35B-A3B · Q6_K
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 4,096 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:55:52-05:00
- Prefill
- 2623.6 tok/s
- Decode
- 103.23 tok/s
- Aggregate
- not measured
- Peak VRAM
- 28.78 GB/GPU · 29.26 GB total
- Peak system RAM
- 40.15 GB
- Errors · duration
- 0 · 15 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes. · Q6_K single-GPU 4K table cell is 103.23 tok/s from the 23:55 card (the 4K point in the complete Q6_K single-GPU series). The earlier 23:26 card at 82.62 tok/s stays on /results and is not averaged into the cell.
amdai-bench/run@1 · single · 32K
Qwen3.6-35B-A3B · Q6_K
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 32,768 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:56:08-05:00
- Prefill
- 1683 tok/s
- Decode
- 71.19 tok/s
- Aggregate
- not measured
- Peak VRAM
- 28.68 GB/GPU · 29.17 GB total
- Peak system RAM
- 44.01 GB
- Errors · duration
- 0 · 31 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes. · Partly offloaded to DDR5 on one GPU (llama.cpp --fit-target 1024). Decode is not an in-VRAM figure.
amdai-bench/run@1 · single · 128K
Qwen3.6-35B-A3B · Q6_K
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 131,072 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:56:39-05:00
- Prefill
- 800.3 tok/s
- Decode
- 54.51 tok/s
- Aggregate
- not measured
- Peak VRAM
- 28.78 GB/GPU · 29.25 GB total
- Peak system RAM
- 45.43 GB
- Errors · duration
- 0 · 134 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes. · Partly offloaded to DDR5 on one GPU (llama.cpp --fit-target 1024). Decode is not an in-VRAM figure.
amdai-bench/run@1 · single · 256K
Qwen3.6-35B-A3B · Q6_K
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 262,144 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-12T23:58:54-05:00
- Prefill
- 471.6 tok/s
- Decode
- 37.78 tok/s
- Aggregate
- not measured
- Peak VRAM
- 28.81 GB/GPU · 29.3 GB total
- Peak system RAM
- 43.69 GB
- Errors · duration
- 0 · 385 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes. · Partly offloaded to DDR5 on one GPU (llama.cpp --fit-target 1024). Decode is not an in-VRAM figure.
amdai-bench/run@1 · split · short
Qwen3.6-35B-A3B · Q6_K
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 4,096 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-13T00:05:21-05:00
- Prefill
- 2572.1 tok/s
- Decode
- 83.5 tok/s
- Aggregate
- not measured
- Peak VRAM
- 15.01 / 14.63 GB/GPU · 29.64 GB total
- Peak system RAM
- 38.22 GB
- Errors · duration
- 0 · 11 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · 32K
Qwen3.6-35B-A3B · Q6_K
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 32,768 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-13T00:05:32-05:00
- Prefill
- 1890.7 tok/s
- Decode
- 75.43 tok/s
- Aggregate
- not measured
- Peak VRAM
- 15.5 / 15.16 GB/GPU · 30.66 GB total
- Peak system RAM
- 38.98 GB
- Errors · duration
- 0 · 26 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · 128K
Qwen3.6-35B-A3B · Q6_K
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 131,072 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-13T00:05:58-05:00
- Prefill
- 1059.7 tok/s
- Decode
- 58.04 tok/s
- Aggregate
- not measured
- Peak VRAM
- 17.21 / 17 GB/GPU · 34.21 GB total
- Peak system RAM
- 41.41 GB
- Errors · duration
- 0 · 100 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · split · 256K
Qwen3.6-35B-A3B · Q6_K
llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 2 GPUs · 262,144 tokens
Measured
Verified
pending re-verify after RAM fix
Credited to AMDai @amdai · local-desktop verified submitter
2026-10-13T00:07:38-05:00
- Prefill
- 666.2 tok/s
- Decode
- 42.2 tok/s
- Aggregate
- not measured
- Peak VRAM
- 19.46 / 19.51 GB/GPU · 38.96 GB total
- Peak system RAM
- 45.15 GB
- Errors · duration
- 0 · 262 s
VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.
amdai-bench/run@1 · dual-instance · 256K
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp · 2 GPUs · 262,144 tokens
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 100 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Two copies (one per card). About 100 tok/s per request.
amdai-bench/run@1 · dual-instance · 256K
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp · 2 GPUs · 262,144 tokens
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- not measured
- Aggregate
- 172 tok/s
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Two concurrent requests across two copies. 172 tok/s total.
amdai-bench/run@1 · dual-instance · 256K
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp · 2 GPUs · 262,144 tokens
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- not measured
- Aggregate
- 177 tok/s
- Peak VRAM
- 27.7 / 29.7 GB/GPU · 57.4 GB total
- Peak system RAM
- 13 GB
- Errors · duration
- 0 · 1236 s
Measured on our rig, 2026-10. 4–8 concurrent, about 177 tok/s total. Soak: 423 requests, 20.6 min, 0 errors. Peak VRAM 27.7 / 29.7 GB (Linux GiB labeled GB). DDR5 about 13 GB.
amdai-bench/run@1 · single · 256K
Qwen 3.8 Flash · UD-Q6_K_XL
Strata v0.1.42 · 2 GPUs · 262,144 tokens
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 32.6 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Prior first-party Flash Q6: Strata ~32.6 tok/s. Card count was not published; recorded on the two-R9700 host. A different run from the 32.8 / 31.8 two-card points.
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 16 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Prior first-party Flash Q6: llama.cpp ~16 tok/s versus Strata ~32.6. Card count was not published; recorded on the two-R9700 host.
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 32.8 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Two-card Flash Q6 at 262K load. Strata 32.8 tok/s at 8K. DDR5-5200.
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 32 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Two-card Flash Q6, Strata, DDR5-3600, 32.0 tok/s at 8K. Pair with 32.8 at DDR5-5200.
amdai-bench/run@1 · single · 32K
Qwen 3.8 Flash · UD-Q6_K_XL
Strata v0.1.42 · 2 GPUs · 32,768 tokens
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 32.8 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Two-card Flash Q6 at 262K load. Strata 32.8 tok/s at 32K.
amdai-bench/run@1 · single · 100K
Qwen 3.8 Flash · UD-Q6_K_XL
Strata v0.1.42 · 2 GPUs · 100,000 tokens
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 31.8 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Two-card Flash Q6 at 262K load. Strata 31.8 tok/s at 100K.
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 18.5 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Two-card Flash Q6, llama.cpp, DDR5-5200, 18.5 tok/s at 8K.
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 18.2 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Two-card Flash Q6, llama.cpp, DDR5-5200, 18.2 tok/s at 32K.
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 17.9 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Two-card Flash Q6, llama.cpp, DDR5-5200, 17.9 tok/s at 100K.
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 15.9 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Two-card Flash Q6, llama.cpp, DDR5-3600, 15.9 tok/s at 8K.
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 15.6 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Two-card Flash Q6, llama.cpp, DDR5-3600, 15.6 tok/s at 32K.
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- 15.3 tok/s
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Two-card Flash Q6, llama.cpp, DDR5-3600, 15.3 tok/s at 100K. Strata prefill drops 9–20% at DDR5-5200 versus 3600 — unexplained; no first-party prefill tok/s published.
amdai-bench/run@1 · single · short
Qwen3.6-35B-A3B · INT4-W4A16
vLLM stilldeadcode/vllm-radiance:0.5.8 · 1 GPU · 4,096 tokens
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- not measured
- Aggregate
- not measured
- Peak VRAM
- not measured
- Peak system RAM
- not measured
- Errors · duration
- 0 · not recorded
Measured on our rig, 2026-10. Band only — no single decode_tps. Image stilldeadcode/vllm-radiance:0.5.8. Exact serve flags not recorded.
llama.cpp · 256K
Qwen3.6-35B-A3B · UD-Q5_K_XL
One R9700, f16 KV, 0% RAM offload. Daily driver. Measured on our rig, 2026-10 — 44.4 tok/s decode at 256K. Official llama-bench c060ca974.
llama.cpp · Vulkan flags not recorded
# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
-m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
-ngl 99 \
-c 262144 \
--cache-type-k f16 \
--cache-type-v f16 \
--host 127.0.0.1 --port 8080
Minimum system RAM needed
42.9 GB
Measured · 42.9 GiB raw · 256K context
local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16
RAM needed at full context
42.9 GB
Measured · 42.9 GiB raw · 256K full context
local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16
Measured peak system RAM on the local desktop: 42.9 GB (Linux GiB labeled GB). Measured, pending re-verify after RAM fix.
Breakdown
- Offloaded weights ·
- 0 GiB
- Separable embed in RAM ·
- 0 GiB
- KV in RAM at selected ·
- 0 GiB
- KV in RAM at full context ·
- 0 GiB
- Runtime / backend ·
- 0.85 GiB
- OS / desktop baseline ·
- 8.00 GiB
- Headroom ·
- 16.0 GiB
llama.cpp · 262K
Qwen 3.8 Flash · UD-Q6_K_XL
llama.cpp Q6, 8 threads. Measured on our rig, 2026-10 — ~16 tok/s writing at 262K.
llama.cpp · Vulkan flags not recorded
# Shape only — llama.cpp Q6, 8 threads on the same Flash file. Exact flags TODO.
./build-vk/bin/llama-server \
-m ~/models/Qwen3.8-Flash-Next-UD-Q6_K_XL.gguf \
-t 8 \
-c 262144 \
--host 127.0.0.1 --port 8080
Minimum system RAM needed
192 GB kit
Estimate · 159.9 GiB raw · 256K context
RAM needed at full context
192 GB kit
Estimate · 159.9 GiB raw · 256K full context
Estimate. Offloaded weights + KV in RAM + runtime + OS/desktop + headroom, rounded up to a 192 GB DIMM kit. The local desktop ships 256 GB DDR5; measured host-RAM usage is not recorded until Local AI numbers arrive.
Breakdown
- Offloaded weights ·
- 125.4 GiB
- Separable embed in RAM ·
- 0 GiB
- KV in RAM at selected ·
- 7.61 GiB
- KV in RAM at full context ·
- 7.61 GiB
- Runtime / backend ·
- 2.88 GiB
- OS / desktop baseline ·
- 8.00 GiB
- Headroom ·
- 16.0 GiB
llama.cpp · 32K
Qwen3.6-35B-A3B · UD-Q5_K_XL
Measured on our rig, 2026-10 — 90.52 tok/s decode at 32K. Official llama-bench c060ca974.
llama.cpp · Vulkan flags not recorded
# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
-m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
-ngl 99 \
-c 32768 \
--cache-type-k f16 \
--cache-type-v f16 \
--host 127.0.0.1 --port 8080
Minimum system RAM needed
37.83 GB
Measured · 37.8 GiB raw · 32K context
local-desktop-20261012-2312-35b-udq5kxl-1gpu-32k-llamacpp-f16
RAM needed at full context
42.9 GB
Measured · 42.9 GiB raw · 256K full context
local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16
Measured peak system RAM on the local desktop: 37.83 GB (Linux GiB labeled GB). Measured, pending re-verify after RAM fix.
Breakdown
- Offloaded weights ·
- 0 GiB
- Separable embed in RAM ·
- 0 GiB
- KV in RAM at selected ·
- 0 GiB
- KV in RAM at full context ·
- 0 GiB
- Runtime / backend ·
- 0.78 GiB
- OS / desktop baseline ·
- 8.00 GiB
- Headroom ·
- 16.0 GiB
llama.cpp · short
Qwen3.6-35B-A3B · Q4_K_M
Measured on our rig, 2026-10 — 117.69 tok/s decode at short context. Prefill 1902.1 tok/s at 32K is a separate bench point on the same file.
llama.cpp · Vulkan flags not recorded
# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
-m ~/models/Qwen3.6-35B-A3B-Q4_K_M.gguf \
-ngl 99 \
-c 4096 \
--cache-type-k f16 \
--cache-type-v f16 \
--host 127.0.0.1 --port 8080
Minimum system RAM needed
33.74 GB
Measured · 33.7 GiB raw · 4K context
RAM needed at full context
39.05 GB
Measured · 39.0 GiB raw · 256K full context
Measured peak system RAM on the local desktop: 33.74 GB (Linux GiB labeled GB). Measured, pending re-verify after RAM fix.
Breakdown
- Offloaded weights ·
- 0 GiB
- Separable embed in RAM ·
- 0 GiB
- KV in RAM at selected ·
- 0 GiB
- KV in RAM at full context ·
- 0 GiB
- Runtime / backend ·
- 0.70 GiB
- OS / desktop baseline ·
- 8.00 GiB
- Headroom ·
- 16.0 GiB
llama.cpp · short
Qwen3.6-35B-A3B · Q5_K_M
Sibling Q5 file. Measured on our rig, 2026-10 — 109.73 tok/s short (24.68 GB VRAM). Official llama-bench c060ca974.
llama.cpp · Vulkan flags not recorded
# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
-m ~/models/Qwen3.6-35B-A3B-Q5_K_M.gguf \
-ngl 99 \
-c 4096 \
--cache-type-k f16 \
--cache-type-v f16 \
--host 127.0.0.1 --port 8080
Minimum system RAM needed
37.32 GB
Measured · 37.3 GiB raw · 4K context
local-desktop-20261012-2311-35b-udq5kxl-1gpu-4k-llamacpp-f16
RAM needed at full context
42.9 GB
Measured · 42.9 GiB raw · 256K full context
local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16
Measured peak system RAM on the local desktop: 37.32 GB (Linux GiB labeled GB). Measured, pending re-verify after RAM fix.
Breakdown
- Offloaded weights ·
- 0 GiB
- Separable embed in RAM ·
- 0 GiB
- KV in RAM at selected ·
- 0 GiB
- KV in RAM at full context ·
- 0 GiB
- Runtime / backend ·
- 0.77 GiB
- OS / desktop baseline ·
- 8.00 GiB
- Headroom ·
- 16.0 GiB
llama.cpp · short
Qwen3.6-35B-A3B · UD-Q5_K_XL
Measured on our rig, 2026-10 — 102.39 tok/s decode at short context. Official llama-bench c060ca974.
llama.cpp · Vulkan flags not recorded
# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
-m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
-ngl 99 \
-c 4096 \
--cache-type-k f16 \
--cache-type-v f16 \
--host 127.0.0.1 --port 8080
Minimum system RAM needed
37.32 GB
Measured · 37.3 GiB raw · 4K context
local-desktop-20261012-2311-35b-udq5kxl-1gpu-4k-llamacpp-f16
RAM needed at full context
42.9 GB
Measured · 42.9 GiB raw · 256K full context
local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16
Measured peak system RAM on the local desktop: 37.32 GB (Linux GiB labeled GB). Measured, pending re-verify after RAM fix.
Breakdown
- Offloaded weights ·
- 0 GiB
- Separable embed in RAM ·
- 0 GiB
- KV in RAM at selected ·
- 0 GiB
- KV in RAM at full context ·
- 0 GiB
- Runtime / backend ·
- 0.77 GiB
- OS / desktop baseline ·
- 8.00 GiB
- Headroom ·
- 16.0 GiB
Strata · 262K
Qwen 3.8 Flash · UD-Q6_K_XL
Strata v0.1.42, no MTP. Measured on our rig, 2026-10 — ~32.6 tok/s writing at 262K. Publisher UD-Q6_K_XL folder ~169 GB — host RAM not measured.
Strata · HIP flags not recorded
# Shape only — exact Strata flags TODO. Recipe strata-flash-q6, no MTP.
# This site’s convention: 127.0.0.1:8083 so llama-server can keep :8080.
curl -s http://127.0.0.1:8083/v1/models
Minimum system RAM needed
192 GB kit
Estimate · 159.9 GiB raw · 256K context
RAM needed at full context
192 GB kit
Estimate · 159.9 GiB raw · 256K full context
Estimate. Offloaded weights + KV in RAM + runtime + OS/desktop + headroom, rounded up to a 192 GB DIMM kit. The local desktop ships 256 GB DDR5; measured host-RAM usage is not recorded until Local AI numbers arrive.
Breakdown
- Offloaded weights ·
- 125.4 GiB
- Separable embed in RAM ·
- 0 GiB
- KV in RAM at selected ·
- 7.61 GiB
- KV in RAM at full context ·
- 7.61 GiB
- Runtime / backend ·
- 2.88 GiB
- OS / desktop baseline ·
- 8.00 GiB
- Headroom ·
- 16.0 GiB
Keep reading
This page is the shopping sheet. The rest of AMDai stays linked — we do not copy orphan content here.
-
R9700 review for local LLMs
Measured 35B llama.cpp review from the local desktop. 256K context. When to buy a second card.
-
35B requirements
VRAM and GPU needs at 256K for Q6/Q5/Q4. Which cards run it fully.
-
Flash requirements
Two R9700s plus 96 GB DDR5 runs Qwen 3.8 Flash Q4 at 256K.
-
Best AMD GPU for local LLMs
R9700 first. 16 GB cards offload. No invented street prices.
-
35B daily-driver recipe
UD-Q5_K_XL on llama.cpp Vulkan, one card, 262K. Launch command and gates.
-
Flash recipe
Qwen 3.8 Flash on Strata. Fast-agent pin, not the 35B overnight matrix.
-
Benchmarks
33-card local desktop matrix. Every cell links a run card.
-
Compare 35B
Cited decisions from the overnight cards — layout and quant.
-
Compare GPUs
R9700 vs 9070 XT and the other published sheets. VRAM, fit estimates, labeled Q3. No invented prices.
-
Radeon vs NVIDIA
Cited specs versus RTX 5090 / 4090 / 3090 / 5080 / 5070 Ti / 5060 Ti 16GB. NVIDIA speed stays empty until cited.
-
One vs two GPU
Same 35B, one R9700 versus a split. Decode winner per row.
-
Run cards
Official amdai-bench/run@1 pages. Every number on this sheet cites one.
-
How we test
amdai-bench/run@1 cards, labeled estimates, citations, corrections.
-
FAQ
Which GPU, Ubuntu vs Windows, and when Q3 is only an estimate.
-
Glossary
Plain-English ROCm, GGUF, Q3–Q6, KV cache, run cards.
-
Fine-tuning on Radeon
Cited LoRA / QLoRA, then export to llama.cpp. Speeds not yet measured on our R9700s. Q3 is a labeled estimate.
-
Q6 / Q5 / Q4 + Q3 estimate
Quant side-by-side. Q3 stays a labeled estimate.
-
Dual / multi-GPU
When a second R9700 helps or hurts — llama.cpp flags, PCIe, PSU. Desktop 2 192 GB is planned, not measured.
-
Multi-GPU llama.cpp how-to
Hands-on --split-mode, --tensor-split, device pins, and two-server combo. 35B split measured on the local desktop.
-
Two R9700s
Split versus two copies. A split is not two chats.
-
Flash daily driver
Qwen 3.8 Flash VRAM and DDR5 offload at Q3–Q6.
-
Qwen compare
Every current 3.6 / 3.8 file × quant on this notebook.
-
Stack Lab
Ubuntu companion. Packages not published yet.
-
First-run setup
Ubuntu 24.04+ to first token, plus an honest Windows path.
-
KV cache
f16 vs q8_0 vs q4_0. Measured 35B VRAM on this card. Estimates labeled.
-
Out of VRAM
Cited llama.cpp Vulkan / HIP OOM strings. Cheapest fix first. Q3 is a labeled estimate.
-
Calculator
Size a GGUF against 32 GB. Client-side, no prices.
-
Budget
Starter through two-R9700 builds. No invented prices.
Newegg GPU listings checked Nov 2, 2026. Newegg GPU-category search (filter N=100007709): Gigabyte AI TOP listed at $2,799.99, with more options from $2,079.97–$3,290.00. ASRock Creator out of stock (last sold $1,699.99). Confirm who ships the card and the current price on the listing. Full store notes: where to buy an R9700.
Check price · no live or invented prices
AMD Radeon AI PRO R9700
- Amazon: check price
- Newegg: check price
- B&H Photo: check price
- Micro Center: check price
- AMD Store: check price
Newegg's R9700 offers have been marketplace sellers (checked 2026-10-30), so that seller's return policy applies, not Newegg's. Open the listing's Warranty & Returns tab before you buy.
As an Amazon Associate I earn from qualifying purchases. Some Newegg links use Rakuten Advertising and some B&H links use B&H’s affiliate program when those IDs are configured. Micro Center and AMD Store links are plain “also available at” links — no tracking. Buttons say “Check price” and never invent or live-quote a price. Full disclosure.
No live or invented prices — dated, sourced snapshot only. Retailer hops use the existing disclosure path.
Compare with other sheets
Head-to-head pages use the same VRAM, bandwidth, and calculator fit already on the sheets. Q3 is a labeled estimate. No invented prices. R9700 against the other published sheets:
- Compare GPUs
- vs PRO W7800
- vs PRO W7900
- vs RX 7900 XTX
- vs RX 9060 XT
- vs RX 9070
- vs RX 9070 XT
- vs Strix Halo
Also: PRO W6800 · PRO W7700 · RX 6700 XT · RX 6800 XT · RX 6950 XT · RX 7600 · RX 7600 XT · RX 7700 XT · RX 7800 XT · RX 7900 GRE · RX 7900 XT · RX 9070 GRE
AMDai Lab Notes digest
Email only — no account. Double opt-in. About once a week, and only when run cards, guides, ROCm problems, or Stack Lab actually changed. Not a daily drip. Issues may include one labeled Sponsored mention. Slots and rates: Sponsor.