Official test-run cards

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:11:50-05:00

Prefill

2761.6 tok/s

Decode

102.39 tok/s

Aggregate

not measured

Peak VRAM

25.45 GB/GPU · 25.97 GB total

Peak system RAM

37.32 GB

Errors · duration

0 · 9 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

unsloth/Qwen3.6-35B-A3B-GGUF · Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

File size · sha256

24.77 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-UD-Q5_K_XL-single-.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/unsloth/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 1920 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:12:00-05:00

Prefill

1937.9 tok/s

Decode

90.52 tok/s

Aggregate

not measured

Peak VRAM

26.16 GB/GPU · 26.7 GB total

Peak system RAM

37.83 GB

Errors · duration

0 · 23 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

unsloth/Qwen3.6-35B-A3B-GGUF · Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

File size · sha256

24.77 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-UD-Q5_K_XL-single-4K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/unsloth/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 30592 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:12:23-05:00

Prefill

942.8 tok/s

Decode

66.43 tok/s

Aggregate

not measured

Peak VRAM

28.22 GB/GPU · 28.82 GB total

Peak system RAM

40.18 GB

Errors · duration

0 · 103 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

unsloth/Qwen3.6-35B-A3B-GGUF · Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

File size · sha256

24.77 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-UD-Q5_K_XL-single-32K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/unsloth/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 128896 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:14:07-05:00

Prefill

519.2 tok/s

Decode

44.4 tok/s

Aggregate

not measured

Peak VRAM

28.73 GB/GPU · 29.34 GB total

Peak system RAM

42.9 GB

Errors · duration

0 · 330 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

unsloth/Qwen3.6-35B-A3B-GGUF · Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

File size · sha256

24.77 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-UD-Q5_K_XL-single-128K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/unsloth/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 259968 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:19:38-05:00

Prefill

2637.3 tok/s

Decode

82.78 tok/s

Aggregate

not measured

Peak VRAM

13.28 / 13.11 GB/GPU · 26.39 GB total

Peak system RAM

37.6 GB

Errors · duration

0 · 9 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

unsloth/Qwen3.6-35B-A3B-GGUF · Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

File size · sha256

24.77 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-UD-Q5_K_XL-both-.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/unsloth/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 1920 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:19:48-05:00

Prefill

1916 tok/s

Decode

74.8 tok/s

Aggregate

not measured

Peak VRAM

13.77 / 13.64 GB/GPU · 27.41 GB total

Peak system RAM

38.16 GB

Errors · duration

0 · 24 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

unsloth/Qwen3.6-35B-A3B-GGUF · Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

File size · sha256

24.77 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-UD-Q5_K_XL-both-4K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/unsloth/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 30592 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:20:12-05:00

Prefill

1071.1 tok/s

Decode

57.94 tok/s

Aggregate

not measured

Peak VRAM

15.47 / 15.48 GB/GPU · 30.94 GB total

Peak system RAM

41.02 GB

Errors · duration

0 · 97 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

unsloth/Qwen3.6-35B-A3B-GGUF · Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

File size · sha256

24.77 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-UD-Q5_K_XL-both-32K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/unsloth/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 128896 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:21:50-05:00

Prefill

671.7 tok/s

Decode

42.03 tok/s

Aggregate

not measured

Peak VRAM

17.8 / 17.99 GB/GPU · 35.74 GB total

Peak system RAM

44.71 GB

Errors · duration

0 · 259 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

unsloth/Qwen3.6-35B-A3B-GGUF · Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

File size · sha256

24.77 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-UD-Q5_K_XL-both-128K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/unsloth/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 259968 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:26:34-05:00

Prefill

2621.7 tok/s

Decode

82.62 tok/s

Aggregate

not measured

Peak VRAM

28.78 GB/GPU · 29.32 GB total

Peak system RAM

40.91 GB

Errors · duration

0 · 18 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q6_K.gguf

File size · sha256

28.82 GB · not recorded

KV · offload

f16 · experts to RAM — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes. · Q6_K single-GPU 4K table cell is 103.23 tok/s from the 23:55 card (the 4K point in the complete Q6_K single-GPU series). The earlier 23:26 card at 82.62 tok/s stays on /results and is not averaged into the cell.

Log: logs/q36-35b-Q6_K-single-.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen36-35b-gguf/Qwen_Qwen3.6-35B-A3B-Q6_K.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 1920 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:27:26-05:00

Prefill

2710.8 tok/s

Decode

109.73 tok/s

Aggregate

not measured

Peak VRAM

24.16 GB/GPU · 24.68 GB total

Peak system RAM

35.83 GB

Errors · duration

0 · 13 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf

File size · sha256

24.13 GB · not recorded

KV · offload

f16 · experts to RAM — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q5_K_M-single-4K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 1920 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:27:39-05:00

Prefill

1902.6 tok/s

Decode

96.21 tok/s

Aggregate

not measured

Peak VRAM

24.87 GB/GPU · 25.39 GB total

Peak system RAM

36.46 GB

Errors · duration

0 · 23 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf

File size · sha256

24.13 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q5_K_M-single-32K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 30592 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:28:02-05:00

Prefill

940.3 tok/s

Decode

69.64 tok/s

Aggregate

not measured

Peak VRAM

26.93 GB/GPU · 27.43 GB total

Peak system RAM

38.65 GB

Errors · duration

0 · 104 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf

File size · sha256

24.13 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q5_K_M-single-128K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 128896 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:29:46-05:00

Prefill

541.7 tok/s

Decode

40.54 tok/s

Aggregate

not measured

Peak VRAM

28.56 GB/GPU · 29.15 GB total

Peak system RAM

42.79 GB

Errors · duration

0 · 310 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf

File size · sha256

24.13 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q5_K_M-single-256K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 259968 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:34:56-05:00

Prefill

2647.2 tok/s

Decode

88.19 tok/s

Aggregate

not measured

Peak VRAM

12.71 / 12.41 GB/GPU · 25.13 GB total

Peak system RAM

37.17 GB

Errors · duration

0 · 10 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf

File size · sha256

24.13 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q5_K_M-both-4K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 1920 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:35:07-05:00

Prefill

1920.9 tok/s

Decode

78.91 tok/s

Aggregate

not measured

Peak VRAM

13.2 / 12.95 GB/GPU · 26.13 GB total

Peak system RAM

37.8 GB

Errors · duration

0 · 23 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf

File size · sha256

24.13 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q5_K_M-both-32K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 30592 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:35:31-05:00

Prefill

1067.3 tok/s

Decode

59.86 tok/s

Aggregate

not measured

Peak VRAM

14.98 / 14.79 GB/GPU · 29.76 GB total

Peak system RAM

40.73 GB

Errors · duration

0 · 96 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf

File size · sha256

24.13 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q5_K_M-both-128K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 128896 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:37:08-05:00

Prefill

664.6 tok/s

Decode

42.33 tok/s

Aggregate

not measured

Peak VRAM

17.45 / 17.3 GB/GPU · 34.54 GB total

Peak system RAM

46.21 GB

Errors · duration

0 · 258 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf

File size · sha256

24.13 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q5_K_M-both-256K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q5_K_M.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 259968 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:41:50-05:00

Prefill

2733.9 tok/s

Decode

117.69 tok/s

Aggregate

not measured

Peak VRAM

20.84 GB/GPU · 21.43 GB total

Peak system RAM

33.74 GB

Errors · duration

0 · 12 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf

File size · sha256

20.75 GB · not recorded

KV · offload

f16 · experts to RAM — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q4_K_M-single-4K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 1920 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:42:03-05:00

Prefill

1902.1 tok/s

Decode

101.88 tok/s

Aggregate

not measured

Peak VRAM

21.55 GB/GPU · 22.17 GB total

Peak system RAM

34.58 GB

Errors · duration

0 · 22 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf

File size · sha256

20.75 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q4_K_M-single-32K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 30592 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:42:26-05:00

Prefill

942.1 tok/s

Decode

72.48 tok/s

Aggregate

not measured

Peak VRAM

23.61 GB/GPU · 24.27 GB total

Peak system RAM

36.5 GB

Errors · duration

0 · 102 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf

File size · sha256

20.75 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q4_K_M-single-128K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 128896 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:44:09-05:00

Prefill

558.5 tok/s

Decode

47.77 tok/s

Aggregate

not measured

Peak VRAM

26.36 GB/GPU · 27.03 GB total

Peak system RAM

39.05 GB

Errors · duration

0 · 292 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf

File size · sha256

20.75 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q4_K_M-single-256K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 259968 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:49:02-05:00

Prefill

2660.4 tok/s

Decode

93.28 tok/s

Aggregate

not measured

Peak VRAM

10.83 / 10.85 GB/GPU · 21.69 GB total

Peak system RAM

33.74 GB

Errors · duration

0 · 9 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf

File size · sha256

20.75 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q4_K_M-both-4K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 1920 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:49:11-05:00

Prefill

1933.1 tok/s

Decode

82.57 tok/s

Aggregate

not measured

Peak VRAM

11.34 / 11.39 GB/GPU · 22.72 GB total

Peak system RAM

34.39 GB

Errors · duration

0 · 22 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf

File size · sha256

20.75 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q4_K_M-both-32K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 30592 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:49:35-05:00

Prefill

1078.4 tok/s

Decode

62.12 tok/s

Aggregate

not measured

Peak VRAM

13.03 / 13.23 GB/GPU · 26.26 GB total

Peak system RAM

38.04 GB

Errors · duration

0 · 95 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf

File size · sha256

20.75 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q4_K_M-both-128K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 128896 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:51:11-05:00

Prefill

673 tok/s

Decode

42.88 tok/s

Aggregate

not measured

Peak VRAM

15.28 / 15.74 GB/GPU · 31.02 GB total

Peak system RAM

40.84 GB

Errors · duration

0 · 257 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf

File size · sha256

20.75 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q4_K_M-both-256K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen-matrix/qwen36-35b-a3b/bartowski/Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 259968 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:55:52-05:00

Prefill

2623.6 tok/s

Decode

103.23 tok/s

Aggregate

not measured

Peak VRAM

28.78 GB/GPU · 29.26 GB total

Peak system RAM

40.15 GB

Errors · duration

0 · 15 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q6_K.gguf

File size · sha256

28.82 GB · not recorded

KV · offload

f16 · experts to RAM — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes. · Q6_K single-GPU 4K table cell is 103.23 tok/s from the 23:55 card (the 4K point in the complete Q6_K single-GPU series). The earlier 23:26 card at 82.62 tok/s stays on /results and is not averaged into the cell.

Log: logs/q36-35b-Q6_K-single-4K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen36-35b-gguf/Qwen_Qwen3.6-35B-A3B-Q6_K.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 1920 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:56:08-05:00

Prefill

1683 tok/s

Decode

71.19 tok/s

Aggregate

not measured

Peak VRAM

28.68 GB/GPU · 29.17 GB total

Peak system RAM

44.01 GB

Errors · duration

0 · 31 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q6_K.gguf

File size · sha256

28.82 GB · not recorded

KV · offload

f16 · experts to RAM — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes. · Partly offloaded to DDR5 on one GPU (llama.cpp --fit-target 1024). Decode is not an in-VRAM figure.

Log: logs/q36-35b-Q6_K-single-32K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen36-35b-gguf/Qwen_Qwen3.6-35B-A3B-Q6_K.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 30592 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:56:39-05:00

Prefill

800.3 tok/s

Decode

54.51 tok/s

Aggregate

not measured

Peak VRAM

28.78 GB/GPU · 29.25 GB total

Peak system RAM

45.43 GB

Errors · duration

0 · 134 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q6_K.gguf

File size · sha256

28.82 GB · not recorded

KV · offload

f16 · experts to RAM — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes. · Partly offloaded to DDR5 on one GPU (llama.cpp --fit-target 1024). Decode is not an in-VRAM figure.

Log: logs/q36-35b-Q6_K-single-128K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen36-35b-gguf/Qwen_Qwen3.6-35B-A3B-Q6_K.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 128896 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:58:54-05:00

Prefill

471.6 tok/s

Decode

37.78 tok/s

Aggregate

not measured

Peak VRAM

28.81 GB/GPU · 29.3 GB total

Peak system RAM

43.69 GB

Errors · duration

0 · 385 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q6_K.gguf

File size · sha256

28.82 GB · not recorded

KV · offload

f16 · experts to RAM — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes. · Partly offloaded to DDR5 on one GPU (llama.cpp --fit-target 1024). Decode is not an in-VRAM figure.

Log: logs/q36-35b-Q6_K-single-256K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen36-35b-gguf/Qwen_Qwen3.6-35B-A3B-Q6_K.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 259968 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-13T00:05:21-05:00

Prefill

2572.1 tok/s

Decode

83.5 tok/s

Aggregate

not measured

Peak VRAM

15.01 / 14.63 GB/GPU · 29.64 GB total

Peak system RAM

38.22 GB

Errors · duration

0 · 11 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q6_K.gguf

File size · sha256

28.82 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q6_K-both-4K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen36-35b-gguf/Qwen_Qwen3.6-35B-A3B-Q6_K.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 1920 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-13T00:05:32-05:00

Prefill

1890.7 tok/s

Decode

75.43 tok/s

Aggregate

not measured

Peak VRAM

15.5 / 15.16 GB/GPU · 30.66 GB total

Peak system RAM

38.98 GB

Errors · duration

0 · 26 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q6_K.gguf

File size · sha256

28.82 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q6_K-both-32K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen36-35b-gguf/Qwen_Qwen3.6-35B-A3B-Q6_K.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 30592 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-13T00:05:58-05:00

Prefill

1059.7 tok/s

Decode

58.04 tok/s

Aggregate

not measured

Peak VRAM

17.21 / 17 GB/GPU · 34.21 GB total

Peak system RAM

41.41 GB

Errors · duration

0 · 100 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q6_K.gguf

File size · sha256

28.82 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q6_K-both-128K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen36-35b-gguf/Qwen_Qwen3.6-35B-A3B-Q6_K.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 128896 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-13T00:07:38-05:00

Prefill

666.2 tok/s

Decode

42.2 tok/s

Aggregate

not measured

Peak VRAM

19.46 / 19.51 GB/GPU · 38.96 GB total

Peak system RAM

45.15 GB

Errors · duration

0 · 262 s

Hardware

GPUs in the box

Radeon AI PRO R9700 32 GB gfx1201 · Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · Gen5 x16

CPU

AMD Ryzen 9 9950X (16C/32T)

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel

Ubuntu 24.04.5 LTS · 7.0.0-38-generic

Mesa / Vulkan · ROCm

Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04

Backend

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file

bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q6_K.gguf

File size · sha256

28.82 GB · not recorded

KV · offload

f16 · no RAM offload — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM

Concurrency · prompt / gen

1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

Log: logs/q36-35b-Q6_K-both-256K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen36-35b-gguf/Qwen_Qwen3.6-35B-A3B-Q6_K.gguf -dev Vulkan0/Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 259968 -r 1 -o jsonl

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

100 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

llama.cpp

Model and config

Repo · file

unsloth/Qwen3.6-35B-A3B-GGUF · Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

File size · sha256

26.6 GB · not recorded

KV · offload

f16 · no RAM offload — 0% system RAM offload

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Two copies (one per card). About 100 tok/s per request.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — one llama-server per card behind a least-connections balancer.
# Exact balancer command not recorded. Device-selection flags were not recorded.
# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
  -m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

not measured

Aggregate

172 tok/s

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

llama.cpp

Model and config

Repo · file

unsloth/Qwen3.6-35B-A3B-GGUF · Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

File size · sha256

26.6 GB · not recorded

KV · offload

f16 · no RAM offload — 0% system RAM offload

Concurrency · prompt / gen

2 · not recorded / not recorded

Measured on our rig, 2026-10. Two concurrent requests across two copies. 172 tok/s total.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — one llama-server per card behind a least-connections balancer.
# Exact balancer command not recorded. Device-selection flags were not recorded.
# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
  -m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

not measured

Aggregate

177 tok/s

Peak VRAM

27.7 / 29.7 GB/GPU · 57.4 GB total

Peak system RAM

13 GB

Errors · duration

0 · 1236 s

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

llama.cpp

Model and config

Repo · file

unsloth/Qwen3.6-35B-A3B-GGUF · Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

File size · sha256

26.6 GB · not recorded

KV · offload

f16 · no RAM offload — 0% system RAM offload

Concurrency · prompt / gen

4 · not recorded / not recorded

Measured on our rig, 2026-10. 4–8 concurrent, about 177 tok/s total. Soak: 423 requests, 20.6 min, 0 errors. Peak VRAM 27.7 / 29.7 GB (Linux GiB labeled GB). DDR5 about 13 GB.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — one llama-server per card behind a least-connections balancer.
# Exact balancer command not recorded. Device-selection flags were not recorded.
# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
  -m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

32.6 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

Strata v0.1.42

Model and config

Repo · file

unsloth/Qwen3.8-Flash-Next-GGUF · UD-Q6_K_XL (folder)

File size · sha256

169 GB · not recorded

KV · offload

f16 · experts to RAM — Hot experts in VRAM; rest on DDR5. Publisher folder ~169 GB.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Prior first-party Flash Q6: Strata ~32.6 tok/s. Card count was not published; recorded on the two-R9700 host. A different run from the 32.8 / 31.8 two-card points.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — exact Strata flags TODO. Recipe strata-flash-q6, no MTP.
# This site’s convention: 127.0.0.1:8083 so llama-server can keep :8080.
curl -s http://127.0.0.1:8083/v1/models

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

16 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

llama.cpp

Model and config

Repo · file

unsloth/Qwen3.8-Flash-Next-GGUF · UD-Q6_K_XL (folder)

File size · sha256

169 GB · not recorded

KV · offload

f16 · experts to RAM — Publisher folder ~169 GB — expect RAM to hold most of it on one 32 GB card.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Prior first-party Flash Q6: llama.cpp ~16 tok/s versus Strata ~32.6. Card count was not published; recorded on the two-R9700 host.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
  -m ~/models/UD-Q6_K_XL \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

32.8 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

Strata v0.1.42

Model and config

Repo · file

unsloth/Qwen3.8-Flash-Next-GGUF · UD-Q6_K_XL (folder)

File size · sha256

169 GB · not recorded

KV · offload

f16 · experts to RAM — Two-card Q6 loaded at 262K. Decode measured at 8K.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Two-card Flash Q6 at 262K load. Strata 32.8 tok/s at 8K. DDR5-5200.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — exact Strata flags TODO. Recipe strata-flash-q6, no MTP.
# This site’s convention: 127.0.0.1:8083 so llama-server can keep :8080.
curl -s http://127.0.0.1:8083/v1/models

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

32 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 3600

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

Strata v0.1.42

Model and config

Repo · file

unsloth/Qwen3.8-Flash-Next-GGUF · UD-Q6_K_XL (folder)

File size · sha256

169 GB · not recorded

KV · offload

f16 · experts to RAM — Two-card Q6 loaded at 262K. Decode measured at 8K.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Two-card Flash Q6, Strata, DDR5-3600, 32.0 tok/s at 8K. Pair with 32.8 at DDR5-5200.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — exact Strata flags TODO. Recipe strata-flash-q6, no MTP.
# This site’s convention: 127.0.0.1:8083 so llama-server can keep :8080.
curl -s http://127.0.0.1:8083/v1/models

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

32.8 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

Strata v0.1.42

Model and config

Repo · file

unsloth/Qwen3.8-Flash-Next-GGUF · UD-Q6_K_XL (folder)

File size · sha256

169 GB · not recorded

KV · offload

f16 · experts to RAM — Two-card Q6 loaded at 262K. Decode measured at 32K.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Two-card Flash Q6 at 262K load. Strata 32.8 tok/s at 32K.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — exact Strata flags TODO. Recipe strata-flash-q6, no MTP.
# This site’s convention: 127.0.0.1:8083 so llama-server can keep :8080.
curl -s http://127.0.0.1:8083/v1/models

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

31.8 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

Strata v0.1.42

Model and config

Repo · file

unsloth/Qwen3.8-Flash-Next-GGUF · UD-Q6_K_XL (folder)

File size · sha256

169 GB · not recorded

KV · offload

f16 · experts to RAM — Two-card Q6 loaded at 262K. Decode measured at 100K.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Two-card Flash Q6 at 262K load. Strata 31.8 tok/s at 100K.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — exact Strata flags TODO. Recipe strata-flash-q6, no MTP.
# This site’s convention: 127.0.0.1:8083 so llama-server can keep :8080.
curl -s http://127.0.0.1:8083/v1/models

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

18.5 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

llama.cpp

Model and config

Repo · file

unsloth/Qwen3.8-Flash-Next-GGUF · UD-Q6_K_XL (folder)

File size · sha256

169 GB · not recorded

KV · offload

f16 · experts to RAM — Two-card Q6 loaded at 262K. Decode measured at 8K.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Two-card Flash Q6, llama.cpp, DDR5-5200, 18.5 tok/s at 8K.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
  -m ~/models/UD-Q6_K_XL \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

18.2 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

llama.cpp

Model and config

Repo · file

unsloth/Qwen3.8-Flash-Next-GGUF · UD-Q6_K_XL (folder)

File size · sha256

169 GB · not recorded

KV · offload

f16 · experts to RAM — Two-card Q6 loaded at 262K. Decode measured at 32K.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Two-card Flash Q6, llama.cpp, DDR5-5200, 18.2 tok/s at 32K.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
  -m ~/models/UD-Q6_K_XL \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

17.9 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

llama.cpp

Model and config

Repo · file

unsloth/Qwen3.8-Flash-Next-GGUF · UD-Q6_K_XL (folder)

File size · sha256

169 GB · not recorded

KV · offload

f16 · experts to RAM — Two-card Q6 loaded at 262K. Decode measured at 100K.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Two-card Flash Q6, llama.cpp, DDR5-5200, 17.9 tok/s at 100K.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
  -m ~/models/UD-Q6_K_XL \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

15.9 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 3600

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

llama.cpp

Model and config

Repo · file

unsloth/Qwen3.8-Flash-Next-GGUF · UD-Q6_K_XL (folder)

File size · sha256

169 GB · not recorded

KV · offload

f16 · experts to RAM — Two-card Q6 loaded at 262K. Decode measured at 8K.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Two-card Flash Q6, llama.cpp, DDR5-3600, 15.9 tok/s at 8K.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
  -m ~/models/UD-Q6_K_XL \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

15.6 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 3600

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

llama.cpp

Model and config

Repo · file

unsloth/Qwen3.8-Flash-Next-GGUF · UD-Q6_K_XL (folder)

File size · sha256

169 GB · not recorded

KV · offload

f16 · experts to RAM — Two-card Q6 loaded at 262K. Decode measured at 32K.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Two-card Flash Q6, llama.cpp, DDR5-3600, 15.6 tok/s at 32K.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
  -m ~/models/UD-Q6_K_XL \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

15.3 tok/s

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

2 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 3600

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

llama.cpp

Model and config

Repo · file

unsloth/Qwen3.8-Flash-Next-GGUF · UD-Q6_K_XL (folder)

File size · sha256

169 GB · not recorded

KV · offload

f16 · experts to RAM — Two-card Q6 loaded at 262K. Decode measured at 100K.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Two-card Flash Q6, llama.cpp, DDR5-3600, 15.3 tok/s at 100K. Strata prefill drops 9–20% at DDR5-5200 versus 3600 — unexplained; no first-party prefill tok/s published.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
  -m ~/models/UD-Q6_K_XL \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Credited to AMDai @amdai · local-desktop verified submitter

2026-10 (date not recorded)

Prefill

not measured

Decode

not measured

Aggregate

not measured

Peak VRAM

not measured

Peak system RAM

not measured

Errors · duration

0 · not recorded

Hardware

GPUs in the box

AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201

GPUs used · PCIe

1 · not recorded

CPU

not recorded

RAM kit / usable / MT/s

256 GB / 249 GB / 5200

Board · BIOS · ReBAR

not recorded · not recorded · not recorded

Software

OS · kernel

Linux · not recorded

Mesa / Vulkan · ROCm

not recorded · not recorded

Backend

vLLM stilldeadcode/vllm-radiance:0.5.8

Model and config

Repo · file

Qwen/Qwen3.6-35B-A3B · INT4-W4A16 (image weights)

File size · sha256

not recorded · not recorded

KV · offload

not recorded · no RAM offload — vLLM image; used_on_our_rig catalog is false.

Concurrency · prompt / gen

1 · not recorded / not recorded

Measured on our rig, 2026-10. Band only — no single decode_tps. Image stilldeadcode/vllm-radiance:0.5.8. Exact serve flags not recorded.

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

# Image only — exact serve flags not recorded.
# stilldeadcode/vllm-radiance:0.5.8
# This site’s convention: 127.0.0.1:8000