Q6_K 131.072K 1×…

Test-run local-desktop-20261012-2356-35b-q6k-1gpu-128k-llamacpp-f16

Official amdai-bench/run@1 card. Credited to AMDai. Verified on the local desktop. Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes.

amdai-bench/run@1 · single · 128K

Qwen3.6-35B-A3B · Q6_K

llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON · 1 GPU · 131,072 tokens

Measured Verified pending re-verify after RAM fix

Credited to AMDai @amdai · local-desktop verified submitter

2026-10-12T23:56:39-05:00

Prefill: 800.3 tok/s
Decode: 54.51 tok/s
Aggregate: not measured
Peak VRAM: 28.78 GB/GPU · 29.25 GB total
Peak system RAM: 45.43 GB
Errors · duration: 0 · 134 s

Hardware

GPUs in the box: Radeon AI PRO R9700 32 GB gfx1201
GPUs used · PCIe: 1 · Gen5 x16
CPU: AMD Ryzen 9 9950X (16C/32T)
RAM kit / usable / MT/s: 256 GB / 249 GB / 5200
Board · BIOS · ReBAR: ASRock X870 Taichi Creator · 4.50 · yes

Software

OS · kernel: Ubuntu 24.04.5 LTS · 7.0.0-38-generic
Mesa / Vulkan · ROCm: Mesa RADV 25.2.8-0ubuntu0.24.04.2 · 7.2.0.70200-43~24.04
Backend: llama.cpp llama.cpp c060ca974 (build 10603, 2026-08-23) Vulkan; cmake Release GGML_VULKAN=ON GGML_NATIVE=ON GGML_OPENMP=ON

Model and config

Repo · file: bartowski/Qwen_Qwen3.6-35B-A3B-GGUF · Qwen_Qwen3.6-35B-A3B-Q6_K.gguf
File size · sha256: 28.82 GB · not recorded
KV · offload: f16 · experts to RAM — llama.cpp --fit-target 1024 MiB placed tensors; experts_to_ram inferred from model size vs peak VRAM
Concurrency · prompt / gen: 1 · 2048 / 128

VRAM sampled every 1 s via amdgpu mem_info_vram_used; Vulkan0 (display) includes desktop baseline ~0.5 GB · DDR5 = MemTotal-MemAvailable + resident page cache of the model files (mincore), 1 s samples; GB = GiB · RAM at DDR5-5200: one transient bad read in 10 earlier sha256 passes; sanity prompt cross-checked · Measured, pending re-verify after RAM fix. DDR5-5200 later proved unstable; a kernel oops at 00:14 CT aborted the sweep. All 33 cards predate the oops. Each card already notes one transient bad read in earlier sha256 passes. · Partly offloaded to DDR5 on one GPU (llama.cpp --fit-target 1024). Decode is not an in-VRAM figure.

Log: logs/q36-35b-Q6_K-single-128K.jsonl

Launch command

Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.

GGML_VK_VISIBLE_DEVICES=0,1 ~/llama.cpp-new/build/bin/llama-bench -m ~/models/qwen36-35b-gguf/Qwen_Qwen3.6-35B-A3B-Q6_K.gguf -dev Vulkan1 -sm layer -ngl 99 -fitt 1024 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 -t 8 -p 2048 -n 128 -d 128896 -r 1 -o jsonl