amdai-bench/run@1 · dual-instance · 256K
Qwen3.6-35B-A3B · UD-Q5_K_XL
llama.cpp · 2 GPUs · 262,144 tokens
Measured
Verified
Credited to AMDai @amdai · local-desktop verified submitter
2026-10 (date not recorded)
- Prefill
- not measured
- Decode
- not measured
- Aggregate
- 177 tok/s
- Peak VRAM
- 27.7 / 29.7 GB/GPU · 57.4 GB total
- Peak system RAM
- 13 GB
- Errors · duration
- 0 · 1236 s
Hardware
- GPUs in the box
- AMD Radeon AI PRO R9700 32 GB gfx1201 · AMD Radeon AI PRO R9700 32 GB gfx1201
- GPUs used · PCIe
- 2 · not recorded
- CPU
- not recorded
- RAM kit / usable / MT/s
- 256 GB / 249 GB / 5200
- Board · BIOS · ReBAR
- not recorded · not recorded · not recorded
Software
- OS · kernel
- Linux · not recorded
- Backend
- llama.cpp
Model and config
- Repo · file
- unsloth/Qwen3.6-35B-A3B-GGUF · Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf
- File size · sha256
- 26.6 GB · not recorded
- KV · offload
- f16 · no RAM offload — 0% system RAM offload
- Concurrency · prompt / gen
- 4 · not recorded / not recorded
Measured on our rig, 2026-10. 4–8 concurrent, about 177 tok/s total. Soak: 423 requests, 20.6 min, 0 errors. Peak VRAM 27.7 / 29.7 GB (Linux GiB labeled GB). DDR5 about 13 GB.
Launch command
Exact command as recorded. Shape-only lines stay not recorded — we do not invent flags.
# Shape only — one llama-server per card behind a least-connections balancer.
# Exact balancer command not recorded. Device-selection flags were not recorded.
# Shape only — exact flags TODO on stacks.
./build-vk/bin/llama-server \
-m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
-ngl 99 \
-c 262144 \
--cache-type-k f16 \
--cache-type-v f16 \
--host 127.0.0.1 --port 8080