llama.cpp on Radeon: Vulkan vs ROCm (HIP)

Decision box

Longer ROCm-guide version: /rocm/vulkan-hip. Glossary: Vulkan · HIP.

Install, speed, split, flags

HIP speed cells stay pending — not measured on the local desktop. Q6 / Q5 / Q4 first. UD-Q3_K_XL is a labeled estimate.

• Install effort

  cited

  Vulkan

  Mesa RADV (or the vendor ICD on Windows) plus libvulkan-dev. cmake -B build-vk -DGGML_VULKAN=ON. No matching /opt/rocm tree required for first tokens. Cited from llama.cpp build.md and this site’s /setup path.

  ROCm / HIP

  Full ROCm userspace, then rocminfo must list this gfx before you compile. cmake -B build-hip -DGGML_HIP=ON -DAMDGPU_TARGETS=<gfx>. Prefix may be /opt/rocm/core-10.0 — ls first. Official matrix: AMD ROCm 10.1.0.

• Decode vs prefill

  pending

  Vulkan

  Measured on the local desktop llama.cpp Vulkan only. Official overnight 35B decode cells live in the table below and on /benchmarks. Those are Vulkan cards, not a HIP verdict. Prefill appears on some Vulkan cards in that same sweep — still not a HIP number.

  ROCm / HIP

  HIP decode and prefill for the house 35B sweep stay pending — not measured on the local desktop. Do not copy a community HIP tok/s onto this page. Measure with amdai-bench.

• Multi-GPU

  cited

  Vulkan

  Cited walk is /setup/multi-gpu: --list-devices, then --device or GGML_VK_VISIBLE_DEVICES. Official default --split-mode layer. Overnight 35B split cards are llama.cpp Vulkan. Combo-preset tok/s is not measured.

  ROCm / HIP

  HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES after rocminfo. Same llama.cpp --split-mode flags. HIP split of the house 35B is pending — not measured on the local desktop. Mixing RDNA gens stays a /multi-gpu question.

• Flash attention and KV type

  cited

  Vulkan

  Overnight 35B llama-bench cards recorded -fa 1 and -ctk f16 -ctv f16. That is the house Vulkan pin. llama-server flags are not on that card — copy -fa only when your build documents it. The shopper page is /flash-attention. q8_0 KV is listed on the calculator; quality and tok/s stay not measured yet.

  ROCm / HIP

  llama.cpp documents the same --cache-type-k/v and -fa flags on the HIP build. HIP -fa / q8_0 KV for the house 35B sweep stay pending — not measured on the local desktop. Do not invent a HIP flash-attn delta. Shopper page: /flash-attention.

• Windows / WSL

  cited

  Vulkan

  Native Windows first token is llama.cpp Vulkan (LunarG SDK, GGML_VULKAN=ON, or a Windows x64 Vulkan release). Cited at /setup/windows. Not measured on the local desktop.

  ROCm / HIP

  ROCm 10.1 lists WSL2 as a tech preview. llama.cpp’s documented GPU path in a Linux guest is HIP (GGML_HIP=ON). We do not invent Vulkan-in-WSL. Native Linux at /setup stays the measured path. Cited at /setup/wsl.

Supported GPUs

Official HIP follows the ROCm 10.1 matrix. Vulkan is the always-works consumer path. gfx1031 HSA_OVERRIDE stays unofficial.

• RDNA 4

  gfx1201 / gfx1200

  Vulkan

  Always-works consumer path. House R9700 (gfx1201) daily driver is Vulkan.

  ROCm / HIP

  Official on the ROCm 10.1 matrix. Compile only after rocminfo prints this gfx. hipErrorNoBinaryForGpu means the binary missed gfx1201.

  Radeon AI PRO R9700 is the measured 32 GB SKU. RX 9070-class is gfx1201 / gfx1200 — Vulkan first, HIP second.

  • R9700
  • RX 9070 XT

• RDNA 3

  gfx110x

  Vulkan

  Always-works consumer path. Mesa RADV + GGML_VULKAN=ON.

  ROCm / HIP

  Official on the ROCm 10.1 matrix for listed gfx1100 / gfx1101 / gfx1102 rows. Pin AMDGPU_TARGETS to the string rocminfo prints. Do not paste gfx1201 onto a 7900.

  Decode on these cards is not a local desktop 35B HIP card. Size Q6/Q5/Q4 on the calculator.

  • RX 7900 XTX
  • RX 7800 XT

• RDNA 2

  gfx1030 / gfx1031

  Vulkan

  Always-works path on used 6000-series. Prefer Vulkan if HIP is angry.

  ROCm / HIP

  Consumer RDNA 2 is unofficial on the current Radeon ROCm row the way W6800 (gfx1030) is official. gfx1031 is not that W6800 row. Community HSA_OVERRIDE_GFX_VERSION=10.3.0 is unofficial — not an AMD how-to.

  House 35B Q6/Q5/Q4 files expect RAM offload on 12–16 GB cards. Both RAM lines stay labeled estimate.

  • RX 6800 XT
  • gfx1031 HSA_OVERRIDE

Measured on the local desktop

Official llama.cpp Vulkan cells from existing run cards. The HIP column stays pending — not measured on the local desktop until a first-party card lands. UD-Q3_K_XL is a labeled estimate. Full matrix: /benchmarks.

Build and run — llama.cpp first

Flags come from llama.cpp build.md and the house 35B recipe. HIP run flags match Vulkan; that is not a measured HIP card.

Vulkan compile

# Vulkan — house daily-driver compile (llama.cpp build.md)
sudo apt install -y build-essential cmake git libvulkan-dev
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build-vk -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build-vk -j"$(nproc)"
./build-vk/bin/llama-cli --list-devices
# Pick RADV GFX1201 / the discrete card, not the iGPU.

HIP compile

# HIP sibling — only after rocminfo lists this gfx (llama.cpp build.md)
cmake -B build-hip -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1201 -DCMAKE_BUILD_TYPE=Release
cmake --build build-hip -j"$(nproc)"
# Prefix may be /opt/rocm/core-10.0 — ls first.
export HIP_PATH=/opt/rocm
export PATH="/opt/rocm/bin:$PATH"
./build-hip/bin/llama-cli --list-devices

House Vulkan serve

# House 35B shape — /recipes/qwen36-35b.llamacpp-vulkan.262k
./build-vk/bin/llama-server \
  -m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

HIP serve (same flags, unmeasured)

# Same GGUF flags as the Vulkan house pin — HIP binary, not a measured HIP card.
./build-hip/bin/llama-server \
  -m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Walkthrough: Run a local Qwen · llama.cpp backend · HIP no binary.

System RAM

Backend choice does not change the shopper RAM labels. House 35B UD-Q5_K_XL on one 32 GB R9700 does not offload. When -ngl leaves layers on the host — typical on a 16 GB card with a house 35B Q4–Q6 file — print Minimum system RAM needed (full system estimate: offloaded weights + OS / cache / headroom, rounded up to a real kit, labeled estimate until measured) and RAM needed at full context (262K with cache; measured numbers only when a card exists).

Measure it on your card

Stack Lab planner is the allowlisted backend A/B. Record an amdai-bench card on your box — Vulkan build-vk versus HIP build-hip, one knob at a time. The house daily recipe is the measured Vulkan pin. There is no invented HIP recipe id. HIP cells on this page stay pending until a first-party local desktop card lands.

FAQ

Which llama.cpp backend should I compile first on a Radeon?

Vulkan. House daily driver is Qwen3.6-35B-A3B UD-Q5_K_XL on one R9700, llama.cpp Vulkan, 262K, f16 KV. HIP is the second compile after rocminfo lists this gfx. Do not pick HIP to chase an unmeasured tok/s.

First-run setup · llama.cpp · Vulkan vs HIP guide

Is HIP faster than Vulkan on RDNA 4?

HIP decode and prefill for the house 35B sweep stay pending — not measured on the local desktop. Measured numbers on this site are Vulkan. Measure both builds with Stack Lab planner and amdai-bench. Do not copy a community HIP figure onto a shopper page.

Stack Lab · amdai-bench · Benchmarks

My used 6700 XT is gfx1031. Do I need HSA_OVERRIDE?

Not as an AMDai how-to. Consumer RDNA 2 is unofficial on the current Radeon ROCm row. gfx1031 is not the official W6800 gfx1030 row. Community HSA_OVERRIDE_GFX_VERSION=10.3.0 is unofficial. Prefer llama.cpp Vulkan. Q6/Q5/Q4 plus a labeled Q3 estimate. Both RAM lines print if the 35B file spills.

gfx1031 HSA_OVERRIDE · Buying a used Radeon · RX 6700 XT

Sources

• llama.cpp repository
• llama.cpp — build.md (GGML_VULKAN / GGML_HIP)
• llama.cpp server README
• AMD — ROCm 10.1.0 compatibility matrix
• AMD — ROCm documentation
• AMD — Radeon / Ryzen ROCm docs
• AMD — ROCm 10.1 blog (WSL2 preview)
• AMDai — Vulkan vs HIP (ROCm guide)
• AMDai — 35B benchmarks