ROCm vs Vulkan vs HIP for local LLMs

Three words that get mixed up

ROCm is the platform: driver, runtime, compiler, libraries, wheels.HIP is the C++ API and kernel language inside that platform.^[2]Vulkan is a Khronos graphics-and-compute API. llama.cpp can speak HIPor Vulkan. They are different compilers pointing at the same VRAM.

llama.cpp lives at ggml-org/llama.cpp and can be built with a HIP backend (GGML_HIP) or a Vulkan backend (GGML_VULKAN).^[1]The consumer default on this site is llama.cpp. Thebackend chooser is the short version of this page.

When llama.cpp Vulkan wins

Vulkan talks to the desktop GPU stack you already needed for a picture on the monitor — RADV on typical Linux, or the vendor ICD on Windows. You do not need a matching/opt/rocm tree to get the first tokens.

• Tokens tonight. HIP CMake cannot find a ROCm that rocminfoinsists is fine. Vulkan configure is the weekend path.
• Windows. A full HIP userspace may not be what you have. Adrenalin + Vulkan often is. ROCm 10.1 adds a WSL2 tech preview; that is still not “just install the Linux tutorial on Windows.”^[5]
• iGPU vs dGPU. llama-cli --list-devices showingRADV GFX1201 next to an iGPU is easier to read than a HIP agent list you have not learned yet. Pick the discrete card.
• This site’s daily driver. Qwen3.6-35B-A3B Unsloth UD-Q5_K_XL onone R9700, llama.cpp Vulkan, 262K context, f16 KV, 0% RAM offload. That is first-party, from our test rig. Measured on our rig, 2026-10 — UD-Q5_K_XL decode100.8 tok/s short context, ~88 at 32K,~50 at 256K. UD-Q5_K_XL VRAM stays TODO.

When HIP / ROCm wins

Install and use HIP/ROCm when you:

• Want PyTorch the way AMD documents it.^[3]
• Want vLLM (or another ROCm serve stack) with matching wheels — batching, several clients, a harness that expects a real server.^[4]
• Care about amd-smi (successor to rocm-smi as of the 10.1 blog) the way a CUDA user cares about nvidia-smi.^[5]
• Need to debug library-level issues (rocBLAS / hipBLASLt kernels, missinggfx1201 tensiles) instead of a Vulkan shader path. Seethe stack.
• Are compiling llama.cpp HIP so one tree matches the ROCm prefix you already maintain (-DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1201). Prefix may be/opt/rocm/core-10.0 — ls first.

AMD’s Radeon/Ryzen ROCm docs are the consumer-facing door for that ecosystem.^[6]vLLM is still advanced on this site — not the first home box. Pins live onversions.

What we run on the test rig

Keep both builds

On Linux it is reasonable to have:

1. A distro or AMD graphics stack with a working RADV ICD.
2. One ROCm userspace for HIP apps.
3. Two llama.cpp build directories: build-hip and build-vk.

Do not declare a winner across different models, different -c, and different days. Measure to learn which layer you broke. Walkthrough:run a local LLM. Fit:calculator. Logs:error paste.