What it is
Pull-and-run daemon with model tags and a local API. Under the hood it is a packaged llama.cpp / ROCm (or Vulkan) runtime you do not compile.
Architecture in plain English
ollama serve on :11434. Native /api/chat plus an OpenAI-shaped /v1. GPU attach is whatever they bundled. You do not pick GGML_VULKAN vs HIP the way you do in a tree you built.
What it does well
Convenience, if a GPU meter moves.
What it does badly
gfx1201 as the first backend; Windows ROCm; silent CPU fallback.
ROCm / Radeon
Official gpu.mdx lists gfx1201 (example GPU RX 9070 XT). Cited pain remains: #12908 / #11555 hipBLASLt TensileLibrary_lazy_gfx1201.dat in the wrong directory; Windows ROCm lagged Linux (6.2 vs 7.2 in #14391). Listing gfx1201 ≠ “works on my R9700 tonight.”
Vulkan. Optional / emerging in their roadmap comments. Prefer a Vulkan llama.cpp you built if ROCm attach fails.
Supported quants
Their library tags. Floor is still Q4. Do not pull a Q2 “to make it fit.”
House floor is still Q4. Prefer Q5. See quants and thecalculator.
Install on an R9700
# Official Linux install is their documented script — then prove GPU use.
# curl -fsSL https://ollama.com/install.sh | sh
# Prefer the Vulkan llama.cpp you already built if this sits on CPU.
Run
ollama serve
# ollama run <tag> # only after rocm-smi / amd-smi moves
curl -s http://127.0.0.1:11434/api/tags
Tuning flags
Do not spray HSA_OVERRIDE_GFX_VERSION. That variable is unofficial on this site. If the tensile file 404s, come back to llama.cpp.
When to choose it
Only if the meter moves. Otherwise llama.cpp.
You want the wrapper and you will read the backend log.