Cited, not measured on the local desktop

Run Ollama on a Radeon GPU

Ollama is a packaged llama.cpp / ROCm (or Vulkan) runtime with a local API on http://127.0.0.1:11434/v1. Ubuntu 24.04+ llama.cpp you compile is still the measured AMDai path — /setup. This page is the easy on-ramp: official install script, house family Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf or library qwen3.6:35b-a3b-q4_K_M. We do not invent an Ollama tok/s.

What Ollama gives you by default

Qwen3.6-35B-A3B library tag q4_K_M. Fit numbers use this site’s 256K total for Q4_K_M (25.6 GB, f16 KV) plus about 1 GB for the vision projector Ollama’s tag includes (estimate). One 32 GB card starts at 32K until you set OLLAMA_CONTEXT_LENGTH=262144.

Ollama default context by card VRAM for Qwen3.6-35B Q4_K_M
Your card(s) Ollama default context 35B Q4_K_M at 256K fully on GPU?
RX 9060 XT / 9070 / 9070 XT (16 GB) 4K No. It spills to system RAM and runs slower.
RX 7900 XTX (24 GB) 32K No at 256K (needs ~26–27 GiB, estimate). Fits at 32K.
Radeon AI PRO R9700 / PRO W7800 (32 GB) 32K Yes at 256K once you set OLLAMA_CONTEXT_LENGTH=262144.
PRO W7900 (48 GB) or two R9700s 256K Yes, by default. A second R9700 is not needed for this model.

llama.cpp first

Try this

Plain llama.cpp — the AMDai daily driver

Ollama is a packaged llama.cpp / ROCm (or Vulkan) runtime you do not compile. The measured path on this site is still a llama.cpp tree you built on Ubuntu 24.04+ — Vulkan first, HIP second. Use that when you want the house flags (-ngl 99, -c 262144, f16 KV, :8080) and the local desktop cards. Ollama is the easy on-ramp. Prefer /setup and /backends/llamacpp when gfx1201 tensiles miss, Windows ROCm stays on CPU, or you need a matching recipe. Link /rocm/vulkan-hip. Not measured inside Ollama.

Easy on-ramp, not the measured path

Ollama — pull-and-run on :11434

Official Linux install is curl -fsSL https://ollama.com/install.sh | sh. AMD extra: the ROCm tarball curl -fsSL https://ollama.com/download/ollama-linux-amd64-rocm.tar.zst | sudo tar x -C /usr. gpu.mdx: Linux ROCm v7; Linux card table includes R9700, RX 9070 XT, 7900 XTX, 7800 XT, and the rest of that list. Windows ROCm table is the 7900 / 7800 / 7700 / 7600 and W79xx class — it does not name the R9700 or 9070 series. Vulkan is additional support and is enabled by default when that backend is installed. Prove the GPU with ollama ps (100% GPU vs 100% CPU) and a meter — silent CPU fallback is the cited failure. Not measured on the local desktop.

Zero to first token

  1. 1 · Cited, not measured on the local desktop

    Linux install on Ubuntu 24.04+

    Official Linux install: curl -fsSL https://ollama.com/install.sh | sh. Manual tarball is also documented. AMD GPU extra: curl -fsSL https://ollama.com/download/ollama-linux-amd64-rocm.tar.zst | sudo tar x -C /usr. gpu.mdx: Ollama requires the AMD ROCm v7 driver on Linux — install or upgrade with amdgpu-install from AMD’s ROCm docs. Ubuntu 24.04.5 is the measured AMDai llama.cpp OS; that is not an Ollama test claim. Start with ollama serve (or systemctl start ollama). Confirm ollama -v and curl -s http://127.0.0.1:11434/api/tags. Keep the bind on 127.0.0.1 unless you set OLLAMA_HOST. Fedora and Arch can run the same script as a cited try — not verified on our rig. Not measured on the local desktop.

  2. 2 · Cited, not measured on the local desktop

    Windows note — short, then back to Linux

    Ollama documents a native Windows app (Windows 10 22H2+). API on http://localhost:11434. Windows ROCm needs an AMD ROCm v7 / HIP7-capable driver, or a Vulkan-capable Radeon driver. Official Windows ROCm card table is 7900 / 7800 / 7700 / 7600 and W79xx — not the R9700 or 9070 series. gpu.mdx Windows table matches that shorter list. Cited Windows R9700 pain: Ollama detects the card, fails to init ROCm, falls back to CPU — /rocm-problems/ollama-r9700-windows. Workaround on that page: llama.cpp Vulkan or OLLAMA_VULKAN=1. Do not invent HSA_OVERRIDE on gfx1201. Official 35B cards on this site are Ubuntu llama.cpp, not Windows Ollama. Not measured on the local desktop.

  3. 3 · Cited, not measured on the local desktop

    How Ollama picks ROCm vs CPU on consumer Radeon

    gpu.mdx Linux ROCm table includes RDNA 4 (9070 XT / R9700 / gfx1201), RDNA 3 (7900 / 7800 / 7700 / 7600), and RDNA 2 names on that list. Listing a card ≠ “works on my R9700 tonight.” If ROCm kernels miss, Ollama falls back to CPU — ollama ps will say 100% CPU. Vulkan is additional support (enabled by default when installed); disable with OLLAMA_VULKAN=0. gfx1201 Linux container pain is hipBLASLt looking for TensileLibrary_lazy_gfx1201.dat in the wrong folder — /rocm-problems/ollama-gfx1201-tensile, do not rewrite it here. gfx1031 (RX 6700 XT) community override is /rocm-problems/gfx1031-hsa-override; this site does not recommend spraying HSA_OVERRIDE_GFX_VERSION on gfx1201. Multi-GPU: official AMD selector is ROCR_VISIBLE_DEVICES (rocminfo UUIDs). Vulkan selector is GGML_VK_VISIBLE_DEVICES. Prefer one card first. When the meter stays idle, leave Ollama for llama.cpp Vulkan/HIP.

  4. 4 · Cited, not measured on the local desktop

    Pull or import Q6, Q5, or Q4 — Q3 is a labeled estimate

    House family is Qwen3.6-35B-A3B. Official library: ollama pull qwen3.6:35b or qwen3.6:35b-a3b-q4_K_M (Q4_K_M, 24 GB, 256K). That library page does not publish Q6_K, UD-Q5_K_XL, or a Q3 tag — import those GGUFs with a Modelfile FROM /path/to/file.gguf and ollama create (Ollama import docs; they do not re-quantize GGUF). Prefer Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf when the calculator says it fits. Q1/Q2 stay off. Stay on the house Qwen3.6 family already on /models. Decode numbers on this site are Ubuntu llama.cpp llama-bench, not a speed measured inside Ollama.

  5. 5 · Cited, not measured on the local desktop

    Stage models on fast NVMe, not a slow bulk mount

    Ollama FAQ: Linux default store is /usr/share/ollama/.ollama/models; Windows is %USERPROFILE%\.ollama\models. Point OLLAMA_MODELS at a fast NVMe directory (the ollama user needs write on Linux — chown -R ollama:ollama). A USB or spinning bulk mount will stall pulls and swaps; we do not invent a GB/s. Size the file on the calculator before the pull. Not measured on the local desktop.

  6. 6 · Cited, not measured on the local desktop

    Context, parallel, and KV — Ollama multiplies per request

    Ollama picks a default context from total VRAM (v0.40.13 source): under 23 GiB you get 4K, 23–47 GiB gets 32K, and 47 GiB or more gets 256K. The FAQ still says 4096. One 32 GB R9700 therefore starts at 32K, not 256K, even though the 35B fits at 256K on that card. Set it yourself: OLLAMA_CONTEXT_LENGTH=262144 ollama serve, then check ollama ps shows CONTEXT 262144. You can also use /set parameter num_ctx or API options.num_ctx. Parallel requests grow that window: FAQ example, 2K context × 4 parallel = 8K plus extra allocation. Required RAM scales by OLLAMA_NUM_PARALLEL × OLLAMA_CONTEXT_LENGTH. That is why the calculator’s Ollama backend sizes KV per request — open /calculator and pick Ollama. Flash Attention: OLLAMA_FLASH_ATTENTION=1 to force on, 0 to disable. KV type: OLLAMA_KV_CACHE_TYPE (default f16; q8_0 or q4_0 only with Flash Attention). House llama.cpp uses f16 KV. The calculator prints Minimum system RAM needed (rounded kit estimate with a breakdown — 192 or 256 GB when a 35B file spills) and RAM needed at full context when we have a measured llama.cpp peak (37.32 GB at 4K, 42.9 GB at 256K on UD-Q5). Those peaks are llama.cpp on the local desktop, not Ollama. Label estimate unless measured.

  7. 7 · Cited, not measured on the local desktop

    Prove the GPU, then point a harness at /v1

    ollama ps is the official check: 100% GPU, 100% CPU, or a split. curl -s http://127.0.0.1:11434/api/tags. OpenAI-shaped local path is http://127.0.0.1:11434/v1 (docs show /v1/chat/completions and /v1/responses). Point Hermes, Open WebUI, Aider, Cline, or Continue at that base — /harnesses/connect and /setup/coding-assistant. House Hermes on the local desktop talks to llama-server :8080, not a measured Ollama attach. Keep 127.0.0.1 unless you mean the LAN (OLLAMA_HOST). Do not present llama.cpp tok/s as an Ollama result.

  8. 8 · Cited, not measured on the local desktop

    When it sits on CPU

    ollama ps first. 100% CPU: ROCm kernels missed, Windows R9700 init failed, or Vulkan never loaded. journalctl -u ollama on Linux. Do not spray HSA_OVERRIDE_GFX_VERSION — unofficial here, and the wrong gfx1201 fix. Tensile 404 → /rocm-problems/ollama-gfx1201-tensile, then llama.cpp. Windows R9700 → /rocm-problems/ollama-r9700-windows. Start from /troubleshoot. The wrapper is not the measured daily driver.

Cited Linux install

Official Ollama Linux script, then prove ollama ps is not 100% CPU. 

curl -fsSL https://ollama.com/install.sh | sh

35B quant picker

Speeds are measured on llama.cpp directly, not inside Ollama. Official library Q4 tag plus imported house GGUFs for Q6 / Q5 / Q3.

35B GGUF quants for Ollama on Radeon, with llama.cpp measurements labeled as not from Ollama
Quant Role Ollama path llama.cpp on the local desktop (not Ollama)
Q6_K Higher-quality 35B sibling No qwen3.6 Q6_K tag on the official Ollama library tags page we checked. Import the Unsloth Q6_K GGUF already sized on /models/qwen36: FROM /path/to/Qwen3.6-35B-A3B-Q6_K.gguf then ollama create qwen36-q6. Not a speed measured inside Ollama. Measured on llama.cpp Vulkan on the local desktop, not inside Ollama: Q6_K 1-GPU 4K decode 103.23 tok/s (23:55 card). 32K+ is partly on DDR5.
UD-Q5_K_XL House default file on one R9700 Import the house GGUF so the quant is the one we measure: FROM ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf then ollama create qwen36-q5. Official library latest / qwen3.6:35b is 23–24 GB — do not assume that blob is UD-Q5_K_XL. Measured on llama.cpp Vulkan on the local desktop, not inside Ollama: 102.39 tok/s decode at 4K, 90.52 at 32K, 66.43 at 128K, 44.4 at 256K. Peak VRAM 25.97 / 29.34 GB; peak RAM 37.32 / 42.9 GB.
Q4_K_M Official Ollama library 35B Q4 Cited library tag qwen3.6:35b-a3b-q4_K_M (24 GB, 256K) on ollama.com/library/qwen3.6/tags. ollama pull qwen3.6:35b-a3b-q4_K_M. Same family as qwen3.6:35b. Measured on llama.cpp Vulkan on the local desktop, not inside Ollama: Q4_K_M 32K decode 117.69 tok/s. Still check both RAM lines.
UD-Q3_K_XL Estimate Cited Q3 option — labeled estimate Hugging Face listing for Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf (16.8 GB). No qwen3.6 Q3 tag on the official library tags page we checked. Import the listed Unsloth Q3 GGUF from /models/qwen36 if you need that size. We do not invent an Ollama Q3 speed. Q1/Q2 stay off. Labeled estimate unless a run card exists — none does for 35B Q3 on the local desktop.

RAM lines (llama.cpp-measured)

Wherever a file spills off the GPU, the two labels stay Minimum system RAM needed and RAM needed at full context. House 35B peaks below are Ubuntu llama.cpp, not Ollama. Ollama multiplies KV per parallel request — size that on the calculator.

Daily driver · 35B llama.cpp · 262K

Minimum system RAM needed

42.9 GB

Measured · 42.9 GiB raw · 256K context

local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16

RAM needed at full context

42.9 GB

Measured · 42.9 GiB raw · 256K full context

local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16

Measured peak system RAM on the local desktop: 42.9 GB (Linux GiB labeled GB). Measured, pending re-verify after RAM fix.

Breakdown
Offloaded weights ·
0 GiB
Separable embed in RAM ·
0 GiB
KV in RAM at selected ·
0 GiB
KV in RAM at full context ·
0 GiB
Runtime / backend ·
0.85 GiB
OS / desktop baseline ·
8.00 GiB
Headroom ·
16.0 GiB

Open in Stack Lab

Stack Lab remains Ubuntu llama.cpp-first. There is no Ollama package and no public download.

Open in Stack Lab

If the app is not installed, packages are not published yet. There is no public download URL. Read the Stack Lab install notes . If Stack Lab is already running on this machine, open the local app .

Sources

How we test — Measured cells cite official amdai-bench/run@1 cards. Estimates stay labeled. Q3 is estimate-only. Q1/Q2 stay off.

Next steps

Real pages on this site. We do not invent a next click.

Last updated

esc

Type to filter. Enter opens the first match. Full-text search lives on /search.