Cited, not measured on the local desktop

Run Open WebUI on a Radeon GPU

Open WebUI is a self-hosted browser chat UI. It does not run the GGUF. Point it at llama.cpp llama-server http://127.0.0.1:8080/v1. Ubuntu 24.04+ Vulkan is the measured path — /setup. House file Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf. Ollama is secondary — /setup/ollama. We do not invent an Open WebUI speed or price.

llama.cpp first

Try this

llama-server /v1 — the first connection

Open WebUI has no models of its own. Official llama.cpp guide: Settings → Admin → Connections → OpenAI, add a connection, URL ending in /v1, Provider llama.cpp. House daily-driver flags: ./build-vk/bin/llama-server -m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --host 127.0.0.1 --port 8080. Their example uses --port 10000; we pin --port 8080 so this site’s harnesses stay on http://127.0.0.1:8080/v1. GPU work is llama.cpp Vulkan (HIP second) on /rocm/vulkan-hip — not an Open WebUI ROCm claim. Docker UI uses http://host.docker.internal:8080/v1. Do not take the :cuda image or a CUDA Ollama blob on a Radeon. Not measured on the local desktop.

Secondary path

Ollama — secondary /v1 only

Open WebUI’s default Docker image looks for Ollama at http://host.docker.internal:11434. That is a convenience, not the house stack. House recipes use llama.cpp. If you already followed /setup/ollama, you can add that URL as an Ollama connection — still prove ollama ps is not 100% CPU, and do not present llama.cpp tok/s as an Ollama or Open WebUI result. Not measured on the local desktop.

Zero to first chat

  1. 1 · Cited, not measured on the local desktop

    Start llama-server first — Vulkan on Ubuntu 24.04+

    Build llama.cpp from /setup, then serve the house file: ./build-vk/bin/llama-server -m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --host 127.0.0.1 --port 8080. Those are the same flags as the 35B recipe. HIP is the ROCm compile on Vulkan vs HIP — not the first token. Confirm curl -s http://127.0.0.1:8080/v1/models before you open a browser. Open WebUI’s llama.cpp page uses --port 10000 and --ctx-size 1024 as an example; house context is 262144. The GPU path is Mesa RADV / Vulkan or AMD ROCm HIP on the llama.cpp process, not inside the chat UI.

  2. 2 · Cited, not measured on the local desktop

    Install Open WebUI — Docker first, pip second, Podman cited

    Official Quick Start: Docker is the recommended path. docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data -e WEBUI_SECRET_KEY=$(openssl rand -hex 32) --name open-webui --restart always ghcr.io/open-webui/open-webui:main. UI is http://127.0.0.1:3000 (container 8080 mapped to host 3000). Keep :main or :slim. Do not pull :cuda on a Radeon. pip / uv: pip install open-webui, then open-webui serve --port 3000 — official default is localhost:8080, which collides with house llama-server; --port 3000 is documented. Their pip note about a ROCm PyTorch wheel is for Open WebUI’s own embeddings / speech stack, not the chat model. Chat tokens stay on llama-server. Podman: official podman run on 3000:8080; reach the host with http://host.containers.internal:8080/v1. Generate WEBUI_SECRET_KEY with openssl rand -hex 32. Not measured on the local desktop.

  3. 3 · Cited, not measured on the local desktop

    Point Connections at llama-server /v1

    Avatar → Settings → Admin → Connections → OpenAI → Add Connection. URL: http://127.0.0.1:8080/v1 on pip, http://host.docker.internal:8080/v1 in Docker, http://host.containers.internal:8080/v1 in Podman. Note the /v1. Auth none / placeholder key if the server has no auth. Advanced → Provider llama.cpp unlocks the Loaded marker and Eject. Models appear after the connection saves. Official LM Studio page uses the same OpenAI-shaped fields at :1234 — secondary only, /setup/lm-studio. House Hermes on the local desktop talks to :8080, not a measured Open WebUI attach.

  4. 4 · Cited, not measured on the local desktop

    Pick Q6, Q5, or Q4 — Q3 is a labeled estimate

    House file is Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf on one R9700. Size 24 GB / 16 GB cards on the calculator before you download. Start at Q6, Q5, or Q4. A cited Q3 file is a labeled estimate. Open WebUI’s llama.cpp page demos an IQ1 DeepSeek pull — ignore that for this site. Stay on the house Qwen3.6 family already on /models. Decode numbers on /benchmarks are Ubuntu llama.cpp llama-bench, not Open WebUI.

  5. 5 · Cited, not measured on the local desktop

    Stage the GGUF on fast NVMe — never /mnt/bulk

    The chat UI does not load weights. llama-server does. Keep the GGUF on fast NVMe (house ~/models), not a spinning disk and not a slow bulk mount such as /mnt/bulk. Prefill stalls with an idle GPU meter are often the disk. Staging is not RAM offload. Docker volume open-webui is UI data (chats, users) — that is not the model file. Not measured on the local desktop.

  6. 6 · Cited, not measured on the local desktop

    Context and both RAM lines stay on the server

    Open WebUI’s UI does not replace llama-server -c. House window is 262144. Size it on the calculator. Wherever a file spills, the two labels stay Minimum system RAM needed (rounded kit estimate with a breakdown — 192 or 256 GB when a 35B file spills) and RAM needed at full context when we have a measured llama.cpp peak (37.32 GB at 4K, 42.9 GB at 256K on UD-Q5). Those peaks are llama.cpp on the local desktop, not Open WebUI. Label estimate unless measured.

  7. 7 · Cited, not measured on the local desktop

    Ollama and LM Studio stay secondary

    If Ollama is already serving, Open WebUI can use http://127.0.0.1:11434 (pip) or host.docker.internal:11434 (Docker). That is their default discovery, not the house recipe. LM Studio :1234 is the same OpenAI-shaped form. Prefer llama-server when you want this site’s flags and cards. Do not present llama.cpp tok/s as an Open WebUI, Ollama, or LM Studio result.

  8. 8 · Cited, not measured on the local desktop

    When the model list is empty

    curl /v1/models on llama-server first. Empty: server not up, or you used 127.0.0.1 from inside Docker — switch to host.docker.internal. Port clash: pip default :8080 vs house llama-server — use --port 3000. :cuda image will not help a Radeon. Timeout on slow loads: official AIOHTTP_CLIENT_TIMEOUT_MODEL_LIST. Start from /troubleshoot. Cited ROCm rows live on /rocm-problems.

Cited commands

House llama-server, then official Open WebUI Docker. pip uses open-webui serve --port 3000 so it does not steal :8080. 

./build-vk/bin/llama-server -m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --host 127.0.0.1 --port 8080
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data -e WEBUI_SECRET_KEY=$(openssl rand -hex 32) --name open-webui --restart always ghcr.io/open-webui/open-webui:main

35B quant picker

Speeds are measured on llama.cpp directly, not inside Open WebUI.

35B GGUF quants for Open WebUI on Radeon, with llama.cpp measurements labeled as not from Open WebUI
Quant Role llama.cpp on the local desktop (not Open WebUI)
Q6_K Higher-quality 35B sibling Measured on llama.cpp Vulkan on the local desktop, not inside Open WebUI: Q6_K 1-GPU 4K decode 103.23 tok/s (23:55 card). 32K+ is partly on DDR5. Size VRAM on the calculator.
UD-Q5_K_XL House default on one R9700 Measured on llama.cpp Vulkan on the local desktop, not inside Open WebUI: 102.39 tok/s decode at 4K, 90.52 at 32K, 66.43 at 128K, 44.4 at 256K. Peak VRAM 25.97 / 29.34 GB; peak RAM 37.32 / 42.9 GB. File Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf.
Q4_K_M Smaller measured 35B sibling Measured on llama.cpp Vulkan on the local desktop, not inside Open WebUI: Q4_K_M 32K decode 117.69 tok/s. Still check both RAM lines.
UD-Q3_K_XL Estimate Cited Q3 option — labeled estimate Hugging Face listing for Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf (16.8 GB). Official Open WebUI llama.cpp page shows an IQ1 DeepSeek example — that is their demo, not a house pick. Q1/Q2 stay off. We do not invent an Open WebUI Q3 speed.

RAM lines (llama.cpp-measured)

Wherever a file spills off the GPU, the two labels stay Minimum system RAM needed and RAM needed at full context. House 35B peaks below are Ubuntu llama.cpp, not Open WebUI.

Daily driver · 35B llama.cpp · 262K

Minimum system RAM needed

42.9 GB

Measured · 42.9 GiB raw · 256K context

local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16

RAM needed at full context

42.9 GB

Measured · 42.9 GiB raw · 256K full context

local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16

Measured peak system RAM on the local desktop: 42.9 GB (Linux GiB labeled GB). Measured, pending re-verify after RAM fix.

Breakdown
Offloaded weights ·
0 GiB
Separable embed in RAM ·
0 GiB
KV in RAM at selected ·
0 GiB
KV in RAM at full context ·
0 GiB
Runtime / backend ·
0.85 GiB
OS / desktop baseline ·
8.00 GiB
Headroom ·
16.0 GiB

Open in Stack Lab

Stack Lab remains Ubuntu llama.cpp-first. There is no Open WebUI package and no public download.

Open in Stack Lab

If the app is not installed, packages are not published yet. There is no public download URL. Read the Stack Lab install notes . If Stack Lab is already running on this machine, open the local app .

Sources

How we test — Measured cells cite official amdai-bench/run@1 cards. Estimates stay labeled. Q3 is estimate-only. Q1/Q2 stay off.

Next steps

Real pages on this site. We do not invent a next click.

Last updated

esc

Type to filter. Enter opens the first match. Full-text search lives on /search.