Out of VRAM? Fix llama.cpp memory errors on a Radeon GPU

Match the log

Out of VRAM is a fit problem: weights plus KV plus compute buffers plus whatever else is already on the card. Short context can load when long context cannot. The existing /troubleshoot#oom-context row is the long-context symptom; this page is the shopper ladder that row now points at. Prefer a smaller -c or q8_0 KV before you drop the weight quant. HIP / ROCm is the cited second compile — same flags, different error prefix (ROCm0 instead of Vulkan0).

• llama.cpp Vulkan

  “ggml_vulkan: Requested buffer size exceeds device memory allocation limit: ErrorOutOfDeviceMemory”

  llama.cpp #14836 (also #15120)

• llama.cpp Vulkan

  “ggml_vulkan: vk::Device::allocateMemory: ErrorOutOfDeviceMemory”

  llama.cpp #9271

• llama.cpp Vulkan

  “ggml_gallocr_reserve_n: failed to allocate Vulkan0 buffer of size …”

  llama.cpp #14836

• llama.cpp Vulkan

  “failed to allocate compute pp buffers”

  llama.cpp #14836

• llama.cpp HIP / ROCm

  “ggml_backend_cuda_buffer_type_alloc_buffer: allocating … MiB on device 0: cudaMalloc failed: out of memory”

  llama.cpp #16725

• llama.cpp HIP / ROCm

  “ggml_gallocr_reserve_n: failed to allocate ROCm0 buffer of size …”

  llama.cpp #16725

• llama.cpp HIP / ROCm

  “alloc_tensor_range: failed to allocate ROCm0 buffer of size …”

  llama.cpp #16725

• llama.cpp HIP / ROCm

  “hipStreamCreateWithFlags: Returned hipErrorOutOfMemory”

  llama.cpp #8618

• Ollama

  “model requires more system memory (X GiB) than is available (Y GiB)”

  Ollama #8734

• LM Studio

  “LM Studio wraps llama.cpp — AMD shoppers still see the Vulkan or HIP strings above (ErrorOutOfDeviceMemory / failed to allocate ROCm0 buffer / hipErrorOutOfMemory).”

  AMD LM Studio playbook + llama.cpp issues

How to check VRAM

Fix ladder — cheapest first

1. 1 · -c / --ctx-size

   Lower -c / --ctx-size first

   Context grows the KV cache. Official CLI is -c / --ctx-size. House daily driver is 262144; a cited OOM at long context is often an 8K or 32K problem, not a “buy a new card” problem. Size the window on the calculator before you change the GGUF. The existing /troubleshoot#oom-context row is this symptom.

   • Long context
   • Calculator
   • Long-context OOM

2. 2 · -ctk / -ctv q8_0 · needs -fa

   Try q8_0 KV (-ctk / -ctv) with -fa on

   Official flags are -ctk / --cache-type-k and -ctv / --cache-type-v. House default is f16. q8_0 shrinks the estimated cache. llama.cpp multi-gpu.md requires flash attention for a quantized V cache — turn -fa on before -ctv q8_0. Quality and tok/s for q8_0 KV stay not measured yet. Do not confuse a KV type with a weight quant.

   • KV cache
   • Flash Attention
   • Quants · KV types

3. 3 · -fa / --flash-attn on

   Turn flash attention on (-fa on)

   Official CLI is -fa / --flash-attn [on|off|auto] (default auto). Overnight house 35B Vulkan llama-bench recorded -fa 1. llama.cpp multi-gpu.md requires FA for --split-mode tensor and/or a quantized V cache. llama-server FA on versus off stays not measured — do not invent a VRAM delta. Shopper page: /flash-attention.

   • Flash Attention
   • Speed up llama.cpp
   • Multi-GPU llama.cpp

4. 4 · -b / --batch-size and -ub / --ubatch-size

   Cut -ub / -b when the compute buffer OOMs

   Official flags are -b / --batch-size and -ub / --ubatch-size. llama.cpp #14836 failed on a large Vulkan compute buffer with -b 4096 -ub 4096 (“failed to allocate compute pp buffers”). Cutting the micro-batch shrinks that transient. Overnight 35B llama-bench cards are not a llama-server -b pin — copy a smaller -ub only when your log shows a compute-buffer allocate.

   • Speed up llama.cpp
   • llama.cpp

5. 5 · --n-cpu-moe / -ot

   MoE only: park experts on CPU (--n-cpu-moe / -ot)

   A llama.cpp MoE trick: keep attention on the GPU and park expert tensors in system RAM. The flags are -ot (tensor override regex) and --n-cpu-moe (how many experts stay on CPU). Exact regex and N from the rig owner are not recorded — do not invent flag values. House 35B on one R9700 does not offload. Flash-class files usually do. Print both RAM lines: Minimum system RAM needed and RAM needed at full context. Those lines are labeled estimate unless a measured card overlays the kit.

   • MoE
   • Flash RAM offload
   • Expert offload

6. 6 · -ngl / --n-gpu-layers

   Partial -ngl — leave some layers on DDR5

   -ngl is how many transformer layers llama.cpp tries to keep on the GPU. -ngl 99 is the house “offload what fits” pin. Partial -ngl is the 16 GB / 12 GB move when Q4 still spills. Leftover layers are system RAM — print Minimum system RAM needed and RAM needed at full context. House daily driver on one R9700 uses -ngl 99 and 0% RAM offload. Partial -ngl tok/s is not measured yet on the local desktop except where a card already names --fit-target.

   • Speed up llama.cpp
   • Offload
   • Calculator

7. 7 · --split-mode / -sm and --tensor-split / -ts

   Split across GPUs (-ts / --split-mode)

   Official default is --split-mode layer (short -sm layer). Overnight 35B split on the local desktop used layer mode. --tensor-split / -ts and --split-mode row are documented and not measured on the local desktop. llama.cpp multi-gpu.md requires FA for --split-mode tensor. One R9700 usually decodes the house 35B faster than the split. Read Dual / multi-GPU before you split. Do not invent a -ts pair.

   • Multi-GPU llama.cpp
   • Dual / multi-GPU
   • One vs split

8. 8 · desktop compositor · browser · second llama-server

   Close other VRAM users, then reload

   llama.cpp Vulkan prints free MiB on load (#14836). A second llama-server, a browser, or a desktop compositor reduces that free figure. Ollama #8734’s cited reply is “close some programs.” Watch rocm-smi / amd-smi while you load. Do not invent a GB leftover for the compositor. Then retry the same flags.

   • Troubleshoot
   • Always-on server
   • R9700

9. 9 · Q6_K → UD-Q5_K_XL → Q4_K_M

   Then step down the weight quant: Q6 → Q5 → Q4

   Last cheap move is a smaller GGUF, not a smaller KV type. House recipes stay Q4 and up. Overnight one-client llama-bench exists for Q6, house UD-Q5, and Q4 on one R9700. UD-Q3_K_XL is a labeled estimate (Hugging Face listing for Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf (16.8 GB)) — do not download Q3 to run. Q1/Q2 stay off.

   • Q6 / Q5 / Q4
   • Make GGUF quants
   • Qwen3.6-35B

Copy-paste after a cited OOM

Vulkan house path. This block is a cheaper-context, q8_0 KV start — not a measured card. Replace the binary and GGUF paths. House recipes stay Q4 and up.

# Cheapest first after a cited OOM. Replace paths.
# House recipes stay Q4 and up. Do not invent --n-cpu-moe N or -ot regex.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 8192 -fa on \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  -b 512 -ub 512 --jinja \
  --host 127.0.0.1 --port 8080

Q6 / Q5 / Q4 plus a labeled Q3

Step down the weight quant only after the cheaper flags. House recipes stay Q4 and up. UD-Q3_K_XL is a labeled estimate — do not download it to run.

• Q6_K

  Qwen3.6-35B-A3B-Q6_K.gguf. Step down from Q6 only after the cheaper flags. Overnight one-client llama-bench exists on one R9700.

• UD-Q5_K_XL

  Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf. House file. Measured peak VRAM 25.97 GB at 4K and 29.34 GB at 256K on one R9700 — in-VRAM at 262144 f16 KV.

• Q4_K_M

  Qwen3.6-35B-A3B-Q4_K_M.gguf. Recommended floor for house recipes. Last weight-quant step on this ladder. Overnight one-client llama-bench exists.

• UD-Q3_K_XL · labeled estimate

  Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf. Labeled estimate (Hugging Face listing for Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf (16.8 GB)). Do not download Q3 to run. Q3 does not buy VRAM. Q1/Q2 stay off.

RAM lines (Ubuntu-measured)

When -ngl leaves layers on the host, or when MoE experts sit in DDR5, the spill is system RAM. Shopper lines stay Minimum system RAM needed and RAM needed at full context. On one R9700 with the house UD-Q5 those lines are a 64 GB kit from the measured 37.32 GB peak at 4K and the measured 42.9 GB peak at 256K (Linux GiB labeled GB). That house card is 0% RAM offload — do not treat it as a Flash or partial-ngl recipe. Other files stay labeled estimate unless a measured card overlays the kit. UD-Q3_K_XL is a labeled estimate. Do not download Q3 to run.

Open in Stack Lab

There is no allowlisted out-of-VRAM deep link. Use the planner. Packages are not published yet and there is no public download.

Sources

• llama.cpp #14836 — ErrorOutOfDeviceMemory / failed to allocate Vulkan0 buffer / compute pp buffers
• llama.cpp #15120 — Requested buffer size exceeds device memory allocation limit: ErrorOutOfDeviceMemory
• llama.cpp #9271 — vk::Device::allocateMemory: ErrorOutOfDeviceMemory
• llama.cpp #16725 — cudaMalloc failed: out of memory / failed to allocate ROCm0 buffer
• llama.cpp #8618 — hipStreamCreateWithFlags: Returned hipErrorOutOfMemory
• Ollama #8734 — model requires more system memory (X GiB) than is available (Y GiB)
• llama.cpp — CLI README (-c, -ctk/-ctv, -fa, -b/-ub, -ngl)
• llama.cpp — multi-gpu.md (--split-mode, --tensor-split, FA for quantized V)
• llama.cpp — llama-server README
• llama.cpp repository
• AMD — Running and Serving LLMs with LM Studio
• LM Studio — lms load (GPU offload and context length)
• radeontop — community Radeon GPU utilization viewer — unofficial for llama.cpp
• ROCm 10.1 blog — amd-smi succeeds rocm-smi
• Qwen3.6-35B-A3B official model card