What's the biggest context I can run on a Radeon GPU with llama.cpp

Native max vs the real ceiling

Context length is the token window of prompt plus chat. llama.cpp sets it with -c / --ctx-size. The Qwen3.6-35B-A3B card lists 262144 native context. A 32 GB card that loads at 4K can still fail at 128K because the KV cache grew. The ceiling finder climbs a lean ladder toward that native max and stops at the first load or serve failure (OOM, alloc, or the process never comes up). It records decode, prefill, peak VRAM per card and total, and peak system RAM on every passing step, then the first fail reason. Do not copy the overnight 102.39 tok/s 4K decode cell as a ctx-ceiling winner. That number is llama-bench, one client, not ctx-ceiling/1.

Lean ladder, one GPU, all GPUs

Lean ladder starts at a short window — the same short-context load that feeds Minimum system RAM needed — and climbs toward the card’s native max. Native max is what the vendor published, not what your VRAM holds. One-GPU arm uses a single card. All-GPUs arm uses every visible device. The first OOM or load failure stops that climb; the last passing step is the winner. AMDai does not invent the step list — run ab-ctx-ceiling and read docs/ctx-ceiling.md in Stack Lab. A winner is not ingested on this site yet.

• Lean / short-context load

  First ladder step. Feeds Minimum system RAM needed after a kit round (64 / 96 / 128 / 192 / 256 GB).

  Winner: not yet measured

  Minimum system RAM needed: 64 GB kit from the measured 37.32 GB llama-bench peak at 4K — labeled estimate until a ctx-ceiling/1 lean step lands

  House 4K llama-bench is not a ctx-ceiling lean card. Breakdown: 0% offload + OS / cache / headroom → 64 GB.

• One GPU

  Climb on a single card toward native max. Stop at the first OOM or load failure.

  Winner: not yet measured

  RAM needed at full context: not yet measured on ctx-ceiling/1. Overnight house peak at 256K is 42.9 GB (llama-bench, not the finder).

  House overnight ran 262144 on one R9700 with UD-Q5, f16 KV. That is not a ctx-ceiling/1 winner.

• All GPUs

  Same climb across every visible device. Peak VRAM is per card and total.

  Winner: not yet measured

  RAM needed at full context: not yet measured on ctx-ceiling/1. Do not copy a split llama-bench cell as an all-GPUs ceiling.

  One vs split decode lives on /compare/one-vs-two-gpu. Those cards are not ctx-ceiling.

Cited house pin and lean load

Vulkan house path. The first block is the recorded llama-server line at native 262144 — not a ctx-ceiling/1 winner. The second is a lean 4K shape for the kit line. Replace the binary and GGUF paths.

# House daily driver as recorded (262144 ran on overnight llama-bench — not a ctx-ceiling winner).
# Official llama.cpp: -c / --ctx-size. Qwen3.6-35B-A3B card native max is 262144.
# Do not invent a ctx-ceiling/1 pin. Measure amdai-bench ab-ctx-ceiling on your box.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja --host 127.0.0.1 --port 8080

# Lean / short-context load — the kit line, not a local desktop pin.
# Size -c on the calculator before you copy it. 4096 is the house 4K llama-bench window.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 4096 --cache-type-k f16 --cache-type-v f16 --jinja \
  --host 127.0.0.1 --port 8080

Q6 / Q5 / Q4 plus a labeled Q3

A smaller GGUF can raise the window that fits, but house recipes stay Q4 and up. UD-Q3_K_XL is a labeled estimate — do not download it to run. Both RAM lines appear. Every ctx-ceiling/1 winner stays not yet measured.

• Q6_K

  Qwen3.6-35B-A3B-Q6_K.gguf

  Ceiling: not yet measured

  Minimum system RAM needed: lean-step peak not recorded on a ctx-ceiling/1 card

  RAM needed at full context: winner peak not recorded on a ctx-ceiling/1 card

  Overnight llama-bench exists. Recommended. A ctx-ceiling winner stays not yet measured.

• UD-Q5_K_XL

  Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

  Ceiling: not yet measured

  Minimum system RAM needed: 64 GB kit from the measured 37.32 GB peak at 4K (llama-bench, not ctx-ceiling/1)

  RAM needed at full context: measured 42.9 GB peak at 256K. ctx-ceiling/1 winner peak stays not yet measured.

  House file. One-client decode 102.39 / 90.52 / 66.43 / 44.4 tok/s at 4K / 32K / 128K / 256K is llama-bench, not a ceiling finder.

• Q4_K_M

  Qwen3.6-35B-A3B-Q4_K_M.gguf

  Ceiling: not yet measured

  Minimum system RAM needed: lean-step peak not recorded on a ctx-ceiling/1 card

  RAM needed at full context: winner peak not recorded on a ctx-ceiling/1 card

  Overnight llama-bench exists. Recommended floor for house recipes. Do not treat Q4 as a smaller-ceiling trick without a card.

• UD-Q3_K_XL · labeled estimate

  Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf

  Ceiling: not yet measured

  Minimum system RAM needed: 32 GB kit (labeled estimate)

  RAM needed at full context: 32 GB kit (labeled estimate)

  Labeled estimate (Hugging Face listing for Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf (16.8 GB)). Do not download Q3 to run. Q3 does not buy a measured ceiling.

Run it yourself

# Stack Lab pack ctx-ceiling/1. Not an allowlisted deep link — open the planner.
# UI hash #ctx-ceiling. Deeplink names context-ceiling / max-context are Stack Lab
# screens, not Lab Notes amdai-stack-lab: buttons. Do not invent those hrefs.
amdai-bench ab-ctx-ceiling

# Export a lab-notes-report/1 pack from Stack Lab, then ingest into Lab Notes.
# Each row must be a validating amdai-bench/run@1 card. Do not hand-type tok/s.
# Companion write-up: Stack Lab docs/ctx-ceiling.md (not in this repo).
npx tsx src/scripts/amdai-bench.ts ingest --dry-run pack.json
npx tsx src/scripts/amdai-bench.ts ingest pack.json
npx tsx src/scripts/amdai-bench.ts ingest pack.json data/runs/stack-lab-import.jsonl
npx tsx src/scripts/amdai-bench.ts validate

Zero to a measured ceiling

1. 1 · Cited Stack Lab ctx-ceiling/1 · house 262K llama-bench · finder not yet ingested

   Keep the house Vulkan pin — do not invent a ceiling

   Vulkan llama-server, Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf, -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja --host 127.0.0.1 --port 8080. Overnight llama-bench ran 262144. That is not a ctx-ceiling/1 winner. House recipes stay Q4 and up. UD-Q3_K_XL is a labeled estimate — do not download it to run.

   • Always-on server
   • Serve Qwen
   • Qwen3.6-35B

2. 2 · Cited Stack Lab ctx-ceiling/1 · house 262K llama-bench · finder not yet ingested

   Separate native max from the lean ladder

   The Qwen3.6-35B-A3B card lists 262144. Lean starts short and climbs. Size -c on the calculator before you copy a 262K line onto a 16 GB card. Long-context shopper numbers live on /long-context.

   • Long context
   • Calculator
   • Glossary · context length

3. 3 · Cited Stack Lab ctx-ceiling/1 · house 262K llama-bench · finder not yet ingested

   Run one GPU and all GPUs as separate arms

   ctx-ceiling/1 records both. Peak VRAM is per card plus total. Do not blend a split llama-bench cell into an all-GPUs ceiling. Dual-card how-to is /multi-gpu. One vs split cards are /compare/one-vs-two-gpu.

   • Dual / multi-GPU
   • One vs split
   • R9700

4. 4 · Cited Stack Lab ctx-ceiling/1 · house 262K llama-bench · finder not yet ingested

   Let OOM or a load failure stop the climb

   The first fail reason is the answer, not a guessed next step. Cited ErrorOutOfDeviceMemory and hipErrorOutOfMemory live on /out-of-vram. Cutting -c is usually cheaper than a new card. KV type is /kv-cache.

   • Out of VRAM
   • KV cache
   • Long-context OOM

5. 5 · Cited Stack Lab ctx-ceiling/1 · house 262K llama-bench · finder not yet ingested

   Map the winner onto the two RAM labels

   Minimum system RAM needed is the lean / short-context load rounded up to a real kit (64 / 96 / 128 / 192 / 256 GB) with a short breakdown — labeled estimate until measured. RAM needed at full context is the winner’s measured peak first, kit round second. House numbers today are 37.32 GB at 4K and 42.9 GB at 256K from llama-bench, not ctx-ceiling/1.

   • Calculator
   • Benchmarks
   • Run cards

6. 6 · Cited Stack Lab ctx-ceiling/1 · house 262K llama-bench · finder not yet ingested

   Run amdai-bench ab-ctx-ceiling — do not invent a winner

   Stack Lab ships ctx-ceiling/1 (amdai-bench ab-ctx-ceiling, UI #ctx-ceiling). Record decode, prefill, peak VRAM per card and total, peak system RAM per passing step, and the first fail reason. Export a lab-notes-report/1 pack and ingest with npx tsx src/scripts/amdai-bench.ts ingest pack.json. Packages are not published yet. Open the planner — do not invent a context-ceiling or max-context deep link. Companion write-up: docs/ctx-ceiling.md in Stack Lab.

   • Stack Lab
   • amdai-bench
   • lab-notes-report/1
   • How we test

RAM lines (Ubuntu-measured)

Minimum system RAM needed on one R9700 with the house UD-Q5 is a 64 GB kit from the measured 37.32 GB llama-bench peak at 4K (lean / short-context). Breakdown: offloaded weights 0% on the house in-VRAM path + OS / cache / headroom → 64 GB. That kit round is a labeled estimate until a ctx-ceiling/1 lean step is ingested. RAM needed at full context uses the measured 42.9 GB peak at 256K as the primary number; the kit-round secondary stays 64 GB. A ctx-ceiling/1 winner peak is not yet measured. UD-Q3_K_XL is a labeled estimate. Do not download Q3 to run.

Open in Stack Lab

ctx-ceiling/1 is the context-ceiling finder. It is not an allowlisted deep link on this site. Use the planner. Packages are not published yet and there is no public download. Anything unmeasured stays not yet measured.

Sources

• AMDai lab-notes-report/1 schema (ingest path for Stack Lab packs)
• AMDai amdai-bench (run@1 cards; ctx-ceiling/1 is not yet ingested)
• Stack Lab docs/ctx-ceiling.md (companion write-up; not checked into Lab Notes) — unofficial for llama.cpp
• llama.cpp — llama-server README (-c / --ctx-size)
• llama.cpp repository
• Qwen3.6-35B-A3B official model card (262144 native context)