Thinking / reasoning mode on a Radeon GPU with llama.cpp

Cited flags

Vulkan llama-server, house 35B, loopback. Replace the binary and GGUF paths. These are examples, not a measured thinking card.

# Turn thinking off. House 35B, Vulkan llama-server, loopback.
# llama-server README: --reasoning on|off|auto (default auto).
# Request body: reasoning_effort none, and/or chat_template_kwargs {"enable_thinking": false}.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja \
  --reasoning off --host 127.0.0.1 --port 8080

# Leave thinking on (Qwen3.6 card default). --reasoning auto detects the template.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja \
  --reasoning auto --host 127.0.0.1 --port 8080

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen",
    "messages": [{"role": "user", "content": "Reply with the single word READY and nothing else."}],
    "max_tokens": 32,
    "reasoning_effort": "none",
    "chat_template_kwargs": {"enable_thinking": false}
  }'

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen",
    "messages": [{"role": "user", "content": "Say READY after a short plan."}],
    "max_tokens": 256,
    "chat_template_kwargs": {"enable_thinking": true}
  }'

# llama-server README: --reasoning-budget N
# -1 unrestricted (default). 0 ends thinking immediately. N>0 is a token budget.
# --reasoning-format none | deepseek | deepseek-legacy (default auto).
# --reasoning-preserve is on by default; --no-reasoning-preserve drops history traces.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja \
  --reasoning on --reasoning-budget 0 --host 127.0.0.1 --port 8080

Zero to a thinking toggle

1. 1 · Cited llama.cpp + Qwen3.6 card · tokens not measured on the local desktop

   Get tokens on loopback before you toggle thinking

   Ubuntu 24.04+ bare metal is the measured AMDai path. Build llama.cpp Vulkan, load Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf on one R9700, and confirm curl http://127.0.0.1:8080/v1/models from /tutorials/serve-qwen. House flags: -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja --host 127.0.0.1 --port 8080. llama.cpp first. Q6/Q5/Q4 recommended; Q3 is a labeled estimate. Q1/Q2 stay off.

   • Ubuntu setup
   • Serve Qwen
   • llama.cpp
   • R9700 measured page

2. 2 · Cited llama.cpp + Qwen3.6 card · tokens not measured on the local desktop

   Official Qwen3.6 thinks by default

   The official Qwen3.6-35B-A3B card thinks before it answers. Thought blocks come first; the final answer follows. The card does not officially support the Qwen3 soft switch /think or /nothink. Do not type those tags and expect a toggle. llama.cpp --reasoning defaults to auto and follows the template. Flash thinking on Strata is not verified here.

   • Qwen3.6-35B
   • Qwen compare
   • Flash daily
   • llama.cpp

3. 3 · Cited llama.cpp + Qwen3.6 card · tokens not measured on the local desktop

   Turn thinking off with cited flags

   llama-server README: --reasoning off (or -rea off). Per request, reasoning_effort none disables thinking when the template honors it. The same README documents chat_template_kwargs, for example {"enable_thinking": false} — that is the official Qwen3.6 instruct path. Use one or more; do not invent a third flag. Confirm with a short curl that returns READY without a long thought block.

   • Serve Qwen
   • Speed up llama.cpp
   • Connect a harness

4. 4 · Cited llama.cpp + Qwen3.6 card · tokens not measured on the local desktop

   Leave it on, or force it on

   House default for official instruct is the card default: thinking on. llama-server --reasoning on forces it; --reasoning auto detects the template. Per request, chat_template_kwargs {"enable_thinking": true}. --reasoning-format deepseek fills message.reasoning_content; none leaves thoughts in message.content. Those are README formats, not a measured quality rank.

   • Harnesses
   • Hermes
   • Open WebUI
   • Coding assistant

5. 5 · Cited llama.cpp + Qwen3.6 card · tokens not measured on the local desktop

   Budget and preserve — cited, not measured

   llama-server README: --reasoning-budget -1 is unrestricted (default), 0 ends thinking immediately, N>0 is a token budget. --reasoning-budget-message injects text when the budget is exhausted. --reasoning-preserve (default on) keeps traces in history when the template supports it; --no-reasoning-preserve does not. The official Qwen3.6 card documents preserve_thinking the same way. AMDai has not measured a budget tok/s or a preserve RAM delta.

   • Long context
   • Speed up llama.cpp
   • amdai-bench

6. 6 · Cited llama.cpp + Qwen3.6 card · tokens not measured on the local desktop

   Both RAM lines — thinking eats context, not a new kit

   Where context and offload appear, AMDai prints Minimum system RAM needed and RAM needed at full context. Overnight UD-Q5 one-GPU peaks are 37.32 GB at 4K and 42.9 GB at 256K — measured llama-bench on the local desktop, Linux GB. Round Minimum system RAM needed to a 64 GB kit for this file on one R9700. Thinking tokens consume the same context window; we do not invent a thinking-token count or a second RAM number. Q3 stays a labeled estimate.

   • Calculator
   • R9700 measured page
   • Long context
   • Flash RAM offload

7. 7 · Cited llama.cpp + Qwen3.6 card · tokens not measured on the local desktop

   When thinking never ends

   Stay on official instruct. Fix --jinja before you change the GGUF. --reasoning off or reasoning_effort none is the cited stop. --reasoning-budget 0 ends the thought block immediately. llama-server also documents reasoning_control plus POST /v1/chat/completions/control with action reasoning_end — unofficial to treat as a harness button until you have tried the flags. Do not grab an abliterated file to “fix” a template. Start from /troubleshoot.

   • Serve Qwen
   • Abliterated
   • Troubleshoot
   • FAQ

8. 8 · Cited llama.cpp + Qwen3.6 card · tokens not measured on the local desktop

   Measure on your box — do not copy a screenshot

   Thinking-token counts and wall-clock stay not measured on the local desktop. Overnight 35B numbers are llama-bench decode, not a thinking A/B. Use amdai-bench and the Stack Lab planner on Ubuntu — packages are not published yet and there is no public download. Do not invent a Stack Lab pack id. Q3 is a labeled estimate.

   • amdai-bench
   • Stack Lab
   • Benchmarks
   • How we test

RAM lines (Ubuntu-measured)

Minimum system RAM needed and RAM needed at full context for the house 35B. Those peaks are llama-bench on the local desktop, not a thinking-token measurement.

Open in Stack Lab

Stack Lab remains Ubuntu-first. Packages are not published yet and there is no public download. Use the planner — do not invent a thinking pack id.

Sources

• llama.cpp — llama-server README (--reasoning, --reasoning-effort, chat_template_kwargs)
• llama.cpp repository
• Qwen3.6-35B-A3B official model card (thinks by default; no /think /nothink)