Parallel slots (-np) on a Radeon GPU with llama.cpp

What a slot is

A slot is one concurrent sequence on llama-server. -np N / --parallel N is the official slot count (default -1 = auto; env LLAMA_ARG_N_PARALLEL). GET /health reports total_slots from that flag. Continuous batching (-cb) is on by default so idle slots can join a running decode. House path is one interactive user until a card says otherwise.

Context per slot

Size context per slot, then the pool. Official --kv-unified-per-slot N is the context limit per parallel slot; when set without -c / --ctx-size, llama.cpp sizes the shared KV pool to n_parallel × N. If you keep one -c and raise -np, treat usable context per client as total ctx / np unless you also raise -c or set --kv-unified-per-slot. --kv-unified (-kvu) shares one KV buffer across sequences (default on when slots are auto). A second slot is still more KV — size it on the calculator before you raise -np. Overnight house cards used -c 262144 and one client: peak VRAM 25.97 GB at 4K and 29.34 GB at 256K on a 32 GB R9700. Those are llama-bench, not a slot A/B.

TTFT versus aggregate decode

More slots raise aggregate decode when several clients stream at once. Each client usually waits longer for first token (TTFT) and sees a slower personal decode. Fewer slots win for one Hermes chat, one coding tool, or thinking mode — hidden thought tokens burn the same window. More slots win when Open WebUI, Hermes, and a phone share one always-on unit. Do not copy the overnight 102.39 tok/s 4K UD-Q5 cell as an -np 4 figure. Do not copy the two-card 172 / 177 tok/s soak as one-GPU slots. Measure concurrency-ab/1.

Cited flags

Vulkan llama-server, house 35B, loopback. Replace the binary and GGUF paths. These are examples, not a measured slot card.

# One interactive user (house daily driver).
# llama-server README: -np / --parallel N (default -1 = auto).
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja \
  -np 1 --host 127.0.0.1 --port 8080

# Shared server: size per-slot context first.
# Official --kv-unified-per-slot N is the context limit per slot.
# When set without -c, the shared KV pool is n_parallel × N.
# Shopper rule: usable context per client ≈ total -c / -np unless you
# raise -c or set --kv-unified-per-slot.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 --kv-unified-per-slot 32768 -np 4 \
  --cache-type-k f16 --cache-type-v f16 --jinja \
  --host 127.0.0.1 --port 8080

Q6 / Q5 / Q4 plus a labeled Q3

Raising -np multiplies KV, not the GGUF. A smaller quant does not invent a slot tok/s. Nothing below Q3.

• Q6_K

  Qwen3.6-35B-A3B-Q6_K.gguf. Overnight one-client llama-bench exists. -np 2/4/8 stays not measured yet.

• UD-Q5_K_XL

  Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf. House file. One-client 102.39 tok/s at 4K is llama-bench, not a slot A/B.

• Q4_K_M

  Qwen3.6-35B-A3B-Q4_K_M.gguf. Overnight one-client llama-bench exists. Raising -np still needs a concurrency-ab/1 card.

• UD-Q3_K_XL · labeled estimate

  Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf. Labeled estimate (Hugging Face listing for Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf (16.8 GB)). Q3 does not buy free slots on a 32 GB card.

Zero to a measured slot count

1. 1 · Cited llama.cpp --parallel · slots not measured on the local desktop

   Start with one slot on the house 35B

   Vulkan llama-server, Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf, -c 262144, f16 KV, -np 1, loopback :8080. That matches the overnight one-client cards. Q6/Q5/Q4 first. UD-Q3_K_XL is a labeled estimate.

   • Always-on server
   • Serve Qwen
   • Qwen3.6-35B

2. 2 · Cited llama.cpp --parallel · slots not measured on the local desktop

   Size context per slot before you raise -np

   Official --kv-unified-per-slot N is the per-slot limit. Usable context per client is total ctx / np unless you raise -c or set that flag. Overnight 256K already peaks at 29.34 GB VRAM on a 32 GB R9700 — a second slot at the same -c will not fit. Calculator first.

   • KV cache
   • Long context
   • Calculator

3. 3 · Cited llama.cpp --parallel · slots not measured on the local desktop

   Leave -np 1 when one chat owns the GPU

   Fewer slots win for one Hermes session, one coding tool, or thinking mode. Hidden thought tokens consume the same window. The house 35B already decodes fast on one R9700 — extra slots only help when several clients stream at once.

   • Thinking mode
   • Tool calling
   • Harnesses

4. 4 · Cited llama.cpp --parallel · slots not measured on the local desktop

   Raise -np only when several clients share one server

   Open WebUI + Hermes + a phone on one systemd unit is the shopper case. Start at -np 2 with a smaller per-slot window (32K is the example, not a measured pin). Confirm GET /health total_slots. Continuous batching stays on unless you pass --no-cont-batching.

   • Always-on server
   • Open WebUI
   • Connect a harness

5. 5 · Cited llama.cpp --parallel · slots not measured on the local desktop

   Do not confuse slots with two GPU copies

   Two llama-server processes on two R9700s are /qwen36-dual (measured soak). -np keeps one process and one weight file. We do not copy the two-card 172 / 177 tok/s totals onto this page as a slot result.

   • 35B on two cards
   • Dual / multi-GPU
   • Multi-GPU llama.cpp

6. 6 · Cited llama.cpp --parallel · slots not measured on the local desktop

   Measure Stack Lab concurrency-ab/1 — do not invent tok/s

   Stack Lab ships concurrency-ab/1 (amdai-bench ab-concurrency): -np 1/2/4/8 with that many simultaneous streaming clients. Record TTFT and aggregate decode. Packages are not published yet and there is no public download. Open the planner — do not invent a concurrency-ab deep link.

   • Stack Lab
   • amdai-bench
   • Benchmarks
   • How we test

RAM lines (Ubuntu-measured)

Minimum system RAM needed on one R9700 with the house UD-Q5 is a 64 GB kit from the measured 37.32 GB peak at 4K. RAM needed at full context uses the measured 42.9 GB peak at 256K. Those are llama-bench cards for one client, GPU RAM plus total system DDR — not a file-cache versus anonymous split, and not an -np 4 measurement. Extra slots grow KV on the same kit. UD-Q3_K_XL is a labeled estimate.

Open in Stack Lab

Pack concurrency-ab/1 is the concurrency A/B. It is not an allowlisted deep link on this site. Use the planner. Packages are not published yet and there is no public download.

Sources

• llama.cpp — llama-server README (-np / --parallel, --kv-unified-per-slot, GET /health total_slots)
• llama.cpp repository
• Qwen3.6-35B-A3B official model card