Sampling settings for local LLMs on a Radeon GPU with llama.cpp: temperature, top-p, top-k, min-p and penalties

What temperature, top-p, top-k, and min-p do

A sampler picks the next token from the logits. Official --temp / --temperature (default 0.80; env LLAMA_ARG_TEMPERATURE) scales that distribution. --top-k keeps the K most likely tokens (default 40; 0 disables; env LLAMA_ARG_TOP_K). --top-p keeps a cumulative-probability nucleus (default 0.95; 1.0 disables; env LLAMA_ARG_TOP_P). --min-p drops tokens below a fraction of the top token (default 0.05; 0.0 disables; env LLAMA_ARG_MIN_P). Those CLI defaults are the llama-server README Sampling params table and common/common.h. They are not a house pin and they are not a Qwen card pin.

Penalties, DRY, and XTC

Official --repeat-penalty penalizes repeating token sequences (CLI default 1.00 = disabled; env LLAMA_ARG_REPEAT_PENALTY). --repeat-last-n is the window (default 64; 0 disables). --presence-penalty and --frequency-penalty are the alpha presence / frequency penalties (default 0.00 = disabled; env LLAMA_ARG_PRESENCE_PENALTY and LLAMA_ARG_FREQUENCY_PENALTY). DRY is off until --dry-multiplier is set (default 0.00); --dry-base default 1.75, --dry-allowed-length 2, --dry-penalty-last-n, --dry-sequence-breaker. XTC is off until --xtc-probability is set (default 0.00); --xtc-threshold default 0.10. The /completion field repeat_penalty is documented as default 1.1 — that is an official README discrepancy versus the CLI 1.00 default. Cite both; do not invent a third number.

Sampler order and seed

Official --samplers lists the chain in order, separated by ';' (default penalties;dry;top_n_sigma;top_k;typ_p;top_p;min_p;xtc;temperature). --sampler-seq / --sampling-seq is the short form (default edskypmxt). The /completion samplers field is an array; the README default listed there is ["dry", "top_k", "typ_p", "top_p", "min_p", "xtc", "temperature"]. If a sampler is omitted it is not used. -s / --seed defaults to -1 (random). --typical / --typical-p and --top-n-sigma are documented; they stay off at the official defaults (1.00 and -1.00).

OpenAI-compatible request fields

llama-server /completion documents temperature (default 0.8), top_k (40), top_p (0.95), min_p (0.05), seed (-1), repeat_penalty, presence_penalty, frequency_penalty, dry_*, xtc_*, typical_p, and samplers. POST /v1/chat/completions is the OpenAI-compatible path: the README says see the OpenAI Chat Completions API and that /completion-specific features such as mirostat are also supported. OpenAI-shaped fields that shoppers send are temperature, top_p, presence_penalty, frequency_penalty, and seed; llama.cpp extras on the same body include top_k, min_p, repeat_penalty, and samplers. Anthropic-shaped POST /v1/messages documents temperature default 1.0 and top_p default 1.0 — those are that endpoint’s documented defaults, not the CLI table.

Cited flags

Vulkan house path. Each row cites the llama-server README. These are official defaults, not a local desktop pin.

• --temp, --temperature N

  temperature

  Scales logits. Official CLI default 0.80 (env LLAMA_ARG_TEMPERATURE). /completion default 0.8. 0 is greedy.

  llama.cpp server README — Sampling params; /completion temperature

• --top-p N

  top_p

  Nucleus sampling. Official default 0.95; 1.0 disables (env LLAMA_ARG_TOP_P).

  llama.cpp server README — --top-p; /completion top_p

• --top-k N

  top_k

  Keep the K most likely tokens. Official default 40; 0 disables (env LLAMA_ARG_TOP_K).

  llama.cpp server README — --top-k; /completion top_k

• --min-p N

  min_p

  Minimum probability relative to the top token. Official default 0.05; 0.0 disables (env LLAMA_ARG_MIN_P).

  llama.cpp server README — --min-p; /completion min_p

• --repeat-penalty N

  repeat_penalty

  Penalize repeating token sequences. CLI default 1.00 = disabled (env LLAMA_ARG_REPEAT_PENALTY). /completion documents default 1.1.

  llama.cpp server README — --repeat-penalty vs /completion repeat_penalty

• --presence-penalty N

  presence_penalty

  Repeat alpha presence penalty. Official default 0.00 = disabled (env LLAMA_ARG_PRESENCE_PENALTY).

  llama.cpp server README — --presence-penalty; /completion presence_penalty

• --frequency-penalty N

  frequency_penalty

  Repeat alpha frequency penalty. Official default 0.00 = disabled (env LLAMA_ARG_FREQUENCY_PENALTY).

  llama.cpp server README — --frequency-penalty; /completion frequency_penalty

• --dry-multiplier N (plus --dry-base, --dry-allowed-length, --dry-penalty-last-n, --dry-sequence-breaker)

  dry_multiplier / dry_base / dry_allowed_length / dry_penalty_last_n / dry_sequence_breakers

  DRY (Don’t Repeat Yourself). Official --dry-multiplier default 0.00 = disabled. --dry-base default 1.75. Sequence breakers default newline, colon, quote, asterisk.

  llama.cpp server README — DRY Sampling params

• --xtc-probability N / --xtc-threshold N

  xtc_probability / xtc_threshold

  XTC token removal. Official --xtc-probability default 0.00 = disabled. --xtc-threshold default 0.10 (threshold > 0.5 disables XTC).

  llama.cpp server README — XTC Sampling params

• --samplers SAMPLERS / --sampler-seq SEQUENCE

  samplers

  Order of the chain. CLI default penalties;dry;top_n_sigma;top_k;typ_p;top_p;min_p;xtc;temperature. Short form default edskypmxt. API array omits unused samplers.

  llama.cpp server README — --samplers / --sampling-seq; /completion samplers

• -s, --seed SEED

  seed

  RNG seed. Official default -1 = random seed.

  llama.cpp server README — -s, --seed; /completion seed

House 35B and Flash as recorded

House daily driver is Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf on llama.cpp Vulkan, one R9700, 262144 context, f16 KV, -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja --host 127.0.0.1 --port 8080. House Flash try is Qwen3.8-Flash-Next-UD-Q6_K_XL.gguf on the recorded llama.cpp / Strata recipes. Neither house recipe records --temp, --top-p, --top-k, --min-p, --repeat-penalty, --presence-penalty, --frequency-penalty, --samplers, DRY, XTC, or --seed. Official llama.cpp defaults then apply unless a client sends request fields. Do not treat a Qwen card row as a house pin.

# House daily driver as recorded (no --temp / --top-p / --min-p / --samplers pin).
# llama-server README Sampling params: --temp 0.80, --top-k 40, --top-p 0.95, --min-p 0.05.
# Those defaults are official docs — not a local desktop pin we will invent.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja --host 127.0.0.1 --port 8080

# House Flash llama.cpp shape as recorded (no sampler pin).
# Qwen 3.8 Flash card values are below — they are not this recipe.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.8-Flash-Next-UD-Q6_K_XL.gguf \
  -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja \
  --host 127.0.0.1 --port 8080

# Official llama.cpp CLI defaults from the server README — not a house pin.
# --samplers penalties;dry;top_n_sigma;top_k;typ_p;top_p;min_p;xtc;temperature
# -s / --seed -1 (random). --repeat-penalty 1.00 disables. DRY / XTC stay off.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --temp 0.80 --top-k 40 --top-p 0.95 --min-p 0.05 \
  --repeat-penalty 1.00 --presence-penalty 0.00 --frequency-penalty 0.00 \
  --samplers 'penalties;dry;top_n_sigma;top_k;typ_p;top_p;min_p;xtc;temperature' \
  --seed -1 --cache-type-k f16 --cache-type-v f16 --jinja \
  --host 127.0.0.1 --port 8080

# Official Qwen3.6-35B-A3B thinking / general-task card values — not a house pin.
# Card: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5,
# repetition_penalty=1.0. llama.cpp maps repetition_penalty to repeat_penalty.
curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen",
    "messages": [{"role": "user", "content": "Reply with the single word READY."}],
    "temperature": 1.0,
    "top_p": 0.95,
    "top_k": 20,
    "min_p": 0.0,
    "presence_penalty": 1.5,
    "repeat_penalty": 1.0
  }'

Official card values only

Thinking versus instruct rows come from each model’s official card. If the card gives no generation set, this page says so. llama.cpp maps the card’s repetition_penalty to --repeat-penalty / repeat_penalty.

• Qwen3.6-35B-A3B

  Thinking · general tasks

  temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

  Official card recommended set. Not a house llama-server pin.

  Official card

• Qwen3.6-35B-A3B

  Thinking · precise coding (e.g. WebDev)

  temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0

  Official card recommended set. Not a house llama-server pin.

  Official card

• Qwen3.6-35B-A3B

  Instruct / non-thinking

  temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

  Official card recommended set. Turn thinking off on /thinking first.

  Official card

• Qwen 3.8 Flash (Qwen3.8-Flash-Next)

  Thinking

  temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0

  Official card recommended set. House Flash recipes do not record these flags.

  Official card

• Qwen 3.8 Flash (Qwen3.8-Flash-Next)

  Instruct / non-thinking

  temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

  Official card recommended set. House Flash recipes do not record these flags.

  Official card

• Qwen3.6 27B

  Thinking · general tasks

  temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0

  Official card recommended set. Presence penalty is 0.0 on this dense card’s general thinking row.

  Official card

• Qwen3.6 27B

  Thinking · precise coding (e.g. WebDev)

  temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0

  Official card recommended set.

  Official card

• Qwen3.6 27B

  Instruct / non-thinking

  temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

  Official card recommended set.

  Official card

• Qwen3.8 27B

  Thinking

  temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0

  Official card recommended set.

  Official card

• Qwen3.8 27B

  Instruct / non-thinking

  temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

  Official card recommended set.

  Official card

• gpt-oss 20B

  Generation samplers

  not on the card

  Official openai/gpt-oss-20b card does not publish recommended temperature / top_p / top_k / min_p values. Do not invent them.

  Official card

• Gemma 3 27B

  Generation samplers

  not on the card

  Official google/gemma-3-27b-it card does not publish a recommended temperature / top_p / top_k set. The generate snippet uses do_sample=False. Do not invent values.

  Official card

• Llama 3.3 70B

  Generation samplers

  not on the card

  Official meta-llama/Llama-3.3-70B-Instruct card does not publish recommended generation temperature / top_p / top_k values. Do not invent them.

  Official card

• GLM 5.3 Flash

  Eval footnotes only

  not a recommended shopper pin

  Official zai-org/GLM-5.3-Flash card publishes eval footnotes (e.g. HLE temperature=1.0 top_p=0.95; DeepSWE temperature=0.95 top_p=1.0). Those are benchmark arms, not a general recommended set. Do not copy them as house defaults.

  Official card

Troubleshooting

Loops, stuck thinking, and broken JSON are usually samplers or templates — not the GPU. No invented quality or speed claims.

• Endless loops or repetition

  Official Qwen cards say you can raise presence_penalty between 0 and 2 to cut endless repetition, and that a higher value may mix languages. That is a card note, not a measured AMDai quality claim. llama.cpp --presence-penalty default is 0.00. House recipes do not record a pin. Do not invent 1.1.

  • Qwen3.6-35B
  • Troubleshoot

• Thinking never ends

  That is a reasoning toggle, not a temperature bug. Official Qwen3.6 thinks by default. Use --reasoning off or enable_thinking: false. The shopper page is /thinking. Thinking-token counts stay not measured on the local desktop.

  • Thinking mode
  • Troubleshoot

• JSON or tool calls break

  Do not raise temperature to “fix” broken JSON. Official GBNF / JSON Schema live on /constrained-decoding. Tool attach is /guides/tool-calling. Grammar decode versus free-form stays not measured yet — run Stack Lab grammar-ab/1 from the planner.

  • Constrained decoding
  • Tool calling
  • Troubleshoot

Q6 / Q5 / Q4 plus a labeled Q3

Samplers do not shrink the GGUF. House recipes stay Q4 and up. UD-Q3_K_XL is a labeled estimate — do not download it to run. Sampler speed stays not measured yet.

• Q6_K

  Qwen3.6-35B-A3B-Q6_K.gguf

  Sampler speed: not measured yet

  Minimum system RAM needed: peak system RAM not recorded on a sampler card

  RAM needed at full context: peak system RAM not recorded on a sampler card

  Overnight llama-bench exists. Sampler speed cost stays not measured yet.

• UD-Q5_K_XL

  Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

  Sampler speed: not measured yet

  Minimum system RAM needed: 64 GB kit from the measured 37.32 GB peak at 4K (no sampler pin on that card)

  RAM needed at full context: measured 42.9 GB peak at 256K. Sampler flags do not add GGUF weight.

  House file. Sampler A/B is not on that llama-bench card.

• Q4_K_M

  Qwen3.6-35B-A3B-Q4_K_M.gguf

  Sampler speed: not measured yet

  Minimum system RAM needed: peak system RAM not recorded on a sampler card

  RAM needed at full context: peak system RAM not recorded on a sampler card

  Overnight llama-bench exists. Recommended floor for house recipes. Sampler speed stays not measured yet.

• UD-Q3_K_XL · labeled estimate

  Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf

  Sampler speed: not measured yet

  Minimum system RAM needed: 32 GB kit (labeled estimate)

  RAM needed at full context: 32 GB kit (labeled estimate)

  Labeled estimate (Hugging Face listing for Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf (16.8 GB)). Do not download Q3 to run. Q3 does not buy a sampler pin.

Zero to a cited sampler pin

1. 1 · Cited llama.cpp --temp / --top-p / --min-p · sampler speed not measured

   Keep the house Vulkan pin — do not invent a sampler flag

   Vulkan llama-server, Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf, -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja --host 127.0.0.1 --port 8080. House recipes do not record --temp, --top-p, --min-p, or --samplers. Official llama.cpp defaults then apply. House recipes stay Q4 and up. UD-Q3_K_XL is a labeled estimate — do not download it to run.

   • Always-on server
   • Serve Qwen
   • Qwen3.6-35B

2. 2 · Cited llama.cpp --temp / --top-p / --min-p · sampler speed not measured

   Copy a sampler set only from the official card

   Qwen3.6-35B, Qwen 3.8 Flash, Qwen3.6 27B, and Qwen3.8 27B publish thinking vs instruct rows. gpt-oss 20B, Gemma 3 27B, and Llama 3.3 70B do not. GLM 5.3 Flash only footnotes eval arms. llama.cpp maps the card’s repetition_penalty to --repeat-penalty / repeat_penalty. Do not invent a missing row.

   • Qwen compare
   • Qwen 3.8 Flash
   • Thinking mode

3. 3 · Cited llama.cpp --temp / --top-p / --min-p · sampler speed not measured

   Send OpenAI-shaped fields on /v1 — or pin the CLI

   Official /v1/chat/completions follows the OpenAI Chat Completions API and also accepts /completion extras (temperature, top_p, top_k, min_p, presence_penalty, frequency_penalty, repeat_penalty, samplers, seed). CLI --samplers order is penalties;dry;top_n_sigma;top_k;typ_p;top_p;min_p;xtc;temperature. DRY and XTC stay off at the official defaults.

   • Always-on server
   • llama.cpp
   • Harnesses

4. 4 · Cited llama.cpp --temp / --top-p / --min-p · sampler speed not measured

   Fix loops, stuck thinking, and broken JSON on the cited pages

   Repetition: Qwen cards allow presence_penalty 0–2. Thinking that never ends: /thinking. JSON that will not parse: /constrained-decoding and /guides/tool-calling. Start from /troubleshoot. Sampler speed cost stays not measured yet.

   • Thinking mode
   • Constrained decoding
   • Tool calling
   • Troubleshoot

5. 5 · Cited llama.cpp --temp / --top-p / --min-p · sampler speed not measured

   Measure on your box — do not invent a sampler tok/s

   AMDai has not measured llama-server decode with versus without a card sampler pin on the local desktop — not measured yet. There is no Stack Lab sampler-pack id on this site. Open the planner. Export a lab-notes-report/1 pack and ingest with npx tsx src/scripts/amdai-bench.ts ingest pack.json. Packages are not published yet.

   • Stack Lab
   • amdai-bench
   • Run cards
   • How we test

RAM lines (Ubuntu-measured)

Sampler flags do not add GGUF weight. Minimum system RAM needed on one R9700 with the house UD-Q5 is a 64 GB kit from the measured 37.32 GB peak at 4K. RAM needed at full context uses the measured 42.9 GB peak at 256K. Those are llama-bench cards, not a sampler A/B. Sampler speed cost stays not measured yet — open the Stack Lab planner rather than guessing. UD-Q3_K_XL is a labeled estimate (Hugging Face listing for Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf (16.8 GB)). Do not download Q3 to run.

Open in Stack Lab

There is no allowlisted sampler A/B deep link on this site. Use the planner. Packages are not published yet and there is no public download. Anything unmeasured stays not measured yet.

Sources

• llama.cpp — llama-server README (Sampling params, /completion fields, /v1/chat/completions)
• llama.cpp — common/common.h sampling defaults
• llama.cpp — common/arg.cpp (--temp, --top-p, --min-p, DRY, XTC, --samplers, --seed)
• OpenAI Chat Completions API (cited by llama-server /v1/chat/completions)
• llama.cpp repository
• Qwen3.6-35B-A3B official model card (thinking vs instruct sampling)
• Qwen3.8-Flash-Next official model card (thinking vs instruct sampling)
• Qwen3.6-27B official model card
• Qwen3.8-27B official model card
• openai/gpt-oss-20b official model card (no recommended sampler set)
• google/gemma-3-27b-it official model card (no recommended sampler set)
• meta-llama/Llama-3.3-70B-Instruct official model card (no recommended sampler set)
• zai-org/GLM-5.3-Flash official model card (eval footnotes only)