Chat templates and --jinja on a Radeon GPU with llama.cpp

What a chat template is

A chat template is the Jinja string that wraps roles, tools, and reasoning before the model sees tokens. Official llama-server README: --chat-template and --chat-template-file default to the template taken from the model’s metadata. gguf-py constants.py names that key tokenizer.chat_template (plus tokenizer.chat_templates and tokenizer.chat_template.{name}). GET /props returns chat_template and chat_template_caps. POST /apply-template formats messages the same way /v1/chat/completions does, without running inference — then you can send the string to raw /completion. Raw /completion does not apply the chat template for you.

How llama-server picks it

Official --jinja / --no-jinja turns the Jinja engine on (default enabled; env LLAMA_ARG_JINJA). function-calling.md requires --jinja for OpenAI-style tools. --chat-template names a built-in (chatml, llama3, gemma, gpt-oss, …) and is only a short list unless --jinja is set first. --chat-template-file loads a Jinja file. Both override the GGUF metadata. --chat-template-kwargs is a JSON object string (env LLAMA_ARG_CHAT_TEMPLATE_KWARGS), for example {"enable_thinking": false}. --reasoning-format none | deepseek | deepseek-legacy (default auto) controls whether thought tags stay in message.content or move to reasoning_content. --skip-chat-parsing dumps reasoning and tool markup into content. Do not invent a house --chat-template-file path.

• --jinja / --no-jinja

  Use the Jinja chat-template engine. Official default enabled (env LLAMA_ARG_JINJA). House pin still passes --jinja. function-calling.md requires it for OpenAI-style tools.

  llama.cpp server README — --jinja (checked 2026-10-14); function-calling.md

• --chat-template JINJA_TEMPLATE

  Override the GGUF metadata with a built-in name (chatml, llama3, gemma, gpt-oss, …). Default: template taken from the model metadata. Short list unless --jinja is set first. env LLAMA_ARG_CHAT_TEMPLATE.

  llama.cpp server README — --chat-template (checked 2026-10-14)

• --chat-template-file JINJA_TEMPLATE_FILE

  Load a Jinja file instead of the GGUF metadata. Same default and built-in list as --chat-template. env LLAMA_ARG_CHAT_TEMPLATE_FILE. Do not invent a house file path.

  llama.cpp server README — --chat-template-file (checked 2026-10-14)

• --chat-template-kwargs STRING

  JSON object string for the template parser, e.g. {"enable_thinking": false}. env LLAMA_ARG_CHAT_TEMPLATE_KWARGS. Same field on /v1/chat/completions.

  llama.cpp server README — --chat-template-kwargs, chat_template_kwargs (checked 2026-10-14)

• --reasoning-format FORMAT

  none leaves thoughts in message.content. deepseek fills reasoning_content. deepseek-legacy keeps tags in content and also fills reasoning_content. Default auto. env LLAMA_ARG_THINK.

  llama.cpp server README — --reasoning-format (checked 2026-10-14)

• --skip-chat-parsing

  Force a pure content parser. Reasoning and tool markup stay in message.content. Default disabled.

  llama.cpp server README — --skip-chat-parsing (checked 2026-10-14)

Symptom to fix

• Garbled or repeated turns; the model talks to itself

  Confirm --jinja. GET /props and read chat_template. Stay on the GGUF metadata unless the log says Chat format: Generic and tools or roles are wrong. --chat-template chatml is the documented YMMV fallback, not a house pin.

  llama.cpp server README — --jinja, GET /props; function-calling.md — chatml YMMV

  Error paste

• The model answers as the user

  The template is not wrapping roles. Compare POST /apply-template output to an official instruct turn. Do not send a raw /completion string that omits the assistant prefix.

  llama.cpp server README — POST /apply-template vs POST /completion

  Harnesses

• Leaked think tags in the visible reply

  Official --reasoning-format deepseek moves thoughts to reasoning_content. none leaves them in content. --skip-chat-parsing dumps markup into content. Qwen3.6 thinks by default — use enable_thinking false or --reasoning off. Shopper page: /thinking.

  llama.cpp server README — --reasoning-format; Qwen3.6-35B card — enable_thinking

  Thinking mode

• Tool calls never fire

  Start llama-server with --jinja and send tools on /v1/chat/completions. Inspect /props for a tools-aware template. function-calling.md: unrecognized templates log Chat format: Generic. Do not invent a --chat-template-file for the house 35B. Shopper page: /guides/tool-calling.

  llama.cpp function-calling.md — --jinja, Chat format: Generic

  Tool calling

• The system prompt is ignored

  Some templates drop or flatten the system role. Read GET /props chat_template for a system branch. Hermes and Open WebUI send system on /v1/chat/completions — raw /completion will not inject it.

  llama.cpp server README — GET /props chat_template; /v1/chat/completions vs /completion

  Open WebUI

How harnesses hit it

Hermes (the house harness), Open WebUI, and coding tools talk to /v1/chat/completions. That path applies the server template. A client that posts a raw string to /completion must wrap roles itself — or call POST /apply-template first. House /v1 is http://127.0.0.1:8080/v1 with model id qwen. Strata tool / template behavior is not verified here — see /backends. Do not invent a harness success rate.

House models — card notes only

Only official cards already on /models. If a card is not quoted for a Jinja string, this page says so.

• Qwen3.6-35B-A3B (house UD-Q5_K_XL) · cited card

  Official card thinks by default. chat_template_kwargs.enable_thinking and preserve_thinking. No /think or /nothink. House llama-server passes --jinja and does not record a template override.

  Qwen3.6-35B-A3B (house UD-Q5_K_XL)

• Qwen 3.8 Flash (house UD-Q6_K_XL) · cited card

  Official Flash card documents enable_thinking (on by default), preserve_thinking, and reasoning_effort via chat_template_kwargs. House Flash recipes do not record a --chat-template pin.

  Qwen 3.8 Flash (house UD-Q6_K_XL)

• Qwen3.6-27B

  On the site. This page does not copy a template string from that card. Do not invent one. Start from GGUF metadata and GET /props.

  Qwen3.6-27B

• Qwen3.8-27B

  On the site. This page does not copy a template string from that card. Do not invent one.

  Qwen3.8-27B

• gpt-oss 20B

  function-calling.md lists Harmony as a native handler. Built-in --chat-template gpt-oss exists. The official openai card is not quoted here for a Jinja string.

  gpt-oss 20B

• Llama 3.3 70B

  function-calling.md lists Llama 3.x as native. Built-in --chat-template llama3 exists. The official Meta card is not quoted here for a Jinja string.

  Llama 3.3 70B

• Gemma 3 27B

  Built-in --chat-template gemma exists. function-calling.md does not name Gemma 3 27B on the native-handler table. Do not invent a file.

  Gemma 3 27B

• GLM 5.3 Flash

  function-calling.md does not name GLM 5.3 Flash on the native-handler table. Built-in list has chatglm3 / chatglm4, not this card. Do not invent a mapping.

  GLM 5.3 Flash

Cited house pin and inspect commands

Vulkan house path. The first block is the recorded llama-server line with --jinja. The rest are official inspect shapes, not local desktop pins. Replace the binary and GGUF paths.

# House daily driver as recorded. --jinja is on the pin.
# llama-server README (checked 2026-10-14): --jinja default enabled.
# No --chat-template / --chat-template-file override is recorded.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja --host 127.0.0.1 --port 8080

# Inspect the live template. Official GET /props (server README).
curl -s http://127.0.0.1:8080/props | python3 -c "import json,sys; p=json.load(sys.stdin); print(p.get('chat_template','')[:400])"

# Official POST /apply-template — format only, no inference.
# Then /completion can take the returned prompt string.
curl -s http://127.0.0.1:8080/apply-template \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"system","content":"You are concise."},{"role":"user","content":"Say READY"}]}'

# Official --chat-template-kwargs (server README). Not a house pin.
# Qwen3.6-35B card: chat_template_kwargs {"enable_thinking": false}.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --jinja \
  --chat-template-kwargs '{"enable_thinking": false}' \
  --cache-type-k f16 --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Q6 / Q5 / Q4 plus a labeled Q3

A template is not a smaller GGUF. House recipes stay Q4 and up. UD-Q3_K_XL is a labeled estimate — do not download it to run. Both RAM lines appear.

• Q6_K

  Qwen3.6-35B-A3B-Q6_K.gguf

  Template quality: not yet measured

  Minimum system RAM needed: peak system RAM not recorded on a template card

  RAM needed at full context: peak system RAM not recorded on a template card

  A template does not change the GGUF. Recommended. Template quality stays not yet measured.

• UD-Q5_K_XL

  Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

  Template quality: not yet measured

  Minimum system RAM needed: 64 GB kit from the measured 37.32 GB peak at 4K (llama-bench, not a template A/B)

  RAM needed at full context: measured 42.9 GB peak at 256K.

  House file. --jinja is on the recorded llama-server line. No --chat-template override.

• Q4_K_M

  Qwen3.6-35B-A3B-Q4_K_M.gguf

  Template quality: not yet measured

  Minimum system RAM needed: peak system RAM not recorded on a template card

  RAM needed at full context: peak system RAM not recorded on a template card

  Recommended floor for house recipes. A smaller quant does not fix a wrong template.

• UD-Q3_K_XL · labeled estimate

  Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf

  Template quality: not yet measured

  Minimum system RAM needed: 32 GB kit (labeled estimate)

  RAM needed at full context: 32 GB kit (labeled estimate)

  Labeled estimate (Hugging Face listing for Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf (16.8 GB)). Do not download Q3 to run. Q3 does not buy a template.

Zero to a cited template pin

1. 1 · Cited llama.cpp --jinja · GGUF tokenizer.chat_template · house recipes pass --jinja

   Keep the house --jinja pin — do not invent a file

   Vulkan llama-server, Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf, -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja --host 127.0.0.1 --port 8080. Official --jinja default is enabled. House recipes do not record --chat-template or --chat-template-file. House recipes stay Q4 and up. UD-Q3_K_XL is a labeled estimate — do not download it to run.

   • Always-on server
   • Serve Qwen
   • Qwen3.6-35B

2. 2 · Cited llama.cpp --jinja · GGUF tokenizer.chat_template · house recipes pass --jinja

   Inspect GET /props before you override

   Official GET /props returns chat_template and chat_template_caps. POST /apply-template shows the wrapped prompt. Read the server log for Chat format. A missing or Generic template is the usual garbled-turn cause — not the Radeon.

   • llama.cpp
   • Error paste
   • Glossary · chat template

3. 3 · Cited llama.cpp --jinja · GGUF tokenizer.chat_template · house recipes pass --jinja

   Use /v1 chat completions from the harness

   Hermes, Open WebUI, and coding tools should hit /v1/chat/completions so the server applies the template. Raw /completion will not. function-calling.md needs --jinja plus a tools array for tool_calls. Thinking leaks are /thinking. Sampler loops are /sampling.

   • Harnesses
   • Hermes
   • Tool calling
   • Coding assistant

4. 4 · Cited llama.cpp --jinja · GGUF tokenizer.chat_template · house recipes pass --jinja

   Copy Qwen kwargs only from the official card

   Qwen3.6-35B and Qwen 3.8 Flash cards document chat_template_kwargs.enable_thinking and preserve_thinking. Flash also documents reasoning_effort. Those are card fields, not a local desktop pin. Other /models pages are not quoted for a Jinja string on this page.

   • Thinking mode
   • Qwen compare
   • Qwen 3.8 Flash

5. 5 · Cited llama.cpp --jinja · GGUF tokenizer.chat_template · house recipes pass --jinja

   Do not swap the quant to fix a template

   Stay on official instruct. Pass --jinja. Compare official GGUF metadata before any abliterated file. The /troubleshoot/errors jinja-template row is this symptom. A Q3 download does not wrap roles.

   • Troubleshoot
   • Abliterated
   • ROCm problems

RAM lines (Ubuntu-measured)

Minimum system RAM needed on one R9700 with the house UD-Q5 is a 64 GB kit from the measured 37.32 GB peak at 4K. RAM needed at full context uses the measured 42.9 GB peak at 256K. Those are llama-bench cards, not a template A/B. A chat template does not shrink the GGUF. UD-Q3_K_XL is a labeled estimate. Do not download Q3 to run.

Open in Stack Lab

There is no Stack Lab chat-template pack id on this site. Use the planner. Packages are not published yet and there is no public download. Anything unmeasured stays not yet measured.

Sources

• llama.cpp 23deec0 — llama-server README (--jinja, --chat-template, --chat-template-file, --chat-template-kwargs, --reasoning-format, GET /props, POST /apply-template)
• llama.cpp 23deec0 — function-calling.md (--jinja, Chat format: Generic, chatml YMMV)
• llama.cpp 23deec0 — gguf-py constants.py (tokenizer.chat_template)
• Qwen3.6-35B-A3B official model card (enable_thinking, preserve_thinking)
• Qwen3.8-Flash-Next official model card (enable_thinking, preserve_thinking, reasoning_effort)