Constrained decoding (JSON / GBNF) on a Radeon GPU with llama.cpp

What a grammar does

Constrained decoding is grammar sampling. llama.cpp only keeps tokens that still match the grammar. GBNF (GGML BNF) is the native format. A JSON Schema is converted to GBNF — that path is official on grammars/README.md and common/json-schema-to-grammar.cpp. The schema is not injected into the prompt except during tool calling, so describe the shape in the prompt. Tool-call attach stays on /harnesses and /setup/coding-assistant; this page is the GBNF / JSON-schema how-to.

GBNF vs JSON Schema

Use a JSON Schema when the output is a JSON object or array: tool arguments, a calculator result, or a recipe export. llama-server accepts -j / --json-schema, -jf / --json-schema-file, the json_schema body field, and /v1/chat/completions response_format ({"type":"json_object"} or a schema-constrained object). Official converter notes: additionalProperties defaults to false; unsupported keywords are skipped silently; a subset of JSON Schema is supported — inspect the grammar if a field disappears.

Use GBNF when the output is not JSON, or when the schema subset cannot express the shape (custom lists, chess notation, a tight enum the converter drops). Pass --grammar, --grammar-file, or the grammar body field. Samples live in llama.cpp grammars/. You can also convert a schema ahead of time and ship the GBNF.

Cost vs free-form

A grammar can change decode speed and can stall on a sloppy rule. llama.cpp documents performance gotchas on grammars/README.md and issue #4218 — write x{0,N} instead of a long chain of x? x? x?. AMDai does not invent an overhead tok/s. Measure free-form versus grammar on Stack Lab grammar-ab/1 and keep the card on /amdai-bench. Quality of the GGUF does not change; the token set does.

Stack Lab ships an opt-in grammar A/B as pack grammar-ab/1 (free-form versus grammar / JSON-schema, amdai-bench metrics). That pack is not allowlisted for Open in Stack Lab on this site. Do not invent a deep-link href for the pack. Open the planner or doctor, then record a card on /amdai-bench. Packages are not published yet.

House launch

the house default is Qwen3.6-35B-A3B Unsloth UD-Q5_K_XL, llama.cpp Vulkan, one R9700, 262144 context, f16 KV, no grammar flag. Add --json-schema or --grammar-file only when the client needs structured JSON. House /v1 is http://127.0.0.1:8080/v1 with model id qwen — same pin as /setup/coding-assistant. A grammar does not add GGUF weight. Fit is still Q6 / Q5 / Q4 on the calculator; UD-Q3_K_XL stays a labeled estimate.

Daily-driver recipe: qwen36-35b.llamacpp-vulkan.262k · Coding attach: /setup/coding-assistant · Harnesses: /harnesses.

# House daily-driver shape — no grammar. Same flags as /tuning.
./build-vk/bin/llama-server \
  -m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

llama.cpp flags

Cited from the llama.cpp server README and grammars/README.md (checked 2026-10-14). Vulkan first; HIP after rocminfo lists the gfx — Vulkan vs ROCm.

• --grammar GRAMMAR

  Inline GBNF string. Official llama.cpp server README. Also the grammar field on completion bodies. Empty / omitted means free-form.

  llama.cpp server README — --grammar GRAMMAR (checked 2026-10-14)

• --grammar-file FNAME

  Read GBNF from a file. Official on llama-cli, llama-completion, and llama-server. Samples live in llama.cpp grammars/.

  llama.cpp server README — --grammar-file FNAME; grammars/README.md

• -j / --json-schema SCHEMA

  JSON Schema string that llama.cpp converts to GBNF. Server README example: {"type":"object"} for any JSON object. llama-cli documents the same -j / --json path on grammars/README.md.

  llama.cpp server README — -j, --json-schema SCHEMA; grammars/README.md

• -jf / --json-schema-file FILE

  JSON Schema from a file. Official server README. Use this when the schema is longer than a one-liner. The file is a constraint, not prompt text.

  llama.cpp server README — -jf, --json-schema-file FILE

• grammar / json_schema body fields

  Completion endpoints accept grammar and json_schema. Default is no grammar. See tests/test-json-schema-to-grammar.cpp for the supported subset.

  llama.cpp server README — grammar and json_schema request fields

• response_format (OpenAI-shaped /v1/chat/completions)

  Official server README: {"type":"json_object"} for JSON, or a schema-constrained object ({"type":"json_object","schema":…} or {"type":"json_schema","schema":…}). Same idea as other OpenAI-inspired servers.

  llama.cpp server README — response_format (checked 2026-10-14)

llama-server with a JSON Schema

# Shape only — grammar decode is not measured on the local desktop.
# Official example schema: {"type":"object"} (server README --json-schema).
./build-vk/bin/llama-server \
  -m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 \
  -c 4096 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --json-schema '{"type":"object"}' \
  --host 127.0.0.1 --port 8080

llama-cli with a GBNF file

# Shape only — GBNF file from llama.cpp grammars/. Not a measured pin.
./build-vk/bin/llama-cli \
  -m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 \
  --grammar-file grammars/json.gbnf \
  -p 'Return one JSON object. Describe the fields in this prompt — the grammar is not injected.'

curl · response_format

# Shape only. Model id qwen matches GET /v1/models on the house server.
# response_format is official on the llama.cpp server README.
curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen",
    "messages": [{"role":"user","content":"Return one JSON object with a name string. Describe that shape here."}],
    "response_format": {"type":"json_object"}
  }'

Shape only. Speed knobs (-ngl, threads, batch, -fa, KV, split, draft) live on /tuning.

Measured 35B — no grammar

Official Qwen3.6-35B-A3B llama.cpp Vulkan cells from existing run cards. These launches have no --grammar / --json-schema. Grammar-on cells stay not measured yet — run Stack Lab grammar-ab/1. UD-Q3_K_XL is a labeled estimate. Full matrix: /benchmarks.

VRAM fit — Q6 / Q5 / Q4 + Q3 estimate

Calculator estimate for Qwen3.6-35B-A3B on llama.cpp, f16 KV. A grammar does not change weights or KV. Leftover is usable VRAM minus total (weights + KV + overhead). UD-Q3_K_XL is a labeled estimate. Do not read leftover as a grammar-speed cell.

One R9700 · 32K

One R9700 · 262K

More cards: R9700 · Q6 / Q5 / Q4 · MoE · Calculator.

System RAM

A grammar or schema does not add GGUF weight. When -ngl leaves layers on the host, the spill is still system RAM. Shopper lines stay Minimum system RAM needed (full-system estimate rounded to a real kit size, with a breakdown — labeled estimate until a card overlays it) and RAM needed at full context (cache included when measured llama.cpp peaks exist). House 35B UD-Q5_K_XL on one 32 GB R9700 does not offload. The 16 GB example below is house 35B Q4_K_M on an RX 7800 XT at 32K selected — estimate unless a measured card overlays the kit.

FAQ

• GBNF or JSON Schema — which should I pass?

  JSON Schema when the output is JSON (tool args, calculator, recipe export). GBNF when the output is not JSON, or the official schema subset cannot express the shape. llama.cpp converts a supported schema to GBNF. Describe the shape in the prompt — the schema is not injected except during tool calling.

  Coding assistant · Harnesses · GBNF

• Does a grammar slow decode on a Radeon?

  AMDai has not measured it. Official llama.cpp notes performance gotchas (issue #4218). Empty cells stay not measured yet — run Stack Lab grammar-ab/1. Do not copy a free-form 35B tok/s as a grammar figure.

  Stack Lab · amdai-bench · Speed up llama.cpp

• Can I constrain a Q3 file?

  UD-Q3_K_XL may appear as a labeled estimate (Hugging Face listing for Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf (16.8 GB)). House files stay Q6 / Q5 / Q4. A grammar does not change that. Q1/Q2 stay off.

  Quants · Q6 / Q5 / Q4 + Q3 estimate · Calculator

Sources

• llama.cpp server README (grammar / JSON schema)
• llama.cpp GBNF guide
• llama.cpp json-schema-to-grammar.cpp
• llama.cpp issue #4218 (grammar performance)
• JSON Schema
• llama.cpp repository
• AMDai — speed up llama.cpp
• AMDai — harnesses
• AMDai — coding assistant
• AMDai — backends
• AMDai — llama.cpp
• AMDai — MoE
• AMDai — Stack Lab
• AMDai — amdai-bench