What it is
A compiled MoE inference engine (Niko1221/Strata) with an OpenAI-shaped HTTP front. It is aimed at Qwen 3.8 Flash-class models that do not fit in one GPU: hot experts in VRAM, the rest in RAM, a lookup table on SSD.
Architecture in plain English
Resident engine plus serve/server.py. Documented APIs include /v1/chat/completions, Anthropic /v1/messages, and OpenAI /v1/responses. AMD path is a HIP build. Default documented listen is :8080; this site’s convention for the fast-agent stack is :8083 so it does not collide with llama-server.
What it does well
Faster Flash Q6 agent loop than llama.cpp on the same rig, once the daily driver is already proven.
What it does badly
First install; undocumented flags; being your only engine.
ROCm / Radeon
HIP download on the v0.1.42 notes. Official notes name RX 7900 / 9070 / AI PRO R9700. gfx12 prompt-path switches default on for gfx1200/gfx1201.
Vulkan. Not documented as a Vulkan engine. Do not treat it as “Vulkan won.”
Supported quants
Publisher Flash file we run is Unsloth UD-Q6_K_XL. Floor is still Q4. Do not use their IQ2/IQ3 examples as a house quant.
House floor is still Q4. Prefer Q5. See quants and thecalculator.
Install on an R9700
# Official AMD HIP build is a download on the v0.1.42 notes.
# Linux update path they document: ./update.sh (setup replaces the engine).
# Exact checkout / pack flags for our rig stay TODO in stacks.yaml.
Run
# This site’s convention: listen on 127.0.0.1:8083 so llama-server can keep :8080.
# TODO: exact strata command line / measured VRAM+RAM from the rig owner.
curl -s http://127.0.0.1:8083/v1/models
Tuning flags
0.1.42 turns gfx12 prompt switches on for gfx1200/gfx1201 (STRATA_GFX12_DEFAULTS=0 disables). Tail-skip and PCIe-share defaults are documented as CUDA-oriented and off on AMD — do not copy NVIDIA env vars onto the R9700 without reading the notes. STRATA_ROUTE_TAIL_SKIP is documented as off on AMD.
When to choose it
Flash agent after llama.cpp is boring. First-party ~32.6 tok/s vs llama.cpp ~16 on that Q6.
Daily driver already works and you want the fast agent.