Flash with RAM offload

Estimate · file size + KV

System RAM at 262K

Minimum system RAM is the 4K load. RAM at full context is 262144 tokens. Spill is weights plus KV that missed the VRAM pool (32 GB on one R9700, 64 GB on two). Every GiB column is an estimate. Linux GiB labeled GB.

Two R9700s · 64 GB VRAM

One R9700 · 32 GB VRAM

More of the file lands on DDR5. Decode at this card count is not measured on the local desktop.

Estimate · 96 / 128 / 192 / 256

Recommended setup per RAM kit

Picks follow the calculator kit rounding. They are not a tok/s promise. 96 GB (2x48) is the pick for most builds. 256 GB is what local desktop ships (~249 GB usable).

From data/runs · Flash only

Decode by quant, cards, and engine

Q3–Q6 × 1 vs 2 R9700s × llama.cpp vs Strata × 4K / 32K / 128K / 256K. A cell fills when a verified Flash run card matches. Until then it stays empty.

• No Flash Q3 decode in data/runs yet. When the approved measured card lands, this grid will show it.
• Flash Q4 decode is not measured on the local desktop.
• Flash Q5 decode is not measured on the local desktop.
• One-card Flash decode cells are empty. Prior 32.6 / 16 points recorded card count as unpublished.
• 4K Flash decode is not in the jsonl yet.
• 128K Flash decode is not in the jsonl yet.

Measured · DDR5-3600 vs 5200

Memory speed vs decode

Same two-card Q6, same 262K load. Only the DIMM clock changes. llama.cpp at 8K is the 15.9 → 18.5 pair. Strata at 8K is the 32.0 → 32.8 pair. Missing contexts stay not measured on the local desktop.

Shape · flags not recorded

Expert-offload launch

llama.cpp places leftover experts with -ot (tensor override) and --n-cpu-moe. Strata’s equivalent is the expert-cache / RAM / SSD tier. Exact regex, N, and Strata flags are not recorded — we will not invent them.

llama.cpp Vulkan

# Shape only — exact -ot tensor regex and --n-cpu-moe N are not recorded from the rig owner.
# After --list-devices, exclude the iGPU if two Vulkan devices appear.
./build-vk/bin/llama-server \
  -m ~/models/UD-Q6_K_XL \
  -ngl 99 \
  -c 262144 \
  --n-cpu-moe not recorded \
  -ot not recorded \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Strata

# Shape only — exact expert-cache / RAM / SSD-tier flags not recorded. No MTP.
# After rocminfo, exclude the iGPU if a second HIP device appears.
# This site’s convention: 127.0.0.1:8083 so llama-server can keep :8080.
curl -s http://127.0.0.1:8083/v1/models

Do this first

Pitfalls