Mixture-of-Experts on a Radeon GPU

MoE vs dense in plain words

Think of MoE as a library of specialists and a small dispatcher. Each token wakes a few experts (the A3B in Qwen3.6-35B-A3B names that active slice) instead of ringing every neuron. Dense 27B files on this site — Qwen3.6 27B and Qwen3.8 27B — run the whole 27B-class stack every token. VRAM follows the file you load plus KV, not the marketing “35B vs 27B” headline. Flash is still MoE, but a much larger one: official Qwen3.8-Flash-Next is 125B total / 6B active plus a 51B n-gram embedding llama.cpp can keep in system RAM. That file usually needs DDR5 expert offload, not a single 32 GB sticker.

Three fit stories

Same shopper question — “will this file sit on one 32 GB R9700?” — three answers. Do not reuse one VRAM number for the other two.

• 35B-A3B on one 32 GB R9700

  35B total / 3B active

  Official Qwen/Qwen3.6-35B-A3B card

  House daily driver: UD-Q5_K_XL on llama.cpp Vulkan, -c 262144, f16 KV, -ngl 99.

  Publisher listing 26.6 GB (Hugging Face Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf). Official overnight card: 25.97 GB VRAM at 4K and 29.34 GB at 256K on one R9700 — in-VRAM. Peak system RAM on that card is 37.32 GB at 4K and 42.9 GB at 256K (Linux GB). That is not an offload story.

  • R9700
  • 35B model page
  • 35B recipe
  • Benchmarks

• Dense 27B is a different file

  27.78B / 27.8B all active

  Official Qwen3.6-27B and Qwen3.8-27B cards

  Cite the 27B rows on /qwen. Do not reuse 35B-A3B VRAM as a 27B rule.

  Dense means every parameter runs. Official Unsloth listings: Qwen3.6-27B Q6_K 22.52 GB, Q5_K_M 19.51 GB, Q4_K_M 16.82 GB; Qwen3.8-27B UD-Q6_K 21.98 GB, UD-Q5_K_M 19.77 GB, UD-Q4_K_M 16.46 GB. Decode on those files is not yet measured by AMDai. Size KV on the calculator. Q3 stays a labeled estimate.

  • Qwen compare
  • Qwen3.6 27B
  • Qwen3.8 27B
  • Compare models

• Flash-class MoE usually needs DDR5

  125B total / 6B active + 51B n-gram

  Official Qwen/Qwen3.8-Flash-Next card

  House fast-agent: UD-Q6_K_XL on Strata after llama.cpp works. Expert offload on DDR5.

  Publisher UD-Q6_K_XL folder ~169 GB; UD-Q5_K_XL 158.3 GB; UD-Q4_K_XL ~111 GB. UD-Q3_K_XL ~90 GB is a labeled estimate. That file does not sit in 32 GB VRAM. Expert offload parks experts in system RAM — read Minimum system RAM needed and RAM needed at full context. Shopper kits on /flash-ram-offload are 128 / 192 / 256 GB, labeled estimate until a card overlays the kit.

  • Flash RAM offload
  • Flash model page
  • Flash daily
  • Strata

Quant ladder

Q6 / Q5 / Q4 first. Q3 is a labeled estimate — no Q3 download or run on this page. Nothing below Q3. Publisher GB only when a listing exists. Dense 27B speed stays not yet measured by AMDai.

35B-A3B · llama.cpp

Dense 27B · cited listings

Flash · expert offload

Measured 35B on one R9700

Official llama.cpp Vulkan cells from existing run cards. Empty stays not measured yet. UD-Q3_K_XL is a labeled estimate. Full matrix: /benchmarks.

House launch

Start here. Exact recipe: qwen36-35b.llamacpp-vulkan.262k.

# House daily-driver shape — /recipes/qwen36-35b.llamacpp-vulkan.262k
./build-vk/bin/llama-server \
  -m ~/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 \
  -c 262144 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Expert offload and system RAM

Expert offload is an MoE-only lever: keep attention on the GPU and park expert tensors in system RAM. llama.cpp flags are -ot (tensor override regex) and --n-cpu-moe (how many experts stay on CPU). Exact regex and N from the rig owner are not recorded — copy the shape on /flash-ram-offload, do not invent values. Wherever offload appears, shopper lines stay Minimum system RAM needed (full system estimate: offloaded weights + OS / cache / headroom, rounded up to a real kit such as 192 or 256 GB, labeled estimate until measured) and RAM needed at full context (262K with cache; measured numbers only when a card exists). House 35B on one R9700 does not offload. Flash kits on this site are 96 / 128 / 192 / 256 GB.

# Shape only — exact -ot tensor regex and --n-cpu-moe N are not recorded from the rig owner.
# After --list-devices, exclude the iGPU if two Vulkan devices appear.
./build-vk/bin/llama-server \
  -m ~/models/UD-Q6_K_XL \
  -ngl 99 \
  -c 262144 \
  --n-cpu-moe not recorded \
  -ot not recorded \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --host 127.0.0.1 --port 8080

Strata expert-cache flags stay on the house Flash path only — shape, not invented values.

# Shape only — exact expert-cache / RAM / SSD-tier flags not recorded. No MTP.
# After rocminfo, exclude the iGPU if a second HIP device appears.
# This site’s convention: 127.0.0.1:8083 so llama-server can keep :8080.
curl -s http://127.0.0.1:8083/v1/models

Shopper kits already on Flash RAM offload: 96 / 128 / 192 / 256 GB. Both labels print on the house recipes below.

Stack Lab and amdai-bench

Open in Stack Lab is allowlisted only. House recipes on this page are the daily 35B llama.cpp Vulkan pin and the Flash Strata pin, plus doctor. There is no allowlisted Stack Lab screen for an amdai-bench MoE expert-offload A/B. Do not invent a deep link. Measure expert offload on your box from /stack-lab planner and /amdai-bench. The gap stays on /wanted until that screen exists.

Two cards

A second card is a split or a second process, not a magic MoE switch. Mixing RDNA generations and two-card 35B vs Flash live on /multi-gpu and the cited how-to /setup/multi-gpu. We do not invent a combo-preset tok/s. House 35B already fits one 32 GB R9700 at UD-Q5_K_XL. Buy the second card for a file that does not fit, or to run two models.

Dual / multi-GPU · Multi-GPU llama.cpp how-to · Speed up llama.cpp

FAQ

MoE vs dense — why does 35B fit 32 GB when 27B looks smaller?

MoE activates a slice. Official Qwen3.6-35B-A3B is 35B total / 3B active. Dense Qwen3.6 27B and Qwen3.8 27B run the whole 27B-class stack. File size plus KV is the VRAM question, not the headline parameter count. Official UD-Q5_K_XL 35B on one R9700 is 25.97 GB VRAM at 4K and 29.34 GB at 256K. Dense 27B listings live on /qwen. Decode on those 27B files is not yet measured by AMDai.

MoE vs dense · Qwen compare · R9700

Which 35B quant should I start with?

House daily driver is UD-Q5_K_XL (26.6 GB listing). The overnight llama.cpp Vulkan sweep also recorded Q6_K, Q5_K_M, and Q4_K_M. UD-Q3_K_XL (16.8 GB listing) is a labeled estimate — not a local desktop run. Nothing below Q3. Size a different card on /can-run and the calculator.

Quants · Q6 / Q5 / Q4 + Q3 estimate · Can my Radeon run it? · Calculator

Is Flash just a bigger 35B?

No. Flash is a different MoE. Official Qwen3.8-Flash-Next is 125B total / 6B active plus a 51B n-gram embedding. Publisher UD-Q6_K_XL is ~169 GB and usually needs DDR5 expert offload. Strata is the house Flash try after llama.cpp works — that citation already lives on /flash-ram-offload. Do not reuse 35B VRAM as a Flash rule.

Flash RAM offload · Flash model page · Expert offload

Is there an Open in Stack Lab MoE expert-offload A/B?

No. That screen is not allowlisted. Do not invent a deep link. Open the daily 35B recipe, the Flash recipe, or doctor. Measure expert offload on your box with /stack-lab planner and an amdai-bench card. The gap is on /wanted.

Stack Lab · amdai-bench · Wanted

Sources

• Qwen3.6-35B-A3B official model card
• unsloth/Qwen3.6-35B-A3B-GGUF
• Qwen3.6-27B official model card
• unsloth/Qwen3.6-27B-GGUF
• Qwen3.8-27B official model card
• unsloth/Qwen3.8-27B-GGUF
• Qwen3.8-Flash-Next official model card
• unsloth/Qwen3.8-Flash-Next-GGUF
• llama.cpp repository
• AMDai — Flash RAM offload
• AMDai — Qwen compare
• AMDai — MoE vs dense