Radeon vs Mac (Apple Silicon) for local LLMs

Memory, bandwidth, stack, Linux

Hardware numbers cite AMD or Apple. Mac and Strix Halo speed stay not measured here. Q6 / Q5 / Q4 first. UD-Q3_K_XL is a labeled estimate.

• Usable model memory

  cited

  Radeon / Strix Halo

  Discrete VRAM is the sticker: 32 GB on the R9700, 24 GB on the 7900 XTX, 64 GB as two R9700s. Strix Halo is one shared LPDDR5x pool — AMD lists max 128 GB. An AMD article on the 128 GB kit says up to 112 GB allocatable by the GPU. The calculator’s 70% unified slice is an estimate, not a BIOS reading.

  Mac

  Apple lists unified memory on the Mac Studio spec pages (M4 Max 36–128 GB, M3 Ultra 96–256 GB, M5 Max 36–128 GB, M5 Ultra 96–512 GB). macOS keeps a slice. We do not invent a Metal usable percent. The calculator’s 70% unified default is an estimate, not a Metal reading.

• Memory bandwidth

  cited

  Radeon / Strix Halo

  Cited AMD peaks: R9700 640 GB/s, 7900 XTX 960 GB/s. Two cards are two pools, not 1280 GB/s on one model. Strix Halo bandwidth on the Halo developer platform is 256 GB/s (128 GB, 8000 MT/s). The AI Max+ 395 product page lists LPDDR5x-8000 and 256-bit — we do not derive a second GB/s from that.

  Mac

  Cited Apple peaks on Mac Studio: M4 Max 410 GB/s (32-core) / 546 GB/s (40-core); M3 Ultra 819 GB/s; M5 Max 460 / 614 GB/s; M5 Ultra 1.2 TB/s. Peak GB/s is not a tok/s.

• Prefill vs decode

  pending

  Radeon / Strix Halo

  Measured on the local desktop llama.cpp Vulkan, one R9700, house 35B only. Decode and prefill live in the table below and on /benchmarks. Strix Halo speed stays not measured here. Dual-card split of the same file is often slower than one card at short context — that is a measured split tax, not a Mac number.

  Mac

  Mac prefill and decode for this 35B file stay not measured here. We will only print a third-party llama.cpp figure with a source link, labeled third-party, and not apples-to-apples. MLX is a different engine. Do not copy a viral screenshot onto this page.

• Software stack

  cited

  Radeon / Strix Halo

  llama.cpp first: Vulkan (GGML_VULKAN=ON) is the house daily driver; HIP (GGML_HIP=ON) after rocminfo lists the gfx. Cited at llama.cpp build.md and /compare/vulkan-vs-rocm. Strata / vLLM stay on /backends.

  Mac

  llama.cpp Metal is the documented macOS GPU compile (GGML_METAL). MLX is Apple’s array framework — a different stack, not a llama.cpp tok/s. We do not invent a Metal-in-Linux path. Asahi Linux exists; it is not measured on the local desktop.

• Upgradeability

  cited

  Radeon / Strix Halo

  A discrete card swaps. A second R9700 is a measured layout on the local desktop. Strix Halo memory is the soldered kit — pick 64 GB or 128 GB at purchase. AMD lists max 128 GB.

  Mac

  Unified memory is configured at purchase on Apple’s spec pages. You do not add a GPU later. A bigger Studio SKU is a different product, not an upgrade slot.

• Power

  cited

  Radeon / Strix Halo

  Board labels: R9700 300 W TBP, 7900 XTX 355 W TBP. Two cards are 2 × 300 W TBP cited — not a dual-card wall figure we measured. Strix Halo default TDP is 55 W; the Halo developer platform lists 120 W. PSU sizing stays on /power. No invented kit wattage.

  Mac

  Apple lists Mac Studio maximum continuous power as 480 W. That is the enclosure, not a chip TDP we invent. Idle or LLM-load watts are not measured here.

• Linux support

  cited

  Radeon / Strix Halo

  Ubuntu 24.04+ is the measured local desktop path. llama.cpp Vulkan first. ROCm 10.1 lists the R9700 / gfx1201 and Ryzen AI Max+ 395 / gfx1151 rows. Windows Vulkan is a cited estimate at /setup/windows.

  Mac

  macOS is Apple’s listed OS on the Studio spec pages. llama.cpp Metal and MLX are macOS stories. Asahi Linux is a cited third-party Apple Silicon Linux project — not measured on the local desktop, not a Metal port.

Vendor specs

Memory, bandwidth, and power. Each cell links the vendor spec page. TBP is AMD’s board label. Mac Studio 480 W is Apple’s maximum continuous enclosure figure, not a chip TDP we invent. Buttons say Check price — we do not show a live or stale figure. Apple SKUs have no /go/ button.

Radeon / APU pages: Radeon AI PRO R9700, Radeon RX 7900 XTX, Two R9700s, Ryzen AI Max+ 395 / Strix Halo. Apple pages: Mac Studio M4 Max (2025), Mac Studio M3 Ultra (2025), Mac Studio M5 Max, Mac Studio M5 Ultra.

Speed

R9700 decode is measured on the local desktop (amdai-bench/run@1, llama.cpp Vulkan, one card). Mac and Strix Halo cells say “not measured here.” A third-party llama.cpp figure may appear later only with a source link, labeled third-party, and not apples-to-apples (different engine, quant, or context). We will not estimate one.

Full 4K / 32K / 128K / 256K matrix including Q6_K: one vs two GPU · benchmarks · run cards.

What fits — Q6 / Q5 / Q4 + Q3 estimate

House Qwen3.6-35B-A3B on llama.cpp, f16 KV, 32K. Discrete kits use the VRAM sticker. Unified kits use the calculator’s 70% slice — an estimate, not a Metal or BIOS reading. UD-Q3_K_XL is a labeled estimate (Hugging Face listing for Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf (16.8 GB)). Nothing below Q3.

System RAM

When -ngl leaves layers on the host, print both shopper labels: Minimum system RAM needed and RAM needed at full context. House 35B UD-Q5_K_XL on one 32 GB R9700 does not offload. A 24 GB 7900 XTX with a Q6 file at 262K is the offload example. Strix Halo / Mac unified is one shared pool — same two labels.

Software reality

llama.cpp first (Vulkan on Radeon, Metal on a Mac). HIP, MLX, and Asahi stay labeled. Longer notes: /backends, /compare/vulkan-vs-rocm, /setup.

• llama.cpp · Vulkan vs Metal

  Radeon

  Consumer default on AMDai. House daily driver is one R9700, Vulkan, UD-Q5_K_XL, 262K, f16 KV. Measured on the local desktop.

  Mac

  llama.cpp documents GGML_METAL on macOS. We have no ingested first-party or cited third-party 35B Metal tok/s for the Studio SKUs on this page, so those cells stay not measured here.

  llama.cpp on AMDai · llama.cpp repository · llama.cpp build.md

• llama.cpp · ROCm / HIP

  Radeon

  HIP is the ROCm compile when rocminfo lists the gfx. Shopper compare: /compare/vulkan-vs-rocm. Try Vulkan first if HIP configure is a mess.

  Mac

  HIP is not an Apple path. Metal is the matching llama.cpp compile on macOS.

  Vulkan vs ROCm · Vulkan vs HIP · What is ROCm

• MLX

  Radeon

  MLX is not a Radeon stack. Do not expect a HIP MLX.

  Mac

  MLX is Apple’s array framework on Apple Silicon. It is not llama.cpp and not a local desktop card. We do not invent an MLX 35B tok/s.

  ml-explore/mlx

• Linux vs macOS

  Radeon

  Ubuntu LTS on AMD’s current ROCm matrix is the path we write and measure. llama.cpp Vulkan is the honest Windows path for tokens on Radeon.

  Mac

  macOS is what Apple lists. Asahi Linux is a cited Apple Silicon Linux project. We do not have an Asahi or macOS tok/s, and we do not invent Vulkan-on-Asahi.

  First-run setup · Asahi Linux

Which should you buy?

Use case first, then memory class, then software. Size vs speed still changes the pick on setup by budget and the calculator. FAQ: /faq.

• Speed on the house 35B

  One R9700 if you want the path we measured.

  House daily driver is Qwen3.6-35B-A3B Unsloth UD-Q5_K_XL on one R9700, llama.cpp Vulkan, 262144 context, f16 KV. Decode in the table is that card. A 7900 XTX is the 24 GB discrete sibling — size Q6/Q5/Q4 on the calculator. Mac Studio and Strix Halo speed stay not measured here. Do not pick a Mac to chase an unmeasured tok/s.

  Daily driver · 35B llama.cpp · 262K

  Minimum system RAM needed

  42.9 GB

  Measured · 42.9 GiB raw · 256K context

  local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16

  RAM needed at full context

  42.9 GB

  Measured · 42.9 GiB raw · 256K full context

  local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16

  Measured peak system RAM on the local desktop: 42.9 GB (Linux GiB labeled GB). Measured, pending re-verify after RAM fix.

  Breakdown

  Offloaded weights ·

  0 GiB

  Separable embed in RAM ·

  0 GiB

  KV in RAM at selected ·

  0 GiB

  KV in RAM at full context ·

  0 GiB

  Runtime / backend ·

  0.85 GiB

  OS / desktop baseline ·

  8.00 GiB

  Headroom ·

  16.0 GiB

  R9700 measured page · Daily-driver recipe · Benchmarks

• Max model size

  Buy the memory class, then size the file.

  32 GB discrete (R9700) holds the house 35B Q5 at 262K without offload on our cards. 24 GB (7900 XTX) is the next discrete bracket — Q4 may fit; Q6 at long context needs the calculator and both RAM lines if layers spill. Two R9700s are 64 GB of VRAM. Strix Halo and Mac Ultra list much larger unified pools (AMD max 128 GB; Apple M3 Ultra up to 256 GB; M5 Ultra up to 512 GB). Unified is one kit with the OS. We do not invent a “fits” claim past the calculator and the vendor memory label. Q3 stays a labeled estimate.

  Minimum system RAM needed

  48 GB kit

  Estimate · 32.7 GiB raw · 256K context

  RAM needed at full context

  48 GB kit

  Estimate · 32.7 GiB raw · 256K full context

  Estimate. Offloaded weights + KV in RAM + runtime + OS/desktop + headroom, rounded up to a 48 GB DIMM kit.

  Breakdown

  Offloaded weights ·

  2.73 GiB

  Separable embed in RAM ·

  0 GiB

  KV in RAM at selected ·

  5.06 GiB

  KV in RAM at full context ·

  5.06 GiB

  Runtime / backend ·

  0.88 GiB

  OS / desktop baseline ·

  8.00 GiB

  Headroom ·

  16.0 GiB

  Calculator · Strix Halo · RX 7900 XTX · Dual / multi-GPU · Quants

• Long context

  The measured 262K path is one R9700, llama.cpp Vulkan, f16 KV.

  Overnight 35B cards publish 4K / 32K / 128K / 256K because KV grows and decode slows. That is a Radeon Vulkan sweep. Mac and Strix Halo long-context tok/s stay not measured here. Size KV on the calculator. Print both RAM lines when the file or the window spills onto the host.

  Daily driver · 35B llama.cpp · 262K

  Minimum system RAM needed

  42.9 GB

  Measured · 42.9 GiB raw · 256K context

  local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16

  RAM needed at full context

  42.9 GB

  Measured · 42.9 GiB raw · 256K full context

  local-desktop-20261012-2314-35b-udq5kxl-1gpu-256k-llamacpp-f16

  Measured peak system RAM on the local desktop: 42.9 GB (Linux GiB labeled GB). Measured, pending re-verify after RAM fix.

  Breakdown

  Offloaded weights ·

  0 GiB

  Separable embed in RAM ·

  0 GiB

  KV in RAM at selected ·

  0 GiB

  KV in RAM at full context ·

  0 GiB

  Runtime / backend ·

  0.85 GiB

  OS / desktop baseline ·

  8.00 GiB

  Headroom ·

  16.0 GiB

  Long context · One vs split · Calculator

• Budget / already-own

  Used 7900 XTX or a kit you already own. We do not print a price.

  Buttons on AMDai say Check price — no live or stale figure. A used 7900 XTX walk is /guides/buying-used-radeon. A Mac you already own should start with llama.cpp Metal; do not buy a Radeon unless you want Linux, a swappable card, or the measured 35B path. Strix Halo is the AMD unified-memory box when you want one kit and no discrete GPU. Q3 stays a labeled estimate. Q1/Q2 stay off.

  Minimum system RAM needed

  32 GB kit

  Estimate · 24.7 GiB raw · 32K context

  RAM needed at full context

  32 GB kit

  Estimate · 24.8 GiB raw · 256K full context

  Estimate. Offloaded weights + KV in RAM + runtime + OS/desktop + headroom, rounded up to a 32 GB DIMM kit.

  Breakdown

  Offloaded weights ·

  0 GiB

  Separable embed in RAM ·

  0 GiB

  KV in RAM at selected ·

  0 GiB

  KV in RAM at full context ·

  0 GiB

  Runtime / backend ·

  0.71 GiB

  OS / desktop baseline ·

  8.00 GiB

  Headroom ·

  16.0 GiB

  Buying a used Radeon · Budget · Affiliate disclosure · Strix Halo