mmap vs --no-mmap vs --mlock: model loading RAM on a Radeon GPU with llama.cpp

What mmap, --no-mmap, and --mlock do

mmap (memory-map) lets the kernel page the GGUF from NVMe instead of allocating a full anonymous copy. Official --load-mode auto is mmap unless a device does not support it. --load-mode mmap forces that path. --load-mode none reads the file into process RAM (the cited --no-mmap replacement — discussion #26469). --load-mode mlock locks a non-mapped copy in RAM so the OS cannot swap or compress it. --load-mode mmap+mlock maps the file and locks those pages (discussion #27912). --load-mode dio uses Direct I/O when available. -lzm / --lazy-mode can leave huge tensors on disk and read rows on demand — it requires mmap (env LLAMA_ARG_LAZY_MODE). llama.cpp PR #20834 folded --mmap / --no-mmap / --mlock into --load-mode. Those old flags are deprecated; do not combine them with --load-mode — only the last flag wins.

When each arm helps

Default mmap is the house-shaped path: the 35B file lives on NVMe and the kernel faults pages in. --no-mmap / --load-mode none can help when mmap progress looks stuck or a platform pages the map out, at the cost of a full RAM copy of the weights. --mlock / --load-mode mlock or mmap+mlock is the “keep this file in RAM” pin when the box is swapping. It does not shrink VRAM. It does not fix ErrorOutOfDeviceMemory — that is /out-of-vram. Flash expert offload is a different RAM line — see /flash-ram-offload. Do not copy the overnight 102.39 tok/s 4K decode cell as a mmap-hit figure. That number is llama-bench, one client, not a load-mode A/B.

The three A/B arms

Stack Lab Model loading A/B is mmap default versus --no-mmap versus --mlock. Official llama.cpp now spells those as --load-mode auto / none / mlock (or mmap+mlock). Every measured cell stays not yet measured until a card is ingested.

• mmap default

  --load-mode auto (official default; mmap unless a device cannot)

  Load time: not yet measured

  TTFT: not yet measured

  Minimum system RAM needed: 64 GB kit from the measured 37.32 GB llama-bench peak at 4K — not a load-mode A/B

  House-shaped path. Overnight cards do not record --load-mode.

• --no-mmap

  --load-mode none (cited replacement for deprecated --no-mmap)

  Load time: not yet measured

  TTFT: not yet measured

  RAM needed at full context: extra RSS for a full weight copy stays not yet measured

  Reads the GGUF into process RAM. Measure Model loading A/B before you keep it.

• --mlock

  --load-mode mlock (no mmap + lock) or mmap+mlock (map + lock)

  Load time: not yet measured

  TTFT: not yet measured

  RAM needed at full context: locked-resident peak stays not yet measured

  Stops swap/compress of the weight copy. Does not free VRAM.

Cited flags

Vulkan house path. The first block is the recorded llama-server line — no --load-mode. The rest are official server-README mappings, not local desktop pins. Replace the binary and GGUF paths.

# House daily driver as recorded (no --load-mode / --mmap / --no-mmap / --mlock pin).
# llama-server README: --load-mode default auto (mmap unless a device cannot).
# That default is official docs — not a local desktop pin we will invent.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja --host 127.0.0.1 --port 8080

# Official llama.cpp --load-mode from the server README — not a house pin.
# auto (default; env LLAMA_ARG_LOAD_MODE) = mmap unless a device cannot.
# none replaces deprecated --no-mmap. mlock replaces --no-mmap --mlock.
# mmap+mlock maps and locks. Measure Model loading A/B before you keep a new pin.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --load-mode auto \
  --cache-type-k f16 --cache-type-v f16 --jinja \
  --host 127.0.0.1 --port 8080

# Cited --no-mmap replacement. Official --load-mode none.
# Deprecated --no-mmap still maps here (discussion #26469). Not a house pin.
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --load-mode none \
  --cache-type-k f16 --cache-type-v f16 --jinja \
  --host 127.0.0.1 --port 8080

# Cited --mlock replacements. Not a house pin. RSS stays not yet measured.
# --load-mode mlock = no mmap + lock (old --no-mmap --mlock).
# --load-mode mmap+mlock = map + lock (discussion #27912).
./build-vk/bin/llama-server \
  -m /path/to/nvme/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf \
  -ngl 99 -c 262144 --load-mode mlock \
  --cache-type-k f16 --cache-type-v f16 --jinja \
  --host 127.0.0.1 --port 8080

Q6 / Q5 / Q4 plus a labeled Q3

Load mode is a mapping knob, not a smaller GGUF. House recipes stay Q4 and up. UD-Q3_K_XL is a labeled estimate — do not download it to run. Both RAM lines appear. Load-mode RSS stays not yet measured.

• Q6_K

  Qwen3.6-35B-A3B-Q6_K.gguf

  Load time: not yet measured

  Minimum system RAM needed: peak system RAM not recorded on a Model loading A/B card

  RAM needed at full context: peak system RAM not recorded on a Model loading A/B card

  Overnight llama-bench exists. mmap / --no-mmap / --mlock load RAM stays not yet measured.

• UD-Q5_K_XL

  Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf

  Load time: not yet measured

  Minimum system RAM needed: 64 GB kit from the measured 37.32 GB peak at 4K (no --load-mode on that card)

  RAM needed at full context: measured 42.9 GB peak at 256K. Extra --no-mmap / --mlock RSS is a labeled estimate on top.

  House file. One-client 102.39 tok/s decode at 4K is llama-bench, not a load-mode A/B.

• Q4_K_M

  Qwen3.6-35B-A3B-Q4_K_M.gguf

  Load time: not yet measured

  Minimum system RAM needed: peak system RAM not recorded on a Model loading A/B card

  RAM needed at full context: peak system RAM not recorded on a Model loading A/B card

  Overnight llama-bench exists. Recommended floor for house recipes. Load A/B still needs Model loading A/B.

• UD-Q3_K_XL · labeled estimate

  Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf

  Load time: not yet measured

  Minimum system RAM needed: 32 GB kit (labeled estimate)

  RAM needed at full context: 32 GB kit (labeled estimate)

  Labeled estimate (Hugging Face listing for Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf (16.8 GB)). Do not download Q3 to run. Q3 does not buy a mmap A/B.

Run it yourself

# Stack Lab Model loading A/B: mmap default vs --no-mmap vs --mlock.
# Not an allowlisted deep link — open the planner.
# Record load time, TTFT, decode, prefill, peak VRAM per card + total,
# and both RAM lines. Do not hand-type tok/s.

# Export a lab-notes-report/1 pack from Stack Lab, then ingest into Lab Notes.
# Each row must be a validating amdai-bench/run@1 card.
npx tsx src/scripts/amdai-bench.ts ingest --dry-run pack.json
npx tsx src/scripts/amdai-bench.ts ingest pack.json
npx tsx src/scripts/amdai-bench.ts ingest pack.json data/runs/stack-lab-import.jsonl
npx tsx src/scripts/amdai-bench.ts validate

Zero to a measured load-mode pin

1. 1 · Cited llama.cpp --load-mode · mmap / --no-mmap / --mlock RAM not yet measured

   Keep the house Vulkan pin — do not invent a load-mode

   Vulkan llama-server, Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf, -ngl 99 -c 262144 --cache-type-k f16 --cache-type-v f16 --jinja --host 127.0.0.1 --port 8080. Official --load-mode default is auto. House recipes do not record --load-mode, --mmap, --no-mmap, or --mlock. House recipes stay Q4 and up. UD-Q3_K_XL is a labeled estimate — do not download it to run.

   • Always-on server
   • Serve Qwen
   • Qwen3.6-35B

2. 2 · Cited llama.cpp --load-mode · mmap / --no-mmap / --mlock RAM not yet measured

   Map shopper flags onto --load-mode

   mmap default is --load-mode auto. --no-mmap is --load-mode none. --mlock with --no-mmap is --load-mode mlock. mmap plus lock is --load-mode mmap+mlock. PR #20834 deprecated the old pair. Do not pass both. Load time and RSS stay not yet measured.

   • Speed up llama.cpp
   • llama.cpp
   • Glossary · mmap

3. 3 · Cited llama.cpp --load-mode · mmap / --no-mmap / --mlock RAM not yet measured

   Print both RAM lines before you lock the file

   Minimum system RAM needed stays the measured 64 GB kit from 37.32 GB at 4K. RAM needed at full context stays 42.9 GB at 256K. Those cards do not record --load-mode. Extra --no-mmap / --mlock RSS is a labeled estimate. Flash expert offload is /flash-ram-offload, not a load-mode pin.

   • Calculator
   • Flash RAM offload
   • KV cache

4. 4 · Cited llama.cpp --load-mode · mmap / --no-mmap / --mlock RAM not yet measured

   Do not use mlock to fix out of VRAM

   ErrorOutOfDeviceMemory and hipErrorOutOfMemory are VRAM. --mlock only pins system RAM. Cut -c, change KV type, or step the quant — see /out-of-vram. A process that swaps or never finishes mapping weights is /troubleshoot#mmap-ram.

   • Out of VRAM
   • System RAM too small
   • R9700

5. 5 · Cited llama.cpp --load-mode · mmap / --no-mmap / --mlock RAM not yet measured

   Run Stack Lab Model loading A/B — do not invent RSS

   Stack Lab ships Model loading A/B (mmap default vs --no-mmap vs --mlock). Record load time, TTFT, decode, prefill, peak VRAM per card and total, and both RAM lines. Export a lab-notes-report/1 pack and ingest with npx tsx src/scripts/amdai-bench.ts ingest pack.json. Packages are not published yet. Open the planner — do not invent a mmap deep link.

   • Stack Lab
   • amdai-bench
   • Run cards
   • How we test

RAM lines (Ubuntu-measured)

Minimum system RAM needed on one R9700 with the house UD-Q5 is a 64 GB kit from the measured 37.32 GB peak at 4K. RAM needed at full context uses the measured 42.9 GB peak at 256K. Those are llama-bench cards with 0% RAM offload and no recorded --load-mode. --no-mmap and --mlock can grow process RSS for the weight file — labeled estimate unless a Model loading A/B card measures the peak. UD-Q3_K_XL is a labeled estimate. Do not download Q3 to run.

Open in Stack Lab

Model loading A/B is the mmap / --no-mmap / --mlock A/B. It is not an allowlisted deep link on this site. Use the planner. Packages are not published yet and there is no public download. Anything unmeasured stays not yet measured.

Sources

• llama.cpp — llama-server README (-lm / --load-mode, --lazy-mode, LLAMA_ARG_LOAD_MODE)
• llama.cpp PR #20834 — --mmap / --no-mmap / --mlock folded into --load-mode
• llama.cpp discussion #26469 — --no-mmap replacement is --load-mode none
• llama.cpp discussion #27912 — --load-mode mlock vs mmap+mlock
• llama.cpp repository
• Qwen3.6-35B-A3B official model card