Fine-tuning on Radeon

What works on Radeon today

LoRA keeps the base weights frozen and trains small adapter matrices. QLoRA loads the base in 4-bit and still trains those adapters. Hugging Face PEFT/TRL document both. Unsloth documents AMD hardware, including Radeon RX 9000 / 7000, Strix Halo, and bitsandbytes QLoRA. AMD’s Radeon playbook walks PEFT + TRL LoRA and QLoRA on a Radeon GPU. After training, merge the adapter, convert to GGUF with llama.cpp, then serve with llama-server. llama.cpp stays first for inference. Training speed is not yet measured on our R9700s.

• PyTorch ROCm wheels

  AMD documents pip install from its ROCm wheel index. Match the gfx extra AMD lists for your card and the ROCm line already on /rocm/versions. Do not invent a wheel pin.

  Cite · AMD — Install PyTorch for ROCm

• Unsloth

  Unsloth’s AMD page: train, fine-tune, RL, and deploy on AMD GPUs. RDNA 4 (RX 9000, gfx1200/1201), RDNA 3 (RX 7000), and Strix Halo are listed Full on Windows + WSL + Linux. QLoRA via bitsandbytes. Install path: uv pip install "unsloth[amd]" after pinning the ROCm torch you already installed.

  Cite · Unsloth AMD docs

• Hugging Face PEFT / TRL

  HF TRL documents LoRA and QLoRA (load_in_4bit). That page is not Radeon-specific. AMD’s Radeon playbook uses Transformers + PEFT + TRL on an AMD Radeon GPU (example: google/gemma-3-4b-it). AMD’s ROCm how-to uses PEFT + TRL on Instinct. Consumer Radeon PEFT/TRL beyond that playbook is not verified on Radeon.

  Cite · AMD AI Playbooks + HF TRL PEFT

• Axolotl

  Axolotl’s own docs list “NVIDIA GPU or AMD GPU” as a requirement. AMD’s ROCm blog walks Axolotl on Instinct (Llama 3), not a consumer Radeon. Axolotl on a home Radeon is not verified on Radeon.

  Cite · Axolotl docs + AMD ROCm blog

• torchtune

  The torchtune README lists AMD ROCm among custom devices. AMD’s notebook is Instinct MI300X. torchtune on a consumer Radeon is not verified on Radeon.

  Cite · torchtune README + AMD notebook

VRAM — LoRA vs QLoRA

Unsloth publishes an absolute-minimum VRAM table: QLoRA is 4-bit, LoRA is 16-bit, and “sometimes more VRAM is required depending on the model.” Those numbers are Unsloth’s table, not a local desktop measurement and not a consumer-Radeon run. AMD’s Radeon playbook (Gemma 3 4B) says LoRA and QLoRA need at least 16 GB dedicated GPU memory and 32 GB system RAM, and that Linux does not fall back to shared GPU memory when VRAM runs out. House 35B MoE is not in Unsloth’s dense-parameter table — we do not invent a 35B training fit.

Map to cards we already list

Card VRAM is the hardware label on each /gpus sheet. Training numbers above stay Unsloth or AMD playbook cites. Speed stays not yet measured on our R9700s.

• 16 GB class

  AMD playbook (Gemma 3 4B): LoRA / QLoRA need at least 16 GB dedicated VRAM and 32 GB system RAM; Linux does not spill to shared GPU memory. Unsloth 14B QLoRA minimum is 8.5 GB; Unsloth 27B QLoRA minimum is 22 GB — above this class. No cited consumer-Radeon training run. Labeled estimate from those tables only.

  Speed · not yet measured on our R9700s

  • RX 9070 XT
  • RX 9070
  • RX 9060 XT
  • RX 7600 XT
  • RX 6800 XT
  • RX 6950 XT / 6900 XT
  • RX 7900 GRE
  • RX 7800 XT

• 20–24 GB class

  Unsloth 27B QLoRA minimum is 22 GB. Unsloth 27B LoRA is 64 GB — not this class. AMD playbook LoRA “typical VRAM” for its 4B example is 24–32 GB. Confirm the tool’s own memory notes. Labeled estimate. No cited local desktop training run.

  Speed · not yet measured on our R9700s

  • RX 7900 XTX
  • RX 7900 XT

• 32 GB R9700

  Unsloth 32B QLoRA minimum is 26 GB; 27B QLoRA is 22 GB. Unsloth 70B QLoRA is 41 GB — above one 32 GB card. House 35B MoE is not in Unsloth’s dense table; do not invent a 35B LoRA fit. ROCm 10.0 training reset note is on /rocm-problems. Speed not yet measured on our R9700s.

  Speed · not yet measured on our R9700s

  • R9700

• Dual R9700

  Unsloth documents AMD multi-GPU on Linux and says Windows has no GPU-collective backend yet. Unsloth 70B QLoRA minimum is 41 GB. We have no cited split-across-two-R9700 training number. Dual-card inference stays on /multi-gpu. Not verified on Radeon as a training layout.

  Speed · not yet measured on our R9700s

  • R9700
  • Dual / multi-GPU

• Strix Halo / shared memory

  Unsloth lists Ryzen AI Max / Strix Halo as Full on Windows + WSL + Linux. AMD’s playbook discusses shared video memory on Ryzen AI Halo. Unsloth does not publish a Strix Halo training VRAM table we can reprint. Not verified on Radeon as a measured training fit.

  Speed · not yet measured on our R9700s

  • Strix Halo

Linux path — Ubuntu, then a wheel

1. Step 1

   Start on Ubuntu 24.04+ with the site ROCm pin

   Ubuntu 24.04+ is the measured AMDai OS. Install ROCm from the pin already on /rocm/versions — do not invent a new version. Confirm the GPU with rocminfo / amd-smi before you download a base model. The install walkthrough is /tutorials/install-rocm. After a version move, rebuild the matching PyTorch ROCm wheel — /rocm/upgrade.

   • Ubuntu setup
   • ROCm versions
   • Upgrade or reinstall ROCm
   • Install ROCm

2. Step 2

   venv, then a ROCm PyTorch wheel

   python3 -m venv .venv && source .venv/bin/activate. Then install torch from AMD’s documented index https://stable.repo.amd.com/rocm/whl-next/ — pick the gfx extra AMD lists for your card (gfx1201 on the R9700) and the ROCm line on /rocm/versions. AMD’s Radeon playbook example for gfx1201 is torch[device-gfx1201] from that index. A plain pip install torch from PyPI is the CUDA build.

   • ROCm versions
   • Image gen (also PyTorch ROCm)

3. Step 3

   Install a cited LoRA / QLoRA tool

   Unsloth: pin the ROCm torch you just installed, then uv pip install "unsloth[amd]" with those pins so PyPI does not swap in CUDA torch. PEFT / TRL: AMD’s Radeon playbook installs transformers, accelerate, peft, trl, and bitsandbytes on Linux. Axolotl and torchtune stay not verified on Radeon for a home card.

   • House models
   • Gemma 3 27B

4. Step 4

   If the R9700 resets under ROCm 10.0 training

   The dated /rocm-problems row: living notes mention a ROCm 10.0 training / fine-tune GPU-reset class on the R9700 and a first mitigation of TORCH_BLAS_PREFER_HIPBLASLT=0. The AMD KI URL is still not recorded. Do not apply that variable to llama.cpp Vulkan inference. Training speed stays not yet measured on our R9700s.

   • Fine-tune reset
   • ROCm problems

5. Step 5

   Windows is a second path

   Inference first-run on Windows is /setup/windows — llama.cpp Vulkan, not measured on the local desktop. WSL2 ROCm is a 10.1 technical preview at /setup/wsl. Unsloth’s AMD page lists Windows + WSL + Linux for RDNA 3/4. AMD’s Radeon playbook: bitsandbytes is not well supported on Windows, so QLoRA is Linux-only there; Windows uses LoRA or full fine-tuning. Unsloth: AMD multi-GPU training is Linux-only. We have not measured training on Windows or WSL.

   • Windows setup
   • WSL2 setup
   • Windows / WSL preview

6. Step 6

   Merge, convert to GGUF, quantize, serve

   AMD’s playbook merges with PEFT merge_and_unload and save_pretrained. Then llama.cpp: python convert_hf_to_gguf.py ./merged --outfile merged-f16.gguf. Quantize with llama-quantize to Q6_K, Q5_K_M, or Q4_K_M. A Q3_K_M row is a labeled estimate — cite /quants; do not run Q2 or Q1. The convert / imatrix / quantize walk is /guides/quantize-gguf. Serve: llama-server -m merged-q5_k_m.gguf --host 127.0.0.1 --port 8080. Size the GGUF on /calculator. Where RAM offload appears, the shopper lines are Minimum system RAM needed and RAM needed at full context.

   • Make GGUF quants
   • Quants
   • Calculator
   • House models
   • llama.cpp

Export back to llama.cpp

Merge the adapter, convert to GGUF, quantize Q6_K / Q5_K_M / Q4_K_M, then serve. Q3_K_M is a labeled estimate. Shopper RAM lines after export, when offload appears, are Minimum system RAM needed and RAM needed at full context — size that on /calculator.

from peft import AutoPeftModelForCausalLM
model = AutoPeftModelForCausalLM.from_pretrained("./adapter")
merged = model.merge_and_unload()
merged.save_pretrained("./merged")

python convert_hf_to_gguf.py ./merged --outfile merged-f16.gguf

llama-quantize merged-f16.gguf merged-q6_k.gguf Q6_K
llama-quantize merged-f16.gguf merged-q5_k_m.gguf Q5_K_M
llama-quantize merged-f16.gguf merged-q4_k_m.gguf Q4_K_M
# Q3_K_M is a labeled estimate — see /quants. Do not run Q2 or Q1.
llama-quantize merged-f16.gguf merged-q3_k_m.gguf Q3_K_M

llama-server -m merged-q5_k_m.gguf --host 127.0.0.1 --port 8080

FAQ

• Can I fine-tune like a 4090 blog post?

  PyTorch ROCm exists. Unsloth and AMD’s Radeon playbook document LoRA / QLoRA on AMD GPUs. That is not a 4090 decode clone. Training speed is not yet measured on our R9700s. The ROCm 10.0 R9700 reset note is a dated workaround, not a measured tokens-per-second. Keep inference on the llama.cpp pin you already run.

  • Site FAQ
  • Fine-tune reset

• Which tool should I start with?

  Cite Unsloth’s AMD page or AMD’s Radeon PEFT/TRL playbook. Axolotl and torchtune document AMD or ROCm, but their AMD how-tos are Instinct — not verified on Radeon for a home card. Do not start from a retired Qwen line.

  • House models
  • Gemma 3 27B

• Can I LoRA the house 35B daily driver?

  Unsloth’s VRAM table is dense parameter sizes. Qwen3.6-35B-A3B is MoE and is not in that table. We do not invent a 35B LoRA or QLoRA VRAM number. Use the calculator for inference GGUFs after you export. Q3 stays a labeled estimate.

  • 35B house model
  • Llama 3.3 70B
  • R9700 measured page
  • Calculator

Sources

• AMD — Install PyTorch for ROCm
• AMD AI Playbooks — Fine-tuning LLMs with PyTorch and ROCm
• Unsloth — Train and run models on AMD GPUs
• Unsloth — Fine-tuning VRAM requirements
• Axolotl documentation
• AMD ROCm blog — Axolotl on AMD GPUs
• torchtune README (AMD ROCm listed)
• AMD — Fine-tune with torchtune
• Hugging Face TRL — PEFT / QLoRA
• AMD ROCm — single-GPU fine-tuning with PEFT / TRL
• llama.cpp convert_hf_to_gguf.py
• llama.cpp tools/quantize