Harnesses & frontends: what to plug your local model into
A harness is the app or agent layer on top of an inference server. llama-server,Strata, vLLM, and Ollama run the weights. The harness talks to them over anOpenAI-compatible API. It is not the GPU stack. If/v1 is down, the harness is a very expensive timeout.
Cited from official repos and docs, checked 2026-10-12. We do not invent features — unclear items stay “not documented.” Exact serve flags for our rig stay TODO instacks.
You → harness (tools, memory, channels, chat UI)
→ HTTP OpenAI-shaped /v1
→ llama-server :8080 / Strata :8083 / vLLM :8000 / Ollama :11434
→ R9700 (Vulkan or HIP)Copy-paste connectPick a harnessComparison table
wtpai1
Used on our rig
William’s daily agent on the two-R9700 box. Pointed at Strata:8083 and llama-server:8080.
- Hermes AgentUsed on our rig
Dedicated agent CLI/desktop on top of local Qwen — not a dozen chat bridges.
Attach after /v1 is up
- 1. Verify ROCm —
rocminfoshowsgfx1201. - 2. Serve official Qwen instruct — Serve Qwen. llama-server default is
127.0.0.1:8080. This site’s Strata convention is:8083. - 3. Prove the API on loopback. Snippets: Connect.
- 4. Watch a GPU meter during that completion. VRAM and compute must move.
- 5. Then point the harness at that base URL. Model id must match
GET /v1/models(placeholder on this site:qwen). - 6. Only after a chat turn works, enable tools that can touch the filesystem or the network.
Radeon Stack Labqwen-reliable is the daily driver (Qwen3.6-35B-A3B Q5).qwen-fast-agent is Qwen 3.8 Flash on Strata. Cards:recommended stacks.
Categories
Agent harnesses
All agentTool loops on top of an OpenAI-compatible /v1. Coding agents, CLIs, and IDE extensions.
Agent harnessesUsed on our rigLocal + hybridCLIDesktop
Hermes Agent
William’s daily agent on wtpai1. Tools, optional multi-agent loop, and provider failover above a local OpenAI-compatible server.
Nous Research
Agent harnessesLocal + hybridWebCLI
OpenHands
Software-agent UI/CLI that talks to any LiteLLM-supported model, including local OpenAI-compatible servers.
All Hands
Agent harnessesLocal + hybridCLI
Aider
Terminal pair-programmer that edits a git repo via any OpenAI-compatible endpoint.
Aider-AI
Agent harnessesLocal + hybridIDECLI
Cline
VS Code / CLI coding agent with a first-class OpenAI Compatible provider.
Cline
Agent harnessesLocal + hybridIDE
Roo Code
VS Code coding agent. Local path is documented for Ollama and LM Studio; OpenAI Compatible is also documented.
Roo Code
Agent harnessesLocal + hybridIDE
Continue
IDE extension with config.yaml providers, including openai (any /v1) and a native llama.cpp provider.
Continue
Agent harnessesLocal + hybridCLI
OpenCode
Terminal coding agent with first-party llama-server provider docs.
OpenCode / SST
Agent harnessesLocal + hybridCLIDesktop
Goose
Local-first agent that can use the OpenAI provider against a custom host, or a declarative JSON provider.
Block
Agent harnessesLocal + hybridCLI
Claude Code
Official CLI speaks the Anthropic Messages API via ANTHROPIC_BASE_URL — not a drop-in OpenAI /v1/chat/completions client.
Anthropic
Agent harnessesLocal + hybridCLI
Codex CLI
Official Codex CLI reads ~/.codex/config.toml. Custom providers use wire_api = responses (the only supported value in the current config reference).
OpenAI
Agent harnessesHybrid-shapedIDE
Cursor
IDE with an Override OpenAI Base URL (BYOK). Not documented as a first-class localhost / air-gapped attach.
Cursor
Agent harnessesLocal + hybridCLIWeb
OpenClaw
Self-hosted multi-channel gateway (Discord, Telegram, Slack, …) in front of a swappable /v1.
OpenClaw Foundation
Chat UIs
All chatTalk to the same server without an agent loop. Browser and desktop frontends.
Chat UIsLocal + hybridWeb
Open WebUI
Self-hosted chat UI. Connects to Ollama and any OpenAI-compatible server.
Open WebUI
Chat UIsLocal + hybridDesktop
LM Studio
Desktop app that both chats and serves an OpenAI-compatible API on port 1234 by default.
LM Studio
Chat UIsLocal + hybridDesktop
Jan
Desktop local-AI app with a built-in OpenAI-compatible server (default 127.0.0.1:1337/v1), powered by llama.cpp.
Menlo / Jan
Chat UIsLocal + hybridWeb
SillyTavern
Character-oriented frontend. Connects to many local backends including llama.cpp and OpenAI-compatible Chat Completion.
SillyTavern
Chat UIsLocal + hybridWeb
LibreChat
Multi-provider chat UI. Custom OpenAI-compatible endpoints live in librechat.yaml.
LibreChat
Chat UIsLocal + hybridDesktopWeb
AnythingLLM
Desktop/Docker workspace chat with a generic OpenAI provider and a dedicated LM Studio provider.
Mintplex Labs
Chat UIsLocal onlyWeb
llama.cpp built-in web UI
llama-server serves a web UI on the same host:port as /v1 (default 127.0.0.1:8080). Disable with --no-webui.
ggml-org
Image / video
All imageDiffusion graphs on ROCm PyTorch. Not an LLM /v1 client.
Overnight research loop (this site’s YAML)
Local Hermes is a practical overnight research agent for the living tracker. It does not publish.
- Leave official instruct serving on
/v1(the same URL you just curled). - Attach Hermes to that endpoint — Hermes notes.
- Point it at the weekday source list on Weekday scrape.
- Ask for a draft diff to the living YAML only. No invented benches. No download stats.
- In the morning: read the diff, run the smoke tests if a pin moved,
npm run build, commit onmain.