Desktop/Docker workspace chat with a generic OpenAI provider and a dedicated LM Studio provider.
What it’s for
Docs + chat with a local /v1. RAG-shaped, not a coding agent.
Local endpoint
llama-server base
http://127.0.0.1:8080/v1
Strata base
http://127.0.0.1:8083/v1
Model name
GENERIC_OPEN_AI_MODEL_PREF / LMSTUDIO_MODEL_PREF
API key
GENERIC_OPEN_AI_API_KEY=sk-123abc
Tool calling
Workspace features; agent completeness against llama-server is not documented.
Long context
GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT in env — set it to the server window, do not invent 262K.
ROCm / Radeon gotchas
Use generic-openai against our llama-server. LOCAL_AI_BASE_PATH in the example is :8080/v1 for LocalAI, which happens to match llama-server’s default port.
Hybrid local + cloud
Multiple LLM_PROVIDER values including cloud OpenAI.
This is the same question as the calculator’sfully localversushybridtoggle. This page does not quote cloud prices.
More on the connect page. Prove http://127.0.0.1:8080/v1/models first.
AnythingLLM → llama-server :8080.env
# docker/.env.example — generic OpenAI provider
LLM_PROVIDER=generic-openai
GENERIC_OPEN_AI_BASE_PATH=http://127.0.0.1:8080/v1
GENERIC_OPEN_AI_MODEL_PREF=qwen
GENERIC_OPEN_AI_API_KEY=sk-123abc
# GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT= # set to the server window; do not invent 262K
Exact llama-server flags for our daily driver stay TODO in stacks.yaml. This is the port only.
AnythingLLM → Strata :8083.env
# docker/.env.example — generic OpenAI provider
LLM_PROVIDER=generic-openai
GENERIC_OPEN_AI_BASE_PATH=http://127.0.0.1:8083/v1
GENERIC_OPEN_AI_MODEL_PREF=qwen
GENERIC_OPEN_AI_API_KEY=sk-123abc
# GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT= # set to the server window; do not invent 262K
Exact Strata flags for our rig stay TODO in stacks.yaml. This is the port only.