Your own swarm of models…
just one at a time.
How it works
-
01
Your app asks for a model.
One OpenAI-compatible endpoint. Put whichever model you like in the
modelfield — the app has expectations; we do not shame them. -
02
The scheduler decides.
Model already loaded? Tokens flow. Not loaded? It stops the current model container, boots yours, and feeds the weights into your very small GPU.
-
03
Your app thinks nothing happened.
Requests queue, responses proxy through, and from the outside all ten of your models appear to exist simultaneously. They do. Sort of.
One honest caveat: swapping models takes seconds to minutes, depending on model size and how fast your disk is. Requests wait in the queue meanwhile. If you want everything answering at once, that's what more VRAM is for — or a generous sponsor.
Capabilities
- Model groups — run exclusively, or co-locate several when VRAM allows.
- Conversation affinity — one chat stays on the loaded model instead of causing swaps during tool calls.
- Router profiles — another model id in the
/v1/modelslist; behind it, your manual pick or an ordered chain that fails over. - Cloud providers — aggregate all your API keys from different services behind unswarm, so you never have to visit multiple websites to fetch new keys when testing a new harness or app.
- Benchmarks — compare runtimes on your own hardware, with saved prompts.
- Usage analytics — latency percentiles, token heatmap, CSV export, live tail.
- Cost tracking — per-token pricing, per-provider subscriptions, or “my power bill was $XX this month.”
- API keys — scoped access control with per-key usage.
- Multi-machine — Go agents dial out to the control plane over WebSocket. No inbound ports.
- Live metrics — GPU (NVIDIA, AMD, Intel), CPU and RAM, straight from
/proc.
Router profiles
A router profile acts like a virtual model: it appears under /v1/models like everything
else, and your harness asks for it like any other id. What answers behind that model is a policy.
Manual — you activate a model from the CLI or the frontend, and it stays.
Auto — a priority-based fail chain: when the active model errors, the next one in priority answers.
The panel below runs both.
manual — you picked it
The harness asks for router/coding. It answers with the model you clicked, and keeps
answering with it until you click a different one.
- The id never moves — your harness calls the profile; the model behind it is what changes.
- Your click is the policy — nothing swaps on its own while you are mid-conversation.
- Swaps still cost — clicking a cloud entry answers right away; clicking a cold local model queues requests while its weights load.
auto — something failed
Same profile id, different policy: the request walks your priority list until something
answers. The useful chains are cloud first, your box last — a 429 from a free tier
at 3 a.m., a 503 from an upstream that no longer exists, and then your own
hosted/ model catching the conversation like it always does.
- Ordered chain — you rank the models; the router walks your ranking, not a guess.
- Consumed errors — a 429 stays inside the profile; your client gets one response, not a stack trace of attempts.
- Hosted last — free tier, paid tier, your box. A chain of local models mostly swaps weights; a chain of providers mostly saves requests.
- Cooldown — a model that just fell over sits out the next few requests.
One control plane.
As many boxes as you own.
Agents connect out to the control plane over WebSocket — the control plane never dials into your machines.
The CLI
Two commands turn your control plane into coding-agent config: it rotates the inference key, reads the models your key is actually allowed to reach, and writes the provider block for pi and OpenCode — base URL, secret, context windows and all.
- pi & OpenCode —
unswarm config generate-piandgenerate-opencodewrite~/.pi/agent/models.jsonand~/.config/opencode/opencode.jsonc. - Rotated, never pasted — the key is rotated to capture a fresh secret; the old one is invalidated.
- Grants respected — only the providers and models your inference key can reach make it into the file.
- Dry-run first —
--dry-runprints the exact config without touching disk;--target projectwrites next to your code.
Quick start
git clone https://github.com/atretador/unswarm.git
cd unswarm
docker compose up -d --build
docker exec -i unswarm-backend-1 dotnet exec -- unswarm-admin set-password --username admin --password YourPasswordBackend, frontend and database in one box to start; add agents when you buy more computers. Development setup (.NET 10 · Go · React 19) is in the README.