Cloud fleet
To serve models across several machines behind one endpoint, run atlas server on a small always-on
box and atlas worker --join on each GPU machine. The same binary and API surface as a single box —
only the topology grows.
How it fits together
Section titled “How it fits together”- Control plane —
atlas server, a single Go binary on a small always-on instance (no GPU). Its state is one directory (SQLite + config); snapshot it and you’ve backed up the platform. Put a load balancer with a TLS cert in front for a public endpoint (see TLS). - Workers —
atlas worker --joinon each GPU machine. Workers dial out to the server, so they need no inbound ports — they sit in private networks with egress only. There’s nothing to expose and no SSH required to operate. - One endpoint — the server authenticates clients and routes requests across all workers.
Joining a worker
Section titled “Joining a worker”Joining is non-interactive (flags/env), so it drops straight into instance start-up scripts:
atlas worker --join wss://atlas.yourco.com/workers/connect --token "$ATLAS_JOIN_TOKEN"--join is the server’s WebSocket URL (wss:// for a TLS-fronted endpoint, ws:// otherwise) and
--token is the join token printed by atlas server; both can come from the environment instead
(ATLAS_SERVER_URL, ATLAS_JOIN_TOKEN). If the server uses a self-signed certificate, pin it with
--tls-pin sha256:<hex> (printed by atlas server --tls-self-signed; not needed for ACME/public-CA
certs). That one line is the integration — install the binary, run it with a join token (kept in
your secrets manager), and the worker appears in the fleet.
Why this shape is convenient
Section titled “Why this shape is convenient”- Elastic pools — because workers self-register and are removed on heartbeat timeout, you can run each pool under an autoscaling group: scale out and a new instance joins in minutes; scale in and the worker stops accepting new requests and lets its in-flight requests finish before disconnecting.
- Spot-friendly — an interrupted worker is just a heartbeat timeout; the scheduler re-places its models on remaining capacity.
- Hybrid is free — since workers dial out, the control plane can be in one place and workers anywhere: an on-prem box and a cloud GPU pool joined to the same server, one endpoint. Nothing here is cloud-specific.
See Scaling & operations for deploy/scale/stop, drain, and weight caching.