Skip to content

Cloud fleet

To serve models across several machines behind one endpoint, run atlas server on a small always-on box and atlas worker --join on each GPU machine. The same binary and API surface as a single box — only the topology grows.

  • Control plane — atlas server, a single Go binary on a small always-on instance (no GPU). Its state is one directory (SQLite + config); snapshot it and you’ve backed up the platform. Put a load balancer with a TLS cert in front for a public endpoint (see TLS).
  • Workers — atlas worker --join on each GPU machine. Workers dial out to the server, so they need no inbound ports — they sit in private networks with egress only. There’s nothing to expose and no SSH required to operate.
  • One endpoint — the server authenticates clients and routes requests across all workers.

Joining is non-interactive (flags/env), so it drops straight into instance start-up scripts:

Terminal window
atlas worker --join wss://atlas.yourco.com/workers/connect --token "$ATLAS_JOIN_TOKEN"

--join is the server’s WebSocket URL (wss:// for a TLS-fronted endpoint, ws:// otherwise) and --token is the join token printed by atlas server; both can come from the environment instead (ATLAS_SERVER_URL, ATLAS_JOIN_TOKEN). If the server uses a self-signed certificate, pin it with --tls-pin sha256:<hex> (printed by atlas server --tls-self-signed; not needed for ACME/public-CA certs). That one line is the integration — install the binary, run it with a join token (kept in your secrets manager), and the worker appears in the fleet.

  • Elastic pools — because workers self-register and are removed on heartbeat timeout, you can run each pool under an autoscaling group: scale out and a new instance joins in minutes; scale in and the worker stops accepting new requests and lets its in-flight requests finish before disconnecting.
  • Spot-friendly — an interrupted worker is just a heartbeat timeout; the scheduler re-places its models on remaining capacity.
  • Hybrid is free — since workers dial out, the control plane can be in one place and workers anywhere: an on-prem box and a cloud GPU pool joined to the same server, one endpoint. Nothing here is cloud-specific.

See Scaling & operations for deploy/scale/stop, drain, and weight caching.