Scaling & operations
In a fleet, models are deployed onto workers and scheduled by the control plane (VRAM-fit placement, auto-start, idle-stop).
Deploy, scale, stop
Section titled “Deploy, scale, stop”atlas deploy <model> # place a model instance on the fleetatlas scale <model> --replicas N # scale a deployed modelatlas stop <model> # stop a deploymentatlas deployments # list deploymentsSee the CLI reference for worker management (atlas workers list,
atlas workers remove).
Drain, spot, and heartbeats
Section titled “Drain, spot, and heartbeats”Workers are stateless executors that self-register and are removed on heartbeat timeout:
- Graceful drain on SIGTERM — a worker stops accepting new requests and lets its in-flight requests finish before exit, so ASG scale-in and spot interruptions don’t drop live requests.
- Spot instances work naturally — an interrupted worker is just a heartbeat timeout; the scheduler re-places its model instances on remaining capacity.
- Heartbeat-timeout removal — vanished workers leave the pool automatically.
Weights
Section titled “Weights”Each worker caches weights on local NVMe or EBS, so a model is pulled once per worker and subsequent
cold boots are fast. Weights are sourced from Hugging Face or an https:// GGUF URL (via the model
catalog); pin them to a fast local disk on each worker so you don’t re-pull the same 140GB on every
restart.
Autoscaling
Section titled “Autoscaling”v1 scaling is “you set ASG desired capacity.” Atlas exposes a queue-depth / utilization signal on
/metrics that an external autoscaler (e.g. an ASG policy) can
consume, rather than Atlas calling cloud APIs itself.