Scaling a service: bigger instances or more of them
The difference between scaling up and scaling out, which one your bottleneck actually calls for, and how to change the size of a Voroa service right now.
What this covers
There are two ways to give a service more capacity. Scaling up gives the single running copy more memory and processor, which is what you want when one request is heavy or when the process is running out of memory. Scaling out runs several copies behind a load balancer, which is what you want when requests are individually cheap but there are too many of them, and it has the side benefit that one copy failing does not take the service down. Scaling out only works if your service keeps no state in its own memory, since a user’s next request may land on a different copy.
You can change any service’s size from its Compute page. Running several copies of one service is available for web services on a paid instance size, and each copy is billed at the plan’s monthly price. Services with scheduled tasks, and preview environments, always run a single copy. Automatic scaling based on load is not available today.
Scale up or scale out: which one you need
| What you see | What it usually means | What to do |
|---|---|---|
| Memory close to the limit, or the process restarts under load | One copy needs more memory | Scale up to a bigger size |
| CPU near 100% while each request is slow | One copy needs more processor | Scale up |
| CPU busy but each request is cheap, and traffic keeps growing | Too many requests for one copy | Scale out to more copies |
| You need the service to survive one copy failing | No spare copy | Scale out to at least 2 copies |
Read the numbers before you change anything: the Compute page shows the last 48 hours of memory and CPU for the busiest copy. See service resource usage.
How to change size or copies
- Open the service and go to Compute.
- To scale up, use the plan card at the top to move to a bigger size. The payment summary shows the prorated amount with GST before you pay.
- To scale out, set the number of copies in the Scaling section.
A bigger size is applied to the running service once payment is confirmed. See instance types for how to pick a size.
Before you scale out
Make sure each copy can serve any request. Keep sessions and uploads out of the container’s memory and disk: store sessions in a database or a Key Value store, and files in object storage. Otherwise a user can be signed out when their next request lands on a different copy.