Overprovisioning is extremely expensive. Under provisioning is risky and can result in catastrophic failures. The modern cloud was supposed to tackle this problem with autoscaling. So why is it still so hard?
- Monolithic traditional load balancer applications cannot start and stop quickly
- Lack of a control plane/data plane architecture in cloud-optimized load balancers
- Even for cloud-first load balancers, containers take too long to spin up
- The risk of cloud lock-in makes building bespoke scaling tied to a single cloud an unacceptable risk.
Real-time scaling requires a fundamental shift to embrace cloud-native concepts and design. This means building app services as distributed systems, optimized for containers and designed to launch scale-out nodes in a minute or less.