Load Balancing

Load balancing is basically the distribution of incoming traffic across a group of backend servers

Advantages of load balancing

  • Enable horizontal scaling: spinning up multiple instances of a service is not possible without a load balancer directing traffic efficiently to the cluster. 
  • Dynamic scaling: it's possible to seamlessly add and remove servers to respond to load.
  • Abstraction: the end user only needs to know the address of the load balancer, not the address of every server in the cluster.
  • Redundancy: distributing load over a cluster means no one server is a single point of failure. Note that the load balancer itself must also not become a single point of failure.
  • Continuous deployment: it's possible to roll out software updates without taking the whole service down, by using the load balancer to take out one machine at a time.

Load balancing algorithms

Load balancing algorithms are generally divided into two groups: static and dynamic. Static algorithms function the same regardless of the state of the back end serving the requests, whereas dynamic algorithms take into account the state of the backend and consider system load when routing requests

Static

Round robin is the simplest and one of the most used algorithms. The load balancer maintains a list of available servers, and routes the first request to the first server, the second request to the second server, and so on. This works well if every server in the pool has roughly the same performance characteristics.

A weighted round robin takes server characteristics into account, so that servers with more resources get proportionally more of the requests.