Lesson 2.4 · 15 min
Load balancing
Run five balancing algorithms against the same traffic, slow one server down, kill another, and see who keeps tail latency and errors low.
The idea in plain words
A load balancer is like the host at a busy restaurant who decides which waiter takes the next table. A good host spreads the work evenly and stops seating tables with a waiter who has gone home.
The problem it solves
Once you run several copies of a server, something has to decide which one handles each request, and stop sending requests to copies that are broken or overloaded.
How it works
- All requests arrive at the load balancer instead of a single server.
- It checks each server regularly (a health check) and drops the ones that fail.
- For each request it picks a healthy server with a rule: in turn, the least busy, or a random pair and the less busy of the two.
- It forwards the request and passes the response back.
Example
Three servers, one is slow today. Taking turns sends it a third of the traffic and those users wait. Choosing the server with the fewest requests in progress sends it less, because its slow requests pile up and make it look busy. That is why the rule matters most when servers or requests are not all alike.
Key terms
- Load balancer
- Spreads incoming requests across a pool of servers.
- Health check
- A regular test request; servers that fail it stop getting traffic.
- Round robin
- Give requests to servers in turn: 1, 2, 3, 1, 2, 3.
- Least connections
- Send the next request to the server with the fewest requests in progress.
- L4 vs L7
- L4 balancers route by IP and port only; L7 balancers read the HTTP request (path, headers).
In this lesson you'll learn
- How round robin, weighted, least connections, power of two choices and hashing pick a server.
- Which one breaks when servers or requests are uneven, or a server fails.
- What to say about L4 vs L7, health checks, and keeping the balancer itself alive.
Sections
- Theory: five algorithms
- Theory: compare them
- Design: L4, L7, health checks
- Design: pick one
- Check yourself
Cheat sheet
| Round robin | equal servers, equal requests |
|---|---|
| Least connections | uneven request cost |
| Power of two | many LBs, no herding |
| Hash | stickiness, cache locality |
| Detection time | ≈ interval × threshold |
The interactive simulations and the quiz load with JavaScript.