Load Balancer Fundamentals

Load Balancer in One Line

A load balancer sits in front of service instances and routes traffic so no single backend becomes a bottleneck or single point of failure.

1. Why Use a Load Balancer?

• Scalability: add/remove instances without changing client behavior.
• Availability: route away from unhealthy nodes.
• Performance: distribute load based on policy and health.
• Safe deployments: rolling, blue/green, canary traffic splits.

Without a load balancer, one server outage or traffic spike can affect the full system.

2. Routing Algorithms (When to Use What)

Algorithm	How it picks backend	Best when	Risk
Round robin	Rotate servers in order	Similar instances	Ignores live load
Weighted round robin	Rotate with capacity weights	Mixed-size instances	Weights can become stale
Least connections	Fewest active connections	Long-lived requests/websockets	Connection count != true load
Least response time	Fastest current responder	Variable latency workloads	Needs continuous telemetry
IP hash / consistent hash	Same client -> same server	Affinity/state locality	Uneven load possible
Random (power of two choices)	Random sample, pick better	Very large pools	Less predictable fairness

Rule of thumb:

• Start with weighted round robin + health checks.
• Move to least-connections/least-latency when workload is uneven.

3. Health Checks and Failure Handling

Production balancers should:

• use active health checks (/health, TCP checks),
• eject failed backends quickly,
• retry with bounded budget,
• avoid retry storms with backoff/circuit breaking,
• support connection draining during deploys.

Interview line:

“I treat load balancing and health checks as one system; routing without good health signals causes cascading failures.”

4. Stateful vs Stateless Balancing

Mode	How it works	Pros	Cons
Stateful affinity	Same user/session tends to hit same backend	Session locality, cache hits	Harder failover and rebalance
Stateless routing	Every request routed independently	Easy scaling and recovery	Needs shared session/state store

Preferred pattern:

• Keep balancer stateless.
• Externalize session/state (Redis/DB) when possible.
• Use sticky sessions only when unavoidable.

5. Layer 4 vs Layer 7 Balancing

Layer	Routes by	Strength	Limitation	Typical use
L4 (transport)	IP + port + connection metadata	High throughput, low overhead	No HTTP-aware routing	TCP/UDP services, simple pass-through
L7 (application)	URL, host, header, cookie, method	Smart routing, auth/policy aware	More CPU/latency overhead	API gateways, multi-tenant HTTP apps

Examples:

• Route /api/* to API service and /static/* to CDN/backend at L7.
• Use L4 for raw TCP services where deep inspection is unnecessary.