Load Balancing is Impossible

Load balancing is something most of us assume is a solved problem. But the idea that load balancing is "solved" could not be further from the truth. If you use multiple load balancers, the problem is even worse. Most of us use "random" or "round-robin" techniques, which have certain advantages but are highly inefficient. Others use more complex algorithms like "least-conns," which can be more efficient but have horrific edge cases. "Consistent hashing" is a very useful technique, but only applies to certain problems. There are several factors that exist both in theory and practice that make efficient load balancing an exceptionally hard problem. For instance:

• Poisson request arrival times
• Exponentially distributed response latency
• Oscillations when sharing data between multiple load balancers

We’ll talk about techniques and algorithms such as Randomized Least-conns, Join-Idle-Queue, and Load Interpretation. Load balancing perfectly may be impossible in the real world, but we can do better than “random,” “round-robin,” and naive “least-conns”.

Talk objectives:

• The audience will talk away with an understanding of why traditional load balancing techniques are sub-optimal, practical techniques for improving it, as well as longer term theoretical approaches. The audience will also understand what makes load balancing such a difficult problem and why it's impossible to solve precisely.

Target audience:

• Operations Architects and Performance-oriented Developers.