Load balancing is the practice of distributing incoming network traffic across several servers so no single one becomes overwhelmed. A load balancer sits in front of a group of identical instances, receives every request, and forwards it to whichever instance should handle it. It delivers two benefits at once, since spreading load raises capacity while routing away from failed instances improves availability. It is the foundational mechanism that makes horizontal scaling practical rather than theoretical.

How Load Balancing Works

The mechanism is straightforward. Clients connect to the load balancer rather than to any individual server, and it selects a destination for each request according to a configured algorithm. It also monitors the health of each instance and stops sending traffic to any that stop responding correctly. That health checking is what turns a group of servers into a resilient pool.

Single Entry Point for Clients

Clients address the load balancer, unaware of how many instances exist behind it. Instances can be added or removed without any client-side change.

Distribution According to an Algorithm

Each request is assigned to an instance based on configured rules, which may consider connection counts, response times, or simple rotation.

Health Checking

The balancer tests each instance periodically and removes unhealthy ones from rotation. This is the feature that converts redundancy into actual availability.

Automatic Reintegration

Recovered instances rejoin the pool once health checks pass again, so recovery needs no manual intervention. Our DevOps services work configures these thresholds carefully.

Common Distribution Algorithms

The algorithm determines how requests get assigned, and the right choice depends on whether your requests are uniform in cost. Where every request takes similar effort, simple rotation works well. Where request cost varies widely, connection-aware or latency-aware methods prevent one instance accumulating all the expensive work.

Round Robin

Requests distributed in sequence across instances. Simple, predictable, and appropriate when instances are identical and request cost is broadly uniform.

Least Connections

Sends each request to the instance handling fewest active connections. Better where request duration varies substantially, since it accounts for actual current load.

Least Response Time

Selects the instance responding fastest. Adapts to instances degrading gradually rather than failing outright, which health checks alone may not catch.

Hash-Based Routing

Routes consistently based on a request attribute such as client address. Useful for cache locality, though it can distribute load unevenly.

Session Handling and State

The main complication in load balancing is state. If an application holds session data in the memory of the instance that created it, subsequent requests must reach the same instance, which undermines even distribution and breaks when that instance fails. Solving this properly is usually a prerequisite for effective load balancing rather than an optimisation afterwards.

Why In-Memory Sessions Cause Problems

State held on one instance means requests must return there. That constrains distribution and loses the session entirely when the instance restarts.

Sticky Sessions as a Workaround

The balancer routes a client consistently to one instance. This works but reduces distribution quality and still loses sessions on instance failure.

Externalising Session State

Storing sessions in a shared cache or database makes every instance equivalent. This is the durable solution and enables genuinely even distribution.

Stateless Token Authentication

Tokens carrying their own validity remove server-side session storage entirely. Our API development work favours this approach for new services.

Load Balancing Compared With Related Components

Several components sit in front of application servers and get conflated, which leads to deploying the wrong one for a given problem. Being clear about what each does prevents both duplicated function and gaps. A load balancer and an API gateway are frequently used together, each handling what it is designed for.

Load Balancer

Distributes traffic across identical instances of the same service and removes unhealthy ones. Concerned with capacity and availability rather than request content.

API Gateway

Applies policy such as authentication and rate limiting, and routes to different services by content. Covered in our API gateway entry.

Reverse Proxy

Forwards requests to backend servers, often handling encryption termination and caching. A load balancer is a reverse proxy with distribution and health checking added.

Content Delivery Network

Caches and serves content from locations near users. Reduces origin load geographically rather than distributing it across instances in one place.

Using Them Together

A typical production setup places a CDN in front, a load balancer distributing across instances, and a gateway applying policy. Our cloud consulting reviews check each is doing only its own job.

FAQs

What is load balancing in simple terms?

Load balancing means spreading incoming requests across several servers so none becomes overwhelmed. A load balancer receives every request, sends it to an available server, and stops sending traffic to any server that fails its health checks.

What is the difference between a load balancer and an API gateway?

A load balancer distributes traffic across identical instances of one service, concerned with capacity and availability. An API gateway inspects request content, applies policy such as authentication and rate limiting, and routes to different services. They are commonly used together.

Which load balancing algorithm should I use?

Round robin suits identical instances handling requests of broadly similar cost. Least connections is better where request duration varies substantially. Least response time helps detect gradually degrading instances. Hash-based routing suits cache locality at the cost of even distribution.

What are health checks in load balancing?

Periodic tests the balancer performs against each instance to confirm it is responding correctly. Failing instances are removed from rotation and reinstated automatically once they recover, which is what turns redundant servers into genuine availability.

What are sticky sessions and should I use them?

Sticky sessions route a client consistently to one instance, which works around session data held in server memory. They reduce distribution quality and still lose sessions when an instance fails, so externalising session state is the better long-term solution.

Do I need a load balancer for a small application?

Only if you run more than one instance, which you should if availability matters, since a single instance means any restart is downtime. Managed cloud load balancers are inexpensive and remove a meaningful category of failure.