One server got slow — garbage collection, nothing fatal. The load balancer's health check hit /health, which returned 200 because the process was alive, so traffic kept arriving. Requests queued, latency climbed, the health check timed out, and the balancer pulled it. Its traffic went to the other three, which were now handling a third more load each, and eleven minutes later there were no healthy servers at all.
Load balancing interview questions get interesting exactly there — not at "what is round robin" but at what happens when the thing meant to protect you accelerates the failure. This guide covers L4 versus L7, the algorithms and when each matters, health checks done properly, sticky sessions, and the cascading-failure question senior rounds always reach.
What load balancing interviews actually test
- Layer 4 vs Layer 7 — what each can see, and therefore do.
- Algorithms — and specifically when round robin is the wrong default.
- Health checks — shallow versus deep, and the trap above.
- State — sticky sessions, and why they are a workaround.
- Failure behaviour — cascading failure, load shedding, and graceful degradation.
Layer 4 vs Layer 7
The foundational distinction, and a guaranteed question.
- Layer 4 (transport) — routes on IP and port without inspecting the payload. Very fast, low latency, protocol-agnostic, and cheap to scale. It cannot route by URL path, cannot terminate TLS meaningfully, and cannot retry an individual HTTP request because it does not know where one ends.
- Layer 7 (application) — parses HTTP, so it can route by path, host or header, terminate TLS, compress, cache, rewrite, rate limit and retry idempotent requests. It costs more CPU and adds latency, and it must understand the protocol.
The practical framing interviewers want: use L7 when routing decisions depend on the content of the request — path-based microservice routing, canary releases by header, per-endpoint rate limits — and L4 when you need raw throughput or are balancing a non-HTTP protocol. Many architectures use both: L4 at the edge for volume, L7 inside for routing.
A detail worth volunteering: gRPC and WebSockets need L7-aware balancing. An L4 balancer pins a long-lived connection to one backend forever, so a gRPC client that opens one connection sends every request to the same server regardless of load. That is a real production surprise and naming it scores well. Our gRPC guide and WebSocket guide cover those cases.
Algorithms, and when round robin fails
- Round robin — the default. Fine when requests cost roughly the same and servers are identical. Fails when request costs vary wildly, because it distributes requests evenly rather than work.
- Weighted round robin — for heterogeneous server sizes.
- Least connections — routes to the backend with fewest active connections. Substantially better when request durations vary, which is most real systems. A strong default answer.
- Least response time — combines active connections with observed latency. Better still, more state to maintain.
- IP hash / consistent hashing — deterministic mapping from client to server. Used for session affinity and cache locality, and consistent hashing specifically so that adding a server does not remap everyone.
- Power of two choices — pick two backends at random and send to the less loaded of the two. Nearly as good as full least-connections with a fraction of the coordination, which is why large systems use it. Naming this is a differentiator.
The insight worth stating: round robin is a request-count optimiser, not a load optimiser. If one endpoint takes 5ms and another takes 5 seconds, even distribution of requests produces wildly uneven distribution of work.
Health checks, and the failure from the intro
This is the highest-value section. A health check decides whether traffic keeps flowing to a backend, so its design determines whether the balancer helps or hurts during an incident.
- Shallow (liveness) — is the process up and accepting connections? Cheap, and it returns 200 from a server that is completely unable to serve real traffic, which is exactly the intro's problem.
- Deep (readiness) — can this instance actually serve? Check the database connection pool, critical dependencies, and whether the instance is still warming up.
- But not too deep — the crucial nuance. If every instance's health check tests a shared database, then when that database blips, every instance fails its check simultaneously and the balancer removes the entire fleet. A partial outage becomes a total one. Say this out loud; it is the senior insight in this topic.
The balanced answer: check what this instance individually controls, degrade rather than fail on shared-dependency problems, and use separate liveness and readiness endpoints — liveness for "restart me", readiness for "stop sending traffic". Add hysteresis (require N consecutive failures before removal, and N successes before return) so a single slow response does not flap a server in and out.
Also know connection draining: on deploy or scale-down, stop sending new requests but let in-flight ones finish before terminating. Without it, every deploy drops requests. Our Kubernetes guide covers readiness probes and the same distinction.
Sticky sessions and state
Session affinity pins a client to one backend, usually by cookie or IP hash. It exists because the application stored session state in server memory.
The answer interviewers want is honest about it being a workaround: sticky sessions break even load distribution, make scale-in lose sessions, prevent clean rolling deploys, and mean one server's failure logs out a subset of users. The better fix is externalising session state — into Redis, a database, or a signed token held by the client — so any server can serve any request. Then you can use whatever algorithm you like.
Concede the legitimate cases too: local caches with expensive warm-up, WebSocket connections which are inherently pinned, and legacy applications you cannot change. That balance reads better than a blanket rejection.
Cascading failure and load shedding
Back to the intro. The senior question is: how do you stop a load balancer from amplifying a failure?
- Capacity headroom — if losing one of four servers pushes the rest past capacity, you were already running too hot. Size so that N−1 (or N−2) can carry peak.
- Load shedding — when overloaded, reject some requests fast rather than queueing everything and failing slowly. A quick 503 for a fraction of traffic keeps the rest healthy; unbounded queueing makes every request slow and then times all of them out.
- Circuit breakers — stop sending to a failing backend for a cool-down period, then probe. Prevents hammering a struggling instance.
- Retries with budgets — the trap is that naive retries multiply load exactly when a system is already overloaded. Cap retries as a percentage of traffic, use exponential backoff with jitter, and only retry idempotent requests.
- Slow start — ramp traffic to a newly added instance gradually, because a cold instance with empty caches and an unwarmed JIT cannot take full load immediately and will fail its health check if you give it a quarter of the fleet's traffic instantly.
- Graceful degradation — shed optional work (recommendations, personalisation) before core work (checkout).
Google's SRE book is the canonical source for this material and is free online; naming load shedding and retry budgets specifically is what marks a senior answer. Our SRE interview questions guide goes deeper.
Beyond one balancer
Expect "what if the load balancer itself fails?" — the answer is that it must not be a single point of failure. Run redundant balancers with a floating virtual IP, use DNS or anycast to distribute across regions, and remember that DNS load balancing is coarse because clients cache records and ignore TTLs, so it is for regional distribution rather than fine-grained balancing.
Know the layers of a real setup: DNS or anycast to a region, an L4 balancer at the edge, an L7 balancer or ingress for routing, and often client-side or service-mesh balancing between internal services. And know that a global server load balancer routes by geography and health, which is the answer to multi-region questions. Our Nginx guide covers the L7 implementation most people meet first.
The SRE book, real proxies, ChatGPT — where each fits
- Google's SRE Book — free, and the chapters on load balancing, overload and cascading failure are exactly the senior material.
- Envoy and HAProxy documentation — the clearest treatment of algorithms, outlier detection and connection draining as configured in practice.
- Running Nginx or HAProxy in front of three containers — highest yield. Kill one, watch the health check react, then make one slow rather than dead and see what your shallow check does.
- Marc Brooker's and AWS's builder articles — excellent on retry budgets, jitter and the power of two choices.
- Your cloud provider's ALB/NLB docs — worth knowing the L7/L4 split as named products, since interviewers often use those terms.
- ChatGPT — good for comparing algorithms and generating config. It will not tell you the health check passed while the server was useless.
- Greenroom — the spoken layer. Ari, the AI interviewer, runs design and incident rounds out loud and pushes the amplification follow-up. Fair tradeoff: Ari will not load-test your fleet.
How to prepare for a load balancing interview
- Week 1: L4 versus L7 and the algorithms. Explain each aloud with the workload it suits, and rehearse why round robin optimises the wrong thing.
- Week 2: health checks. Build shallow and deep versions, then simulate a slow-but-alive server and a shared-dependency blip and observe both behaviours.
- Week 3: failure. Implement circuit breaking, retry budgets with jitter, and load shedding, and be able to argue why a fast 503 beats a slow queue.
- Final week: multi-layer architecture — DNS, edge L4, internal L7 — then two spoken mocks with cascading-failure follow-ups.
Building the rest of the stack? The Nginx guide covers the proxy, the caching guide covers reducing the load in the first place, the distributed systems guide covers the theory, and the system design guide covers assembling them.
Frequently asked questions
What are the most common load balancing interview questions?
The most common questions cover the difference between Layer 4 and Layer 7 load balancing and what each can route on, the balancing algorithms and when round robin is a poor default, how to design health checks and why a shallow check is dangerous, why sticky sessions are a workaround rather than a solution, how a load balancer can amplify a failure into a cascading outage, and what happens if the load balancer itself fails.
What is the difference between Layer 4 and Layer 7 load balancing?
Layer 4 balancing routes on IP address and port without inspecting the payload, making it very fast, protocol-agnostic and cheap to scale, but unable to route by URL path or retry individual HTTP requests. Layer 7 balancing parses the application protocol, so it can route by path, host or header, terminate TLS, rate limit and retry idempotent requests, at the cost of more CPU and latency. gRPC and WebSockets specifically need L7-aware balancing.
Why is round robin often a poor load balancing algorithm?
Round robin distributes requests evenly, not work. If one endpoint takes five milliseconds and another takes five seconds, an even split of request counts produces a wildly uneven split of actual load, so some servers saturate while others idle. Least connections is a better default because it accounts for request duration, and power of two choices — picking two backends at random and using the less loaded — gets most of that benefit with far less coordination.
How should you design a health check for a load balancer?
Use separate liveness and readiness endpoints: liveness answers whether the process should be restarted, readiness whether it should receive traffic. Make the readiness check deep enough to catch an instance that is alive but unable to serve, since a shallow check returns 200 from a server drowning in garbage collection. But do not check shared dependencies, because if every instance tests the same database, one database blip fails every health check and removes the entire fleet.
What are sticky sessions and why are they discouraged?
Sticky sessions pin a client to one backend, usually by cookie or IP hash, because the application stores session state in server memory. They are discouraged because they break even load distribution, lose sessions when instances scale in, complicate rolling deployments, and mean one server failing logs out a subset of users. The better fix is externalising session state to Redis, a database, or a signed token so any server can serve any request.
How do you prevent a load balancer from causing cascading failure?
Keep enough capacity headroom that losing one or two instances does not push the rest past capacity. Shed load by rejecting a fraction of requests quickly rather than queueing everything and failing slowly. Use circuit breakers to stop hammering a failing backend, cap retries with a budget and use exponential backoff with jitter since naive retries multiply load during overload, and ramp traffic to newly added instances gradually with slow start.