---
title: Distributed Systems Interview Questions and Answers (2026)
description: Real distributed systems interview questions — CAP, consistency models, consensus, idempotency and exactly-once delivery — with answers and a four-week prep plan.
url: https://usegreenroom.app/blog/distributed-systems-interview-questions
last_updated: 2026-07-31
---
← Back to blog
System Design
# Distributed systems interview questions and answers
July 31, 2026 · 13 min read
"Your service calls the payment provider," the interviewer said, "and the request times out. Did the payment happen?" The candidate said no, it timed out. "How do you know?" And that is the question — because a timeout tells you that you did not receive a response, and absolutely nothing about whether the work was done.
Distributed systems interview questions are almost all versions of that: you cannot distinguish slow from dead, and every design decision follows from accepting it. This guide covers CAP honestly, consistency models, consensus, and the practical questions — idempotency, exactly-once delivery, and distributed transactions — that senior rounds actually spend their time on.
## What distributed systems interviews actually test
- The fundamental constraints — the fallacies, partial failure, and why timeouts are ambiguous.
- CAP and consistency models — stated precisely rather than as a slogan.
- Consensus and replication — leader election, quorums, Raft at a conceptual level.
- Practical correctness — idempotency, exactly-once, ordering, clocks.
- Distributed transactions — two-phase commit, sagas, and the outbox pattern.
The five bands of distributed systems interview questions, with practical correctness dominating senior rounds.
## The constraints everything follows from
Start with partial failure, because it is what makes distributed systems different from a single program: a component can fail while others continue, and you often cannot tell which happened. A timeout is compatible with the request never arriving, arriving and being processed but the response being lost, or still being processed right now.
Hence the answer to the intro: you cannot know, so you design so it does not matter — an idempotency key, or a reconciliation process that asks the provider what actually happened. Saying "I'd make the operation idempotent and then retry safely" answers the question completely.
Know the fallacies of distributed computing well enough to use them: the network is not reliable, latency is not zero, bandwidth is not infinite, the network is not secure, topology changes, there is more than one administrator, transport cost is not zero, the network is not homogeneous. Interviewers rarely ask for the list, but every wrong answer in this topic assumes one of them.
## CAP, stated precisely
The most commonly mangled concept in system design interviews. Stated properly: when a network partition occurs, a distributed system must choose between consistency and availability. That is all it says.
The corrections worth making out loud, because they distinguish you immediately:
- You do not "pick two of three." Partition tolerance is not optional — networks partition, so the real choice is what you do when they do.
- It says nothing about normal operation. When there is no partition, you can have both consistency and availability, which is the overwhelming majority of the time.
- Consistency here means linearizability, a very strong guarantee — not the C in ACID, which is a different property entirely. Conflating them is a common error.
- PACELC extends it usefully: if there is a Partition, choose Availability or Consistency; Else, choose Latency or Consistency. That second clause explains most real system design, because latency-versus-consistency is the tradeoff teams actually make daily.
Then the consistency models, from strong to weak: linearizable (every read sees the most recent write, as if there were one copy), sequential, causal (causally related operations are seen in order — often the sweet spot), read-your-writes and other session guarantees, and eventual (replicas converge given no new writes, with no bound on when). Being able to say "read-your-writes is usually what users actually notice" is a practical, credible answer.
## Consensus, quorums and replication
You will not be asked to implement Raft, but you should explain it conceptually: a cluster elects a leader; the leader accepts writes and replicates them to followers; an entry commits once a majority has acknowledged it; if the leader fails, a new election happens with terms preventing two leaders from both being authoritative. Raft is designed to be understandable, and saying that its value over Paxos is comprehensibility rather than capability is a nice touch.
The related mechanics interviewers do ask about:
- Why a majority? Because two majorities of the same cluster must overlap, so a new leader is guaranteed to see any committed entry. That one sentence is the whole intuition.
- Quorum reads and writes — with N replicas, if R + W > N, reads and writes overlap and a read sees the latest write. Dynamo-style systems expose this as a tunable.
- Why odd cluster sizes — five nodes tolerate two failures, six also tolerate two, so the sixth adds cost without fault tolerance.
- Split brain — two partitions each believing they lead, which is what quorum requirements prevent by making it impossible for both sides to have a majority.
- Synchronous vs asynchronous replication — the durability-versus-latency tradeoff, and why asynchronous replication means a failover can lose recently acknowledged writes.
## Idempotency, delivery semantics and clocks
This band is where senior interviews spend most of their time, because it is what you deal with in production.
Delivery semantics. At-most-once means no duplicates and possible loss. At-least-once means no loss and possible duplicates, and is what almost every real system provides. Exactly-once delivery is not achievable over an unreliable network — the honest answer, and the one interviewers are fishing for, is that you get exactly-once processing by combining at-least-once delivery with idempotent handling or deduplication. Kafka's "exactly-once" is transactional processing within Kafka, not magic across system boundaries.
Idempotency is therefore the central practical skill: an idempotency key stored with the result so a retry returns the stored outcome; natural idempotency where possible (setting a value rather than incrementing it); and deduplication windows keyed on message ID. Our Kafka guide and RabbitMQ guide cover the messaging layer.
Clocks and ordering. Physical clocks on different machines disagree and can jump backwards, so ordering by wall-clock timestamp is unreliable — a favourite gotcha. Know logical clocks: a Lamport clock gives a total order consistent with causality but cannot tell you whether two events were truly concurrent; a vector clock can, at the cost of size. Mention that Google's TrueTime uses bounded uncertainty with atomic clocks to make Spanner's guarantees possible, since interviewers like the concrete example.
## Distributed transactions
The scenario is always some version of: an order service, a payment service and an inventory service must all succeed or all fail.
- Two-phase commit — a coordinator asks everyone to prepare, then commits if all agree. Correct, and it blocks: if the coordinator dies after prepare, participants hold locks indefinitely. Rarely used across services for that reason.
- Sagas — a sequence of local transactions, each with a compensating action to undo it. Choreographed (services react to events) or orchestrated (a coordinator drives the steps). You get eventual consistency and must design compensations that make business sense — you cannot un-send an email, so you send a correction.
- The outbox pattern — the answer to "how do you update the database and publish an event atomically?" Write the event to an outbox table in the same local transaction, then a separate process publishes it. This solves the dual-write problem and naming it unprompted is a strong signal.
- Avoid the need — the senior answer. Redraw service boundaries so the transaction is local, because a distributed transaction is often a sign the decomposition is wrong.
Our microservices guide covers the architectural context and the sharding guide covers the same problem inside one database.
## DDIA, the papers, ChatGPT — where each fits
- Designing Data-Intensive Applications — the single best preparation for this entire topic. If you read one book, read this one.
- The Raft paper — genuinely readable, unlike most systems papers, and the visualisations at raft.github.io make leader election click in ten minutes.
- Jepsen reports — Kyle Kingsbury testing real databases' consistency claims. Excellent for understanding what these guarantees mean in practice and where they break.
- Google's Spanner and Dynamo papers — the two ends of the consistency spectrum, and both are referenced in interviews by name.
- Building something that retries — highest yield practical exercise. Implement an idempotency key end to end and force duplicate delivery; the concept stops being abstract.
- ChatGPT — good for explaining consensus and comparing models. It will not ask how you know the payment did not happen.
- Greenroom — the spoken layer. Ari, the AI interviewer, runs design rounds out loud and pushes the ambiguity follow-up your reading never asks. Fair tradeoff: Ari will not verify your implementation.
> The core truth: distributed systems interviews reward candidates who accept ambiguity rather than assume it away. A timeout tells you nothing about whether the work happened — design for that, say exactly-once processing rather than exactly-once delivery, and state CAP precisely.
## How to prepare for a distributed systems interview
- Week 1: constraints. Rehearse partial failure and the timeout ambiguity until answering "how do you know?" is automatic.
- Week 2: CAP and consistency models, stated precisely, including PACELC and why linearizability is not ACID's C.
- Week 3: consensus. Work through the Raft visualisation, explain leader election and quorum overlap aloud in ninety seconds each.
- Final week: practical correctness — implement an idempotency key, describe a saga with real compensations, name the outbox pattern — then two spoken mocks.
Building the rest of the stack? The system design guide covers the round format, the sharding and caching guides cover the components, and the Kafka guide covers the log these guarantees are usually built on.
## Frequently asked questions
### What are the most common distributed systems interview questions?
The most common questions cover what a timeout actually tells you about whether work completed, how to state CAP precisely and why it does not mean picking two of three, the consistency models from linearizable to eventual, how consensus and quorums work at a conceptual level, why exactly-once delivery is impossible and what you get instead, how to make an operation idempotent, and how to coordinate a transaction across several services.
### What does the CAP theorem actually say?
It says that when a network partition occurs, a distributed system must choose between consistency and availability. It does not mean picking two of three properties, because partition tolerance is not optional — networks partition regardless of your preference. It also says nothing about behaviour when there is no partition, when you can have both. The consistency it refers to is linearizability, which is a different property from the C in ACID.
### Is exactly-once delivery possible in a distributed system?
No. Over an unreliable network you cannot guarantee a message is delivered exactly once, because the sender cannot distinguish a lost message from a lost acknowledgement. What you can achieve is exactly-once processing, by combining at-least-once delivery with idempotent handling or deduplication keyed on a message identifier. Systems advertising exactly-once semantics, such as Kafka transactions, provide it within their own boundaries rather than across external systems.
### How do you make an operation idempotent?
Attach an idempotency key generated by the caller for each logical operation, store it alongside the result, and return the stored result if the same key arrives again rather than performing the work twice. Prefer naturally idempotent operations where possible, such as setting a value rather than incrementing it. For message consumers, deduplicate on a message identifier within a retention window that comfortably exceeds your retry period.
### What is the saga pattern and when do you use it?
A saga replaces a distributed transaction with a sequence of local transactions, each having a compensating action that semantically undoes it if a later step fails. It can be choreographed, with services reacting to each other's events, or orchestrated, with a coordinator driving the steps. You accept eventual consistency, and compensations must make business sense — you cannot unsend an email, so you send a correction instead.
### Why can't you order distributed events by timestamp?
Physical clocks on different machines drift apart and can be adjusted backwards by time synchronisation, so a timestamp from one machine is not reliably comparable with one from another, and two events can appear out of order. Logical clocks solve this: a Lamport clock produces a total order consistent with causality, and a vector clock can additionally identify whether two events were genuinely concurrent, at the cost of growing with the number of nodes.
These rounds reward saying "I cannot know that" and designing around it. Greenroom runs mock system design interviews out loud with Ari and scores structure, clarity and depth. Free to start. Curious how it works? See how AI mock interviews work.