Midway through a TCS data engineering interview, a candidate had just finished a confident, correct answer about Spark's DAG scheduler and lazy evaluation. Then the interviewer asked: "What's the default HDFS block size?" A pause. "We don't really use HDFS, we read from S3." The interviewer wasn't testing whether the candidate's team happened to use S3. He was testing whether they understood the storage layer their Spark jobs sit on top of at every other company in the room's client portfolio — and the honest "I don't use it day to day" landed very differently from "128MB, up from 64 in Hadoop 1, and here's why it changed."
That's the trap this guide is built to close. Hadoop interview questions haven't disappeared just because Spark now does most of the actual compute at product companies — they've concentrated into a specific, testable core: HDFS architecture, the MapReduce programming model, YARN, fault tolerance, and — increasingly — the honest relationship between Hadoop and the tools that came after it. This post covers all of it, with real answers, not just definitions.
What Hadoop interview questions actually test in 2026
Nobody is asking you to write a MapReduce job by hand anymore, and an interviewer who does is testing something narrow — usually whether you understand why a distributed compute model needed to look that way before anything better existed. The real target of most Hadoop interview questions today is whether you understand the storage and resource-scheduling layer that a large share of "big data" infrastructure — including plenty of Spark clusters — still runs on, even at companies that migrated their compute engine years ago.
Three ecosystem pieces come up constantly: HDFS (how data is actually stored and replicated), MapReduce (the original processing model, and the reasoning that still underlies distributed compute even where MapReduce itself is gone), and YARN (how a shared cluster decides who gets which resources, for which job, right now). A fourth theme threads through nearly every senior-level round: fault tolerance — what happens when a machine holding your data, or running your job, simply dies mid-task.
HDFS architecture: NameNode, DataNode, blocks and replication
HDFS (Hadoop Distributed File System) splits every file into fixed-size blocks — 128MB by default in Hadoop 2 and 3 (64MB in Hadoop 1, a version-history detail interviewers use to check whether you've actually worked with it or just memorized a number) — and scatters those blocks across a cluster of machines.
The NameNode is the single master that holds the filesystem's metadata in memory: the directory tree, which blocks make up which file, and which DataNodes hold each block's replicas. It does not store any actual file data. The DataNodes are the workers — they store the real blocks on local disk and send periodic heartbeats to the NameNode confirming they're alive and reporting which blocks they hold.
Replication factor defaults to 3: every block is stored on three different DataNodes, usually rack-aware — two copies on one rack, one on another — so a single machine failure, or even a whole rack failure, doesn't lose data. This is the direct mechanism behind Hadoop's core selling point: fault tolerance through redundancy, not through any single component being unbreakable.
The follow-up interviewers actually ask: "What happens if the NameNode dies?" In classic single-NameNode Hadoop, the whole filesystem becomes unreachable — the NameNode is a single point of failure, because it's the only place the block-to-location mapping lives. Modern Hadoop (2.x onward) fixes this with NameNode High Availability: an active and a standby NameNode, kept in sync via a shared edit log on JournalNodes, with automatic failover managed by ZooKeeper. Knowing that this was a real, painful gap Hadoop had to be redesigned to fix is a stronger answer than reciting the HA setup as if it always existed.
MapReduce: map, shuffle, reduce and the combiner
MapReduce is the programming model Hadoop shipped with: break a big computation into a map phase (transform each input record independently, in parallel, close to where the data physically lives — "move the computation to the data, not the data to the computation") followed by a shuffle-and-sort phase (group all values by key across the whole cluster) followed by a reduce phase (aggregate each key's values into a final result).
Word count is the canonical teaching example, and interviewers still use it because it's the fastest way to check you actually understand the phases rather than just naming them: the mapper emits (word, 1) for every word in its input split; the shuffle groups all the 1s for the same word onto the same reducer, across however many machines held the original data; the reducer sums them into a final count per word.
The shuffle is the expensive part — the same reason it's expensive in Spark today. Moving keyed data across the network, writing intermediate results to disk between phases, and sorting it all is I/O-heavy and slow, which is the single biggest reason Spark's in-memory model eventually replaced MapReduce for new development.
Combiners are the answer to "how do you reduce that shuffle cost?" A combiner runs a mini-reduce locally on the mapper's output, before anything crosses the network — in word count, summing partial counts for a word within one mapper's split before shipping them to the shuffle. It's an optimization, not a guarantee (Hadoop may or may not run it, and it must be associative and commutative to be safe to skip), and forgetting that "may not run" detail is exactly the kind of gap a good interviewer probes for.
YARN: ResourceManager, NodeManager and the ApplicationMaster
YARN (Yet Another Resource Negotiator) is Hadoop's cluster resource manager, split out as its own layer in Hadoop 2 specifically so MapReduce wasn't the only thing that could run on a Hadoop cluster — which is exactly the architectural decision that later let Spark, Tez, and other engines run on the same YARN-managed cluster instead of Hadoop needing to be replaced wholesale.
The ResourceManager is the cluster-wide master — one per cluster — that tracks how much CPU and memory every node has available and arbitrates which job gets which resources. The NodeManager runs on every worker node, manages containers (the actual CPU/memory-bounded execution units) on that machine, and reports resource usage back to the ResourceManager. The ApplicationMaster is the one genuinely non-obvious piece: a per-application coordinator, spun up in its own container when a job starts, that negotiates resources for that specific job with the ResourceManager and manages its own task scheduling — meaning YARN itself doesn't need to know anything about MapReduce, Spark, or Tez internals; it just grants containers to whichever ApplicationMaster asks.
The follow-up interviewers actually ask: "Why was YARN split out from MapReduce in the first place?" In Hadoop 1, the JobTracker did both resource management and job scheduling/monitoring for MapReduce specifically — a single component doing two jobs, and a scalability bottleneck past a few thousand nodes. Splitting resource management (ResourceManager/NodeManager) from application-specific execution (ApplicationMaster) is what let Hadoop clusters run more than one kind of processing engine at all.
Fault tolerance and how Hadoop scales
Hadoop's whole design premise is that commodity hardware fails constantly at scale, so the system has to expect it rather than prevent it. Three mechanisms do most of the work:
Block replication (covered above) means losing a DataNode doesn't lose data — the NameNode notices the missing heartbeats, sees which blocks that node held, and instructs other DataNodes to re-replicate those blocks to restore the replication factor.
Speculative execution handles a slow-but-not-dead task, not a dead one: if one task is running far behind its peers (a "straggler," often caused by a slow or overloaded machine rather than a genuine failure), the framework can launch a duplicate copy of that task on another node and simply use whichever finishes first, killing the other. This is a direct ancestor of the same "one task takes forever while 199 finish in seconds" symptom that shows up in Spark data-skew debugging today.
Horizontal scaling — adding more commodity machines rather than buying bigger ones — is the answer to "how does Hadoop scale?" Both HDFS (more DataNodes, more storage and replication capacity) and YARN (more NodeManagers, more schedulable resources) scale by adding nodes, not by upgrading individual machines, which is the entire economic argument for Hadoop existing in 2006: cheap, replaceable hardware instead of expensive, reliable hardware.
Why Spark replaced MapReduce — but not Hadoop itself
This is the nuance that trips up more candidates than any single architecture question, and it's worth stating plainly: Spark commonly runs on top of Hadoop's HDFS and YARN — it doesn't replace them. "We moved off Hadoop to Spark" is a sentence a lot of engineers say loosely, and a good interviewer will ask you to be precise about what that actually means at your last company.
MapReduce writes intermediate results to disk between every map and reduce phase — durable, but slow. Spark keeps intermediate data in memory across a whole DAG of transformations by default, only spilling to disk under memory pressure, which is why an equivalent Spark job routinely runs 10-100x faster on iterative workloads (the kind machine learning training and multi-step ETL both are). That's the real reason Spark displaced MapReduce for new development — not that HDFS or YARN were themselves the bottleneck.
What Spark did not need to reinvent: a distributed, replicated filesystem, or a cluster resource scheduler. A huge share of production Spark clusters (spark-submit --master yarn) still read from HDFS and get their executors scheduled by YARN — Spark supplies a better compute engine on top of storage and scheduling infrastructure Hadoop already solved. Cloud-native setups increasingly swap HDFS for S3/GCS/ADLS object storage and YARN for Kubernetes, which is the real shift happening — away from Hadoop's specific implementations of storage and scheduling, not away from needing storage and scheduling at all. See our Apache Spark interview questions guide for the architecture on the compute side of that split.
Where this still gets asked — and where it doesn't
At most Indian product companies and global tech firms, the practical stack has moved to Spark, Databricks, or cloud-native warehouses, and interviewers there rarely open with raw HDFS block-size trivia. But Hadoop interview questions are still a real, active part of data engineering rounds at Indian service companies — TCS, Infosys, Wipro, Cognizant, and similar consulting shops running legacy client data platforms that were built on Hadoop a decade ago and never fully migrated off it. If your interview loop is with a service company staffing a data engineering role, or a bank/insurance client project running an older on-prem stack, assume HDFS and YARN fundamentals are in scope, not optional trivia.
At product companies, the honest framing that actually lands well: know the architecture cold enough to explain why Spark's model is faster and what it inherited from Hadoop, rather than pretending Hadoop doesn't exist. That answer signals real systems understanding; "we don't use Hadoop" as a full answer signals you memorized a migration headline without understanding what changed underneath it.
GeeksforGeeks lists, official docs, and ChatGPT — where they fall short
Most Hadoop prep starts the same way: a GeeksforGeeks-style question dump, the Apache Hadoop documentation, or asking ChatGPT to "explain HDFS architecture." All three are genuinely useful for building the mental model this post covers — and none of them prepare you for the actual interview, which is verbal, live, and built entirely around follow-ups a static page can't predict.
A question dump gives you the definition of a combiner. It won't ask you, out loud, with someone watching you think, "you said replication factor 3 — what happens to the other two copies the instant a DataNode dies?" ChatGPT will explain the NameNode/DataNode split in a clean paragraph you can read silently — it won't notice you called YARN "the scheduler for MapReduce" instead of "a general-purpose resource manager that MapReduce happens to run on," the exact imprecision a real interviewer catches immediately.
Ari, the AI interviewer behind Greenroom, runs a spoken mock interview that asks the follow-up a real Hadoop interviewer would actually ask next — not a script, a genuine response to what you said. Compare that to a friend's WhatsApp PDF of "50 Hadoop interview questions" — useful for recognition, useless for producing an answer live under a follow-up you didn't see coming.
Practise the architecture questions out loud
You can read this whole guide, nod at every section, and still stumble the first time someone asks you live to trace what happens between a DataNode dying and the NameNode noticing. Reading is recognition; interviewing is production under pressure with a stranger watching you think. Greenroom runs spoken data-engineering mock interviews, asks real follow-ups on HDFS, YARN, and fault tolerance, and gives feedback on how clearly you explained your reasoning — not just whether the final answer was technically right.
Pair this with our guides on Apache Spark interview questions if the compute-engine half of the loop matters more for your role, Apache Kafka interview questions if the pipeline streams data in before it ever lands in HDFS, and PySpark interview questions for code-level practice on the engine that most often runs on top of this exact storage layer today.
Frequently asked questions
What are the most common Hadoop interview questions?
The most common are explaining the NameNode/DataNode architecture and what happens if a NameNode fails, the default HDFS block size and replication factor and why they're set that way, the map/shuffle/reduce phases of MapReduce and what a combiner optimizes, the role of YARN's ResourceManager, NodeManager, and ApplicationMaster, and — increasingly — how Spark relates to Hadoop rather than simply replacing it.
What is the difference between HDFS and MapReduce?
HDFS is Hadoop's storage layer — it splits files into blocks, replicates them across DataNodes, and tracks metadata on a NameNode. MapReduce is a processing model that runs computations against data stored in HDFS (or elsewhere), splitting work into map, shuffle, and reduce phases. HDFS answers where the data lives and how it is kept safe; MapReduce answers how you compute something over that data at scale.
What is the role of YARN in Hadoop?
YARN is Hadoop's cluster resource manager. It was split out from MapReduce in Hadoop 2 specifically so more than one processing engine could share a cluster: a ResourceManager tracks available CPU and memory cluster-wide, NodeManagers manage containers on each worker node, and a per-job ApplicationMaster negotiates resources for that specific job. This split is exactly what let engines like Spark run on the same YARN-managed clusters that used to be MapReduce-only.
Is Hadoop still relevant if companies use Spark now?
Yes, in a specific and often misunderstood way: Spark commonly runs on top of Hadoop's HDFS and YARN rather than replacing them outright — moving to Spark frequently means the compute engine changed while the underlying storage and scheduling layer stayed Hadoop. Cloud-native teams are increasingly swapping HDFS for object storage (S3, GCS, ADLS) and YARN for Kubernetes, which is a real shift away from Hadoop's specific components — but the storage-plus-scheduling architecture Hadoop established is still the pattern most of the industry follows.
What is the default HDFS block size and replication factor?
The default block size is 128MB in Hadoop 2 and 3 (64MB in the original Hadoop 1). The default replication factor is 3 — every block is stored on three different DataNodes, typically rack-aware, so a single machine or rack failure doesn't cause data loss.
Do Indian service companies like TCS and Infosys still ask Hadoop questions?
Yes — Hadoop interview questions remain common at Indian service-company data engineering interviews (TCS, Infosys, Wipro, Cognizant and similar consulting shops), because many client data platforms they maintain were originally built on Hadoop and haven't fully migrated to newer stacks. Product companies ask Hadoop questions less often directly, but still expect you to know how Spark relates to the HDFS/YARN layer it frequently runs on.