PyTorch interview questions and answers

"Write me a training loop," the interviewer said, which sounds like the easiest question in machine learning until you are typing it in a shared editor with someone watching. The candidate wrote eleven tidy lines. The interviewer read them and asked, mildly, "What happens on the second batch?" The answer was: the gradients from the first batch were still there, quietly being added to.

PyTorch interview questions are largely a test of whether you have written that loop from memory enough times to know where it bites. The framework is deliberately unopinionated, which means it will happily let you accumulate gradients, run dropout at test time, or leave a tensor on the wrong device. This guide covers autograd, the loop, the data pipeline, and the bugs interviewers ask about by name.

What PyTorch interviews actually test

PyTorch interview questions show up in ML engineer, research engineer and applied scientist loops, and cluster into five areas:

• Tensors and autograd — the computation graph, requires_grad, detach, no_grad.
• The training loop — written from memory, with the classic bugs.
• Modules and the data pipeline — nn.Module, Dataset, DataLoader, collation.
• Performance — GPU transfer, mixed precision, distributed training, profiling.
• Debugging — shape errors, device errors, memory leaks and OOM.

Because PyTorch is the framework, the concepts underneath are fair game too — the deep learning interview questions guide is effectively half of this preparation.

Autograd: the questions behind the magic

Expect "how does autograd work?" The answer: PyTorch builds a dynamic computation graph as operations execute, recording how each tensor was produced. Calling .backward() on a scalar walks that graph in reverse, applying the chain rule, and accumulates gradients into the .grad attribute of every leaf tensor with requires_grad=True. The graph is then freed — which is why calling backward() twice raises an error unless you pass retain_graph=True.

The distinctions interviewers probe:

• detach() vs no_grad() — detach() returns a single tensor cut out of the graph; torch.no_grad() is a context manager that stops recording entirely for everything inside it. Use no_grad() for inference and validation because it also saves memory.
• Why .item() matters — accumulating total_loss += loss keeps the whole graph alive for every batch and leaks memory across an epoch. Use loss.item(). This is asked as a memory-leak question surprisingly often.
• Leaf vs non-leaf tensors — gradients populate .grad only on leaves by default; for an intermediate you need retain_grad().
• Why gradients accumulate at all — it is deliberate, and it is what makes gradient accumulation across micro-batches and multi-loss setups possible. Which is exactly why you must clear them yourself.

The training loop and its classic bugs

You will be asked to write this from memory. Here it is with the four bugs interviewers hunt for annotated:

for epoch in range(num_epochs):
    model.train()                      # BUG 1 if missing: dropout/BN stay in eval mode
    for x, y in train_loader:
        x, y = x.to(device), y.to(device)

        optimizer.zero_grad()          # BUG 2 if missing: gradients accumulate across batches
        logits = model(x)
        loss = criterion(logits, y)   # CrossEntropyLoss wants raw logits, not softmax
        loss.backward()
        optimizer.step()

        running += loss.item()        # BUG 3 if you drop .item(): the graph leaks

    model.eval()                       # BUG 4 if missing: dropout active during validation
    with torch.no_grad():
        for x, y in val_loader:
            ...

Be ready to explain each. The zero_grad() one is the most-asked question in PyTorch interviewing, and the complete answer says why the default is accumulation rather than just "you have to call it." The CrossEntropyLoss point is a favourite trap — it applies log_softmax internally, so adding a softmax in your model's forward pass silently degrades training rather than erroring.

The natural follow-up is gradient accumulation for a batch too large for memory: call backward() on each micro-batch, and only step() and zero_grad() every N micro-batches — scaling the loss by 1/N so the gradient magnitude matches a true large batch.

nn.Module, Dataset and DataLoader

On modules: know that nn.Module tracks submodules and parameters via attribute assignment, which is why a list of layers must be nn.ModuleList rather than a plain Python list — a plain list means those parameters never reach the optimizer and never train. That is a classic interview question and a classic real bug.

Know that you call model(x), not model.forward(x), because __call__ runs registered hooks around the forward pass. Know the difference between parameters() and buffers() — buffers (like batch norm running stats) are saved in the state_dict but not optimized. And know that saving state_dict() is preferred over pickling the whole model, because the latter binds you to the exact class definition and file layout.

On data: a Dataset implements __len__ and __getitem__; a DataLoader handles batching, shuffling and parallel loading. Interviewers ask about num_workers (parallel loading processes — and why 0 makes GPU training input-bound), pin_memory (faster host-to-device transfer), and writing a custom collate_fn for variable-length sequences, which is where padding happens. Our Python interview questions guide covers the language layer.

GPU, mixed precision and distributed training

For applied and infrastructure-leaning roles this section decides the round:

• Device management — tensors and model must be on the same device; the error message for this is one every practitioner recognises instantly, and interviewers ask what causes it.
• Mixed precision (AMP) — autocast runs suitable ops in fp16/bf16 for speed and memory, while GradScaler scales the loss to stop small fp16 gradients underflowing to zero. Explaining why the scaler exists is the differentiator.
• DataParallel vs DistributedDataParallel — DataParallel is single-process multi-threaded, bottlenecks on one GPU and is effectively deprecated; DDP runs one process per GPU with gradient all-reduce and is what you should say you use.
• OOM debugging — reduce batch size, gradient accumulation, gradient checkpointing (recompute activations to trade compute for memory), mixed precision, and check you are not accumulating graphs via the .item() bug above.
• torch.compile — graph capture and kernel fusion for speedups on recent versions. Naming it reads as current.

PyTorch vs TensorFlow, and the honest comparison

You will be asked. The fair answer in 2026: PyTorch dominates research and has become the default in most production stacks too, largely because eager execution makes debugging ordinary Python debugging. TensorFlow retains strengths in mature deployment tooling — TF Serving, TFLite for mobile, TFX pipelines — and a lot of enterprise code is written in it and is not being rewritten.

The mature framing: they have converged substantially (TF added eager mode, PyTorch added graph compilation and better serving), so the choice is usually driven by what the team already runs and what the deployment target is, not by a technical superiority argument. Refusing to trash the other framework reads well. Our TensorFlow interview questions guide covers the other side.

The docs, Karpathy, Kaggle, ChatGPT — where each fits

• The official PyTorch tutorials and docs — genuinely excellent, and the autograd mechanics page answers most of the hard questions directly.
• Karpathy's Zero to Hero series — builds autograd from scratch, which makes the gradient questions unbluffable.
• Writing the training loop from memory, daily — highest yield item here, and it takes five minutes. The round frequently opens with exactly this.
• Kaggle or any real project — where you meet OOM, device errors and silent eval-mode bugs for yourself.
• GeeksforGeeks-style question dumps — fine for definitions, weak for the "what happens on the second batch?" family.
• ChatGPT — good for reviewing code and explaining an API. It will not read your eleven tidy lines and ask what happens on the second batch.
• Greenroom — the spoken layer. Ari, the AI interviewer, runs the round out loud and pushes the consequence follow-up when your answer stops at the definition. Fair tradeoff: Ari will not run your notebook.

How to prepare for a PyTorch interview

• Week 1: autograd — build a tiny autograd engine, then explain detach, no_grad, leaf tensors and graph freeing aloud.
• Week 2: write the training loop from memory every day, then break it four ways and diagnose each from symptoms alone.
• Week 3: data pipeline and performance — custom Dataset and collate_fn, then AMP and a DDP script, plus one deliberate OOM you fix three different ways.
• Final week: one project explained end to end in first person, the PyTorch-vs-TensorFlow answer rehearsed without partisanship, and two full spoken mocks.

Going deeper? The deep learning interview questions guide covers the theory underneath, the NLP and computer vision guides cover the domains, and the LLM production engineer guide covers serving models at scale.

Frequently asked questions

What are the most common PyTorch interview questions?

The most common questions are writing a training loop from memory, explaining why optimizer.zero_grad() is necessary and why gradients accumulate by default, the difference between detach() and torch.no_grad(), why model.train() and model.eval() matter for dropout and batch normalisation, how Dataset and DataLoader work including num_workers and collate_fn, and how you would debug an out-of-memory error.

Why do you need to call zero_grad() in PyTorch?

PyTorch accumulates gradients into the .grad attribute rather than overwriting them, so without clearing them each iteration the gradients from previous batches are added to the current ones and the updates become meaningless. The accumulation default is deliberate because it enables gradient accumulation across micro-batches and multi-loss setups, which is exactly why clearing them is left to you.

What is the difference between detach() and torch.no_grad()?

detach() returns a single tensor that is cut out of the computation graph, while torch.no_grad() is a context manager that disables gradient recording for every operation inside it. Use no_grad() for inference and validation loops, since it also avoids storing intermediate activations and therefore saves significant memory. Use detach() when you need one specific tensor to stop propagating gradients.

Why does model.eval() matter in PyTorch?

model.eval() switches modules that behave differently between training and inference. Dropout stops zeroing units, and batch normalisation uses its accumulated running statistics instead of the current batch's statistics. Forgetting it during validation or inference produces noisy, batch-dependent predictions that are silently wrong rather than raising an error, which makes it one of the most commonly shipped bugs.

What is the difference between DataParallel and DistributedDataParallel?

DataParallel is single-process and multi-threaded, replicating the model each forward pass and gathering outputs on one GPU, which creates a bottleneck and is effectively deprecated. DistributedDataParallel runs one process per GPU and synchronises gradients with an all-reduce operation, which scales far better across multiple GPUs and machines and is the recommended approach for any serious multi-GPU training.

Should I learn PyTorch or TensorFlow for interviews in 2026?

PyTorch dominates research and has become the default in most production stacks, so it is the safer primary choice. TensorFlow retains real strengths in deployment tooling such as TF Serving and TFLite, and substantial enterprise codebases still use it. The frameworks have converged considerably, so the honest interview answer is that the choice depends on the team's existing stack and deployment target rather than technical superiority.