NLP interview questions and answers

"How many tokens is the word unbelievable?" the interviewer asked. The candidate said one — it's a word. "Try it in a tokenizer." It is usually three or four, depending on the model, and that fact quietly explains why the candidate's character-limit logic had been off by 30% in production for a month.

NLP interview questions in 2026 sit awkwardly across two eras: the classical pipeline everyone still gets asked about, and the LLM stack everyone actually builds on. Good loops test both, plus the judgment to know when you need a fine-tune and when you need a better prompt. This guide covers tokenization, embeddings, attention, the fine-tuning-versus-RAG decision, and evaluation.

What NLP interviews actually test

NLP interview questions appear in ML engineer, AI engineer, data scientist and research loops. Five bands:

• Text representation — tokenization, embeddings, and why both are less obvious than they look.
• Architecture — attention and transformers, encoder vs decoder, positional information.
• Adaptation — pretraining, fine-tuning, LoRA, prompting, RAG, and choosing between them.
• Evaluation — the hardest part of modern NLP, and the band most candidates underprepare.
• Production concerns — latency, cost, hallucination, context limits.

Tokenization: the question from the intro

Subword tokenization is asked constantly because it explains so many downstream behaviours. Word-level vocabularies cannot handle unseen words; character-level sequences are too long. Byte Pair Encoding and WordPiece split the difference: start from characters and iteratively merge the most frequent adjacent pairs until you reach a target vocabulary size. Frequent words end up as single tokens; rare ones decompose.

The consequences interviewers want you to connect:

• No out-of-vocabulary problem — anything can be spelled out from smaller pieces.
• Token count ≠ word count — which breaks naive cost estimates, context-window arithmetic and truncation logic, exactly as in the opening story.
• Non-English text costs more — languages underrepresented in the training corpus fragment into more tokens per word, which is a real fairness and cost issue and a great thing to raise unprompted.
• Numbers and code tokenize oddly — part of why models historically struggled with arithmetic.

Classical preprocessing still gets asked: stemming (crude suffix chopping) versus lemmatization (dictionary-based, returns a real word), stop-word removal, and — importantly — why you would not do these for a transformer, since the pretrained model expects raw text tokenized its way. Knowing that the classical steps are largely obsolete for LLM pipelines, and saying so with a reason, is a maturity signal.

Embeddings and the shift to contextual representations

The progression to narrate: one-hot vectors (huge, sparse, no notion of similarity) → TF-IDF (frequency weighted by rarity, still no semantics) → word2vec and GloVe (dense static vectors where similar words sit close) → contextual embeddings from transformers.

The key limitation of static embeddings, and the reason for the shift: bank gets one vector, whether it is a river bank or a savings bank. Contextual models produce a different vector per occurrence based on surrounding text. That one example answers the question completely.

Practical follow-ups worth having ready: cosine similarity as the standard comparison metric and why magnitude is usually ignored; that embeddings inherit social biases from training data, with the classic analogy demonstrations; and — very common now — vector databases, approximate nearest neighbour search, and the tradeoff that ANN buys speed with a small recall loss. Our LangChain interview questions guide covers the retrieval tooling.

Attention and transformers

Expect "explain attention." The answer in plain terms: each token produces a query, a key and a value. Attention scores every token against every other by comparing queries to keys, softmaxes those scores into weights, and returns a weighted sum of values. So each position's new representation is a blend of the whole sequence, weighted by relevance.

Implementing it is a common request, and it is shorter than people expect:

import torch, math
import torch.nn.functional as F

def scaled_dot_product_attention(q, k, v, causal=False):
    """q, k, v: (batch, heads, seq_len, d_k)"""
    d_k = q.size(-1)

    # Compare every query against every key -> (batch, heads, seq, seq)
    scores = q @ k.transpose(-2, -1) / math.sqrt(d_k)

    if causal:
        # Position i must not see i+1.. — mask BEFORE the softmax
        seq = scores.size(-1)
        mask = torch.triu(torch.ones(seq, seq, device=q.device), diagonal=1).bool()
        scores = scores.masked_fill(mask, float("-inf"))

    weights = F.softmax(scores, dim=-1)
    return weights @ v

Two details interviewers listen for: the division by sqrt(d_k) happens before the softmax, and the causal mask uses -inf rather than zero — because zeroing a score still leaves it competing in the softmax, while -inf sends its weight to exactly zero.

The follow-ups that matter:

• Why scale by √d_k? Dot products grow with dimension; without scaling the softmax saturates and gradients vanish.
• Why multi-head? Different heads attend to different relationship types — syntactic, positional, coreference — in parallel subspaces.
• Why positional encodings? Attention is permutation-invariant; without position information the sentence is a bag of tokens. Know that modern models often use rotary embeddings (RoPE) rather than the original sinusoidal scheme.
• Why did transformers beat RNNs? Parallelism across the sequence during training (RNNs are sequential) plus direct long-range connections instead of information squeezed through a recurrent bottleneck.
• What is the cost? Self-attention is O(n²) in sequence length, which is why context windows were historically limited and why FlashAttention and sparse variants exist.

Know the architecture families: encoder-only (BERT — bidirectional, good for classification and retrieval), decoder-only (GPT family — causal masking, generation, what almost all modern LLMs are), and encoder-decoder (T5, translation). Being able to say why a causal mask is necessary — so position i cannot see the future it is meant to predict — is a common checkpoint.

Fine-tuning vs prompting vs RAG

The most practically important question in a modern NLP loop, usually framed as a scenario: "a client wants a chatbot over their internal documentation — what do you build?"

The answer that scores works from cheapest to most expensive and matches the technique to the problem:

• Prompting first. Zero-shot, then few-shot examples. Cheap, instant to iterate, often sufficient. Start here and say why.
• RAG when the problem is knowledge. The model lacks facts, or the facts change, or you need citations. Retrieve relevant chunks and put them in the context. This is the right answer for the documentation chatbot, and saying so confidently is the point of the question.
• Fine-tuning when the problem is behaviour. Consistent format, a specific tone, a narrow classification task, or reducing prompt length and latency at scale. Fine-tuning teaches style and task shape far better than it teaches facts — that distinction is the crux.
• LoRA / PEFT — train small low-rank adapter matrices instead of all weights, cutting memory and cost enormously while keeping most of the benefit. Name it; it is the default fine-tuning approach now.

Then the RAG depth questions, because interviewers go there: chunking strategy and why chunk size trades context against precision, why chunk overlap exists, hybrid search combining keyword and vector retrieval, reranking a larger candidate set with a cross-encoder, and evaluating retrieval separately from generation — because if retrieval fails, no amount of prompt engineering saves the answer. Our LLM production engineer guide and prompt engineering guide go deeper.

Evaluation, the band candidates underprepare

Know the classical metrics and their honest limitations: accuracy/precision/recall/F1 for classification; BLEU for translation (n-gram overlap, penalises valid paraphrase); ROUGE for summarisation (recall-oriented overlap, same weakness); perplexity for language modelling (how surprised the model is, not whether the output is useful).

Then the modern problem, which is the real question: how do you evaluate open-ended generation where there is no single right answer? A complete answer covers human evaluation as the gold standard and why it does not scale; LLM-as-judge with its known biases (position bias, verbosity bias, self-preference) and the mitigations of randomising order and using rubrics; task-specific automated checks such as exact-match on extracted fields, schema validation, or unit tests for generated code; and a golden dataset with regression testing so you can tell whether a prompt change helped or hurt.

The line that lands: "I'd rather have fifty well-chosen examples I check every deploy than a single aggregate score nobody trusts."

Hugging Face, papers, courses, ChatGPT — where each fits

• The Hugging Face NLP course — free, current, and closest to what these interviews actually ask.
• Attention Is All You Need, plus The Illustrated Transformer — read the paper, then Jay Alammar's visual walkthrough. Together they make the attention questions straightforward.
• Speech and Language Processing (Jurafsky & Martin) — free online and the reference for the classical half nobody expects to be asked about and everybody is.
• Building a small RAG system yourself — highest yield. Chunk documents, embed them, retrieve, generate, then watch it fail on a question whose answer spans two chunks. That failure is the interview.
• Tokenizer playgrounds — five minutes with one permanently fixes the token-versus-word confusion.
• ChatGPT — genuinely useful for exploring NLP concepts, and unable to ask why your retrieval returned the wrong chunk.
• Greenroom — the spoken layer. Ari, the AI interviewer, runs the round out loud and pushes the design follow-up when your answer jumps to fine-tuning before considering retrieval. Fair tradeoff: Ari will not train your adapter.

How to prepare for an NLP interview

• Week 1: representation — tokenize real text in a playground, implement TF-IDF by hand, and explain the static-versus-contextual embedding shift aloud with the bank example.
• Week 2: transformers. Implement scaled dot-product attention from scratch, then explain multi-head, positional encoding and causal masking in ninety seconds each.
• Week 3: build a small RAG pipeline end to end, break its retrieval deliberately, and practise the fine-tune-versus-RAG scenario answer out loud.
• Final week: evaluation — build a fifty-example golden set for a task you know, plus two spoken mocks with scenario follow-ups.

Going deeper? The deep learning interview questions guide covers the training theory, the PyTorch guide covers the framework, the AI engineer interview questions guide covers the applied stack, and the computer vision guide covers the other major domain.

Frequently asked questions

What are the most common NLP interview questions?

The most common questions cover subword tokenization such as byte pair encoding and why token count differs from word count, the shift from static embeddings like word2vec to contextual ones, explaining attention and why it is scaled by the square root of the key dimension, why transformers replaced RNNs, when to choose prompting versus RAG versus fine-tuning, and how you evaluate open-ended generation where there is no single correct answer.

How does subword tokenization like BPE work?

Byte pair encoding starts from individual characters and repeatedly merges the most frequent adjacent pair until the vocabulary reaches a target size. Frequent words become single tokens while rare words decompose into pieces, which eliminates the out-of-vocabulary problem. The practical consequences are that token count does not equal word count, breaking naive context and cost arithmetic, and that underrepresented languages fragment into more tokens per word.

What is the difference between fine-tuning and RAG?

Fine-tuning updates model weights and is the right choice when the problem is behaviour — a consistent output format, a specific tone, or a narrow task shape. RAG retrieves relevant documents at query time and puts them in the context, which is the right choice when the problem is knowledge, particularly when facts change frequently or you need citations. Fine-tuning teaches style far better than it teaches facts.

How does attention work in a transformer?

Each token produces a query, key and value vector. Attention compares every query against every key to score relevance, applies softmax to turn those scores into weights, and returns a weighted sum of the value vectors, so each position's representation blends the whole sequence by relevance. Scores are divided by the square root of the key dimension because dot products grow with dimensionality and would otherwise saturate the softmax and vanish gradients.

Why did transformers replace RNNs for NLP?

Transformers process an entire sequence in parallel during training, whereas RNNs must process tokens sequentially, which makes transformers vastly faster to train on modern hardware. They also connect any two positions directly through attention rather than forcing information through a recurrent bottleneck, which handles long-range dependencies much better. The cost is that self-attention scales quadratically with sequence length.

How do you evaluate a large language model or open-ended generation?

Combine several imperfect signals. Human evaluation is the gold standard but does not scale. LLM-as-judge scales but carries position, verbosity and self-preference biases, mitigated by randomising order and using explicit rubrics. Add task-specific automated checks such as exact match on extracted fields, schema validation or unit tests for generated code. Above all maintain a golden dataset so you can regression-test whether a prompt change actually helped.