Interview guide · Technology roles
Data scientist interview questions
In short
Data scientist interview questions should test four things: sound statistics and experiment design, practical machine learning judgment, hands-on coding in SQL and Python, and the ability to turn analysis into decisions people act on. Below are 34 questions for data scientists, machine learning engineers and AI engineers, with what strong answers include, red flags, sample answers and a scorecard.
The title "data scientist" covers very different jobs. At one company it means product analytics and A/B testing; at another it means building and deploying machine learning models; at a third it now means evaluating and improving LLM-based features. Before you write a single question, decide which of those your team needs, and weight the interview accordingly. A candidate who is excellent at causal inference may be a poor fit for a role that is mostly feature engineering and model serving, and the reverse is just as common.
The questions below are grouped so you can assemble a loop: openers, statistics and experimentation, machine learning, data and coding, business impact and communication, and questions for senior, ML engineering and AI engineering candidates. Probe every answer with "how did you know it worked?" Strong data scientists talk about baselines, validation, trade-offs and what the business did differently because of their work. Weak ones talk about algorithms. Pair the conversation with a realistic exercise built from the kind of data the person will actually use.
What to look for in a data scientist
- Statistical judgmentMost costly data science mistakes are statistical: peeking at A/B tests, leakage, confounding, or misread significance. Fancy models do not fix them.
- Problem framingThe best data scientists start by asking what decision the analysis will change and what a simple baseline would achieve. That habit saves months of work on the wrong problem.
- Practical ML skillChoosing metrics, validating properly, handling imbalance and monitoring after launch matter more than knowing the newest architecture.
- Strong data and coding fundamentalsData scientists spend much of their time in SQL and Python cleaning, joining and checking data. Sloppy code produces wrong answers quietly.
- Clear communicationResults only matter if non-technical leaders understand them well enough to act. Look for plain explanations with honest uncertainty.
- Responsibility about impactModels that touch people, such as hiring, credit or pricing, can encode bias. Good candidates raise fairness, privacy and misuse without being prompted.
Opening questions
Walk me through a project you are proud of, from the business question to what changed afterwards.
- Why ask it
- Shows end-to-end ownership and whether the candidate measures impact.
- A strong answer
- States the decision or metric the work targeted, the data, the baseline, the approach, how they validated it, and a measured outcome such as a lift, a cost saving or a decision made.
- Red flags
- Describes the model in detail but cannot say whether anyone used it.
How would you describe the split of your work between analytics, experimentation, modeling and engineering?
- Why ask it
- Clarifies which kind of data scientist they are.
- A strong answer
- A specific split with examples, and awareness of what they want more or less of.
- Red flags
- Claims equal expertise in everything.
What does your typical stack look like, and what would you change about it?
- Why ask it
- Shows hands-on depth and opinions grounded in use.
- A strong answer
- Names languages, libraries, warehouse, orchestration, experiment tracking and deployment tools they used daily, with real trade-offs.
- Red flags
- A list of buzzwords without specifics.
Tell me about a time your analysis showed something stakeholders did not want to hear.
- Why ask it
- Integrity and communication under pressure.
- A strong answer
- Checked the result carefully, presented it clearly with uncertainty, and stood by it while staying open to new evidence.
- Red flags
- Changed the analysis to match expectations.
Why this role, and what kind of problems do you want to work on next?
- Why ask it
- Fit between the candidate's interests and the actual job.
- A strong answer
- Specific interest in the domain and problem types the team works on.
- Red flags
- Only interested in research when the job is applied analytics, or the reverse.
Statistics and experimentation
Explain a p-value to a product manager in two sentences.
- Why ask it
- Tests understanding and plain communication together.
- A strong answer
- If there were truly no difference, the p-value is the probability of seeing a result at least this extreme by chance. A small p-value means the result would be surprising under no effect; it is not the probability that the change works.
- Red flags
- Says it is the probability the null hypothesis is true.
Our checkout conversion is 10%. We want to detect a one-point absolute lift. Roughly how many users per group do we need, and what drives that number?
- Why ask it
- Practical power analysis.
- A strong answer
- Uses significance level (often 0.05), power (often 80%), baseline rate and minimum detectable effect; arrives at roughly 15,000 users per group; explains that smaller effects need far more users, roughly four times as many for half the effect size.
- Red flags
- Plans to run the test until it looks significant.
A PM checks the A/B test dashboard every day and wants to stop as soon as p < 0.05. What is the problem, and what would you do?
- Why ask it
- Peeking inflates false positives and is one of the most common experimentation errors.
- A strong answer
- Explains that repeated looks raise the false positive rate; proposes a fixed sample size and duration set in advance, or a sequential testing method designed for continuous monitoring.
- Red flags
- Sees no problem with stopping early.
Customers who use our mobile app churn half as much as those who do not. Does the app reduce churn?
- Why ask it
- Tests causal reasoning and confounding.
- A strong answer
- Not necessarily; engaged customers may both use the app and stay. Suggests an experiment, or methods like matching, difference-in-differences or instrumental variables, with their assumptions.
- Red flags
- Accepts the correlation as causal.
You test 20 variations of an email subject line and one wins at p = 0.04. How confident are you?
- Why ask it
- Multiple comparisons.
- A strong answer
- Not very; with 20 tests at 0.05, about one false positive is expected. Mentions corrections such as Bonferroni or false discovery rate control, or a confirmation test.
- Red flags
- Ships the winner without question.
When would you report a median instead of a mean?
- Why ask it
- Basic but revealing about real data.
- A strong answer
- For skewed or heavy-tailed data, such as revenue per user or time to fill, where a few large values distort the mean; often reports both with context.
- Red flags
- Always uses the mean.
An experiment shows a significant lift overall but a drop for new users. What do you do?
- Why ask it
- Segment analysis and heterogeneous effects.
- A strong answer
- Checks whether the segment was pre-specified and has enough power, considers Simpson's paradox and novelty effects, and discusses the business trade-off before deciding.
- Red flags
- Reports only the overall number, or slices until something looks interesting.
Machine learning
You are building a fraud model where 0.5% of transactions are fraud. Which metrics do you use, and why not accuracy?
- Why ask it
- Class imbalance and metric choice.
- A strong answer
- Accuracy is misleading because predicting no fraud scores 99.5%. Uses precision, recall, precision-recall AUC, and chooses a threshold based on the cost of false positives versus missed fraud.
- Red flags
- Reports accuracy or ROC AUC alone with no link to costs.
Tell me about a time you found data leakage in a model, yours or someone else's.
- Why ask it
- Leakage is the most common reason models fail in production.
- A strong answer
- A specific example, such as a feature recorded after the outcome or random splits on time-dependent data, how they spotted it (results too good to be true), and how they fixed validation.
- Red flags
- Has never encountered leakage or cannot explain it.
How would you validate a model that forecasts weekly call volume?
- Why ask it
- Time series validation differs from random cross-validation.
- A strong answer
- Time-based splits or rolling-origin evaluation, baselines like last year's same week or a seasonal naive forecast, error metrics such as MAPE or MAE, and checks around holidays.
- Red flags
- Uses random k-fold cross-validation.
Explain the bias-variance trade-off and how you see it in practice.
- Why ask it
- Core concept; the practical part matters more than the definition.
- A strong answer
- Simple models underfit, complex models overfit; diagnoses with learning curves and train versus validation error; uses regularization, more data or simpler features.
- Red flags
- Recites a definition with no practical example.
When would you choose gradient-boosted trees over a neural network for tabular data?
- Why ask it
- Model selection judgment.
- A strong answer
- For most tabular problems, boosted trees are a strong, fast, interpretable-enough default; neural networks make more sense with very large data, unstructured inputs or specific architectures.
- Red flags
- Always uses deep learning regardless of data.
How do you explain a model's predictions to the people affected by them?
- Why ask it
- Interpretability for stakeholders and, in some uses, regulation.
- A strong answer
- Global feature importance, SHAP or similar local explanations, partial dependence, and plain-language examples, with caveats about correlation versus cause.
- Red flags
- Says the model is a black box and that is fine.
Your model's precision dropped 15% two months after launch. How do you investigate?
- Why ask it
- Production monitoring and drift.
- A strong answer
- Checks data pipelines and schema changes first, then input distribution drift, label delay and concept drift; compares with a baseline; sets up monitoring and retraining triggers.
- Red flags
- Retrains immediately without diagnosing.
Data and coding
In SQL, how would you find each customer's second purchase and whether it happened within 30 days of the first?
- Why ask it
- Window functions are everyday data science SQL.
- A strong answer
- Uses ROW_NUMBER or LAG partitioned by customer and ordered by purchase date, then compares dates; mentions ties and customers with only one purchase.
- Red flags
- Cannot use window functions or writes many self-joins without noticing duplicates.
A key column is missing for 30% of rows. How do you decide what to do?
- Why ask it
- Missing data handling is judgment, not a default.
- A strong answer
- Investigates why it is missing (random, by segment, by process), considers whether missingness itself is informative, then chooses between dropping, imputing or adding a missing flag, and checks the effect on results.
- Red flags
- Always fills with the mean.
Your pandas job on 50 million rows runs out of memory. What do you try?
- Why ask it
- Practical engineering sense.
- A strong answer
- Pushes work into the warehouse with SQL, reads only needed columns, uses efficient data types, processes in chunks, or uses a distributed or columnar engine.
- Red flags
- Asks for a bigger laptop as the only answer.
How do you make your analysis reproducible for someone else on the team?
- Why ask it
- Team productivity and trust in results.
- A strong answer
- Version control, environment management, parameterized notebooks or scripts, documented data sources and dates, tests on key transformations, and experiment tracking.
- Red flags
- Results live in an unnamed notebook on their laptop.
What do you look for when reviewing a colleague's analysis or model code?
- Why ask it
- Code review shows standards and collaboration.
- A strong answer
- Correct joins and filters, leakage, validation design, edge cases, readability and whether conclusions match the evidence.
- Red flags
- Only comments on style.
Business impact and communication
A sales leader asks you to build a model to predict which leads will close. What questions do you ask first?
- Why ask it
- Problem framing before modeling.
- A strong answer
- What decision the score will drive, how leads are handled today, what a good outcome is worth, what data exists at scoring time, how success will be measured, and whether a simple rule would do.
- Red flags
- Starts with which algorithm to use.
Tell me about a model or analysis that was never used. Why not?
- Why ask it
- Honest reflection on adoption.
- A strong answer
- Specific reasons, such as no clear owner, a workflow that did not fit or a metric nobody trusted, and what they now do differently.
- Red flags
- Blames stakeholders for not understanding.
How would you present a result with wide uncertainty to an executive who wants a single number?
- Why ask it
- Communicating uncertainty is a core skill.
- A strong answer
- Gives a best estimate with a plain range, explains what it means for the decision, and suggests how to reduce uncertainty if needed.
- Red flags
- Hides the uncertainty or drowns the executive in statistics.
An HR team wants a model that predicts which applicants will quit in their first 90 days, to use in screening. What would you consider?
- Why ask it
- Tests fairness and legal awareness when models affect people.
- A strong answer
- Checks for disparate impact across protected groups, avoids proxies such as zip code, considers whether the model should inform retention support rather than reject applicants, and knows that tools like this can fall under rules such as NYC Local Law 144 bias audits and the EU AI Act's high-risk category for employment.
- Red flags
- Treats it as a pure accuracy problem.
Describe a time you pushed back on a request for a complex model.
- Why ask it
- Judgment about when simple is better.
- A strong answer
- Showed that a rule, a dashboard or a simple regression met the need faster and with less risk, with a result.
- Red flags
- Always builds what is asked, or always builds the most complex thing.
Senior, ML engineering and AI engineering questions
How would you evaluate an LLM feature that summarizes customer support conversations?
- Why ask it
- Core skill for AI engineer interview questions in 2026.
- A strong answer
- Builds a labeled evaluation set from real conversations, defines criteria such as accuracy, completeness and harmful errors, uses human ratings and calibrated automated grading, tracks regressions on every prompt or model change, and monitors in production.
- Red flags
- Judges quality by reading a few examples.
When would you use retrieval-augmented generation instead of fine-tuning a model?
- Why ask it
- Architecture judgment for LLM work.
- A strong answer
- Retrieval suits changing or proprietary knowledge and citations; fine-tuning suits consistent style, format or narrow tasks; often combined; mentions cost, latency and evaluation for both.
- Red flags
- Fine-tunes for every problem.
How do you decide whether to deploy a model as a batch job or a real-time service?
- Why ask it
- Machine learning engineer interview question on serving.
- A strong answer
- Depends on decision latency, feature freshness, volume and cost; batch is simpler and cheaper when predictions can be precomputed.
- Red flags
- Defaults to real time without reason.
How would you set standards for a team of five data scientists?
- Why ask it
- Leadership for senior and lead roles.
- A strong answer
- Shared code and review standards, experiment design reviews, a template for project write-ups with impact measurement, reusable data assets, and mentoring.
- Red flags
- No view on standards or wants everyone to work like them.
How do you prioritize between requests from several teams?
- Why ask it
- Senior data scientists manage demand.
- A strong answer
- Estimates value and effort, aligns with company goals, agrees priorities with stakeholders openly and says no with reasons.
- Red flags
- Works on whoever asks loudest.
Sample answers
With a 10% baseline, a one-point lift to 11%, a 5% significance level and 80% power, the standard two-proportion calculation gives roughly 15,000 users per group, so about 30,000 total. The main drivers are the minimum effect we care about and the baseline rate. Sample size scales with one over the effect squared, so if we wanted to detect half a point we'd need about four times as many users. I'd also check how long that takes with our traffic. If we get 3,000 checkout visitors a day, that's about a week and a half, and I'd round up to two full weeks so we cover weekday and weekend behavior. Then I'd fix the duration in advance and not stop early.
Accuracy is almost useless here, because a model that says nothing is ever fraud scores 99.5%. I'd look at precision and recall, and the precision-recall curve, since that focuses on the rare class. The real decision is the threshold, and that comes from costs. If a missed fraud costs us about $400 on average and a false alarm costs a $5 manual review plus some customer friction, I'd pick the threshold that minimizes expected cost, and I'd check what review volume that creates for the operations team. I'd validate on a later time period than training, because fraud patterns shift, and monitor precision weekly after launch since labels arrive with a delay.
Interview scorecard
| Competency | Weight | Strong (4–5) | Weak (1–2) |
|---|---|---|---|
| Statistics and experimentation | 20% | Designs sound experiments, explains significance and power correctly, and spots confounding and peeking. | Misreads p-values, ignores power or treats correlation as cause. |
| Machine learning judgment | 20% | Chooses metrics by cost, validates properly, catches leakage and plans monitoring. | Focuses on algorithms and accuracy with weak validation. |
| Data and coding | 20% | Fluent SQL and Python, careful with joins and missing data, writes reproducible code. | Struggles with window functions or produces fragile, untested work. |
| Problem framing and business impact | 20% | Starts from the decision, uses baselines and can show measured impact. | Builds models without asking how they will be used. |
| Communication | 10% | Explains results and uncertainty in plain language. | Technical jargon or false certainty. |
| Responsibility and collaboration | 10% | Raises fairness and privacy issues and works well with engineers and stakeholders. | Ignores impact on people and works in isolation. |
Score each competency from 1 to 5 straight after the interview, before discussing the candidate with anyone else, then multiply by the weight.
How to structure a data scientist interview loop
A practical loop for a mid-level data scientist is five steps over two to three weeks. A recruiter screen confirms the type of data science the candidate does and their expectations. A 60-minute technical screen covers statistics and SQL with a shared editor. Then comes either a take-home exercise capped at three to four hours or a live case of similar scope, followed by a final round of three or four interviews: ML and modeling, coding, a product or business case, and a behavioral interview with the hiring manager.
Build exercises from realistic data, such as a messy extract of experiment results or a churn table with a leaked feature planted in it, and give every candidate the same task and rubric. Score independently before the debrief. Recruiters running these searches will find our technical recruiter interview questions useful for assessing their own team, and a structured technical interview format keeps panels consistent.
Mistakes interviewers make with data science candidates
The most common mistake is testing algorithm trivia and puzzle-style coding instead of the actual job. Very few data scientists implement gradient descent from scratch at work; most spend their days writing SQL, checking data and explaining results. Weight the interview to match.
The second is an unpaid take-home that takes a weekend. Strong candidates with jobs and families drop out. Cap exercises at a few hours, tell candidates the time limit, or offer a live alternative. A short, realistic work sample test predicts performance better than a long one.
The third is mixing up roles. If the job is mostly dashboards and metric definitions, the data analyst interview questions will serve you better, and you should title and pay the role accordingly. If the job is mostly production systems, borrow questions from the software engineer interview questions and weight engineering skills more heavily.
Adapting the interview for ML engineers and AI engineers
For machine learning engineer roles, reduce the statistics and experimentation share and add system design: feature pipelines, model serving, monitoring, retraining and cost. Ask candidates to design a system end to end, such as a recommendation service with a 100-millisecond latency budget, and probe failure modes.
For AI engineer roles building on large language models, the core skill is evaluation. Ask how they built test sets, measured quality, caught regressions after prompt or model changes, and controlled cost and latency. Candidates who can only describe prompts they wrote, with no measurement, are early in the craft. For any role whose models affect people, such as screening, pricing or credit, include at least one question on fairness and regulation. Hiring and HR uses are a growing area of rules, including bias audits under NYC Local Law 144 and the EU AI Act, so look for candidates who treat AI bias as an engineering requirement rather than an afterthought.
Frequently asked questions
What are common data science interview questions?
Expect questions on p-values and A/B test design, causal inference, class imbalance and metric choice, data leakage, model validation, SQL window functions, missing data, and how past projects changed business decisions.
How are machine learning engineer interview questions different?
They focus more on software engineering and production: feature pipelines, model serving, latency, monitoring, retraining and system design, with less weight on experimentation and statistics.
What should I ask in AI engineer interview questions?
Ask how the candidate evaluates LLM outputs, builds test sets, chooses between retrieval and fine-tuning, controls cost and latency, handles hallucinations and monitors quality after launch.
Should a data scientist interview include a take-home assignment?
A short, realistic exercise is useful. Cap it at three to four hours, use the same task and rubric for every candidate, and offer a live alternative for people who cannot spare the time.
How many interview rounds should a data scientist go through?
Usually four to six touchpoints: a recruiter screen, a technical screen, an exercise or live case, and a final round of three or four interviews covering modeling, coding, business sense and behavior.