AI engineer interview questions
AI engineer interviews at the top AI labs ask AI system design (LLM serving, retrieval, agents, evals) and ML design, on top of coding. Here are 26 AI system design questions and 31 ML design questions asked at Anthropic, OpenAI, Nvidia, Databricks, Perplexity and Amazon and more, most asked first, each dated by when it was last asked.
AI system design 26
All 26 →LLM serving, retrieval, agents, evals and the infrastructure behind them.
- Design a video generation service on a scarce GPU pool · OpenAISep 2026 · very often
- Batched LLM inference API over GPUs · AnthropicSep 2026 · very often
- Design a distributed AI model downloader · AnthropicAug 2026 · often
- Design an AI prompt playground · Anthropic, OpenAIAug 2026 · often
- Design a GPU scheduling platform · OpenAI, Anthropic, DatabricksSep 2026 · rarely
- Rule-driven file review agent · HarveySep 2026 · rarely
- Design durable execution for long-running agent queries · PerplexitySep 2026 · rarely
- Design an AI chatbot app · OpenAI, Microsoft AI, MicrosoftJul 2026 · rarely
- Design a streaming AI feature for a large audience · OpenAIAug 2026 · once
- Design a text-to-SQL chatbot with access control · NvidiaJul 2026 · once
- Design a RAG system over company docs · Anthropic, Databricks, CohereMar 2026 · once
- Design an agent workflow that produces a feature-length video · xAIJan 2026 · once
- KV cache eviction for million-token contexts · Hugging FaceOnce
- Design an LLM inference serving platform · Anthropic, OpenAI, NvidiaOnce
- Design ChatGPT streaming with resumable responses and long conversations · OpenAI
ML design 31
All 31 →Models, data, training and evaluation for a product problem.
- ML fundamentals deep dive for ML roles · Google, Amazon, MetaOct 2026 · often
- Transformer and attention fundamentals · AmazonSep 2026 · often
- Design a harmful content detection system · Databricks, MetaAug 2026 · sometimes
- GPU and inference performance fundamentals · NvidiaAug 2026 · sometimes
- Matmul performance modeling on a fixed GPU · AnthropicSep 2026 · rarely
- Design a personalized search ranking system · Airbnb, UberMay 2026 · rarely
- ML configuration system with inheritance and validation · AnthropicApr 2026 · rarely
- Design a feed recommendation system · Perplexity, FigmaApr 2026 · rarely
- ML design for retrieval: embeddings, hybrid search and ranking · OpenAIMar 2026 · rarely
- Debug a GRPO training script and defend the RL math · AnthropicMar 2026 · rarely
- QKV attention programming with einsum fluency · AnthropicFeb 2026 · rarely
- Self-contained ML take-home with live review · AnthropicFeb 2026 · rarely
- Prompt design with evaluation scaffolding · AnthropicJan 2026 · rarely
- Deep learning fundamentals: optimization, drift and calibration · NvidiaJan 2026 · rarely
- Research engineering training discussion · HarveySep 2026 · once
About AI engineer interviews
What do AI engineer interviews ask?
Besides coding, AI engineer loops ask AI system design (serve a model, build retrieval or an agent, design an evaluation harness) and ML design (frame the problem, the data, the model and how you evaluate it).
Which companies ask AI system design questions?
Most often: Anthropic, OpenAI, Nvidia, Databricks, Perplexity and Amazon.
What are the most asked AI engineer interview questions?
Design a video generation service on a scarce GPU pool, Batched LLM inference API over GPUs and Design a distributed AI model downloader.