Instagram LinkedIn Topmate ZohoMail
Interview preparation for Generative AI
Top 21: GenAI System Design
Interview Questions Answer These and Walk Into Your
Interview with 90% Confidence
A For AI Academy | Data & AI/ML Expert | Generative AI | Interview Expert | Udemy
Instagram LinkedIn Topmate ZohoMail
Interview preparation for Generative AI
01. You are building a ChatGPT-like product expecting 1M concurrent
users. Your single GPU server handles 500 req/s but starts timing out
at peak. What is the correct architectural fix?
02. You run a RAG pipeline for a legal tech startup (like Casetext).
Users report answers citing cases that were overruled 6 months ago.
What is the root cause and fix?
03. Anthropic's Claude API costs $15/M output tokens. Your B2B
SaaS uses Claude for every user question, and your token bill is
$80K/month. 60% of questions are repetitive (pricing FAQs, how-to
guides). How do you cut costs by 50%+?
04. You deploy GPT-4 for a customer support bot at a fintech company
(like Stripe). After 3 months, customers report the bot giving wrong
refund policies. Your model hasn't changed. What is happening and
how do you detect it?
05. You are the ML platform lead at a company like Notion AI. You
need to roll out a new Claude 3.5 Sonnet integration replacing GPT-4,
but you cannot risk a quality regression for your 4M users. What is
the safest deployment strategy?
06. You are building a multi-agent coding assistant (like Cursor or
Devin). An agent loop calls tools (file read, bash exec, web search)
and after 10 steps starts looping infinitely, burning $50 in tokens
before anyone notices. How do you prevent this?
07. Your RAG-powered enterprise search (like Glean) serves 500
Fortune 500 companies. Legal requires that Company A's documents
are never retrievable by Company B. Your single shared Pinecone
index is causing occasional cross-tenant data leaks. What is the
correct fix?
08. You run an LLM API platform (like Together AI or Fireworks AI). A
customer's poorly written agent sends 10,000 requests in 60 seconds
with no retry backoff, crashing your inference cluster. How do you
build a robust rate limiting system that is fair and prevents this?
A For AI Academy | Data & AI/ML Expert | Generative AI | Interview Expert | Udemy
Instagram LinkedIn Topmate ZohoMail
Interview preparation for Generative AI
09. You are the infra lead for a consumer GenAI app (like Perplexity).
Your LLM responses take 8–12 seconds to generate but users expect
instant feedback. Your current setup sends the full response only
after generation is complete, causing high bounce rates. How do you
fix this?
10. You fine-tuned Llama 3 on your company's internal support data.
After 2 months, users say answers feel generic and the model seems
to have forgotten company-specific terminology. You cannot afford to
retrain from scratch monthly. What is the right fine-tuning
architecture?
11. You are building a code review assistant (like GitHub Copilot) that
ingests entire repositories (200K+ tokens). Users report the assistant
missing context from files loaded early in the prompt. What is the
problem and the fix?
12. Your company runs a medical information chatbot. After launch,
regulators flag that the model occasionally outputs dangerous drug
dosage information. You need real-time output moderation without
adding more than 50ms latency. How do you build this?
13. You are scaling a document Q&A product (like Notion AI or Guru)
from 10K to 10M users. Your Postgres database storing embeddings
is falling over at 5M vectors. What is the right migration path for your
vector storage and retrieval infrastructure?
14. You are the ML lead at a startup deploying LLMs to 50 enterprise
clients. Each client wants answers based only on their own private
data, but all share the same GPU infrastructure. Your current setup
has occasional cross-client data leaks in responses. What
architectural change eliminates this risk?
15. You are building a GenAI product on top of OpenAI's API. OpenAI
has a major outage and your product is completely down. How do
you architect your LLM gateway to guarantee 99.9% uptime across
provider failures?
A For AI Academy | Data & AI/ML Expert | Generative AI | Interview Expert | Udemy
Instagram LinkedIn Topmate ZohoMail
Interview preparation for Generative AI
16. You are building a GenAI feature for a CRM (like Salesforce
Einstein) with 50M records. Embedding all 50M records upfront is too
slow and too expensive. How do you architect selective, cost-efficient
indexing?
17. You are launching a GenAI feature that generates personalized
marketing emails for 10M users (like Klaviyo AI). Each email requires
one LLM call. Calling the LLM one-by-one would take 28 hours. How
do you complete all 10M generations within a 2-hour batch window?
18. You built a customer-facing AI chatbot for an e-commerce
company (like Shopify). Users are jailbreaking the assistant — getting
it to roleplay, reveal system prompt instructions, and output
competitor prices. How do you harden the system against prompt
injection and jailbreaks in production?
19. You are the founding ML engineer at a startup. You need to
choose between fine-tuning Llama 3 vs. using RAG with GPT-4 +
Pinecone. Your use case is answering questions from a 10,000-page
internal knowledge base that updates weekly. Which approach is
correct and why?
20. You are building a GenAI pipeline where GPT-4 calls three
external tools: a SQL database, a web scraper, and a Slack API.
During testing, the SQL tool returns malformed JSON and the entire
pipeline crashes with a 500 error. How do you build a resilient
tool-calling architecture that handles partial failures gracefully?
21. You are running an LLM application in production and your p99
latency suddenly spikes from 2s to 18s. Your model and infrastructure
have not changed. How do you systematically diagnose and fix the
latency regression?
A For AI Academy | Data & AI/ML Expert | Generative AI | Interview Expert | Udemy