📌 Problem Statement
In the competitive field of technical interviews, especially for Computer Science
students, the ability to practice with company-specific and topic-relevant
questions is crucial. Current platforms often provide generic questions that lack
personalization or structured reasoning.
The goal of this project is to develop an intelligent system that generates high-
quality, company-specific technical questions using Large Language Models (LLMs).
The system should ensure:
Relevance to the user’s target company and topic,
Correct difficulty level,
Semantic alignment with known industry questions,
Efficient generation using scalable and adaptable architectures.
Design and Architecture
We explored three primary architectures for generating high-quality interview
questions:
1. Vector Database + RAG (Retrieval-Augmented Generation)
This architecture uses vector embeddings and semantic retrieval to ground LLM
outputs.
Step 1: Store Data
All historical questions are embedded into vectors using models like FAISS.
Step 2: User Query
User input is converted into a vector form.
Step 3: Retrieve (RAG)
Top-N semantically similar items are fetched using cosine similarity.
Step 4: LLM Response
The LLM generates new questions using both the original query and the retrieved
items as context.
2. RAG on Fine-Tuned LLM with Knowledge Graph (KG)
This architecture enriches the RAG method using a Knowledge Graph to store and
relate entities like companies, topics, and difficulties.
Step 1: Build Knowledge Graph
Entities such as Company, Topic, Question, and Difficulty are modeled as nodes.
Step 2: Process Input
User input is mapped to graph nodes to query related topics and questions.
Step 3: Retrieve
Context is gathered via the graph, then fed into the LLM for context-aware
generation.
3. Fine-Tuned LLM
This approach trains a smaller LLM on labeled prompt-response pairs to tailor
outputs.
Step 1: Base Model Selection
Lightweight models like GPT-2, Mistral, or Gemma 2B are selected for cost-effective
fine-tuning.
Step 2: Fine-Tuning
Prompt-response datasets are used to train the model (using an 80/20
train/validation split).
Step 3: Evaluation
Generated outputs are evaluated using metrics like BLEU, ROUGE, and manual
relevance checks.
🧪 Proposed Methodology
We propose a hybrid approach combining RAG with a fine-tuned LLM, backed by a
lightweight knowledge graph.
Key Steps:
Data Collection & Embedding
Curate a dataset of company-specific questions.
Embed questions using an embedding model (e.g., Sentence-BERT).
Knowledge Graph Design
Construct a semantic graph linking companies, topics, difficulty levels, and
historical questions.
Context Retrieval (RAG)
Given a user query (e.g., “Amazon, Trees, Medium”), retrieve similar past questions
via the graph and vector DB.
Prompt Enrichment
Combine the user query with the retrieved context into a structured prompt.
LLM Generation
Use a fine-tuned LLM (or LoRA-adapted lightweight model) to generate 1–3 new
questions.
Evaluation
Use Recall@K and manual review to validate output quality.
Incorporate user feedback from CS students for continuous improvement.