Smart Meeting Assistant - 30 Minute Presentation Guide

📋 Table of Contents
1. Introduction & Demo

2. Project Overview & Architecture

3. Technology Stack Deep Dive

4. Frontend Implementation

5. Backend Implementation

6. Key Features & Flow

7. Challenges & Solutions

8. Q&A Preparation

1. Introduction & Demo (3-4 minutes)

Opening Statement

"Good [morning/afternoon] everyone. Today I'm excited to present my Smart Meeting Assistant - an AI-
powered real-time meeting transcription and Q&A platform that transforms how we capture and
interact with meeting content."

Quick Demo Points (Show Live or Recording)

1. Home Page Entry

"Users can enter their name or join anonymously"

"The system automatically handles both cases"

2. Meeting Room Interface

"Real-time video calling with [Link]"

"Live transcription panel on the right"

"Clean, professional UI with gradient backgrounds"

3. Live Transcription

"As participants speak, transcripts appear in real-time"

"Each message shows speaker name, timestamp, and content"

"Auto-scrolls to latest message"

 4. AI Assistant Q&A
"Say 'Hey Assistant' followed by your question"

"The AI bot responds based on meeting context"

"Example: 'Hey Assistant, what are the action items?'"

Hook Statement

"This isn't just another video chat app - it's an intelligent meeting companion that listens, transcribes,
and answers questions based on your conversation context."

2. Project Overview & Architecture (4-5 minutes)

High-Level Architecture

"Let me walk you through the system architecture using this sequence diagram."

Component Overview

Frontend ([Link] + React)

↓
Backend API ([Link] API Routes)
↓
[Link] Services (Video + Chat)
↓
Python Vision Agent (Gemini AI)

Key Components Explanation

1. User Entry Flow

Point to diagram: User → HomePage

"When a user visits the application, they're greeted with a simple name entry form. The system has built-
in intelligence:"

If name is provided → use that name

If blank → automatically set to "Anonymous"

This ensures no user is blocked from joining

2. Authentication & Token Management

Point to diagram: HomePage → TokenAPI

"Once a user enters or skips the name, the system:"

Creates a unique user ID (lowercase, hyphenated)

Sends request to /api/token endpoint

Backend generates a secure Stream token with 24-hour validity

Includes clock skew handling (10-second offset) to prevent authentication issues

Technical Detail:

javascript

const iatTime = now - 10; // Prevents clock skew issues

const token = [Link]({
user_id: userId,
validity_in_seconds: 24 * 60 * 60,
iat: iatTime
} );

3. Stream Client Initialization

Point to diagram: MeetingPage → StreamUser → StreamProvider

"With the token, we initialize two critical clients:"

Video Client: Handles video calls, screen sharing, controls

Chat Client: Manages real-time messaging (currently prepared for future features)

"The StreamProvider component wraps the entire meeting interface, providing context to all child
components."

4. Meeting Room Architecture

Point to diagram: StreamProvider → MeetingRoom

"The MeetingRoom component is the heart of the application:"

Creates or joins an existing call using the call ID

Automatically starts closed captions for transcription

Manages participant lifecycle (join/leave events)

Renders the video layout and transcript panel side-by-side

5. Live Transcription System

Point to diagram: MeetingRoom → TranscriptPanel

"[Link] provides closed caption events that we capture in real-time:"

Listen for call.closed_caption events

Extract speaker name, text, and timestamp

Store in React state array

Auto-scroll to latest entry

Beautiful UI with speaker avatars and timestamps

6. Vision Agent Integration

Point to diagram: VisionAgent

"Here's where it gets really interesting - the Python backend:"

A Vision Agent bot joins the call as a virtual participant

Uses Google's Gemini Realtime API for speech understanding

Listens silently to entire conversation

Maintains full meeting transcript in memory

Activates ONLY when someone says "Hey Assistant"

"The agent has strict behavioral rules:"

python

instructions="""
CRITICAL RULES - FOLLOW EXACTLY:
1. YOU MUST NEVER SPEAK unless someone says "Hey Assistant"
2. DO NOT respond to conversations between users
3. ONLY RESPOND when you explicitly hear "Hey Assistant"
"""

7. Q&A System

Point to diagram: User ↔ VisionAgent

"When 'Hey Assistant' is detected:"

1. Extract the question after "Hey Assistant"

2. Build context from entire meeting transcript

3. Generate response using Gemini AI

 4. Speak answer back into the call

5. Response appears in video as audio

Architecture Benefits

"This architecture provides:"

Scalability: [Link] handles infrastructure

Real-time Performance: WebRTC for video, WebSocket for events

AI Intelligence: Context-aware responses

Modularity: Each component has single responsibility

Security: Token-based authentication

3. Technology Stack Deep Dive (5-6 minutes)

Frontend Stack

[Link] 14 (App Router)

"I chose [Link] for several reasons:"

1. Server-Side Rendering (SSR)

Better SEO (though not critical for this use case)

Faster initial page load

Environment variable security

2. File-based Routing

/app/[Link] → Home page

/app/meeting/[id]/[Link] → Dynamic meeting routes

/app/api/token/[Link] → API endpoint

3. API Routes
Backend functionality without separate server

Token generation endpoint lives in the same codebase

Simplified deployment

Code Example:
 javascript

// Dynamic routing in [Link] App Router

// File: app/meeting/[id]/[Link]
const callId = [Link]; // Automatically extracted from URL

React 18

"React powers our interactive UI with:"

Hooks for State Management

useState for local component state

useEffect for side effects (API calls, event listeners)

useRef for preventing duplicate operations

Component Architecture
HomePage: Name entry form

MeetingPage: Orchestrates authentication

StreamProvider: Context provider for Stream clients

MeetingRoom: Video call interface

TranscriptPanel: Live transcript display

Tailwind CSS

"For styling, I used Tailwind CSS because:"

Utility-first approach speeds up development

No CSS file management

Built-in responsive design

Easy theming with custom colors

Example:

javascript

className="bg-gray-800 rounded-2xl border border-gray-700 shadow-2xl"

[Link] Video SDK

"[Link] is the backbone of our real-time features:"

Key Features Used:

1. Video Calling

javascript

const myCall = [Link]('default', callId);

await [Link]();
await [Link]();

2. Closed Captions / Transcription

javascript

await [Link]({ language: 'en' });

3. Event System

javascript

[Link]('call.closed_caption', handleClosedCaption);
[Link]('call.session_ended', handleSessionEnd);

4. Pre-built UI Components

SpeakerLayout : Video grid layout

CallControls : Mute, video toggle, leave buttons

StreamTheme : Consistent theming

"Why [Link] over alternatives like Twilio or Agora?"

More generous free tier

Better documentation

Built-in closed captions support

React SDK is production-ready

Backend Stack

Python 3.x
"The AI agent is built with Python because:"

Rich AI/ML ecosystem

Vision Agents library support

Async/await for concurrent operations

Easy integration with Google Gemini

Vision Agents Framework

"This is a specialized framework for building AI agents that interact with real-time media:"

Core Concepts:

1. Edge Connections
Connects to [Link] infrastructure
Handles media streaming

Manages real-time events

2. Agent Definition

python

agent = [Link](
edge=[Link](),
agent_user=User(id="meeting-assistant-bot"),
instructions="...", # Behavior rules
llm=[Link](fps=0)
)

3. Event-Driven Architecture

Subscribe to specific events

React to participants joining/leaving

Process speech transcriptions

Handle errors gracefully

Google Gemini Realtime API

"For AI capabilities, I integrated Google's Gemini:"

Why Gemini over GPT or Claude?

Real-time audio processing

 Low latency responses

Multimodal understanding (text + audio)

Cost-effective for this use case

Configuration:

python

llm=[Link](fps=0) # fps=0 means audio-only mode

Event Handling System

"The agent subscribes to multiple event types:"

1. Session Events

python

@[Link]
async def handle_session_started(event: CallSessionStartedEvent):
meeting_data["is_active"] = True
# Initialize chat channel for future features

2. Participant Events

python

@[Link]
async def handle_participant_joined(event):
# Log participant name
# Update internal state

3. Transcription Events

python

@[Link]
async def handle_transcript(event: RealtimeUserSpeechTranscriptionEvent):
# Store in meeting_data["transcript"]
# Check for "Hey Assistant" trigger

4. Error Events
 python

@[Link]
async def handle_errors(event: PluginErrorEvent):
if event.is_fatal:
# Handle fatal errors

Development & Deployment Tools

Environment Configuration

"Both frontend and backend use environment variables:"

Frontend (.[Link]):

bash

NEXT_PUBLIC_STREAM_API_KEY=your_key
NEXT_PUBLIC_CALL_ID=default_meeting_id
STREAM_API_SECRET=your_secret

Backend (.env):

bash

STREAM_API_KEY=your_key
STREAM_API_SECRET=your_secret
CALL_ID=meeting_id
GOOGLE_API_KEY=gemini_key

Version Control

Git for source control

Separate frontend and backend directories

.gitignore for node_modules and .env files

4. Frontend Implementation Walkthrough (8-10 minutes)

Component 1: Home Page ( app/[Link] )

"Let's start with the entry point of our application."

Code Walkthrough

javascript

export default function Home() {

const [username, setUsername] = useState("");
const router = useRouter();

const handleJoin = () => {

const nameToUse = [Link]() === "" ? "Anonymous" : [Link]();
const meetingId = [Link].NEXT_PUBLIC_CALL_ID;
[Link](`/meeting/${meetingId}?name=${encodeURIComponent(nameToUse)}`);
}

Key Points:

1. State Management
Single state variable for username
Simple controlled input component

2. Name Validation
Trim whitespace

Default to "Anonymous" if empty

URL encode for safe navigation

3. Navigation

Use [Link] useRouter hook

Programmatic navigation to meeting page

Pass name as query parameter

UI Design:

Centered card layout

Dark theme with gradient background

Responsive design

Clear call-to-action button

Component 2: Meeting Page ( app/meeting/[id]/[Link] )

"This is the orchestrator component that handles authentication and setup."

Code Walkthrough

Step 1: Extract URL Parameters

javascript

const params = useParams();

const searchParams = useSearchParams();
const callId = [Link]; // From URL: /meeting/[id]
const name = [Link]("name") || "anonymous";

Step 2: Create User Object

javascript

useEffect(() => {
setUser({
id: [Link]().replace(/\s+/g, "-"),
name,
} );
}, [name]);

"Why lowercase and hyphenate?"

[Link] user IDs should be URL-safe

Consistent format prevents issues

Example: "John Doe" → "john-doe"

Step 3: Fetch Authentication Token

javascript
 useEffect(() => {
if (!user) return;

fetch("/api/token", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: [Link]({ userId: [Link] }),
})
.then((res) => [Link]())
.then((data) => {
if ([Link]) setToken([Link]);
else setError("No token returned");
})
.catch((err) => setError([Link]));
}, [user]);

"This is a critical security step:"

Never expose API secrets in frontend

Backend generates token server-side

Token is valid for 24 hours

If token fetch fails, show error UI

Step 4: Render Based on State

javascript

if (error) return <ErrorUI />;

if (!token || !user) return <LoadingUI />;
return (
<StreamProvider user={user} token={token}>
<MeetingRoom callId={callId} onLeave={handleLeave} userId={[Link]} />
</StreamProvider>
);

"Progressive rendering ensures users see appropriate feedback at each stage."

Component 3: Stream Provider ( app/components/[Link] )

"This component initializes the Stream clients and provides them via React Context."

Code Walkthrough
 javascript

export default function StreamProvider({ children, user, token }) {

const { videoClient } = useStreamClients({ apiKey, user, token });

if (!videoClient) {
return <LoadingSpinner />;
}

return (
<StreamVideo client={videoClient}>
{children}
</StreamVideo>
);
}

"Simple but crucial - it wraps children with Stream's context provider."

Component 4: Custom Hook ( app/hooks/[Link] )

"This is where the actual client initialization happens."

Code Walkthrough

Video Client Initialization:

javascript

const tokenProvider = () => [Link](token);

myVideoClient = new StreamVideoClient({
apiKey,
user,
tokenProvider,
} );

Key Points:

Token provider is a function (for future token refresh)

User object contains { id, name }

Client creation is asynchronous

Chat Client Initialization:

javascript
 myChatClient = [Link](apiKey);

if (![Link]) {
await [Link](user, token);
}

"The chat client uses singleton pattern:"

getInstance returns existing instance if available

Only connect if not already connected

Prevents duplicate connections on re-renders

Cleanup Handling:

javascript

return () => {
isMounted = false;
// Don't disconnect on unmount - causes issues with reconnection
};

"Notice we DON'T disconnect on unmount:"

React Strict Mode causes double mounting in dev

Disconnecting causes authentication loops

Clients remain connected throughout session

Component 5: Meeting Room ( app/components/[Link] )

"This is the main meeting interface."

Code Walkthrough

Call Initialization:

javascript
 const myCall = [Link]('default', callId);

await [Link]({
data: {
created_by_id: userId,
},
} );
await [Link]();
await [Link]({ language: 'en' });

Step-by-step:

1. Create call reference with type and ID

2. Get existing call or create new one

3. Join the call (establishes WebRTC connection)

4. Start closed captions for transcription

Event Handling:

javascript

[Link]("call.session_ended", () => {
[Link]("Session ended");
onLeave?.();
} );

Ref Pattern for Preventing Duplicates:

javascript

const joinedRef = useRef(false);

const leavingRef = useRef(false);

useEffect(() => {
if ([Link]) return; // Prevent double join
[Link] = true;
// ... initialization
}, [client, callId, userId]);

"Why use refs instead of state?"

Refs don't trigger re-renders

Prevent duplicate operations

 Essential for React Strict Mode compatibility

Layout Structure:

javascript

<div className="grid grid-cols-1 lg:grid-cols-[1fr_420px] gap-6">

<div>
<SpeakerLayout /> {/* Video grid */}
<CallControls onLeave={handleLeaveClick} />
</div>
<TranscriptPanel /> {/* Live transcripts */}
</div>

"Responsive design:"

Single column on mobile

Two-column on desktop (video + transcript)

Video takes remaining space, transcript fixed 420px

Component 6: Transcript Panel ( app/components/[Link] )

"The live transcription interface."

Code Walkthrough

Event Listening:

javascript
 useEffect(() => {
if (!call) return;

const handleClosedCaption = (event) => {

if (event.closed_caption) {
const newTranscript = {
text: event.closed_caption.text,
speaker: event.closed_caption.user?.name || "Unknown",
timestamp: new Date(event.closed_caption.start_time).toLocaleTimeString(),
};
setTranscripts((prev) => [...prev, newTranscript]);
}
};

[Link]("call.closed_caption", handleClosedCaption);
return () => [Link]("call.closed_caption", handleClosedCaption);
}, [call]);

Key Points:

1. Event Subscription
Listen for call.closed_caption events

Extract text, speaker, timestamp

Add to transcript array

2. Data Structure

javascript

{
text: "Hello everyone",
speaker: "John Doe",
timestamp: "2:45:30 PM"
}

3. Cleanup

Remove event listener on unmount

Prevents memory leaks

Auto-scroll Feature:

javascript
 const transcriptEndRef = useRef(null);

useEffect(() => {
[Link]?.scrollIntoView({ behavior: "smooth" });
}, [transcripts]);

// In JSX:
<div ref={transcriptEndRef} />

"Every time transcripts update, scroll to bottom smoothly."

UI Components:

1. Header
Shows "Live Transcript" title

Message count
Green "Live" indicator with pulse animation

2. Empty State
Shows when no transcripts yet
Microphone icon

Instructional text

3. Transcript Items

Speaker avatar (first letter of name)

Speaker name in blue
Timestamp in gray

Message text
Hover effects for interactivity

Frontend Data Flow Summary

"Let me summarize the complete frontend flow:"

 User enters name
↓
HomePage creates user ID
↓
Navigate to /meeting/[id]?name=...
↓
MeetingPage fetches token from API
↓
useStreamClients initializes Video & Chat clients
↓
StreamProvider wraps MeetingRoom with context
↓
MeetingRoom joins call & starts captions
↓
TranscriptPanel listens for caption events
↓
Display transcripts in real-time

5. Backend Implementation Walkthrough (6-8 minutes)

API Route: Token Generation ( app/api/token/[Link] )

"This is our only backend API endpoint - and it's critical for security."

Code Walkthrough

Step 1: Initialize Stream Server Client

javascript

const serverClient = new StreamClient(apiKey, apiSecret);

"This client has elevated privileges:"

Can create users

Can generate tokens

Should NEVER be exposed to frontend

Step 2: Create/Upsert User

javascript
 const newUser = {
id: userId,
role: "admin",
name: userId,
};
await [Link]([newUser]);

"upsertUsers means:"

Create user if doesn't exist

Update user if already exists

No errors for duplicate creation

Step 3: Generate Token with Clock Skew Fix

javascript

const now = [Link]([Link]() / 1000);

const iatTime = now - 10; // 10 seconds in past

const token = [Link]({

user_id: userId,
validity_in_seconds: 24 * 60 * 60, // 24 hours
iat: iatTime, // Issued At time
} );

"This was a critical bug fix:"

Problem:

JWT tokens have an iat (issued at) claim

If server clock is slightly ahead of Stream's clock

Token validation fails with "Token used before issued"

Solution:

Subtract 10 seconds from current time

Ensures token's iat is always in the past

Accounts for clock drift between servers

Step 4: Return Token

 javascript

return [Link]({ token });

Error Handling:

javascript

try {
// ... token generation
} catch (error) {
[Link]("Token generation error:", error);
return [Link](
{ error: "Failed to generate token", details: [Link] },
{ status: 500 }
);
}

Python Vision Agent ( backend/[Link] )

"Now let's look at the AI agent backend."

Architecture Overview

"The agent runs as a separate Python process that:"

1. Connects to the same Stream call

2. Joins as a virtual participant

3. Listens to all audio

4. Processes speech with Gemini

5. Responds when triggered

Code Walkthrough

Agent Initialization:

python
 agent = [Link](
edge=[Link](),
agent_user=User(
id="meeting-assistant-bot",
name="Meeting Assistant"
),
instructions="""[System prompt]""",
llm=[Link](fps=0),
)

Components:

edge: Connection to Stream infrastructure

agent_user: Bot's identity in the call

instructions: Behavioral rules (system prompt)

llm: AI model for processing/generation

Critical Instructions:

python

instructions="""
CRITICAL RULES - FOLLOW EXACTLY:
1. YOU MUST NEVER SPEAK unless someone says "Hey Assistant"
2. DO NOT respond to conversations between users
3. DO NOT acknowledge anything users say to each other
"""

"These rules are essential:"

Without them, bot would interrupt conversations

Would respond to rhetorical questions

Would create awkward meeting dynamics

"This took significant prompt engineering to get right!"

Event Handlers

1. Session Started Handler:

python
 @[Link]
async def handle_session_started(event: CallSessionStartedEvent):
meeting_data["is_active"] = True
[Link]("🎙️ Meeting started")

# Initialize chat channel for future messaging features

channel = [Link]("messaging", call_id)
await [Link]()
meeting_data["channel"] = channel

"When the meeting starts:"

Set active flag

Initialize chat channel (prepared for future features)

Log the event

2. Participant Joined Handler:

python

@[Link]
async def handle_participant_joined(event: CallSessionParticipantJoinedEvent):
if [Link] == "meeting-assistant-bot":
return # Ignore self
participant_name = [Link]
[Link](f"👤 Participant joined: {participant_name}")

"Track who's in the meeting:"

Ignore the bot itself

Log other participants

Could be used for attendance tracking

3. Transcription Handler (Most Important):

python
 @[Link]
async def handle_transcript(event: RealtimeUserSpeechTranscriptionEvent):
if not [Link] or len([Link]()) == 0:
return

speaker = event.participant_id
transcript_text = [Link]

# Store transcript
meeting_data["transcript"].append({
"speaker": speaker,
"text": transcript_text,
"timestamp": [Link]
})

[Link](f"📝 [{speaker}]: {transcript_text}")

"Every speech segment is captured:"

Filter empty transcripts

Store speaker, text, timestamp

Build up meeting transcript in memory

This becomes context for Q&A

4. Q&A Trigger Detection:

python
 # Inside handle_transcript function
if transcript_text.lower().startswith("hey assistant"):
question = transcript_text[13:].strip() # Remove "hey assistant"

if question:
# Build context from transcript
context = "MEETING TRANSCRIPT:\n\n"
for entry in meeting_data["transcript"]:
context += f"[{entry['speaker']}]: {entry['text']}\n"

prompt = f"""
{context}

USER QUESTION: {question}

Answer based ONLY on the meeting transcript above.

Be concise and helpful.
"""

await agent.simple_response(prompt)

"How Q&A works:"

1. Detect "hey assistant" trigger (case-insensitive)

2. Extract question after trigger phrase

3. Build context from entire meeting transcript

4. Create prompt with context + question

5. Call simple_response to generate answer

6. Agent speaks the answer into the call

Example Flow:
 User: "Hey Assistant, what did John say about the budget?"

Context Built:
[John]: "The budget is $50,000"
[Jane]: "That seems reasonable"
[John]: "We should allocate 30% to marketing"

Prompt to Gemini:
"MEETING TRANSCRIPT:
[John]: The budget is $50,000
[Jane]: That seems reasonable
[John]: We should allocate 30% to marketing

USER QUESTION: what did John say about the budget?

Answer based ONLY on the meeting transcript above."

Gemini Response:
"John mentioned that the budget is $50,000 and suggested allocating 30% to marketing."

[Agent speaks this response in the call]

5. Session Ended Handler:

python

@[Link]
async def handle_session_ended(event: CallSessionEndedEvent):
meeting_data["is_active"] = False
[Link]("🛑 Meeting ended")
[Link](f"📊 Final Stats:")
[Link](f" - Transcript entries: {len(meeting_data['transcript'])}")

"Cleanup and logging when meeting ends."

Agent Lifecycle

Joining the Call:

python
 await agent.create_user() # Register bot user with Stream
call = [Link]("default", call_id)

async with [Link](call):

[Link]("🎙️ MEETING ASSISTANT ACTIVE!")
await [Link]() # Wait until meeting ends

"The async with context manager:"

Automatically handles join/leave

Ensures cleanup on exit

Manages connection lifecycle

Meeting Summary Feature

Post-Meeting Summary:

python

def print_meeting_summary():
print("\n📋 MEETING SUMMARY")
print(f"📝 Transcript ({len(meeting_data['transcript'])} entries):")
for entry in meeting_data['transcript']:
print(f"[{entry['speaker']}]: {entry['text']}")

"When agent exits, prints full meeting transcript to console."

Future Enhancement:

Could send this via email

Store in database

Generate PDF report

Create action items list

Backend Architecture Benefits

"Why separate Python backend instead of all [Link]?"

1. Vision Agents Framework

 Python-only library
Specialized for real-time media AI agents

Rich event system

2. AI/ML Ecosystem

Better Python support for Gemini, GPT, etc.

NumPy, TensorFlow if we add ML features
Audio processing libraries

3. Separation of Concerns
Frontend handles UI/UX

Node backend handles web API

Python backend handles AI processing

4. Independent Scaling

Can deploy agent separately

Multiple agents for large meetings

Doesn't affect frontend performance

6. Key Features & Flow Analysis (4-5 minutes)

Feature 1: Anonymous Join

"Users can join without creating an account:"

Implementation:

javascript

const nameToUse = [Link]() === "" ? "Anonymous" : [Link]();

Benefits:

Zero friction onboarding

Privacy for users who prefer anonymity

Still unique via system-generated ID

Technical Detail:

Anonymous users still get unique IDs

 Multiple anonymous users don't conflict

Transcript shows "Anonymous" as speaker name

Feature 2: Real-Time Transcription

"Powered by [Link]'s closed captions API:"

How it works:

1. Stream processes audio using speech-to-text

2. Generates closed caption events

3. Frontend receives events via WebSocket

4. Displays in transcript panel

5. Backend Vision Agent also receives events

Accuracy:

Uses industry-standard speech recognition

Supports multiple languages (we use English)

Real-time with < 1 second latency

Feature 3: AI Q&A System

"Context-aware question answering:"

Trigger Mechanism:

Wake word: "Hey Assistant"

Natural language questions

Example: "Hey Assistant, summarize the last 5 minutes"

Context Building:

Full meeting transcript in memory

Speaker attribution

Timestamps for temporal queries

AI Model:

Google Gemini Realtime API

Optimized for conversational responses

Audio-native processing

Response Types:

1. Factual: "What did John say about X?"

2. Summarization: "Summarize the discussion"

3. Action Items: "What are the next steps?"

4. Clarification: "Who agreed to do the presentation?"

Feature 4: Professional UI/UX

Design Principles:

1. Dark Theme
Reduces eye strain

Professional appearance

Modern aesthetic

2. Gradient Backgrounds

Visual interest
Depth perception

Brand identity

3. Responsive Layout

Mobile-first approach
Tablet optimization
Desktop maximization

4. Loading States
Spinner animations

Progress feedback
Error boundaries

5. Smooth Transitions
 Fade-in animations
Scroll behavior

Hover effects

Feature 5: Session Management

Join Flow:

1. Enter name (or skip)

2. Generate user ID
3. Fetch auth token
4. Initialize clients
5. Join call
6. Start captions
7. Begin transcription

Leave Flow:

1. Click "Leave" button

2. Stop closed captions
3. Leave call
4. Cleanup event listeners
5. Navigate to home page

Edge Cases Handled:

Duplicate join attempts (ref pattern)

Network disconnections (Stream handles reconnection)

Browser refresh (re-authentication)

Token expiration (24-hour validity)

Complete User Journey

"Let me walk through the entire user experience:"

Minute 0:00 - Entry

User visits homepage

Sees clean, simple form

 Enters name "Sarah" or leaves blank

Minute 0:05 - Authentication

Clicks "Join Meeting"

Sees loading spinner

System generates token in background

Token creation takes ~200ms

Minute 0:07 - Meeting Join

Meeting room loads

Video initializes

Sees own camera feed

Transcript panel ready

Minute 0:10 - Start Speaking

Sarah: "Hello everyone"

Transcript appears: "[Sarah]: Hello everyone"

Timestamp: "2:45:30 PM"

Minute 5:00 - Q&A

Sarah: "Hey Assistant, what have we discussed so far?"

Agent processes request

Generates summary from transcript

Speaks response in 3-4 seconds

Minute 30:00 - Leave

Sarah clicks "Leave"

Call disconnects gracefully

Returns to home page

Backend logs meeting summary

7. Challenges & Solutions (3-4 minutes)

Challenge 1: Token "Used Before Issued" Error

Problem:

Error: Token validation failed

Reason: iat (issued at) claim is in the future

Root Cause:

Clock skew between server and [Link] servers

Even 1-2 second difference causes rejection

JWT validation is strict about timestamps

Solution Attempts:

1. ❌ NTP sync (requires server access)

2. ❌ Longer validity (doesn't solve root cause)
3. ✅ Subtract 10 seconds from iat

Code:

javascript

const now = [Link]([Link]() / 1000);

const iatTime = now - 10; // THE FIX

const token = [Link]({

user_id: userId,
validity_in_seconds: 24 * 60 * 60,
iat: iatTime, // Use past time
} );

Why this works:

Token claims to be issued 10 seconds ago

Even if clocks differ by 5 seconds, still valid

Stream accepts tokens with iat in the past

No downsides (token still valid for full 24 hours)

Learning:
 Always account for distributed system clock drift

JWT validation is strict

Small time offsets can break authentication

Challenge 2: React Strict Mode Double Mounting

Problem:

In development, React mounts components twice

Caused duplicate call joins

Multiple WebRTC connections

Transcript duplicates

Symptoms:

JOIN CALL
LEAVE CALL (cleanup)
JOIN CALL (again)

Solution: Ref Pattern

javascript

const joinedRef = useRef(false);

useEffect(() => {
if ([Link]) return; // Prevent double execution
[Link] = true;

// ... join call logic

}, [client, callId]);

Why refs work:

Refs persist across re-renders

Don't trigger re-renders when changed

Perfect for tracking "did this already" state

Alternative Solutions Tried:

 1. ❌ State variable (causes infinite loop)
2. ❌ Disable Strict Mode (bad practice)
3. ✅ Ref flag

Challenge 3: Vision Agent Interrupting Conversations

Problem:

Initial agent responded to every question

Even rhetorical questions

Even questions directed at other participants

Made meetings awkward

Example of Bad Behavior:

User A: "What do you think about this idea?"

Agent: "I think it's a great approach because..."
User A: "I was asking User B... 😐"

Solution: Strict Behavioral Instructions

python

instructions="""
CRITICAL RULES - FOLLOW EXACTLY:
1. YOU MUST NEVER SPEAK unless someone says "Hey Assistant"
2. DO NOT respond to conversations between users
3. DO NOT acknowledge anything users say to each other
"""

Iterations Required:

Version 1: Agent spoke 80% of the time (too much)

Version 2: Agent asked if it should respond (annoying)

Version 3: Agent explained it was staying quiet (unnecessary)

Version 4: Silent until triggered ✅

Key Learning:
 AI agents need VERY explicit behavioral rules

Default LLM behavior is too conversational

Negative instructions ("DON'T do X") are essential

Test with real conversations, not just scripts

Challenge 4: Chat Client Reconnection Issues

Problem:

On component remount, tried to reconnect chat client

StreamChat complained about duplicate connections

Console errors flooded logs

Sometimes caused authentication loops

Solution: Singleton Pattern

javascript

myChatClient = [Link](apiKey);

if (![Link]) {
await [Link](user, token);
}

How it works:

getInstance reuses existing connection

Only connect if userID is null

Prevents duplicate connection attempts

Cleanup Strategy:

javascript

return () => {
isMounted = false;
// DON'T disconnect on unmount
// Causes issues with reconnection
};
"Counter-intuitive but necessary:"

Usually you clean up in useEffect return

But here, disconnecting causes more problems

Stream clients should live for session duration

Challenge 5: Transcript Auto-Scroll

Problem:

New transcripts appeared but stayed off-screen

Users had to manually scroll

Poor UX for long meetings

Solution: Ref + ScrollIntoView

javascript

const transcriptEndRef = useRef(null);

useEffect(() => {
[Link]?.scrollIntoView({ behavior: "smooth" });
}, [transcripts]);

// In render:
{[Link](...)}
<div ref={transcriptEndRef} />

Why this works:

Invisible div at end of transcript list

Ref points to this div

When transcripts change, scroll to ref

behavior: "smooth" for nice animation

Alternative Approaches:

1. ❌ [Link] (scrolls entire page)

2. ❌ [Link] (not smooth)
3. ✅ scrollIntoView with smooth
Challenge 6: Environment Variable Management

Problem:

API keys needed in multiple places

Different keys for frontend vs backend

Risk of exposing secrets

Deployment configuration complexity

Solution: Structured .env Files

Frontend (.[Link]):

bash

# Public variables (exposed to browser)

NEXT_PUBLIC_STREAM_API_KEY=xyz
NEXT_PUBLIC_CALL_ID=meeting123

# Private variables (server-only)

STREAM_API_SECRET=secret123

Backend (.env):

bash

STREAM_API_KEY=xyz
STREAM_API_SECRET=secret123
CALL_ID=meeting123
GOOGLE_API_KEY=gemini_key

Best Practices:

1. Never commit .env files to Git

2. Use .[Link] with dummy values

3. Prefix public vars with NEXT_PUBLIC_

4. Keep secrets server-side only

5. Different keys for dev/staging/prod

Technical Debt & Future Improvements

Current Limitations:

1. Single Meeting ID
Currently hardcoded in environment

Should support dynamic meeting creation

2. No Persistence
Transcripts lost when agent stops

Should save to database

3. No User Authentication
Anyone with link can join

Should add password protection

4. Limited Chat Features

Chat client initialized but not used

Could add text messaging

5. No Recording

Can't replay meetings

Should add cloud recording

Planned Enhancements:

1. Meeting Management
Create meeting via UI
Schedule future meetings

Recurring meetings

2. Transcript Export

Download as PDF
Email summary

Integration with Google Docs

3. Advanced AI Features
Action item extraction

Sentiment analysis
Speaker identification
 Multi-language support

4. Analytics Dashboard

Meeting duration
Participant engagement

Speaking time distribution

Keyword extraction

8. Q&A Preparation

Technical Questions

Q: Why [Link] instead of building WebRTC yourself?

A: "[Link] provides production-ready infrastructure that would take months to build:

TURN/STUN servers for NAT traversal

Scalable media servers

Built-in transcription

Mobile SDKs

99.99% uptime SLA

Building this from scratch would require:

DevOps expertise

Significant infrastructure costs

Ongoing maintenance

Security audits

For a project like this, [Link]'s free tier is perfect and lets me focus on the AI features rather than low-level
video infrastructure."

Q: How does the transcription work technically?

A: "[Link] uses their own speech-to-text engine that:

1. Receives audio from participants via WebRTC

2. Processes in real-time on their servers

 3. Generates closed caption events

4. Sends via WebSocket to all connected clients

The events look like:

javascript

{
type: 'call.closed_caption',
closed_caption: {
text: 'Hello everyone',
user: { id: 'user123', name: 'John' },
start_time: '2024-01-11T12:30:45Z'
}
}

Both the frontend TranscriptPanel and backend Vision Agent subscribe to these events, so they both have the
same transcript data."

Q: What happens if the Python agent crashes?

A: "Great question. Currently:

Frontend continues to work normally

Users can still have video calls

Transcription still works in UI

Only Q&A feature breaks

Improvements needed:

1. Automatic agent restart

2. Health check endpoint

3. Fallback error message in UI

4. Alert to administrator

Production solution:

Run agent in container (Docker/Kubernetes)

Restart policy: always

Monitoring with Prometheus

 Auto-scaling for multiple meetings"

Q: How do you handle multiple simultaneous meetings?

A: "Currently, the system supports one meeting (hardcoded call ID). For multiple meetings:

Architecture changes needed:

1. Meeting Management Service

Create meeting API endpoint

Generate unique call IDs

Store meeting metadata

2. Agent Pool
Multiple Python agents running

Each handles one meeting

Dynamic agent assignment

3. Frontend Changes
Meeting creation UI
Meeting list/dashboard

Join via unique link

Code example for dynamic meetings:

javascript

// Create meeting
POST /api/meetings
Response: { meetingId: 'abc123', joinUrl: '/meeting/abc123' }

// Python agent
call_id = [Link]('CALL_ID') # From agent startup args

Scalability:

1 agent per meeting

Agents can run on different machines

Load balancer distributes agents

Horizontal scaling as needed"

Q: Why [Link] instead of plain React or Vite?

A: "[Link] provides several advantages:

1. API Routes: Backend endpoint in same codebase

2. File-based Routing: No react-router configuration

3. Server-Side Rendering: Better initial load (though not critical here)

4. Environment Variables: Built-in support with NEXT_PUBLIC_ prefix

5. Production Ready: Optimized builds out of the box

6. Deployment: Vercel one-click deploy

For this project specifically:

API route for token generation is essential

Dynamic routing for /meeting/[id]

Environment variable management

Professional developer experience

Alternatives considered:

Vite + Express: More setup, separate server

Create React App: Deprecated, no backend

Plain React: Would need separate backend anyway"

Q: How secure is the authentication?

A: "The authentication follows JWT best practices:

Security measures:

1. API Secret Never Exposed

Only exists in backend

Not in frontend code

Not in browser

2. Short-Lived Tokens
 24-hour validity
Can reduce for higher security

Automatic expiration

3. User-Specific Tokens

Each user gets unique token

Token tied to user ID
Can't be reused by others

4. HTTPS in Production
Encrypted transmission

No token interception

Potential improvements:

1. Token Refresh: Issue new tokens before expiry

2. Rate Limiting: Prevent token generation spam

3. IP Whitelisting: Restrict token generation to known IPs

4. Meeting Passwords: Add extra auth layer

5. OAuth Integration: Google/GitHub login

Current risk assessment:

Low risk for demo/internal use

Medium risk for public deployment

High risk for sensitive meetings

For production:

Add meeting passwords

Implement user authentication (OAuth)

Add audit logging

Regular security audits"

Q: What's the latency like?

A: "Latency breakdown:
Video Transmission:

WebRTC: 100-300ms (excellent for real-time)

No perceptible delay in conversation

Transcription:

Speech-to-text processing: 500-1000ms

Acceptable for transcript display

Not suitable for real-time subtitles

AI Response:

Gemini processing: 2-4 seconds

Acceptable for Q&A use case

Much faster than GPT-4 (5-8 seconds)

Total Q&A Latency:

User says "Hey Assistant" → Transcription (1s)

→ Agent receives event (0.1s)
→ Build context (0.1s)
→ Gemini processes (2-3s)
→ Agent speaks response (0.5s)
= ~4-5 seconds total

Optimization opportunities:

1. Streaming Responses: Start speaking before full answer generated

2. Context Caching: Pre-build context between questions

3. Faster Model: Use Gemini Flash instead of Pro

4. Edge Deployment: Run agent closer to users"

Business/Product Questions

Q: Who is the target audience?

A: "Primary target audiences:

1. Remote Teams
 Daily standups
Planning meetings

Retrospectives

2. Educational Institutions

Online classes
Student group projects
Office hours

3. Healthcare
Telemedicine consultations

Medical transcription
Patient notes

4. Legal

Client consultations
Depositions

Case discussions

Key value propositions:

Automatic transcription saves manual note-taking

AI assistant can answer questions without interrupting meeting

Search through past meetings

Action item extraction"

Q: How would you monetize this?

A: "Potential business models:

Freemium Tier:

5 meetings/month

30-minute max duration

Basic transcription

24-hour transcript retention

Pro Tier ($10/month):

 Unlimited meetings

No duration limit

Advanced AI features

1-year transcript retention

Export to PDF/Docs

Business Tier ($50/month):

All Pro features

Custom AI training on company data

Integrations (Slack, Teams, Google Workspace)

Admin dashboard

SSO authentication

Priority support

Enterprise Tier (Custom):

Self-hosted option

Unlimited users

Custom features

SLA guarantees

Dedicated support

Additional Revenue Streams:

API access for developers

White-label solution

Per-minute transcription pricing

Custom AI model training"

Q: What are the main competitors?

A: "Competitive landscape:

Direct Competitors:
 1. [Link]
Pros: Established, accurate transcription

Cons: No video, expensive, limited AI

2. [Link]

Pros: Good integrations, CRM features

Cons: Complex UI, privacy concerns

3. Grain

Pros: Video clips, good UX

Cons: Focused on sales, expensive

Our Differentiators:

1. Real-time Q&A: Unique AI assistant feature

2. Open Source Potential: Could offer self-hosted version

3. Simpler UX: Just join and talk

4. Custom AI: Can train on company data

5. Price: More affordable for small teams

Indirect Competitors:

Zoom (basic transcription)

Google Meet (auto-captions)

Microsoft Teams (transcription)

Competitive Advantages:

More powerful AI features

Better user experience

Lower cost

Privacy-focused (self-hosted option)"

Implementation Questions

Q: How long did this take to build?

A: "Development timeline:
Week 1: Planning & Setup (10 hours)

Architecture design

Technology research

[Link] account setup

Boilerplate code

Week 2: Frontend Core (15 hours)

[Link] setup

Homepage & routing

Stream integration

Basic video calling

Week 3: Transcription (12 hours)

Transcript panel UI

Event handling

Auto-scroll feature

UI polish

Week 4: Backend Agent (20 hours)

Python environment setup

Vision Agents learning curve

Gemini integration

Q&A system

Behavioral prompt engineering

Week 5: Bug Fixes & Polish (10 hours)

Token clock skew fix

React Strict Mode issues

UI improvements

Documentation

Total: ~70 hours

Time breakdown:

Frontend: 40%

Backend: 30%

Bug fixing: 20%

Documentation: 10%

Longest challenges:

Token authentication issues: 8 hours

Agent behavioral tuning: 6 hours

React remounting bugs: 4 hours"

Q: What would you do differently if starting over?

A: "Key lessons learned:

Technical Decisions:

1. Start with simpler AI

Should've used basic chat API first

Vision Agents learning curve was steep

Could've validated concept faster

2. Add database from day one

Transcript persistence is essential
Should've used Prisma + PostgreSQL

Would enable more features

3. Better error handling

Need more graceful degradation

Should catch all async errors
Better user error messages

4. Automated testing
Integration tests for call flow

Mock [Link] in tests

E2E tests with Playwright
Process Improvements:

1. Design phase

Should've created Figma mockups

User flow diagrams first

Would've saved refactoring time

2. Incremental deployment
Should've deployed MVP earlier

Get user feedback sooner

Iterate based on real usage

3. Documentation
Should've documented as I built
Architecture decisions

API contracts

But overall:

Project was successful

Learned many new technologies

Built something functional and impressive

Good foundation for future features"

Q: How would you scale this to 100,000 users?

A: "Scaling strategy:

Infrastructure:

1. Frontend
CDN for static assets (Cloudflare)

Edge caching for API routes

Deploy to Vercel (auto-scaling)

2. Backend API
Separate [Link] server
Load balancer (AWS ALB)
 Auto-scaling groups
Rate limiting (Redis)

3. Python Agents
Container orchestration (Kubernetes)
Agent pool management

Horizontal scaling
Health checks & auto-restart

4. Database
PostgreSQL with read replicas
Meeting metadata

User accounts
Transcript storage

5. Message Queue
RabbitMQ or SQS
Decouple agent assignment

Handle burst traffic

Cost Estimation:

100,000 users
Average 2 meetings/month per user
Average 30 minutes per meeting

= 200,000 meetings/month
= 100,000 hours/month

[Link]: ~$0.005/minute = $30,000/month

Gemini API: ~$0.10/1K tokens = $10,000/month
Infrastructure: $5,000/month
Total: ~$45,000/month

Revenue needed: $50,000+/month

Price point: $10/month
Conversion rate: 5%
= Need 100,000 * 5% = 5,000 paying users

Architecture diagram:
 [Cloudflare CDN]
↓
[Load Balancer]
↓
┌──────────────────┴──────────────────┐
↓ ↓
[[Link] Cluster] [API Server Cluster]
↓ ↓
└──────────────────┬──────────────────┘
↓
[Message Queue]
↓
[Agent Pool Manager]
↓
┌──────────────────┼──────────────────┐
↓ ↓ ↓
[Agent Pod 1] [Agent Pod 2] [Agent Pod N]
└──────────────────┴───────────────────┘
↓
[PostgreSQL]
```"

---

## Conclusion & Future Vision (2 minutes)

**"To summarize:"**

### What We Built

- **Full-stack video calling platform** with AI assistance
- **Real-time transcription** with professional UI
- **Context-aware Q&A system** using Gemini AI
- **Production-ready architecture** with scalability in mind

### Technical Achievements

- Integrated 4 major technologies ([Link], [Link], Vision Agents, Gemini)
- Solved complex real-time synchronization challenges
- Built responsive, beautiful UI
- Implemented secure authentication

### Learning Outcomes

- WebRTC and real-time communication
- AI agent behavioral design
- Async event-driven architecture
- Production deployment considerations
### Future Vision
**"This project is just the beginning. I envision it evolving into:"**

1. **Meeting Intelligence Platform**

- Automatic action item extraction
- Sentiment analysis
- Speaking time analytics
- Meeting effectiveness scores

2. **Team Productivity Suite**

- Calendar integration
- Slack/Teams notifications
- CRM integration
- Custom AI training on company data

3. **Enterprise Solution**
- Self-hosted option for security
- SSO integration
- Compliance features (HIPAA, GDPR)
- Advanced admin controls

4. **AI-Powered Insights**
- Meeting summary emails
- Trend analysis across meetings
- Predictive scheduling
- Participant engagement tracking

**"Thank you for your time. I'm happy to answer any questions or dive deeper into any technical aspect you'd like to
explore further!"**

---

## Appendix: Quick Reference

### Key Technologies

- **Frontend**: [Link] 14, React 18, Tailwind CSS
- **Backend**: [Link] (API), Python 3.x (Agent)
- **Video**: [Link] Video SDK
- **AI**: Google Gemini Realtime API
- **Framework**: Vision Agents

### Repository Structure

```
project/
├── app/
│ ├── api/
│ │ └── token/
 │ │ └── [Link] (Token generation API)
│ ├── components/
│ │ ├── [Link] (Main meeting interface)
│ │ ├── [Link] (Stream context provider)
│ │ └── [Link] (Live transcript panel)
│ ├── hooks/
│ │ └── [Link] (Client initialization)
│ ├── meeting/
│ │ └── [id]/
│ │ └── [Link] (Dynamic meeting page)
│ └── [Link] (Home page)
├── backend/
│ ├── [Link] (Vision Agent)
│ ├── .env (Backend config)
│ └── [Link] (Python dependencies)
└── .[Link] (Frontend config)

Environment Variables Reference

bash

# Frontend
NEXT_PUBLIC_STREAM_API_KEY=your_key
NEXT_PUBLIC_CALL_ID=meeting_id
STREAM_API_SECRET=your_secret

# Backend
STREAM_API_KEY=your_key
STREAM_API_SECRET=your_secret
CALL_ID=meeting_id
GOOGLE_API_KEY=gemini_key

Key API Endpoints

POST /api/token - Generate Stream authentication token

Useful Commands

bash
 # Frontend
npm run dev # Start development server
npm run build # Production build
npm start # Start production server

# Backend
python [Link] # Start Vision Agent
pip install -r [Link] # Install dependencies

Important Links

[Link] Docs: [Link]

Vision Agents: [Link]

Gemini API: [Link]

END OF PRESENTATION GUIDE

Total estimated presentation time: 30-35 minutes Adjust pacing based on audience engagement and questions