Title: Dialog Generation using Deep Learning
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tokenizer = AutoTokenizer.from_pretrained("microsoft/DialoGPT-small")
model = AutoModelForCausalLM.from_pretrained("microsoft/DialoGPT-small")
print("🤖 Chatbot: Hello! Type 'bye' to end the chat.\n")
chat_history_ids = None
for step in range(5): # limit to 5 interactions for demo
user_input = input("You: ")
if user_input.lower() == "bye":
print("🤖 Chatbot: Goodbye! 👋")
break
new_input_ids = [Link](user_input + tokenizer.eos_token, return_tensors='pt')
bot_input_ids = [Link]([chat_history_ids, new_input_ids], dim=-1) if chat_history_ids is
not None else new_input_ids
chat_history_ids = [Link](
bot_input_ids,
max_length=1000,
pad_token_id=tokenizer.eos_token_id,
temperature=0.7,
top_p=0.9,
do_sample=True
)
bot_output = [Link](chat_history_ids[:, bot_input_ids.shape[-1]:][0],
skip_special_tokens=True)
print(f"🤖 Chatbot: {bot_output}\n")
OUTPUT:
The program creates a simple chatbot using deep learning — specifically, a pre-trained
DialoGPT model from Microsoft.
It generates human-like responses to user input.
🧠 Concepts Behind the Code
DialoGPT is a transformer-based language model (similar to GPT-2), trained on
millions of conversations from Reddit.
It can generate replies word-by-word to continue a conversation.
We use Hugging Face’s transformers library, which makes it easy to use pre-
trained models.
🔍 Code Explanation (Line by Line)
1️⃣ Importing Libraries
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
AutoTokenizer: Converts text → numbers (tokens) that the model understands.
AutoModelForCausalLM: Loads a Causal Language Model (used for text generation
tasks).
torch: PyTorch library used for tensor operations and model inference.
2️⃣ Loading Pre-trained Model and Tokenizer
tokenizer = AutoTokenizer.from_pretrained("microsoft/DialoGPT-small")
model = AutoModelForCausalLM.from_pretrained("microsoft/DialoGPT-small")
Loads a small version of DialoGPT.
from_pretrained() downloads the model and tokenizer from Hugging Face’s model
hub.
You can replace "small" with "medium" or "large" for more powerful models.
3️⃣ Display Welcome Message
print("🤖 Chatbot: Hello! Type 'bye' to end the chat.\n")
Just a friendly welcome line for the user.
4️⃣ Initialize Conversation History
chat_history_ids = None
This variable stores previous messages between user and bot.
The model uses it to maintain context — so it can remember what was said earlier.
5️⃣ Main Chat Loop
for step in range(5): # limit to 5 interactions for demo
Limits the chat to 5 exchanges (you can increase this number).
Each loop represents one turn in the conversation.
6️⃣ Get User Input
user_input = input("You: ")
Takes the user’s text input from the console.
7️⃣ Exit Condition
if user_input.lower() == "bye":
print("🤖 Chatbot: Goodbye! 👋")
break
If the user types ‘bye’, the loop stops and the program ends.
8️⃣ Encode User Input
new_input_ids = [Link](user_input + tokenizer.eos_token,
return_tensors='pt')
Converts the user’s sentence into tokens (numerical format).
Adds a special end-of-sentence token (eos_token) to mark message end.
return_tensors='pt' → returns a PyTorch tensor.
9️⃣ Combine with Chat History
bot_input_ids = [Link]([chat_history_ids, new_input_ids], dim=-1) if
chat_history_ids is not None else new_input_ids
If this is not the first message, combine new input with previous chat history.
This helps the model remember the entire conversation.
🔟 Generate Response
chat_history_ids = [Link](
bot_input_ids,
max_length=1000,
pad_token_id=tokenizer.eos_token_id,
temperature=0.7,
top_p=0.9,
do_sample=True
)
Let’s break this down:
[Link](...) → tells the model to produce text (a reply).
max_length=1000 → maximum total tokens allowed (longer context).
pad_token_id → ensures proper padding with end-of-sentence token.
temperature=0.7 → controls creativity (lower = more focused, higher = more
random).
top_p=0.9 → nucleus sampling, keeps the most likely 90% of words.
do_sample=True → enables random sampling (so answers vary each time).
The model’s output (chat_history_ids) now contains the conversation so far + new bot
response.
11️⃣ Decode and Display Response
bot_output = [Link](chat_history_ids[:, bot_input_ids.shape[-1]:]
[0], skip_special_tokens=True)
print(f"🤖 Chatbot: {bot_output}\n")
Converts token IDs → human-readable text.
skip_special_tokens=True removes tokens like <eos>.
Only prints the new part of the model’s output (not the whole chat history).
💬 Example Interaction
🤖 Chatbot: Hello! Type 'bye' to end the chat.
You: Hi there!
🤖 Chatbot: Hello! How are you doing?
You: I am fine.
🤖 Chatbot: That’s great to hear!
You: bye
🤖 Chatbot: Goodbye! 👋