📋 Complete Overview: Adding Memory to Your LLM Chat System

1. The Original Problem

Your original chat system had no memory. Each time a user sent a message, the LLM
treated it as a completely new conversation. It couldn't remember:

What the user asked before

What responses it gave
The context of the ongoing conversation

Example of the problem:

User: "Give me logs from /var/log/[Link]"
AI: [Provides logs]

User: "What was my previous question?"

AI: "This conversation just started" ❌ (Wrong!)

2. The Solution Architecture

I implemented a conversation memory system that:

Stores all previous messages (user questions + AI responses)

Retrieves this history when processing new queries
Includes the history in the prompt sent to the LLM
Maintains the history throughout the entire processing chain

3. Key Components Added

A. Memory Storage (Global Variable)
pythonconversation_history: List = []
MAX_HISTORY_LENGTH = 10
What it does:

conversation_history: A list that stores all conversation messages

MAX_HISTORY_LENGTH: Limits memory to last 10 exchanges (20 messages total: 10 user
+ 10 AI)

Why a list?

Simple to append new messages

Easy to retrieve recent history
Can be cleared when needed

Message Format:
We use LangChain's message objects:

HumanMessage(content="user's question") - For user messages

AIMessage(content="AI's response") - For AI responses

B. Helper Function: format_history_for_template()

pythondef format_history_for_template(history: List) -> str:
if not history:
return "This is the start of the conversation.\n\n"

history_text = "Previous conversation:\n"

for msg in history[-10:]:
role = "User" if isinstance(msg, HumanMessage) else "Assistant"
history_text += f"{role}: {[Link]}\n"
history_text += "\n"
 return history_text
What it does:

Converts message objects into readable text

Formats it as "User: ... / Assistant: ..."
Takes only the last 10 messages to avoid overwhelming the LLM

Example output:
Previous conversation:
User: Give me logs from /var/log/[Link]
Assistant: I've searched for logs in /var/log/[Link]...
User: What was my previous question?

4. Modified Functions - Step by Step

Function 1: promptGenerator() - Entry Point
Original code:
pythonasync def promptGenerator(x: Dict[str, any]):
initialTemplate = formTemplateForModel()
prompt = [Link]({"user_input": x["user_input"]})
return {"prompt": prompt, "human": x["user_input"]}
Modified code:
pythonasync def promptGenerator(x: Dict[str, any]):
initialTemplate = formTemplateForModel()

# 1. Format history as text

history_text = ""
if conversation_history:
history_text = "Previous conversation:\n"
for msg in conversation_history[-10:]:
role = "User" if isinstance(msg, HumanMessage) else "Assistant"
history_text += f"{role}: {[Link]}\n"
history_text += "\n"
else:
history_text = "This is the start of the conversation.\n\n"

# 2. Debug output
print(f"\n📝 Current conversation history length: {len(conversation_history)}")

# 3. Create prompt with history

prompt_input = {
"user_input": x["user_input"],
"history_text": history_text # ← New: Include history
}
prompt = [Link](prompt_input)

# 4. Store current user message AFTER creating prompt

conversation_history.append(HumanMessage(content=x["user_input"]))

return {"prompt": prompt, "human": x["user_input"], "history":

conversation_history.copy()}
Key changes:

Format history as text before creating the prompt

Pass history to the template via history_text
Store user message in memory AFTER generating prompt (to avoid duplication)
Return history to pass it down the chain

Why store AFTER generating prompt?

If we store before, the current message would appear twice in the prompt
We want history to contain only PREVIOUS messages, not the current one

Function 2: processUserQuery() - LLM Invocation

Original code:
pythonasync def processUserQuery(x: Dict[str, any]):
results = await [Link](x["prompt"])
return {"results": results, "userQuery": x["human"]}
Modified code:
pythonasync def processUserQuery(x: Dict[str, any]):
prompt_with_history = x["prompt"]

print(f"\n🤖 Sending to LLM with {len([Link]('history', []))} history messages")

results = await [Link](prompt_with_history)

return {"results": results, "userQuery": x["human"], "history":

[Link]("history", [])}
Key changes:

Debug output to show how many history messages are being sent
Pass history forward in the return statement

Why pass history forward?

The next functions in the chain need access to history

This maintains history throughout the entire pipeline

Function 3: tool_call() - Tool Execution

Modified to include history in all return statements:
pythonasync def tool_call(x: Dict[str, Any]):
results = x["results"]
user_query = x["userQuery"]
history = [Link]("history", []) # ← Get history from input

tool_calls = results.additional_kwargs.get("tool_calls", [])

context = [Link]

if not tool_calls:
return {
"toolOutput": None,
"userQuery": user_query,
"context": context,
"history": history # ← Pass history forward
}

# ... tool execution code ...

return {
"toolOutput": tool_output,
"userQuery": user_query,
"context": context,
"history": history # ← Pass history forward
}
Key change:

Preserve history in all return paths so it reaches the next template

Function 4: analyzeQuery() - Final Response
Original code:
pythonasync def analyzeQuery(x: Dict[str, any]):
results = await [Link](x)
print([Link])
Modified code:
pythonasync def analyzeQuery(x: Dict[str, any]):
results = await [Link](x)
print([Link])

# Store AI response in conversation history

conversation_history.append(AIMessage(content=[Link]))

# Trim history if too long

if len(conversation_history) > MAX_HISTORY_LENGTH * 2:
conversation_history[:] = conversation_history[-(MAX_HISTORY_LENGTH * 2):]

return results
Key changes:

Store AI response in memory after generating it

Trim old messages to prevent memory overflow

Why trim?

LLMs have token limits (context window)

Keeping too much history can cause errors or slow responses
We keep only the most recent 10 exchanges (20 messages)

5. Template Modifications
A. formTemplateForModel() in [Link]
Original:
pythondef formTemplateForModel():
template = ChatPromptTemplate.from_messages([
("system", "You are a helpful SOC Analyst bot..."),
("ai", "Okay, can you clarify your requirements?"),
("system", "Your requirement is to analyze..."),
("human", "{user_input}")
])
return template
Modified:
pythondef formTemplateForModel():
template = ChatPromptTemplate.from_messages([
("system",
"You are a helpful SOC Analyst bot. You will provide analysis data over
the Splunk SIEM.\n\n"
"{history_text}" # ← New: Include history here
),
("ai", "Okay, can you clarify your requirements?"),
("system", "Your requirement is to analyze..."),
("human", "{user_input}")
])
return template
Key change:

Added {history_text} placeholder in the system message

This is where the formatted conversation history gets inserted

B. formParserTemplate() in [Link]
Modified similarly:
pythondef formParserTemplate():
template = ChatPromptTemplate.from_messages([
(
"system",
(
"You are a security operations analyst assistant..."
"7. Consider the conversation history to provide contextually
relevant responses.\n\n"
"{history_text}" # ← New: Include history here
)
),
(
"human",
"User Query: {userQuery}\n..."
),
])
return template
Why modify both templates?

First template: Used for initial query processing

Second template: Used for analyzing tool outputs
Both need access to conversation history for context

6. Chain Modification
Original chain:
pythonagent_chain = RunnableSequence(
RunnableLambda(promptGenerator),
RunnableLambda(processUserQuery),
RunnableLambda(tool_call),
secondTemplate,
RunnableLambda(analyzeQuery)
)
Modified chain:
pythonagent_chain = RunnableSequence(
RunnableLambda(promptGenerator),
RunnableLambda(processUserQuery),
RunnableLambda(tool_call),
RunnableLambda(lambda x: {
**x,
"history_text": format_history_for_template([Link]("history", []))
}) | secondTemplate, # ← Format history before second template
RunnableLambda(analyzeQuery)
)
Key change:

Added a lambda function before secondTemplate

Converts history from message objects to formatted text
Uses the spread operator **x to preserve all other data

7. User Commands Added

pythonif [Link]() == "clear":
conversation_history.clear()
 print("🧹 Conversation history cleared!")
continue

if [Link]() == "history":
print("\n📜 Conversation History:")
if not conversation_history:
print(" (empty)")
for i, msg in enumerate(conversation_history):
role = "User" if isinstance(msg, HumanMessage) else "Assistant"
print(f" [{i+1}] {role}: {[Link][:100]}...")
continue
Commands:

clear: Wipes conversation memory (fresh start)

history: Shows all stored messages
quit: Exits the chat

8. Data Flow Diagram

User Query: "What was my previous question?"
↓
[promptGenerator]
├─ Retrieves conversation_history
├─ Formats as text: "Previous conversation:\nUser: Give logs..."
├─ Creates prompt with history included
└─ Stores current query in history
↓
[processUserQuery]
├─ Sends prompt (with history) to LLM
└─ Returns LLM response + history
↓
[tool_call]
├─ Checks if tools needed
└─ Passes history forward
↓
[Lambda + secondTemplate]
├─ Formats history again for second template
└─ Creates final prompt with history
↓
[analyzeQuery]
├─ Gets final LLM response
├─ Stores AI response in conversation_history
└─ Prints response to user

9. Why This Approach Works

Initial Attempt (Failed):
I first tried using MessagesPlaceholder:
pythonMessagesPlaceholder(variable_name="history", optional=True)
Why it failed:

Some LLM providers (like Groq) don't fully support MessagesPlaceholder

The history wasn't being properly serialized into the prompt

Final Approach (Success):

Convert history to plain text and inject it directly into system prompt:
python"{history_text}"
Why this works:

✅ Works with ALL LLM providers

✅ Explicit and predictable
✅ Easy to debug (you can see exactly what's sent)
✅ No dependency on special LangChain features

10. Example Flow

Conversation:
User: "Give me logs from /var/log/[Link]"
AI: [Executes Splunk search and returns logs]

User: "What was my previous question?"

What happens internally:

Storage after first query:

pythonconversation_history = [
HumanMessage(content="Give me logs from /var/log/[Link]"),
AIMessage(content="I've executed a Splunk search...")
]

When second query arrives:

pythonhistory_text = """Previous conversation:

User: Give me logs from /var/log/[Link]
Assistant: I've executed a Splunk search...

"""

Prompt sent to LLM:

System: You are a helpful SOC Analyst bot...

Previous conversation:
User: Give me logs from /var/log/[Link]
Assistant: I've executed a Splunk search...RetryClaude does not have the ability to
run the code it generates yet.