Chapter 6: Modeling of the Experimental

System
6.1. Introduction to System Modeling
This chapter presents a comprehensive mathematical and architectural model of the emotion-
aware chatbot system. The modeling approach provides a rigorous formalization of the
experimental framework, enabling precise specification of system components, data flows,
algorithms, and interactions. The model serves as both a design specification and a foundation
for implementation, ensuring reproducibility and facilitating systematic evaluation.
The modeling framework encompasses multiple abstraction levels:
• Conceptual Model: High-level system architecture and component relationships
• Mathematical Model: Formal specifications of algorithms, optimization objectives, and
evaluation metrics
• Architectural Model: Detailed component design and interface specifications
• Process Model: Sequential workflows and state transitions
• Data Model: Data structures, transformations, and flow representations

6.2. System Architecture Model

6.2.1. High-Level Architecture
The emotion-aware chatbot system follows a modular, pipeline-based architecture that
separates concerns into distinct, independently manageable components. The architecture is
formally defined as a tuple:
S=⟨ I , E , R , O , M ⟩

where:
• I : Input Processing Module
• E : Emotion Recognition Module
• R : Response Generation Module
• O : Output Formatting Module
• M : Memory/Context Management Module

Component Interaction Model:

The system processes user input through a sequential pipeline with feedback loops:
I E M R O
x user → x processed → e detected → c context → r response → y output

where:
 • x user : Raw user input text
• x processed : Preprocessed and tokenized input
• e detected : Detected emotion class
• c context : Conversation context (including emotion history)
• r response: Generated response text
• y output : Formatted output to user

6.2.2. Component Specifications

[Link]. Input Processing Module I
Functionality: Preprocesses raw text input for emotion recognition.
¿
Input: Raw text string x ∈ Σ where Σ is the character alphabet.
Output: Tokenized sequence T ={t 1 ,t 2 ,... , t n } where t i ∈ V , and V is the vocabulary.
Processing Pipeline:
¿ ¿
1. Normalization Function Norm : Σ → Σ :

x norm =lowercase ( x )

2. Tokenization Function Tokenize : Σ ¿ →V ¿ :

T =BERT-WordPiece ( x norm )

Where BERT-WordPiece applies the WordPiece tokenization algorithm, splitting text into
subword tokens.

3. Truncation and Padding:

{
T [ 1: L ] if |T|> L
Truncate ( T , L )= T ∪ {[PAD] }L−|T| if |T|< L
T if |T|=L

where L=128 is the maximum sequence length.

4. Encoding Function:

Encode ( T )={v ( t 1 ) , v ( t 2 ) ,... , v ( t L ) }

where v :V → Z maps tokens to vocabulary indices.

Formal Specification:

I ( x )=Encode ( Truncate ( Tokenize ( Norm ( x ) ) , L ) )

[Link]. Emotion Recognition Module E
Functionality: Classifies emotional state from preprocessed input.
Input: Token sequence encoding x ∈ Z L

Output: Emotion probability distribution p ∈ R|Y | and predicted class y ¿ ∈Y

Model Architecture:
The emotion recognition module implements a fine-tuned BERT encoder with a classification
head:
1. BERT Encoder:

H=BERT ( x )=[ h [ CLS] ,h 1 , ... , hL ]

where:
( L+1 ) × d
– H ∈R is the hidden state matrix
– d=768 is the hidden dimension
d
– h [ CLS] ∈ R is the classification token representation
2. Classification Head:
c=W c h [ CLS] +bc

where:
|Y |×d
– W c∈R is the classification weight matrix
|Y |
– b c ∈ R is the bias vector
– c ∈ R|Y | is the logit vector
3. Probability Distribution:

[ ]
exp ( c 1 ) exp ( c|Y |)
p=Softmax ( c )= |Y | , ... , |Y |
∑ exp ( c j ) ∑ exp ( c j )
j=1 j=1

4. Prediction:
¿
y =arg max p [ y ]
y ∈Y

Keyword Override Mechanism:

The system includes a rule-based override to handle obvious emotional expressions:

KeywordMatch ( x , K ) = { k ¿ if ∃ k ∈ K : contains ( x ,k )
∅ otherwise
where K is a set of keyword-emotion mappings.
Final Prediction Logic:

e detected =
{
KeywordMatch ( x , K ) if KeywordMatch ( x , K ) ≠∅
y
¿
otherwise

Formal Specification:
E ( x )=( p , e detected )

[Link]. Memory/Context Management Module M

Functionality: Maintains conversation history and emotional context.
State Representation:
The context manager maintains a conversation state:
State=⟨ H , E history ,T ⟩

where:
• H={h1 , h2 ,... , hk } is the conversation history (last k turns)
• E history={e1 , e 2 , ... , ek } is the emotion sequence
• T is the timestamp of last interaction

Update Function:
Update (State , x , e ) =⟨ H ∪ {x }, E history ∪ {e }, now( )⟩

Context Extraction:

{
⟨ e k , e k−1 ⟩ if k ≥ 2
ExtractContext ( State )= ⟨ e k , neutral ⟩ if k =1
⟨ neutral , neutral ⟩ if k =0

This extracts the current emotion and previous emotion for context-aware response
generation.
Formal Specification:

M (State ,e )=Update ( State , x , e ) ,ExtractContext ( Update ( State , x , e ) )

[Link]. Response Generation Module R

Functionality: Generates empathetic responses based on detected emotion.
Input: Detected emotion e ∈ Y and context c=⟨ e current ,e previous ⟩
Output: Response text r ∈ Σ ¿
Response Selection Model:
The system employs a template-based response generation approach:
1. Template Repository:
T= ⋃ T y
y ∈Y

where T y is the set of response templates for emotion y .

2. Template Selection Function:

SelectTemplate ( e , c )=
{RandomSelect ( T e ) if template-based
SemanticMatch ( e , c ,T e ) if semantic-based

3. Semantic Matching (Optional): For semantic-based selection, the system uses sentence
embeddings:

Embed ( t )=SentenceTransformer ( t )

Similarity ( t 1 ,t 2 )=cos ( Embed ( t 1 ) , Embed ( t 2 ) )

SemanticMatch ( e , c , T e )=arg max Similarity ( t , c )

t ∈T e

4. Response Generation:

r =SelectTemplate ( e , ExtractContext ( State ) )

Response Strategy Mapping:

Each template is associated with a response strategy:
• Strategy ( r ) ∈ {Validation , Positive reinforcement , Empathy , Problem solving ,... }

Formal Specification:
R ( e ,c )=SelectTemplate ( e , c )

[Link]. Output Formatting Module O

Functionality: Formats response for API output.
Input: Response text r ∈ Σ ¿
Output: JSON-formatted output y ∈ JSON
Formatting Function:
y=JSON ( { response : r , emotion: e , timestamp : t })

Formal Specification:
 O ( r , e )=FormatJSON ( r , e )

6.3. Mathematical Model of Emotion Classification

6.3.1. Problem Formulation
Emotion Classification as a Supervised Learning Problem:
N
Given a dataset D={( x i , y i ) }i=1 where:

• x i ∈ X is a text input (conversational utterance)

• y i ∈Y is the corresponding emotion label
• Y ={ y 1 , y 2 , ..., y C } is the set of C=8 emotion classes

The goal is to learn a function f : X → Y that maps text inputs to emotion classes.
Probabilistic Formulation:
Instead of learning a deterministic function, we learn a conditional probability distribution:
P ( y∨x ;θ )

where θ represents the model parameters.

6.3.2. BERT-Based Classification Model

Architecture Specification:
1. Token Embedding:

E=[ e [ CLS ] , e1 ,... , e L ]

where e i ∈ Rd is the embedding of token i.

2. Position Embedding:

P= [ p 0 , p1 ,... , pL ]

where pi ∈ Rd encodes position information.

3. Input Representation:
( 0)
X =E + P
4. Transformer Layers: The BERT model applies LBERT =12 transformer layers:

For each layer l=1 , ... , LBERT :

Multi-Head Self-Attention:
 ( )
T
QK
Attention ( Q , K ,V )=softmax V
√ dk
( l−1 ) Q ( l−1 ) K ( l−1) V
Q h= X W h , K h= X W h , V h=X Wh

H h=Attention ( Qh , K h , V h )
O
H=Concat ( H 1 , ... , H H ) W

where H=12 is the number of attention heads, d k =d / H=64 .

Feed-Forward Network:

FFN ( x )=GELU ( x W 1 +b 1) W 2+b 2

Layer Normalization and Residual Connection:

X =LayerNorm ( H + X )
(l) ( l−1)

X =LayerNorm ( FFN ( X ) + X )
(l) (l ) (l )

5. Classification Head:

h final=X (
L BERT )
[0]
(taking [CLS] token)
c=W c h final+ bc

p=Softmax ( c )

Parameter Count:
• BERT-base: ~110M parameters
• Classification head: |Y |× d +|Y |=8 ×768+8=6 , 152 parameters
• Total: ~110M parameters

6.3.3. Training Objective

Loss Function:
The model is trained to minimize the cross-entropy loss:
N
−1
L ( θ )= ∑ ∑
N i=1 y ∈Y
1 [ y i= y ] log P ( y∨x i ; θ ) + λ ∥ θ ∥22

where:
• 1 [ ⋅ ] is the indicator function
 • λ is the L2 regularization coefficient (weight decay)
2
• ∥ θ ∥2 is the L2 norm of model parameters

Optimization:
The parameters are updated using the AdamW optimizer:
^t
m
θt +1=θ t−α t
√ v^ t + ϵ
where:
• α t is the learning rate at step t
• ^ t and ^v t are bias-corrected first and second moment estimates
m
• ϵ is a small constant for numerical stability

Learning Rate Schedule:

Linear warmup followed by linear decay:

{
t
α max × if t< T warmup
T warmup
αt=

(
α max × 1−
t−T warmup
T total−T warmup ) otherwise

where:
−5
• α max=2 ×10 (maximum learning rate)
• T warmup is the number of warmup steps
• T total is the total number of training steps

6.3.4. Evaluation Metrics Model

Confusion Matrix:
For multi-class classification, we construct a confusion matrix:
C ×C
CM ∈ Z
where CM [ i , j ] represents the number of instances with true label y i predicted as y j.
Per-Class Metrics:
For each class y ∈Y :

• True Positives: T P y =CM [ y , y ]

• False Positives: F P y = ∑ CM [ y ' , y ]
y '≠ y
 • False Negatives: F N y = ∑ CM [ y , y ' ]
y '≠ y

Precision:
T Py
P y=
T P y+ F P y

Recall:
T Py
Ry =
T P y+ F N y

F1 Score:
2 × Py × R y
F 1y =
Py + Ry

Macro-Averaged Metrics:
1
Pmacro =
C
∑ Py
y ∈Y

1
Rmacro =
C
∑ Ry
y∈Y

1 2×Py×Ry
F 1macro =
C
∑ F 1 y = C1 ∑ Py + Ry
y ∈Y y ∈Y

Accuracy:

∑ T P y ∑ CM [ y , y ]
Accuracy= y ∈Y = y ∈Y
N N

6.4. Data Flow Model

6.4.1. Training Data Flow
Training Pipeline Model:
Raw Dataset (CSV)
↓
Data Loading Module
↓
Preprocessing (Normalization, Tokenization)
↓
Dataset Splitting (Train/Val/Test: 70/15/15)
↓
DataLoader (Batching, Shuffling)
↓
BERT Tokenizer (WordPiece)
↓
Model Forward Pass
↓
Loss Computation
↓
Backward Pass (Gradient Computation)
↓
Parameter Update (AdamW Optimizer)
↓
Model Checkpointing

Formal Data Transformation:

Raw Data: Draw ={( x i , y i ) }i=1

raw N
1.

Preprocessing: D processed ={( I ( x i ) , y i ) }i=1

raw N
2.

3. Dataset Split:
N train
– D train ={( x i , yi ) }i=1
N val
– D val={( x i , y i ) }i=1
N test
– D test ={( x i , y i ) }i=1
|B|
4. Batch Construction: For each batch B={ ( x i , y i ) }i =1:

– Stack tokenized sequences: X B ∈ Z|B|× L

– Create attention masks: M B ∈{0 , 1}|B|× L
– Stack labels: y B ∈ Y |B|

6.4.2. Inference Data Flow

Real-Time Inference Pipeline:
User Input (Text)
↓
Input Processing (Tokenization, Padding)
↓
BERT Forward Pass
↓
Logit Computation
↓
Softmax (Probability Distribution)
↓
Emotion Prediction (argmax)
↓
Keyword Override Check
↓
Context Manager Update
↓
Response Template Selection
↓
Response Formatting
↓
API Response (JSON)

Inference Time Model:

InferenceTime=T preprocess +T model +T response

where:
• T preprocess ≈ 10−20 ms (tokenization)
• T model ≈ 150−200 ms (BERT inference on CPU)
• T response ≈ 5−10 ms (template selection)

Total: ~200 ms per request (real-time capable)

6.4.3. Multi-Turn Conversation Flow

State Machine Model:
The conversation can be modeled as a finite state machine:
States : S={Sinitial , S active , S ended }

State Transitions:
1. Initial State Sinitial :

– State: State=⟨ ∅ , ∅ , t 0 ⟩
– Action: Wait for first user input
first input
– Transition: Sinitial → S active
2. Active State Sactive:

– State: State=⟨ H , E history ,T ⟩ where |H|> 0

– Actions:
• Process user input
• Detect emotion
• Update context
• Generate response
continue end
– Transition: Sactive → S active or Sactive → S ended
3. Ended State Sended :

– State: Final conversation state

 – Action: Save conversation log
Emotion Transition Model:
For context-aware responses, we track emotion transitions:
$$e_{\text{prev}} \rightarrow e_{\text{current}}}}$$
Examples:
• sadness → joy : Positive transition, respond with encouragement
• joy → sadness: Negative transition, respond with empathy
• neutral → anger : Escalation, respond with de-escalation

6.5. Component Interaction Model

6.5.1. Module Dependency Graph
Dependency Structure:
The system follows a hierarchical dependency structure:
Output Formatting Module
↓ (depends on)
Response Generation Module
↓ (depends on)
Memory/Context Management Module
↓ (depends on)
Emotion Recognition Module
↓ (depends on)
Input Processing Module

Interface Specifications:
1. I → E Interface:

– Input: Token sequence encoding x ∈ Z L

– Output: Emotion probability distribution p ∈ R|Y |, predicted emotion e ∈ Y
2. E → M Interface:

– Input: Detected emotion e ∈ Y , user input x ∈ Σ ¿

– Output: Updated context c=⟨ e current ,e previous ⟩
3. M → R Interface:

– Input: Current emotion e , context c

¿
– Output: Response text r ∈ Σ
4. R →O Interface:

– Input: Response text r , detected emotion e

 – Output: JSON-formatted output y

6.5.2. Communication Protocols

Request-Response Model:
For the Flask API, the system implements a RESTful request-response protocol:
Request Format:
{
"text": "user input text",
"conversation_id": "optional_conversation_id"
}

Response Format:
{
"response": "bot response text",
"emotion": "detected_emotion",
"confidence": 0.95,
"timestamp": "2024-01-01T12:00:00Z"
}

Error Handling Model:

ProcessRequest ( req )=¿

6.6. Algorithm Models

6.6.1. Emotion Classification Algorithm
Algorithm: Emotion Classification
Input: Text x, Model M(θ), Keyword Set K
Output: Emotion e, Probability Distribution p

1. // Preprocessing
2. x_processed ← Normalize(x)
3. tokens ← Tokenize(x_processed)
4. encoding ← Encode(tokens)
5.
6. // Keyword Override Check
7. if KeywordMatch(x, K) ≠ ∅ then
8. e ← KeywordMatch(x, K)
9. return (e, OneHot(e))
10. end if
11.
12. // Model Inference
13. h_CLS ← BERT(encoding)[0] // [CLS] token
14. logits ← W_c · h_CLS + b_c
15. p ← Softmax(logits)
16. e ← argmax(p)
17.
18. return (e, p)

Time Complexity:

• Tokenization: O (|x|)
BERT Forward Pass: O ( L ×d × LBERT ) = O ( 128 ×7682 ×12 ) ≈ O ( 109 ) operations
2
•
• Classification Head: O ( d ×|Y |) = O ( 768× 8 ) = O ( 103 ) operations
• Total: Dominated by BERT inference, approximately O ( 109 ) operations

Space Complexity:

• Model parameters: O ( 110 ×10 6 ) floats ≈ 440 MB

• Activations: O ( L× d × LBERT ) ≈ O ( 128× 768 ×12 ) ≈ 4 MB per forward pass

6.6.2. Response Selection Algorithm

Algorithm: Template-Based Response Selection
Algorithm: Template-Based Response Selection
Input: Emotion e, Context c, Template Set T
Output: Response r

1. // Get emotion-specific templates

2. T_e ← T[e]
3.
4. // Selection Strategy
5. if |T_e| = 1 then
6. r ← T_e[0]
7. else if SelectionMode = "random" then
8. r ← RandomSelect(T_e)
9. else if SelectionMode = "semantic" then
10. embeddings ← SentenceTransformer(T_e)
11. context_emb ← SentenceTransformer(c)
12. similarities ← CosineSimilarity(embeddings, context_emb)
13. r ← T_e[argmax(similarities)]
14. end if
15.
16. return r

Time Complexity:
• Random selection: O ( 1 )
• Semantic selection: O (|T e|× d embed ) ≈ O ( 10× 384 ) = O ( 103 ) operations
6.6.3. Context Update Algorithm
Algorithm: Context Management
Input: Current State s, New Input x, Detected Emotion e
Output: Updated State s', Context c

1. // Update History
2. s'.history ← [Link] ∪ {x}
3. s'.emotions ← [Link] ∪ {e}
4. s'.timestamp ← CurrentTime()
5.
6. // Maintain Window Size
7. if |s'.history| > MAX_HISTORY then
8. s'.history ← s'.history[-MAX_HISTORY:]
9. s'.emotions ← s'.emotions[-MAX_HISTORY:]
10. end if
11.
12. // Extract Context
13. if |s'.emotions| ≥ 2 then
14. [Link] ← s'.emotions[-1]
15. [Link] ← s'.emotions[-2]
16. else if |s'.emotions| = 1 then
17. [Link] ← s'.emotions[-1]
18. [Link] ← "neutral"
19. else
20. [Link] ← "neutral"
21. [Link] ← "neutral"
22. end if
23.
24. return (s', c)

Time Complexity: O ( 1 ) (assuming constant-time set operations)

Space Complexity: O ( MA X H ISTORY × (|x|+|e|) ) ≈ O ( 10× 500 ) = O ( 5000 ) bytes per

conversation

6.7. Performance Model

6.7.1. Training Performance Model
Training Time Estimation:
N epochs × N batches × T batch
TrainingTime=
Parallelism
where:
• N epochs=3 (number of training epochs)
 N train
• N batches=⌈ ⌉ (number of batches per epoch)
B
• B=8 (batch size)
• T batch ≈ 500−1000 ms (time per batch on GPU)
• Parallelism = 1 (single GPU)
Example Calculation:
• N train=47 samples
• N batches=⌈ 47/8 ⌉ =6 batches per epoch
• T batch =500 ms (on GPU)
• Total Training Time: 3 ×6 × 0.5=9 seconds (very fast due to small dataset)
Memory Requirements:
Memory =ModelParams+ Activations+ OptimizerState

• Model parameters: 440 MB

• Activations per batch: B× L× d × 4 bytes ≈ 8 ×128 × 768× 4 ≈ 3 MB
• Optimizer state (AdamW): 2 ×ModelParams ≈ 880 MB
• Total: ~1.3 GB (GPU memory)

6.7.2. Inference Performance Model

Latency Breakdown:
Latency=T preprocess +T inference+ T postprocess

where:
• T preprocess=10−20 ms (tokenization)
• T inference=150−200 ms (BERT forward pass on CPU)
• T postprocess=5−10 ms (response selection)

Throughput Model:
For batch processing:
Bbatch
Throughput=
Latency ( B batch )

where Bbatch is the batch size.

CPU Inference:
• Single request: ~200 ms (5 requests/second)
• Batch of 8: ~500 ms (16 requests/second)
GPU Inference (if available):
• Single request: ~50 ms (20 requests/second)
• Batch of 16: ~100 ms (160 requests/second)

6.8. Validation and Reproducibility Model

6.8.1. Experiment Reproducibility
Determinism Guarantees:
1. Random Seed Setting:

Seed=42

Applied to:

– Python random module

– NumPy random number generator
– PyTorch random number generator
– Dataset shuffling
2. Dataset Splitting:

Split ( D ,seed ) =DeterministicSplit ( D ,seed )

Ensures identical train/val/test splits across runs.

3. Model Initialization:

θ0 =InitializeFromPretrained ( bert −base−uncased , seed )

Uses pretrained weights (deterministic) + deterministic classifier head initialization.

6.8.2. Cross-Validation Model

K-Fold Cross-Validation:
For K=5 folds:
K
test
D= ⋃ D k
k=1

where Dtest
k is the test set for fold k .

Performance Aggregation:
K
1
μ́= ∑μ
K k=1 k
 √
K
1
σ μ= ∑ ( μ − μ́ ) 2
K−1 k=1 k

where μk is the performance metric (e.g., F1 score) on fold k .

6.9. System Constraints and Assumptions

6.9.1. Constraints
Computational Constraints:
• Maximum sequence length: L=128 tokens
• Maximum conversation history: MA X H ISTORY =10 turns
• Memory limit: ~2 GB RAM (for CPU inference)
Data Constraints:
• Input text length: |x|≤500 characters (truncated if longer)
• Emotion classes: Fixed set Y with |Y |=8
• Label distribution: Assumed balanced (or balanced via sampling)
Temporal Constraints:
• Inference latency: ≤ 500 ms for real-time interaction
• Training time: Acceptable for dataset size (minutes to hours)

6.9.2. Assumptions
Model Assumptions:
1. Independence Assumption: Each utterance is classified independently (no cross-
utterance dependencies in emotion detection, though context is used for response
generation)
2. Stationarity: Emotion distribution in training data matches test data distribution
3. Language: English language input only
4. Emotion Taxonomy: Discrete emotion classification (8 classes) adequately captures
emotional states
System Assumptions:
1. Single User: One conversation per session (no multi-user concurrency model)
2. Synchronous Communication: Request-response pattern (no asynchronous messaging)
3. Stateless Server: No persistent session storage (context maintained per request or in
client)
6.10. Summary
This chapter presented a comprehensive modeling framework for the emotion-aware chatbot
system, encompassing:
1. System Architecture Model: Modular component design with formal specifications
2. Mathematical Model: BERT-based emotion classification with complete mathematical
formulations
3. Data Flow Model: Training, inference, and multi-turn conversation flows
4. Component Interaction Model: Interface specifications and communication protocols
5. Algorithm Models: Detailed algorithms for emotion classification, response selection,
and context management
6. Performance Model: Training and inference time/memory analysis
7. Validation Model: Reproducibility guarantees and cross-validation procedures
The models provide a rigorous foundation for implementation, evaluation, and reproducibility,
ensuring that the experimental system can be precisely specified, implemented, and validated.