BCSE334L

Predictive Analytics
Dr. Uma Priya D
Assistant Professor Sr. Gr.-I
School of Computer Science and Engineering
Vellore Institute of Technology, Vellore
Module-III : Predictive Modeling
• Propensity models

• Cluster models

• Collaborative filtering

• Applications and fundamental limitations

• Statistical Modeling
• Formal Definition
• Model Comparison
• Classification

Dr. Uma Priya D 2

Predictive Modeling
• Predictive modeling is the process of using data and statistical algorithms to
predict the probability of an outcome. The goal is to use existing or historical data
to make educated guesses about future events or behaviors.

• Predictive models analyze input data to estimate how likely an outcome is, e.g.,
the chance a customer will buy a product.

• Example: If you're building a spam filter, the model predicts whether a new email
is "spam" or "not spam" based on patterns learned from previous emails.

Dr. Uma Priya D 3

Predictive Modeling - Types
• Propensity models(Prediction)

• Clustering Models (Segmentation)

• Collaborative Filtering (Recommendation)

Dr. Uma Priya D 4

Predictive Modeling – Propensity Models
• Predicts the likelihood of a specific outcome

• These models answer "What is the probability that X will happen?“

• Helps target the right customers at the right time

• Examples:
• Will a customer buy this product?

• Will a user click on this ad?

• Will a subscriber cancel next month?

Dr. Uma Priya D 5

 Propensity Models - Example
Customer Age Income (k$) Clicked Email (0/1) Bought (0/1)
A 22 30 0 0
B 25 40 1 1
C 35 60 1 1
D 45 80 0 0
E 30 50 1 1
F 28 45 0 0

1. What is the probability that someone buys the product given they clicked the email?
2. What is the probability that someone buys the product if they did NOT click the email?
3. Which feature seems most predictive of buying behavior? Age/Income/Clicked Email
4. Will the new customer buy the product?
Age: 27
Income: $48k
Clicked Email: 1
Dr. Uma Priya D 6
 Propensity Models - Example
Customer Age Income (k$) Clicked Email (0/1) Bought (0/1)
A 22 30 0 0
B 25 40 1 1
C 35 60 1 1
D 45 80 0 0
E 30 50 1 1
F 28 45 0 0

1. What is the probability that someone buys the product given they clicked the email? P(Buy/ClickedEmail) = 3/3 = 100
2. What is the probability that someone buys the product if they did NOT click the email? P(Buy/NotClicked) = 0/3 = 0
3. Which feature seems most predictive of buying behavior? Age/Income/Clicked Email
4. Will the new customer buy the product? Yes
Age: 27
Income: $48k
Clicked Email: 1
Dr. Uma Priya D 7
Predictive Modeling – Clustering Models
• Groups similar individuals or items into segments or clusters based on shared
traits or behaviors.

• These models answer "Who behaves similarly?“

• Examples:
• What are the different types of shoppers?

• Which users engage with similar content?

• How can we group customers with similar buying patterns?

Dr. Uma Priya D 8

Predictive Modeling – Clustering Models
Popular Algorithms

• K-Means Clustering – You choose how many clusters (k); algorithm groups data
accordingly

• Hierarchical Clustering – Builds a tree of clusters

• DBSCAN – Groups data based on density (doesn’t need you to pick k)

Dr. Uma Priya D 9

Predictive Modeling – Clustering Models
Example:
Customer Income Spending Score
A 25 80
B 30 85
C 75 15
D 78 12
E 28 78

How many clusters can be formed?

Dr. Uma Priya D 10

Predictive Modeling – Clustering Models
Example:
Customer Income Spending Score
A 25 80
B 30 85
C 75 15
D 78 12
E 28 78

How many clusters can be formed?

1. High-spending, Low-income: A, B, E
2. High-Income, Low-spending: C, D

Dr. Uma Priya D 11

Predictive Modeling – Collaborative Filtering
• Predicts what a user might like based on other users' preferences.

• These models answer "What should we recommend next?“

• Examples:
• What product should we suggest to this customer?

• What movie might this person enjoy?

• Which article is most relevant for this user?

Dr. Uma Priya D 12

Predictive Modeling – Collaborative Filtering
• Example – User movie Rating
User Movie A Movie B Movie C Movie D
Sam 5 4 1
Alex 4 5 1
Mia 4 2 1
Raj 5 4 2

1. Compare the preferences of Raj and Sam

2. Based on Raj's rating, should we recommend Movie C to Sam?
3. What would you recommend to Mia based on Alex’s preferences?

Dr. Uma Priya D 13

Predictive Modeling – Collaborative Filtering
• Example – User movie Rating
User Movie A Movie B Movie C Movie D
Sam 5 4 1
Alex 4 5 1
Mia 4 2 1
Raj 5 4 2

1. Compare the preferences of Raj and Sam: Similar preferences

2. Based on Raj's rating, should we recommend Movie C to Sam? No
3. What would you recommend to Mia based on Alex’s preferences? Recommend Movie A to Mia,
since Alex (who is relatively similar) liked it.

Dr. Uma Priya D 14

Propensity models
• Propensity models are what most people think of when they hear “predictive
analytics”.

• Propensity models make predictions about a customer’s future behavior.

• However, keep in mind that even propensity models are abstractions and do
not necessarily predict absolute true behavior.

• It helps marketers understand if people respond to particular offers without

the need to launch promotional campaigns.
15
Propensity Models
• Model 1: Predicted customer lifetime value

• Model 2: Predicted share of wallet

• Model 3: Propensity to engage

• Model 4: Propensity to unsubscribe

• Model 5: Propensity to buy

• Model 6: Propensity to churn

Model 1: Predicted customer lifetime value
• CLV (Customer Lifetime Value) is a prediction of all the value a business will derive from their entire
relationship with a customer.

• “All-Star” customers can be extremely valuable to your business.

• Algorithms can predict how much a customer will spend with you long before customers themselves
realizes this.

• At the moment a customer makes their first purchase you may know a lot more than just their initial
transaction record: you may have email and web engagement data for example, as well as demographic
and geographic information.

• By comparing a customer to many others who came before them, you can predict with a high degree of
accuracy their future lifetime value.

17
Model 1: Predicted customer lifetime value
• CLV isn't just calculated based on the past — it’s predicted using data like:

• Transaction history (e.g., how much and how often they buy)

• Demographics (age, location, income)

• Engagement data (email opens, website visits, app usage)

• Customer behavior patterns (purchase cycles, time since last order)

By comparing a new customer to historical customers with similar behavior, you can estimate:

• “How valuable is this customer likely to be over the next year, or five years?”

18
Model 1: Predicted customer lifetime value
CustomerID Age Region FirstPurchase($) WebVisits EmailOpens PredictedCLV($)

C001 25 North 50 5 3 300

C002 40 South 120 10 7 850
C003 30 East 80 7 5 500
C004 22 West 30 3 2 150
C005 35 North 90 6 4 600
C006 45 South 150 12 9 1100

1. Who are the high CLV customers?

2. Does age or region impact CLV?

3. Do people who open more emails tend to have higher CLV?

Dr. Uma Priya D 19
Model 2: Predicted share of wallet
• Predicted share of wallet refers to the amount of the customer’s total
spending that a business captures in the products and services that it offers.
• Increasing the share of a customer’s wallet a company receives is often a
cheaper way of boosting revenue than increasing market share.
• For example, if a customer spends $100 with you on groceries, is this 10% or
90% of their grocery spending for a given year? Knowing this allows you to see
where future revenue potential is within your existing customer base and to
design campaigns to capture this revenue.

20
 Model 2: Predicted share of wallet
CustomerID TotalCategorySpend ($) SpendWithUs ($) CurrentSOW (%) PredictedSOW (%)
C001 1000 200 20% 35%
C002 800 300 37.5% 50%
C003 1200 600 50% 60%
C004 700 100 14.3% 25%
C005 1500 1200 80% 82%
1. Which customer has the most growth potential? C001
2. Which customer is already close to
Current SOW: 20%, Predicted SOW: 35%
saturation? C005
Growth gap = 35% - 20% = 15%
Current SOW: 80%, Predicted SOW: 82%
This translates to a potential $150 increase in spend.
Already spending most of their total category
Also C004 has a good opportunity:
budget with you — very little room to grow (only
Current SOW: 14.3%
2%).
Predicted: 25% → $175 potential spend vs $100 now → $75
increase
21
Top potential: C001, then C004
Model 2: Predicted share of wallet
CustomerID TotalCategorySpend ($) SpendWithUs ($) CurrentSOW (%) PredictedSOW (%)
C001 1000 200 20% 35%
C002 800 300 37.5% 50%
C003 1200 600 50% 60%
C004 700 100 14.3% 25%
C005 1500 1200 80% 82%
3. How much additional revenue can you expect if predictions
Summary
hold true?
Increase Over 1. Focus campaigns on C001 and C004 — large
CustomerID Predicted Spend ($)
Current ($)
room to grow.
C001 35% of 1000 = 350 350 - 200 = 150
2. Maintain loyalty for C005 — already a top
C002 50% of 800 = 400 400 - 300 = 100
C003 60% of 1200 = 720 720 - 600 = 120 spender.
C004 25% of 700 = 175 175 - 100 = 75 3. Medium potential in C002 and C003.
C005 82% of 1500 = 1230 1230 - 1200 = 30

22
Model 3: Propensity to engage
• A propensity to engage model predicts the likelihood that a person will engage in
some activity.

• It allows you to evaluate the propensity of your leads and customers to show
proactive behavior. It may be a propensity score showing which website visitors are
expected to click on an ad or which citizens will possibly vote for a given party in
elections.

• For example, predict how likely it is that a customer will click on your email links.

• Armed with this information you can decide not to send an email to a certain “low
likelihood to click” segment.
23
Model 3: Propensity to engage
CustomerID PastEmailsOpened TotalEmailsSent Age WebVisitsLastWeek PropensityToClick (%)
U001 5 10 25 3 70%
U002 0 8 42 1 15%
U003 7 10 34 5 85%
U004 2 9 29 2 40%
U005 1 12 50 0 10%

• U003 has a high propensity (85%) — definitely include in the email campaign.

• U002 and U005 have very low propensities — might exclude them to save cost and reduce spam
complaints.

24
Model 4: Propensity to unsubscribe
• A propensity to unsubscribe model tells you which customers not to touch: if there are high value
customers you are at risk of losing to unsubscribe, you need to find other ways to reaching out to them
that are not by email.

• For example, you can predict how likely it is that a customer will unsubscribe from your email list at
any given point in time.

• Armed with this information you can optimize email frequency. For “high likelihood to unsubscribe”
segments, you should decrease send frequency; whereas for “low likelihood to unsubscribe” segments,
you can increase email send frequency.

• You could also decide to use different channels (like direct mail or LinkedIn) to reach out to “high
likelihood to unsubscribe” customers.

25
Model 5: Propensity to buy
• The propensity to buy model tells you which customers are ready to make their purchase, so you
can find who to target. Goal: Target the right customers with the right offers, at the right time.
• Moreover, once you know who is ready and who is not helps you provide the right aggression in
your offer.
• Those that are likely to buy won’t need high discounts (You can stop cannibalizing your margin)
while customers who are not likely to buy may need a more aggressive offer, thereby bringing you
incremental revenue.
• For example, a “propensity to buy a new vehicle” model built with only data the automotive
manufacturer has in their database can be used to predict percent of sales.
• By incorporating demographic and lifestyle data from third parties, could improve the result to 70%
in the top five deciles. the accuracy of that model can be improved. That is, if the first model
predicts 50% sales in the top five deciles (there are ten deciles), then the later.

26
Model 6: Propensity to churn
• Companies often rely on customer service agents to “save” customers who call to say they are taking
their business elsewhere.
• But by this time, it is often too late to save the relationship. The propensity to churn model tells you
which active customers are at risk, so you know which high value, at risk customers to put on your
watch list and reach out.
• Armed with this information, you may be able to save those customers with preemptive marketing
programs designed to retain them.

• Often propensity models can be combined to make campaign decisions.

• For example, you may want to do an aggressive customer win back campaign for customers who
have both a high likelihood to unsubscribe and a high predicted lifetime value.

27
Model 6: Propensity to churn
CustomerID ChurnProb (%) CLV ($) UnsubscribeProb (%)
C301 85 1200 60
C302 10 500 15
C303 75 2000 80

1. Who should get a win-back offer? C303

ChurnProb = 75% → High
CLV = $2000 → Very High
UnsubscribeProb = 80% → High (don’t push too hard via email)
2. Who should receive multi-channel outreach instead of emails? C301 and C303
UnsubscribeProb > 50% for both → risk of email fatigue
3. Should we invest in saving C301? Try saving via non-email channel; offer moderate retention incentive.

Dr. Uma Priya D 28

Clustering Models
• Clustering is the term used in predictive analytics to describe customer segmentation.

• Like classification, it is used to group or segment data.

• But unlike classification, clustering doesn’t use predefined labels. Instead, it automatically
discovers patterns or groupings in the data.

• Clustering segments customers into similar groups based on their behavior, demographics, or any
other features.

• Marketers can use clustering to discover customer segments like:

• Bargain shoppers

• High spenders

• Infrequent visitors

Dr. Uma Priya D 29

Clustering Models - Types
Three types of cluster models:

• Behavioral clustering

• Product based clustering (also called category-based clustering)

• Brand based clustering

Dr. Uma Priya D 30

Clustering Models – Behavioral Clustering
• Groups customers based on how they act — their habits, preferences, and
behaviors.

• Examples:
• Frequency of visits or purchases

• Types of content they engage with

• Average spend per visit

Dr. Uma Priya D 31

Clustering Models – Product-based Clustering
• Clusters customers based on what types of products or categories they buy.

• Examples:
• Someone buying mostly electronics

• Someone focused on baby products or fitness gear

• Helps in cross-selling and personalized recommendations within categories

Dr. Uma Priya D 32

Clustering Models – Brand-based Clustering
• Groups customers by their brand preferences.

• Examples:
• Customers loyal to Apple vs. Samsung

• Preference for Nike over Adidas

• Design brand-specific marketing campaigns, loyalty programs, or partnerships

Dr. Uma Priya D 33

 Clustering Models – Example
Last_Purchase_Day
Customer_ID Age Gender Total_Orders Avg_Spend Product_Category Preferred_Brand
s
C1 28 F 8 500 5 Electronics Apple
C2 35 M 2 150 45 Grocery LocalMart
C3 23 F 5 200 20 Fashion Zara
C4 40 M 1 100 100 Grocery LocalMart
C5 31 F 6 300 10 Electronics Samsung
C6 45 M 3 120 60 Home Ikea
C7 26 F 9 550 3 Fashion H&M
C8 30 M 4 200 25 Grocery FreshBasket
C9 29 F 5 250 15 Electronics Apple
C10 38 M 1 90 120 Home Ikea
C11 34 F 7 400 6 Fashion Zara
C12 27 M 2 180 40 Grocery FreshBasket
Dr. Uma Priya D 34
Clustering Models – Example
You're working with an online retail store. The goal is to segment customers into
meaningful groups to improve targeted marketing and personalization strategies.
1. Behavioral Clustering
1. Which cluster should receive loyalty points or exclusive offers?
2. Which customers are likely to churn based on their recent inactivity?
2. Product-based Clustering
1. What types of products are most popular with each segment?
2. How can we personalize marketing based on product category preferences?
3. Brand-based Clustering
1. Which brands have the most loyal customer base?
2. How can this help in brand partnership or co-branded campaigns?

Dr. Uma Priya D 35

Clustering Models – Example
You're working with an online retail store. The goal is to segment customers into meaningful
groups to improve targeted marketing and personalization strategies.
Behavioral Clustering
1. Which cluster should receive loyalty points or exclusive offers?
Customers who are highly active, have frequent recent purchases, and spend more should be rewarded.
C1: Avg_Spend = 500, Total_Orders = 8, Last_Purchase_Days = 5
C7: Avg_Spend = 550, Total_Orders = 9, Last_Purchase_Days = 3
C11: Avg_Spend = 400, Total_Orders = 7, Last_Purchase_Days = 6

2. Which customers are likely to churn based on their recent inactivity?

Customers with low purchase frequency and long time since last purchase may churn.
C4: Last_Purchase_Days = 100, Total_Orders = 1
C10: Last_Purchase_Days = 120, Total_Orders = 1
C6: Last_Purchase_Days = 60, Total_Orders = 3

Dr. Uma Priya D 36

Clustering Models – Example
You're working with an online retail store. The goal is to segment customers into
meaningful groups to improve targeted marketing and personalization strategies.
Product-based Clustering
1. What types of products are most popular with each segment?

• Fashion: C3, C7, C11 → Popular with younger, high-spend customers

• Electronics: C1, C5, C9 → Popular across females with high spend
• Grocery: C2, C4, C8, C12 → Preferred by medium to low spenders
• Home: C6, C10 → Older customers with low engagement

2. How can we personalize marketing based on product category preferences?

• Fashion lovers (C3, C7, C11) → Send latest trends, flash sales
• Electronics shoppers (C1, C5, C9) → Offer bundle deals, new launches
• Grocery buyers (C2, C4, C8, C12) → Weekly offers, subscription services
• Home category (C6, C10) → Furniture, decor sale alerts

Dr. Uma Priya D 37

Clustering Models – Example
You're working with an online retail store. The goal is to segment customers into
meaningful groups to improve targeted marketing and personalization strategies.
Brand-based Clustering
1. Which brands have the most loyal customer base?
Loyalty inferred from frequency + spend + brand consistency:
• Apple: C1, C9 → High orders & spend
• Zara: C3, C11 → Younger, stylish customers
• LocalMart: C2, C4 → Budget shoppers with low activity
• Ikea: C6, C10 → Low purchase frequency, consistent preference
• FreshBasket: C8, C12 → Grocery segment
Apple, Zara, and FreshBasket seem to have loyal segments.
2. How can this help in brand partnership or co-branded campaigns?
• Apple + Loyalty Offers: Tech reward programs
• Zara + Flash Sales: Target young fashion segment
• FreshBasket + Subscription Bundles: Ideal for recurring grocery buyers
• Ikea + Festive Campaigns: Tap into home-decor shopping trends

Dr. Uma Priya D 38

Collaborative Filtering
• Collaborative Filtering is a popular technique used in Recommendation Systems. It predicts a
user’s interests by collecting preferences or taste information from many other users.

• If two users have shown similar reactions to certain items, they are likely to agree on other
items as well.

• Applications:
• Movies: Netflix, IMDb, etc.

• Shopping – Amazon, Flipkart, etc.

• Music – Spotify, etc.

• Apps – App Stores, etc.

• News – Google News, etc.

Dr. Uma Priya D 39

Collaborative Filtering - Types
1. User-User Collaborative Filtering – Recommends items based on User
similarity.

2. Item-Item Collaborative Filtering - Recommends items that are similar to

items the user already likes.

• Common Similarity Indexes:

• Jaccard Similarity

• Cosine Similarity

• Pearson Correlation Coefficient

Dr. Uma Priya D 40

Collaborative Filtering - Types
1. User-User Collaborative Filtering

Simij= similarity(useri , userj)

Dr. Uma Priya D 41

Collaborative Filtering - Types
2. Item-Item Collaborative Filtering

Simij= similarity(itemi , itemj)

Dr. Uma Priya D 42

Collaborative Filtering – Similarity Index
Jaccard Similarity
• Used when data is binary (e.g. liked/disliked, purchased/not purchased

• A: Set of items liked by User A

• B: Set of items liked by User B

User Item A Item B Item C Item D
U1 1 0 1 1
U2 1 1 0 1
• U1 likes {A, C, D}

• U2 likes {A, B, D}

Dr. Uma Priya D 43

 Collaborative Filtering – Similarity Index
Cosine Similarity
• Measures similarity between two vectors using the angle between them.

• A⋅B is the dot product

• ∣∣A∣∣ and ∣∣B∣∣ are the magnitudes of vectors A and B

User Item A Item B Item C

U1 4 5 0
U2 5 3 2

Dr. Uma Priya D 44

Collaborative Filtering – Similarity Index
Pearson Correlation Coefficient
• Measures the linear correlation between two users' ratings.

• ​: ratings of User X and Y for item i

• : average ratings of User X and Y

Item U1 Rating U2 Rating

A 4 5
B 3 3
C 5 4

Dr. Uma Priya D 45

Collaborative Filtering – Similarity Index
When to use which Similarity Index
• Jaccard – Binary Interactions.

• Cosine – Ratings without considering mean

• Pearson – Ratings, with mean normalization

Dr. Uma Priya D 46

Collaborative Filtering - Example
• Predict a missing rating using user-user collaborative filtering or item-item collaborative
filtering

Users Item 1 Item 2 Item 3 Item 4

User 1 5 3 0 ?
User 2 4 3 5 2
User 3 2 2 0 4
User 4 5 0 4 1

Dr. Uma Priya D 47

Statistical Modeling
• A statistical model is a collection of probability distributions (or density functions) that could
potentially describe the data.
• Think of it as the “universe” of possible mathematical patterns your data could follow.
Parametric Model
• A parametric model is a statistical model where each distribution is determined by a finite
number of parameters.
• It can be expressed as:

• θ = parameter vector (finite-dimensional)

• Θ⊆Rd = parameter space (possible values of θ)
• d = number of parameters

Dr. Uma Priya D 48

Parametric Model
Example – Gaussian Model

• If we assume data comes from a univariate Gaussian distribution, the model is:

Where

• μ∈R = mean

• σ>0 = standard deviation

• Both μ and σ are parameters.

The parameter space here is:

Dr. Uma Priya D 49

Non-Parametric and Semi-Parametric Model
• A non-parametric model is a set of probability distributions with infinite dimensional
parameters, and might be written as 𝒫 = {all distributions}.

• Do not assume a fixed number of parameters.

• More flexible but may require more data to estimate.

• A semi-parametric model also has infinite dimensional parameters, but is not dense in the
space of distributions.

• For example, a mixture of Gaussians with one Gaussian at each data point is dense in the
space of distributions. Formally, if 𝑑 is the dimension of the parameter, and 𝑛 is the number
of samples, if 𝑑 → ∞as 𝑛 → ∞ and 𝑑 /𝑛→ 0 as 𝑛 → ∞, then the model is semi-parametric.

Dr. Uma Priya D 50

 Parametric, Non-Parametric and Semi-Parametric Model

Assume height grows linearly with age for Linear age + gender (still parametric) Gender is still modeled as a fixed numbe (parametric).
everyone, ignoring gender. Adds a fixed offset for gender but still Age is modeled flexibly using a smooth curve (spline),
Mathematically: assumes a straight-line age–height which can capture rapid childhood growth, teenage
Height=β0+β1×Age relationship. spurts, and plateaus.
Example: boys might be, on average, 3 cm
This is simple but unrealistic — it ignores taller than girls at any age. Mathematically:
that growth slows after teenage years. Mathematically: Height=β0+f(Age)+β1×Gender
Dr. Uma Priya D 51
Model Comparison
• When we have multiple candidate models, we need criteria to choose the “best” one.

• Exploratory analysis: Formulate all possible models and see which describes the data best.

• Confirmatory analysis: Test a specific model (or set of models) you hypothesized before data
collection.

Common Approaches:

• Goodness-of-fit measures: e.g., R2 in linear regression for variance explained.

• Likelihood-based methods: Likelihood-Ratio Test for nested models (simpler model =

complex model with restrictions).

• Information criteria: AIC, BIC for non-nested model comparison.

Dr. Uma Priya D 52

Model Comparison - Example
To predict a person’s height (heighti).

Model 1 (simpler model)

• heighti = b0 + b1⋅agei + εi

• b0​ = intercept (predicted height when age = 0).

• b1​ = slope (how much height changes per year of age).

• εi​ = error term (unexplained variation).

Model 2 (more complex model)

• heighti = b0 + b1⋅agei + b2⋅genderi + εi

• Adds gender as a predictor (genderi is 0 for male, 1 for female).

• b2​ adjusts predicted height depending on gender.

Dr. Uma Priya D 53

Model Comparison – Example(Contd…)
• Model comparison is about deciding which model explains the data better.

• In exploratory analysis: We try all possible predictors (e.g., age, gender, nutrition) and pick
the best.

• In confirmatory analysis: We test whether adding gender (predetermined before data

collection) improves predictions.

• R² (coefficient of determination): Measures how much variance in height is explained.

• Model 1: R² may be moderate (age explains some height variation).

• Model 2: R² likely higher (age + gender together explain more variation).

• Nested Model: Model 1 is nested inside Model 2 because Model 1 can be obtained by setting
b2=0.

Dr. Uma Priya D 54

Model Comparison - Example(Contd…)
• Use a Likelihood Ratio Test (or an F-test in regression) to check if adding gender significantly
improves the fit.
• Null hypothesis: b2=0 (gender does not help predict height).

• If p-value < significance level, we reject null — Model 2 is better.

• Interpretation
• If Model 2 is significantly better:

• Age and sex both are important predictors of height.

• Error term is smaller — predictions are more precise.

• If not significantly better:

• Age alone may be sufficient for prediction.

Dr. Uma Priya D 55

Model Classification
• Categorizes models based on structure, assumptions, and variables.

Criteria Used:

• By completeness:
• Complete models: Number of equations = number of endogenous variables.

• Incomplete models: Number of equations ≠ number of endogenous variables.

• By type of variables:
• General Linear Model (GLM): Continuous dependent variables.

• Generalized Linear Model (GzLM): Allows other types, e.g., logistic regression.

• Multilevel Models: Data with hierarchical/nested structure.

• Structural Equation Models (SEM): Complex relationships among observed and latent variables.

Dr. Uma Priya D 56

Model Classification – Example (Contd…)
Step 1 — Switch from Regression to Classification

• Instead of predicting the actual height in cm, we predict whether someone is tall or not tall.

• Let’s define:
• Yi=1 if the person is tall (e.g., above 170 cm).

• Yi=0 if not tall.

Step 2 — Model Formulation

• Model 1 (simpler logistic model)

• pi​ = probability person i is tall.

• b1​ shows how odds of being tall change with age.
• Model 2 (more complex logistic model)

• Adds gender (0 = male, 1 = female) as an extra predictor.

Dr. Uma Priya D 57
Model Comparison – Example(Contd…)
• Models can be classified as complete or incomplete based on whether the number of
equations matches the number of endogenous variables.
• Here, we have one endogenous variable (Yi) and one equation → both models are “complete.”

• The logistic regression is an example of a generalized linear model.

• Interpretation
• If adding gender significantly improves classification accuracy and likelihood, Model 2 is better.

• If not, Model 1 is sufficient.

Dr. Uma Priya D 58