Unit – II

Scores and Rankings - The purpose of scoring and ranking systems in data science is to
convert raw data into clear, comparable values so that we can make decisions easily.
 Scoring functions are measures that reduce multi-dimensional records to a single value,
highlighting some particular property of the data.
 A familiar example of scoring functions is those used to assign student grades in courses
such as mine. Students can then be ranked (sorted) according to these numerical scores,
and later assigned letter grades based on this order.
 Each student receives a single combined score, often scaled between 0 and 100.
 These scores typically come from a linear combination of the input variables, perhaps
giving 8% weight to each of five homework assignments, and 20% weight to each of
three exams.

There are several things to observe about such grading rubrics, which we will use as a model
for more general scoring and ranking functions:

1. Degree of arbitrariness – Every teacher grades differently

There is no single fixed rule for grading.

 Some teachers give more importance to the final exam.

 Some average all marks after converting them to 100.
 Some use Z-scores or other formulas.

Each teacher believes their method is correct, but actually these choices are somewhat
subjective.
In data analysis also, scoring models are often based on assumptions chosen by the designer.

2. Lack of validation data – No “correct answer” for grades

There is no gold standard that tells a teacher:

“This student’s correct grade is B+.”

Students may complain and ask for more marks, but there is no objective truth to compare
against.

So teachers cannot scientifically test whether their grading system is perfect or not.
Similarly, many scoring models in data science lack true validation data.

3. General robustness – Different systems still give similar results

Even though grading methods are different:

 Top students usually remain top students.

 Average students stay in the middle.
 Weak students rarely become toppers suddenly(optional).
This shows that grading systems are robust – small changes in the scoring method do not
drastically change the final ranking

There are many different techniques to convert raw data into useful and understandable
scores.

Raw data by itself is often not informative. So we apply methods like:

 The Body Mass Index (BMI)

 Averaging

 Normalization

 Z-score standardization

 Weighted scoring

 Ranking functions

to produce scores that clearly show performance, importance, or comparison.

The Body Mass Index (BMI)

Raw data:

 Height

 Weight

By themselves, these two numbers do not clearly tell whether a person is healthy or not.
The body mass index (BMI) is a score or statistic designed to capture whether
your weight is under control. It is defined as
BMI = mass / height2
where mass is measured in kilograms and height in meters.
 As I write this, I am 68 inches tall (1.727 meters) and feeling slightly fat at
150
lbs (68.0 kg). Thus my BMI is 68.0=(1.7272) = 22.8. This isn't so terrible, however,
because commonly accepted BMI ranges in the United States de_ne:
_ Underweight: below 18.5.
_ Normal weight: from 18.5 to 25.
_ Overweight: from 25 to 30.
_Obese: over 30.
o The BMI is an example of a very successful statistic/scoring function in Health Field
Scoring and ranking systems help transform complex data into simple numbers that
guide accurate and fair decisions.
Step 1: Raw Dataset (Student Performance)
Student Test (100) Assignment (50) Attendance (%)
A 78 40 90
B 85 45 95
C 60 30 80
D 90 35 85
Different scales → cannot compare directly.

Step 2: Normalize each column (0 to 1)

Xnorm= (X−Min) / (Max−Min)
Test Marks
Min = 60, Max = 90
Student Test_norm
A (78−60)/30 = 0.60
B (85−60)/30 = 0.83
C 0.00
D 1.00
Assignment
Min = 30, Max = 45
Student Asg_norm
A (40−30)/15 = 0.67
B 1.00
C 0.00
D (35−30)/15 = 0.33

Attendance
Min = 80, Max = 95
Student Att_norm
A (90−80)/15 = 0.67
B 1.00
C 0.00
D (85−80)/15 = 0.33

Step 3: Create Scoring Formula

Final Score=Testnorm+Asgnorm+Attnorm/3
Student Final Score
A (0.60+0.67+0.67)/3 = 0.65
B (0.83+1+1)/3 = 0.94
C (0+0+0)/3 = 0.00
D (1+0.33+0.33)/3 = 0.55

Step 4: Ranking
Rank Student Score
1 B 0.94
2 A 0.65
Rank Student Score
3 D 0.55
4 C 0.00

Final Answer
We first created scores from raw data, then sorted them to get ranking.
This is how scoring leads to ranking in data science.
 4.2 Developing Scoring Systems
Scores are functions that map the features of each entity to a numerical value of merit. This
section will look at the basic approaches for building effective scoring systems, and evaluating
them.

A score is just a number that tells us how good or important something is.

Example: BMI score tells how healthy a person is. Ie) So, a scoring system = a formula that
converts many features into one useful number.
4.2.1 Gold Standards and Proxies
Gold Standard – The “true answer”
A gold standard is the value we believe is the correct answer.
Example:
 For body fat, the real gold standard is measuring fat using medical machines.
 BMI was created by comparing height & weight to these true fat values.
If we have gold standard data, we can:
 Adjust our formula so that the score matches the true values.

Problem: Gold standards are hard to get

In real life:
 Teachers don’t know the true grade a student deserves.
 Companies don’t know the true importance of a customer.
So we use a proxy.

Proxy – A substitute for the truth

A proxy is something easy to measure that is closely related to what we want.
Examples:
True value (unknown) Proxy used
Real student ability GPA in other subjects
Real body fat BMI
Real importance of a celebrity Autograph price, awards, magazine rankings
Proxies help us test and improve scoring systems.

4.2.2 Scores vs Rankings

Score
A score is a number.
Example: GPA = 8.2
Ranking
A ranking is the order.
Example: Student is 12th out of 200.
Usually:
 We compute scores first
 Then we sort them to get rankings.
Which is better – Score or Rank?
It depends.
Situation Better
You want to know exact performance Score
You want to compare with others Rank

Rankings are permutations ordering n entities by merit, generally constructed by sorting

the output of some scoring system. Popular examples of rankings/rating systems include:

o Football/basketball top twenty:

o University academic rankings:
o Google PageRank/search results:
o Class rank:

Since scores and rankings are duals of each other, which provides a more meaningful
representation of the data? As in any comparison, the best answer is that it depends, on
issues like:

o Will the numbers be presented in isolation (without any comparison or context) ?

Will people see just the number, or will they also see how it compares with others?

Example:

 Score = 39.18 → hard to understand.

 Rank = 111 out of 351 → immediately meaningful.

So this question is asking whether the number will be shown by itself or along with
context (ranking).
o What is the underlying distribution of scores?

Rank difference does not show how big the real difference is
If Team A is rank 1 and Team B is rank 2, we don’t know:
 Are they almost equal?
 Or is Team A far better?
Similarly, difference between rank 1 & 2 looks same as 111 & 112,
but in reality their scores may be very close or very far apart.
So: Small score change can cause big rank change.
o Do you care about the extremes or the middle?
Middle ranks are very sensitive
Most scores form a bell-shaped curve.
That means:
 Many teams have very similar scores in the middle.
 A tiny change in score can move a team from 111 → 120.
So:
 Falling from 1st to 10th is serious.
 Falling from 111th to 120th is usually not important.

4.2.3 Recognizing Good Scoring Functions

Good scoring functions are good because they are easily interpretable and generally
believable. Properties are ;
1. Easily computable: Example is BMI ie) Weight / Height2
2. Easily understandable: People should know what it means
3. Monotonic interpretations of variables:
Monotonic behaviour – Each variable must affect score logically
Every input should affect the result in the expected direction.
Variable Effect
Weight ↑ BMI ↑
Height ↑ BMI ↓
If heavier people get lower BMI, the formula is wrong.

4. Produces generally satisfying results on outliers:

Works well for extreme cases – Stars should be on top
If your scoring says a weak student is rank 1, the system is broken.
Example (Student Ranking)
If the student who always fails comes first, there is a bug.

5. Uses systematically normalized variables:

Uses normalized values – All features must be on same scale
6. Breaks ties in meaningful ways:
Avoid same scores
If too many people get the same score, ranking is useless.

Summary:

A good scoring function must be:

✔ Easy to calculate
✔ Easy to understand
✔ Logically correct
✔ Matches reality
✔ Uses normalized data
✔ Produces very few ties
What is Normalization?

Normalization means converting different variables into a common scale so that no variable
dominates the model simply because of its unit or size.

Why do we need normalization?

Suppose we build a model using:

Feature Typical Value
City area (sq km) 5 – 100
Population 25,000 – 8,000,000

Population values are extremely larger than area values.

So population will dominate the learning algorithm, even if area is equally important.

This causes:

 Numerical instability

 Very small coefficient changes causing huge prediction changes

Hence we normalize.

Z-Score Normalization

The most common normalization method is the Z-score transform.

Where:

 ai = original value

 μ = mean of the feature

 σ = standard deviation

Example – Converting Values to Z-Scores

Consider the dataset:

Student Marks
A 50
B 60
C 70
D 80
E 90
Step 1: Compute Mean
μ=50+60+70+80+905=70

Step 3: Compute Z-Scores

Marks Z-score
(50−70)/14.14
50
= −1.41
60 −0.71
70 0
80 +0.71
90 +1.41

The outcome / meaning of these Z-scores is how each student’s mark stands relative to the
class average.
Marks Z-score Outcome / Interpretation
The student scored 1.41 standard deviations below the mean → very low
50 −1.41
performance
60 −0.71 Below average performance
70 0 Exactly average
80 +0.71 Above average performance
90 +1.41 1.41 standard deviations above the mean → very high performance

Advanced Ranking Techniques

What is Advanced Ranking?

Normal ranking =
Score each item → sort → get ranking

But sometimes we don’t have clear scores. Instead we have:

 pairwise comparisons (A vs B)

 many different rankings

 network of preferences

So we use advanced techniques.

I. Elo Rating Formula
r′(A)=r(A)+k(SA−μA)

This formula updates player A’s rating after one match.

Meaning of Each Term

Symbol Meaning
r(A) Old rating of player A
r′(A) New rating of player A
Adjustment factor (speed of
k
change)
SA Actual result of the match
μA Expected result for player A

Values of SA
Result SA
A wins 1
A loses −1

Meaning of μA
This tells how much A was expected to win.
Situation μA
A and B equal strength 0
close to
A much stronger than B
+1
close to
A much weaker than B
−1

Role of k
k value Effect
Small (10–20) Ratings change slowly (stable)
Large (40–60) Ratings change fast (unstable, wild swings)

Example 1 – Expected Win

Player Rating
A 1800
B 1400

A is much stronger → μA≈0.9

Let k=40
Case: A wins
r′(A)=1800+40(1−0.9) = 1800+40(0.1)=1804
A gains only 4 points (because he was expected to win).

Example 2 – Big Upset

Same players, but A loses.
r′(A)=1800+40(−1−0.9) =1800 −76=1724
A loses 76 points → big penalty for unexpected loss.

Final Meaning
The Elo system does not reward winning alone

II. Merging Rankings – Borda Method

Suppose 5 movies A, B, C, D, E ranked by 4 critics:
Critic 1 Critic 2 Critic 3 Critic 4
A B A A
B A B B
C C C D
D D E C
E E D E

Borda Scoring
Rank 1 → 1 point
Rank 2 → 2 points
Rank 3 → 3 points ….
Add total points for each movie.
Movie Total Points
A 5
B 8
C 12
D 16
E 19
Final Ranking:
A>B>C>D>E
This is the consensus ranking(one combined ranking that best reflects all individual rankings.)

III. Digraph-Based RankingBottom of Form

We convert preferences into a directed graph.

 Each item = node

 If A is better than B → draw arrow A → B

Goal: Find a ranking order that violates the fewest arrows.

Example

Preferences:

 A>B
 B>C
 C>A
 A>D
Graph:

A→B
B→C
C → A ❌ (cycle)
A→D

This cycle A → B → C → A means no perfect ranking exists.

Solution Approach (Heuristic)

For each node compute:

d(v)=out-degree−in-degree

Node Out In d(v)

2
A 1 (C) +1
(B,D)
B 1 (C) 1 (A) 0
C 1 (A) 1 (B) 0
D 0 1 (A) −1

Now sort by highest d(v):

A>B>C>D

This order breaks the fewest arrows, so it is a good ranking.

IV. PageRank
Used by Google to rank web pages.

Core Idea
A page is important if:
1. Many pages link to it, and
2. Those pages are themselves important.
So not all votes are equal.

Example
Links:
 A→B
 C→B
 D→B
 Wikipedia → B
B has many incoming links, especially from Wikipedia → B is very important.

Intuition
Situation Importance
Blog links to you Small vote
Wikipedia links to you Big vote
Many strong pages link to you Huge importance
PageRank repeats this process until scores stabilize.

Real-World Meaning
PageRank does not ask:
“Who is better than whom?”
It asks:
“Who is most referenced by other important entities?”