COMPREHENSIVE REFERENCE & STUDY GUIDE

Fundamentals of Statistics & Theory of Attributes

Scope: Core Foundations to Association Document Standard: Academic /
Subject: Applied Statistics & Probability
Metrics Technical

1. Definition, Importance, Scope, and Limitations of Statistics

1.1 Definition of Statistics

The term Statistics originates from the Latin word 'Status' or the Italian word 'Statista', historically referring to state craft and
administrative facts. In modern analysis, statistics is understood in two distinct senses:

• Plural Sense (Numerical Data): Refers to numerical facts systematically collected or observed in any field of inquiry.
According to Horace Secrist, statistics in the plural sense are "aggregates of facts affected to a marked extent by
multiplicity of causes, numerically expressed, enumerated or estimated according to reasonable standards of accuracy,
collected in a systematic manner for a predetermined purpose, and placed in relation to each other."
• Singular Sense (Statistical Methods): Refers to the body of scientific methods, techniques, and theories used to
collect, organize, summarize, present, analyze, and interpret numerical data. As defined by Croxton & Cowden,
"Statistics may be defined as the science of collection, presentation, analysis, and interpretation of numerical data."

1.2 Importance of Statistics

Statistics serves as an essential quantitative engine across virtually all scientific, commercial, and policy disciplines:

• Economic Planning & Policy Formulation: Enables governments and central banks to model inflation, forecast
national income (GDP), evaluate unemployment rates, and allocate financial resources effectively.
• Business Management & Operations: Crucial for market research, demand forecasting, inventory optimization,
quality control (Six Sigma), and risk management in banking and insurance.
• Scientific Research & Medicine: Validates experimental findings, assesses clinical trial efficacy, analyzes
epidemiological trends, and tests hypotheses under uncertainty.
• Data Science & Artificial Intelligence: Forms the mathematical foundation for machine learning algorithms, predictive
modeling, and statistical inference from big data.

1.3 Scope of Statistics

The scope of statistics is conventionally divided into two major branches:

BRANCH CORE OBJECTIVE KEY TOOLS & TECHNIQUES

Descriptive Statistics Summarizing, organizing, and describing the Measures of Central Tendency (Mean,
main features of a dataset without drawing Median, Mode), Dispersion (Variance,
conclusions beyond the observed data. SD), Histograms, Boxplots.

Inferential Statistics Drawing conclusions, generalizations, and Hypothesis Testing ($z$-test, $t$-test,
predictions about a larger population based on ANOVA), Confidence Intervals,
sample data. Regression Analysis.

Statistical Theory & Methodology Reference Manual Page 1 of 6

1.4 Limitations of Statistics

Critical Limitations of Statistical Analysis

• Deals with Aggregates Only: Statistics does not study isolated individual items (e.g., the income of a single
worker has no statistical significance unless compared within a group).
• Requires Quantitative Expressibility: Direct statistical methods apply only to measurable numerical data.
Qualitative characteristics (e.g., honesty, intelligence) must be transformed via qualitative rating scales or
analyzed using the Theory of Attributes.
• Truth Only on Average: Statistical laws are probabilistic rather than deterministic; they hold true only under large-
sample averages over the long run.
• Susceptible to Misuse: Without domain knowledge and proper contextual understanding, statistical results can
be manipulated to produce misleading or biased conclusions ("Lies, damned lies, and statistics").

2. Concepts of Statistical Population and Sample

In statistical theory, the fundamental purpose of empirical research is to investigate properties of a comprehensive
aggregate by examining a carefully chosen portion of it.

Statistical Population (Universe) Statistical Sample

The total collection of all individuals, items, A smaller, finite subset selected from a population
observations, or entities that possess common according to a specific sampling design, analyzed to
observable characteristics under a specific statistical infer characteristics of the entire population.
inquiry. • Representativeness: Must accurately reflect population
• Finite Population: Contains a countable number of diversity without systematic bias.
units (e.g., students in a university, cars produced in a • Efficiency: Provides reliable estimates at a fraction of
day). the cost, time, and destructive testing required for a full
• Infinite Population: Contains an uncountable or census.
unlimited number of units (e.g., rolls of a die, stars in the • Sampling Frame: A list of population units from which
sky). the sample is drawn.
• Real vs. Hypothetical: Physically existing items vs.
theoretical outcomes of an experiment.

2.1 Parameters vs. Statistics

A central distinction in inferential statistics is the difference between summary values calculated for a population versus
those calculated for a sample:

MEASURE / POPULATION PARAMETER (THEORETICAL / SAMPLE STATISTIC (OBSERVED /

FEATURE TARGET) ESTIMATOR)

Definition Fixed, true numerical value describing a population. Random variable calculated from sample
data.

Notation Scheme Greek Letters (e.g., μ, σ, N, P) Roman Letters (e.g., x, s, n, p)

Mean μ = (1 / N) Σ Xi x = (1 / n) Σ xi

Statistical Theory & Methodology Reference Manual Page 2 of 6

 MEASURE / POPULATION PARAMETER (THEORETICAL / SAMPLE STATISTIC (OBSERVED /
FEATURE TARGET) ESTIMATOR)

Variance σ2 = (1 / N) Σ (Xi − μ)2 s2 = (1 / (n − 1)) Σ (xi − x)2

Proportion P=X/N p=x/n

3. Scales of Measurement

Data collection requires converting empirical observations into numerical or symbolic values. According to psychologist
Stanley Smith Stevens (1946), measurement scales are categorized into four distinct levels based on their mathematical
and structural properties.

MATHEMATICAL PERMISSIBLE OPERATIONS &

SCALE LEVEL REAL-WORLD EXAMPLES
PROPERTIES STATISTICS

1. Nominal Classification only. Mutually Counting, Equality (=, ≠). Gender, Blood Group, Marital
exclusive and exhaustive Statistics: Mode, Frequency, Chi- Status, Country Codes, Zip
categories without order or square (χ2). Codes.
magnitude.

2. Ordinal Classification + Distinct Comparison (>, <), Ordering. Likert Scales (Satisfied/Neutral),
Order/Rank. Distance Statistics: Median, Percentiles, Socioeconomic Class,
between consecutive ranks Spearman Rank Correlation (rs). Education Level, Stage of
is unknown or unequal. Cancer.

3. Interval Classification + Rank + Addition & Subtraction (+, −). Ratios Temperature in Celsius/
Equal Distance between are meaningless. Fahrenheit, IQ Scores, SAT
units. Arbitrary Zero Point Statistics: Mean, Standard Deviation, Scores, Calendar Years.
(no absolute absence). Pearson Correlation (r).

4. Ratio Classification + Rank + All Arithmetic (+, −, ×, ÷). Meaningful Height, Weight, Annual Income,
Equal Distance + ratio comparisons. Distance, Age, Kelvin
Absolute / True Zero Point Statistics: Geometric Mean, Temperature.
(indicates total absence). Coefficient of Variation (CV).

Summary Rule of Hierarchy

The scales form a strict hierarchy: Nominal < Ordinal < Interval < Ratio. Any statistical method applicable to a lower level can be
applied to a higher level, but higher-level methods cannot be validly applied to lower-level scales (e.g., calculating the arithmetic
mean of nominal zip codes produces meaningless numbers).

4. Theory of Attributes

4.1 Concept and Dichotomy

When qualitative variables cannot be measured on a continuous numerical scale (e.g., literacy, employment status, eye
color, disease presence), data are studied using the Theory of Attributes (pioneered by G. Udny Yule and M.G. Kendall).

Statistical Theory & Methodology Reference Manual Page 3 of 6

An attribute is a qualitative characteristic. The process of dividing a population into two complementary classes—those
possessing the attribute and those lacking it—is called dichotomy.

• Positive Attributes: Denoted by capital Roman letters (A, B, C, ...).

• Negative Attributes: Denoted by Greek letters (α, β, γ, ...).
• Class Frequencies: The number of individuals possessing a specific combination of attributes, denoted in parentheses
(e.g., (A), (B), (AB), (αβ)). Total frequency is N.

4.2 Class Frequencies and Relations

For two attributes A and B, the population of size N is partitioned into a 2 × 2 contingency grid comprising 9 fundamental
class frequencies:

ATTRIBUTE CLASSIFICATION B (POSITIVE) Β (NEGATIVE) TOTAL ROW FREQUENCY

A (Positive) (AB) (Aβ) (A) = (AB) + (Aβ)

α (Negative) (αB) (αβ) (α) = (αB) + (αβ)

Total Column Frequency (B) = (AB) + (αB) (β) = (Aβ) + (αβ) N = (A) + (α) = (B) + (β)

Order of Classes Formula

For n attributes, the total number of classes of all orders is 3n, and the total number of ultimate class frequencies (containing n
attributes) is 2n. For 2 attributes, there are 32 = 9 total classes and 22 = 4 ultimate class frequencies ((AB), (Aβ), (αB), (αβ)).

4.3 Consistency of Data

A set of class frequencies is said to be consistent if none of the ultimate class frequencies are negative. A negative
frequency indicates contradictory or logically impossible empirical data.

Condition of Consistency: Every ultimate class frequency ≥ 0

Conditions for Two Attributes (A and B):

Since (AB) ≥ 0, (Aβ) = (A) − (AB) ≥ 0, (αB) = (B) − (AB) ≥ 0, and (αβ) = N − (A) − (B) + (AB) ≥ 0, the fundamental bounding
inequality for consistency is:

max( 0, (A) + (B) − N ) ≤ (AB) ≤ min( (A), (B) )

4.4 Independence and Association of Attributes

Two attributes A and B are defined as independent if the proportion of A's among B's is identical to the proportion of A's
among non-B's (β's).

Condition for Independence: (AB) / (B) = (Aβ) / (β) = (A) / N

Expected Frequency under Independence: (AB)exp = [ (A) × (B) ] / N

Statistical Theory & Methodology Reference Manual Page 4 of 6

Depending on the observed value of (AB) relative to its expected value, attributes exhibit one of three states:

• Independence: (AB) = [ (A)(B) ] / N. Presence of A gives no information about B.

• Positive Association: (AB) > [ (A)(B) ] / N. Presence of A increases likelihood of B (e.g., Education and High Income).
• Negative Association (Disassociation): (AB) < [ (A)(B) ] / N. Presence of A decreases likelihood of B (e.g.,
Vaccination and Disease Contracting).

4.5 Measures of Association

To quantify the intensity and direction of association between two attributes, three primary metrics are used:

1. Yule's Coefficient of Association (Q)

Introduced by G. Udny Yule, this is the standard metric for 2 × 2 tables:

Q = [ (AB)(αβ) − (Aβ)(αB) ] / [ (AB)(αβ) + (Aβ)(αB) ]

• Range: −1 ≤ Q ≤ +1
• Q = +1: Complete/Perfect Positive Association.
• Q = −1: Complete/Perfect Negative Association.
• Q = 0: Attributes are Completely Independent.

2. Yule's Coefficient of Colligation (Y)

An alternative measure proposed by Yule that provides a less extreme scale than Q:

Y = [ 1 − √k ] / [ 1 + √k ], where k = [ (Aβ)(αB) ] / [ (AB)(αβ) ]

Mathematical Relation between Q and Y

The relationship connecting Yule's Coefficient of Association (Q) and Colligation (Y) is:
Q = 2Y / ( 1 + Y2 )

3. Chi-Square (χ2) Test of Independence

A rigorous hypothesis-testing framework used to assess whether an observed discrepancy from independence is
statistically significant:

χ2 = Σ [ ( O − E )2 / E ] = N [ (AB)(αβ) − (Aβ)(αB) ]2 / [ (A)(α)(B)(β) ]

Where O represents observed frequencies, and E represents expected class frequencies under the null hypothesis of
independence.

Statistical Theory & Methodology Reference Manual Page 5 of 6

5. Quick Reference Summary Matrix

CORE CONCEPT PRIMARY DEFINITION / FORMULA KEY MATHEMATICAL PROPERTY

Statistics (Singular) Science of collecting, organizing, analyzing, and Inductive reasoning under uncertainty.
interpreting data.

Sample Statistic x = (1/n) Σ xi, s2 = [1/(n−1)] Σ (xi − x)2 Unbiased estimator of population parameter
μ.

Ratio Scale Equal units + True Absolute Zero. Supports all arithmetic (+, −, ×, ÷) and
geometric mean.

Data Consistency max(0, (A)+(B)−N) ≤ (AB) ≤ min((A), (B)) Ensures no ultimate class frequency is
negative (≥ 0).

Independence (AB) = [ (A)(B) ] / N Zero association between qualitative

attributes.

Yule's Association Q = [ (AB)(αβ) − (Aβ)(αB) ] / [ (AB)(αβ) + (Aβ)(αB) ] Bounded metric in [−1, +1].
(Q)

Statistical Theory & Methodology Reference Manual Page 6 of 6