Binary Logistic Regression: Comprehensive Explanation

Definition and Introduction

Logistic Regression is a supervised machine learning algorithm designed for binary classification
tasks—predicting the probability that an instance belongs to one class or another. Instead of
predicting continuous outcomes (as in linear regression), it models categorical dependent
variables, typically coded as 0 or 1. The algorithm estimates the relationship between
independent variables and the log-odds of the dependent binary outcome by using the logistic
function (sigmoid), which maps any real-valued number into a probability between 0 and 1.

Types of Logistic Regression

• Binomial Logistic Regression: Handles dependent variables with two possible outcomes
(e.g., pass/fail, 0/1).

• Multinomial Logistic Regression: For dependent variables with three or more unordered
categories (e.g., cat, dog, sheep).

• Ordinal Logistic Regression: For dependent variables with ordered categories (e.g., low,
medium, high).

Assumptions

1. Independent observations: No correlation exists between observations or explanatory

variables.

2. Binary dependent variable: The outcome must be binary; for multiple categories,
specialized functions (e.g., Softmax) are used.

3. Linearity of log-odds: The log-odds of the outcome are linearly related to the
independent variables.

4. No significant outliers: Outliers can distort the model.

5. Large sample size: Sufficient data is needed to estimate reliable coefficients.

Logistic (Sigmoid) Function

The logistic (sigmoid) function transforms linear combinations of predictors to probabilities

between 0 and 1, with the formula:
1
𝜎(𝑡) =
1 + 𝑒 −𝑡
where 𝑡 is the linear combination 𝑏0 + 𝑏1 𝑥1 + ⋯ + 𝑏𝑛 𝑥𝑛 . The resulting S-shaped curve
smoothly transitions probabilities from 0 to 1, defining the likelihood of belonging to the
positive class.

A threshold (commonly 0.5) decides classification: values above belong to class 1; below belong
to class 0.

Key Terminologies
𝑝
• Odds: Ratio of the probability that the event occurs to it not occurring, i.e., 1−𝑝.

𝑝
• Log-odds (Logit): Natural logarithm of the odds, log( ). Logistic regression models this
1−𝑝
as a linear function of predictors.

• Coefficient: Estimated parameters indicating the effect size of independent variables on

the log-odds.

• Intercept: Baseline log-odds when all predictors are zero.

• Maximum Likelihood Estimation: Method to find coefficients maximizing the likelihood

of observing the given data.

Model Equations

• Logit form:
𝑝
log⁡( ) = 𝑏0 + 𝑏1 𝑥1 + ⋯ + 𝑏𝑛 𝑥𝑛
1−𝑝

• Probability form:

𝑒 𝑏0+𝑏1𝑥1+⋯+𝑏𝑛𝑥𝑛
𝑝=
1 + 𝑒 𝑏0+𝑏1𝑥1+⋯+𝑏𝑛𝑥𝑛

Logistic vs. Linear Regression

• Linear regression targets a continuous dependent variable; logistic regression targets a

binary/categorical dependent variable.

• Logistic regression outputs probabilities mapped through sigmoid; linear regression

outputs continuous numeric predictions.

Business Applications
 • Framingham Heart Study: Predicting 10-year coronary heart disease risk via logistic
regression modeling multiple demographic, behavioral, and medical factors.

• Spam filtering, loan default prediction, voting behavior classification: Logistic

regression models classify outcomes based on predictors.

Steps for Logistic Regression Analysis (e.g., in R)

1. Exploratory Data Analysis (EDA).

2. Convert binary target into factor variable.

3. Identify and remove highly correlated variables (via correlation plots).

4. Build initial model using logistic regression.

5. Check multicollinearity (e.g., Variance Inflation Factor).

6. Use ANOVA to identify important features.

7. Refine model with significant variables only.

8. Interpret odds ratios to understand the magnitude of effects.

Evaluation Metrics

1. Confusion Matrix: Table with True Positives (TP), True Negatives (TN), False Positives
(FP), False Negatives (FN).
𝑇𝑃+𝑇𝑁
2. Accuracy: 𝑇𝑃+𝑇𝑁+𝐹𝑃+𝐹𝑁, overall correct predictions.

• Note: Can be misleading with imbalanced classes.

𝑇𝑃
3. Precision (Positive Predictive Value): 𝑇𝑃+𝐹𝑃, proportion of predicted positives which are
correct.
𝑇𝑃
4. Recall (Sensitivity or True Positive Rate): 𝑇𝑃+𝐹𝑁, percentage of actual positives correctly
identified.

5. F1 Score: Harmonic mean of precision and recall:

𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 × 𝑅𝑒𝑐𝑎𝑙𝑙
𝐹1 = 2 ×
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑅𝑒𝑐𝑎𝑙𝑙

𝐹𝑃
6. ROC Curve: Plots True Positive Rate (Recall) vs False Positive Rate (FPR = 𝐹𝑃+𝑇𝑁) for
different classification thresholds, illustrating trade-offs.
 7. AUC (Area Under the Curve): Integral of ROC curve; values near 1 imply excellent model
discrimination, 0.5 means random guessing.

Interpretation of ROC and AUC

• The closer the ROC curve follows the left and top borders of the plot, the better the
model.

• A perfect classifier reaches the top-left corner (TPR=1, FPR=0).

• A random classifier shows a diagonal line where TPR equals FPR.

• Higher AUC values indicate stronger classifiers that better separate classes across
thresholds.

Decision Tree: In-Depth Study Guide

1. Decision Tree Definition and Hierarchical Structure

A decision tree is a graphical, hierarchical structure used to model decisions and their possible
consequences, including chance event outcomes. It starts with a main question—the root
node—and branches out based on the answers to a series of questions, forming a "tree" of
decisions that lead to final predictions (endpoints are called leaf nodes). Decision trees are
popular in both classification (categorical outcomes) and regression (continuous outcomes),
making them versatile tools in business analytics.

Key Terms

• Root Node: The initial node at the top, representing the entire dataset or the main
decision point.

• Branches: Lines connecting nodes, representing possible answers or outcomes for each
decision.

• Internal Nodes: Decision points (non-terminal), where the data is split further based on
an attribute.

• Leaf Nodes (Terminal Nodes): The final nodes at the ends of branches, representing the
final prediction or outcome.

2. Analogical Example (Coffee Drinking Decision)

Imagine deciding whether to drink coffee:

• The tree starts by asking time of day (root node).

 • If it's morning, check tiredness: If tired, branch to "Drink coffee" (leaf), else "No coffee."

• If it's afternoon, same logic applies: Are you tired? If yes: "Drink coffee"; if not: "No
coffee."

This shows how decision trees make decisions by sequentially testing features (attributes),
branching until reaching a clear outcome.

3. Types of Decision Trees

• Classification Trees: Used for categorical outcomes. For example, detecting if an email is
spam or not based on email features.

• Regression Trees: Predict continuous values, like estimating house prices based on
features such as size and location.

4. How Decision Trees Work (Branching Process)

1. Begin at the root node (main question or feature selection).

2. Each node asks a yes/no or categorical question about a feature.

3. Based on the answer, the dataset is split (partitioned) into two or more subsets.

4. The process repeats recursively for each branch, each time selecting the best attribute
to split the data further.

5. Construction workflow: Continue branching (splitting) until no further informative

questions remain, or until nodes become "pure" (i.e., all data at that node belong to the
same class, or a stopping criterion is met).

6. The final split leads to the leaf node, containing the predicted class (in classification
trees) or numerical value (in regression trees).

5. Business Applications of Decision Trees

• Spam Detection: Classification trees can decide if incoming emails are "spam" or "not
spam" by splitting on email features (e.g., presence of certain words).

• Price Estimation: Regression trees estimate values such as house prices, car values, or
insurance premiums by branching based on features.
6. Random Forest: Ensemble of Trees

A Random Forest is an ensemble machine learning technique that constructs a multitude of

decision trees and combines their results for improved accuracy and reliability.

• Definition: A collection (ensemble) of multiple decision trees working together. Each

tree gives a prediction, and the final outcome is chosen by majority voting (classification)
or averaging (regression).

• Trains each tree on a unique random subset of the data, and for each split, randomizes
the subset of features considered, making each tree different.

• The approach is based on bagging (bootstrap aggregating): Each tree is built on a

bootstrap sample (random sample with replacement), and feature selection is
randomized at each node.

• The diversity among trees reduces overfitting and improves generalization.

Real-World Analogy (Vacation Decision by Friends)

Picture a group of friends (trees) making an important decision (which vacation spot to pick):
Each friend considers different factors (random subsets of data and features) and gives a
suggestion; the group then uses a majority vote (classification) or takes an average (regression)
to choose the best option. This makes the group decision more robust than any single person's
recommendation.

How Random Forest Works: Step-by-Step

1. Random Subset Training: For each decision tree, a random subset of data is picked (with
replacement), and at each split, only a random subset of features is considered.

2. Independent Tree Construction: Each decision tree is built independently using its
random data and features, learning unique patterns.

3. Prediction Aggregation: For new data:

• Classification: Each tree predicts a class; the majority class is the final prediction.

• Regression: Each tree predicts a value; the average value is the prediction.

4. Overfitting Reduction: Random sampling and aggregation prevent the model from fitting
the training data too closely, improving its accuracy on unseen data.
Assumptions of Random Forest

• Each decision tree is built independently from random samples/features for maximum
diversity.

• The model assumes there is enough data variety to allow diversity among trees.

• Aggregation (ensemble) improves accuracy by combining various independent solutions.

7. Ensemble Learning: Bagging vs. Boosting

Ensemble learning is a methodology that combines several models to produce a stronger

classifier.

• Bagging (Bootstrap Aggregating):

• Models are trained independently on random, unique subsets of the training

data.

• Final prediction is made by averaging (regression) or voting (classification) the

predictions from all models.

• Goal: Reduce variance in predictions (helps with models prone to overfitting, like
decision trees).

• Example: Random Forest uses bagging.

• Boosting:

• Models are trained sequentially, where each model tries to correct the errors of
the previous one.

• Later models focus more on previously misclassified data by earlier models.

• Goal: Reduce bias (by making the ensemble pay more attention to hard-to-
classify cases).

Bagging vs. Boosting: Key Differences

• Training: Bagging models are built independently; boosting models are built
sequentially.

• Focus: Bagging builds diverse models to reduce variance; boosting improves accuracy by
reducing bias and focusing on errors.
 • Interaction: In bagging, models do not interact; in boosting, each new model learns from
the mistakes of the one before.

Quick Review

• Decision tree structure: Root node → branches → internal nodes → leaf nodes.

• Classification vs. Regression trees: Classification = categorical target; regression =

continuous target.

• Decision process: Ask a series of feature-based questions, branching and splitting data
until every path reaches a leaf (outcome).

• Random forest: Multiple trees, random data/features, results combined

(majority/average), robust predictions.

• Bagging vs. Boosting: Bagging = parallel and independent, reduces variance (RF);
Boosting = sequential and dependent, reduces bias.

K-Nearest Neighbour (KNN) – Full Concept Guide

Definition & Core Principle

K-Nearest Neighbour (KNN) is one of the simplest supervised machine learning algorithms used
for both classification and regression problems. Its main principle is similarity-based
prediction: when a new data point arrives, KNN finds the most similar cases (the 'k' nearest
neighbours) from the stored dataset and predicts the label (category or value) for the new point
based on those neighbours.

• Similarity means: Data points that are close to each other in feature space typically have
similar outcomes. KNN will assign the new instance to the same group as its closest
neighbours.

• KNN is heavily used where pattern recognition and instance-based reasoning are
appropriate.

Key Terms

• Non-parametric: KNN makes no explicit assumptions about the underlying data

distribution (unlike, say, logistic regression).
 • Lazy Learner: KNN “learns” only at prediction time; it does not build a model during
training. Instead, it stores the entire dataset, deferring computation until new data must
be classified.

• Storage of Dataset: Every instance is stored; the full dataset is used for every prediction,
requiring significant memory for large datasets.

• Action at Classification Time: The heavy computation (distance calculations, finding

nearest neighbours) only occurs when a new query is made, not during a separate
training phase.

Business Example

Consider predicting subscription churn for a streaming service:

• New user arrives. Their age and activity data is known.

• KNN checks the ages and behaviours of all existing users. If most of the 'K' closest users
(in terms of age or behaviour) churned, it predicts the new user is also at risk of churn.

• Key idea: customers who are similar are likely to behave similarly.

The KNN Algorithm: Step-by-Step

1. Select the optimal value of K – The number of neighbours to use (see below).

2. Calculate Distances – For a new data point, compute the distance to every other point in
the stored dataset (see distance metrics below).

3. Find the K Nearest Neighbours – Sort the distances and pick the K closest data points.

4. Majority Voting (Classification): For classification tasks, KNN checks the most common
class among the K neighbours and assigns that label.

• Mean/Averaging (Regression): For regression, KNN predicts the average value of

the K neighbours.

Choosing K

• Effect on Outliers/Noise:

• A low K (e.g., K=1) is sensitive to noise and outliers; predictions may be unstable.
 • A higher K smooths predictions, reducing influence of any single unusual point,
but may also reduce how fine-grained the predictions are.

• Trade-off:

• Very small K: risk of overfitting (model is too sensitive to random noise).

• Very large K: risk of underfitting (model is too simple, misses important

boundaries).

Statistical Methods for Selecting K:

• Cross-Validation: Split data into folds, test different K values on each, and pick the K with
the best average performance.

• Elbow Method: Plot error rate or accuracy as you vary K. The 'elbow' point (where
improvement flattens out) suggests a good K.

• Odd Values: Especially in binary classification, odd K avoids tie situations when voting.

Distance Metrics in KNN

Distance defines the notion of “nearness” or “similarity.”

1. Euclidean Distance (for continuous data):

𝑛
𝑑𝑒𝑢𝑐𝑙𝑖𝑑 (𝑥, 𝑦) = √∑ (𝑥𝑖 − 𝑦𝑖 )2
𝑖=1

• The straight-line distance between two points in n-dimensional space.

• Used in applications like image recognition or gesture classification.

2. Manhattan Distance (for grid data):

𝑛
𝑑𝑚𝑎𝑛ℎ𝑎𝑡𝑡𝑎𝑛 (𝑥, 𝑦) = ∑ ∣ 𝑥𝑖 − 𝑦𝑖 ∣
𝑖=1

• The distance you’d walk along city blocks — only horizontal/vertical moves.

• Relevant for GPS navigation or robotics movement.

3. Minkowski Distance:

• Generalization that includes both Euclidean (p=2) and Manhattan (p=1).

• Formula: 𝑑𝑚𝑖𝑛𝑘𝑜𝑤𝑠𝑘𝑖 (𝑥, 𝑦) = (∑𝑛𝑖=1 ∣ 𝑥𝑖 − 𝑦𝑖 ∣𝑝 )1/𝑝

 • You can set 'p' as needed for domain-specific problems.

4. Hamming Distance (categorical/binary data):

• Counts the number of positions at which the corresponding values are different.

• Used in text analytics (e.g., typo/spelling detection), DNA sequence analysis,

error detection in comms (bit strings).

Algorithm Operation

• Classification tasks:

• Assigns a new data point to the class most common among its K nearest
neighbours (majority voting).

• Regression tasks:

• Predicts the mean value of the target variable among the K nearest neighbours.

• KNN’s “action phase” is always at the point of prediction—it stores the data
unprocessed until prediction is needed.

Key Points for Exams

• KNN is a memory-based, non-parametric, lazy learner suited for pattern-based, local

“vote”-driven classification/regression.

• Selection of K is crucial: low K = sensitivity to noise (overfitting); high K = smoothing, but

possible underfitting. Use cross-validation/elbow method.

• Know how to select/apply the correct distance metric for problem type and data.

• Application in business analytics: segmentation, churn prediction, recommendation

engines.

Naïve Bayes: Comprehensive Study Guide

1. Definition and Introduction

• Naïve Bayes classifiers are a family of simple, yet effective, supervised machine learning
algorithms designed for classification problems.
 • The algorithm is called "naïve" because it assumes all features in the data are
independent of each other—a strong and rarely true assumption, but it often works well
in practice.

• It is a probabilistic classifier: it models the probability that a sample belongs to a

particular class and chooses the class with the highest calculated probability.

• Naïve Bayes is favored for tasks like text classification due to its speed and efficiency,
even with high-dimensional datasets.

2. The Bayes Theorem Basis

• The core of Naïve Bayes is Bayes' Theorem, which relates the probability of a hypothesis
(class) given observed data (features) to the prior probability of the class and the
likelihood of observing the data given the class.
𝑃(𝑋 ∣ 𝐶) ⋅ 𝑃(𝐶)
𝑃(𝐶 ∣ 𝑋) =
𝑃(𝑋)

• 𝑃(𝐶 ∣ 𝑋) is the posterior probability (probability of class 𝐶 given features 𝑋).

• 𝑃(𝑋 ∣ 𝐶) is the likelihood (probability of features 𝑋 given class 𝐶).

• 𝑃(𝐶) is the prior probability (probability of class 𝐶 in the data).

• 𝑃(𝑋) is the total probability of features 𝑋 under all classes (constant for a given
observation).

• In practice, we compute 𝑃(𝐶 ∣ 𝑋) for each class and assign the class with the highest
probability.

3. Naïve Assumption — Feature Independence

• The algorithm assumes that every feature contributes to the classification

independently of the others, even though in reality features are often correlated.

• Example: For fruit classification, size, color, and shape are treated as contributing
independently to classifying 'apple', even though color and size might in reality
be related.

• This assumption allows the computation of the likelihood 𝑃(𝑋 ∣ 𝐶) as the product of
individual probabilities:

𝑃(𝑋 ∣ 𝐶) = 𝑃(𝑥1 ∣ 𝐶) × 𝑃(𝑥2 ∣ 𝐶) × … × 𝑃(𝑥𝑛 ∣ 𝐶)

 • The naïve aspect makes model building much faster and simpler, reducing computational
complexity and the number of parameters required for training.

4. How Probability Drives Prediction

• Probabilistic Reasoning: For any new instance, the classifier calculates the probability of
that instance belonging to each possible class.

• Using Bayes' Theorem and the independence assumption, it computes these

probabilities using previously observed (training) data.

• Prediction: The class with the highest posterior probability is assigned to the instance.
This means Naïve Bayes does not produce hard rules but rather scores for each class and
picks the best-scoring class.

5. Applications

• Spam Filtration: Filtering emails as "spam" or "not spam" based on word occurrences or
other features.

• Sentiment Analysis: Classifying text as positive, negative, or neutral sentiment by

analyzing word frequency and context.

• Article/News Classification: Assigning topics (business, sports, politics) to articles based

on the occurrence of specific words or phrases.

• Other examples: language detection, recommendation systems, and real-time

classification tasks.

6. Steps in Classification

A. Preparation:

1. Convert all features to a suitable format (categorical or continuous).

2. Calculate the prior probabilities for each class from training data.

3. For discrete features: calculate the frequency of each feature value for each class.

• For continuous features: often assume a normal (Gaussian) distribution within

each class and estimate means/variances.

B. Prediction:

1. For a new instance, calculate the probability of its feature values under each class,
assuming independence (multiply the probabilities for individual features together for
each class).
 2. Multiply by the prior probability of each class.

3. Assign the class with the highest resulting probability.

7. Probability Model for Class Assignment

• For a new data point 𝑋 = (𝑥1 , 𝑥2 , … , 𝑥𝑛 ), calculate for each class 𝐶𝑘 :

𝑛

𝑃(𝐶𝑘 ∣ 𝑋) ∝ 𝑃(𝐶𝑘 ) ∏ 𝑃(𝑥𝑖 ∣ 𝐶𝑘 )

𝑖=1

• Assign the class with the highest value of 𝑃(𝐶𝑘 ∣ 𝑋).

• If features are continuous, 𝑃(𝑥𝑖 ∣ 𝐶𝑘 ) is estimated using the Gaussian distribution

for each class.

• For discrete features, use frequency/counts from training data.

8. Evaluating Classifier Performance

• Accuracy: The proportion of correct predictions (true positives + true negatives) out of
all predictions.

• Confusion Matrix: Table layout for results —

• TP (True Positive): Classified as positive and is actually positive.

• TN (True Negative): Classified as negative and is actually negative.

• FP (False Positive): Classified as positive, but is actually negative.

• FN (False Negative): Classified as negative, but is actually positive.

• Precision: Accuracy for positive predictions:

𝑇𝑃
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =
𝑇𝑃 + 𝐹𝑃

• Recall (Sensitivity): Ability to find all positive examples:

𝑇𝑃
𝑅𝑒𝑐𝑎𝑙𝑙 =
𝑇𝑃 + 𝐹𝑁

• F1 Score: Harmonic mean of Precision and Recall:

 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 × 𝑅𝑒𝑐𝑎𝑙𝑙
𝐹1 = 2 ×
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑅𝑒𝑐𝑎𝑙𝑙

• ROC Curve & AUC:

• ROC curve plots True Positive Rate (Recall) vs. False Positive Rate (𝐹𝑃/(𝐹𝑃 +
𝑇𝑁)) at different thresholds. A curve closer to the top-left corner indicates better
performance.

• AUC (Area Under the ROC Curve): A higher value (closer to 1) means better
distinction between classes.

Quick Review and Tips

• Naïve Bayes is prized for its simplicity, speed, and solid performance, especially on text
data.

• Always check the independence assumption—while often violated, the algorithm can
still perform surprisingly well.

• For numerical data, converting to Gaussian/normal approximations inside classes is

standard.

• Pay attention to evaluation metrics beyond accuracy, especially in cases of imbalanced

datasets.

Theory and Evaluations of Classifiers: Complete Study Guide

Classification Metrics Overview

• Accuracy: Overall correctness, i.e., proportion of correctly predicted instances (both

positive and negative) out of total.

• Precision: Of all predicted positive instances, fraction which are correct. Important to
minimize false positives.

• Recall (Sensitivity or True Positive Rate): Of all actual positive instances, fraction
correctly identified. Important to minimize false negatives.

• F1 Score: Harmonic mean of precision and recall providing a balance metric:

𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 × 𝑅𝑒𝑐𝑎𝑙𝑙
𝐹1 = 2 ×
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑅𝑒𝑐𝑎𝑙𝑙
Confusion Matrix

Actual Positive Actual Negative

Predicted Positive True Positive (TP) False Positive (FP)

Predicted Negative False Negative (FN) True Negative (TN)

• TP: Correct positive prediction

• TN: Correct negative prediction

• FP: Incorrectly predicted positive

• FN: Missed positive case

Useful to calculate metrics above.

ROC Curve and AUC

• ROC curve plots True Positive Rate (TPR) vs False Positive Rate (FPR) across classification
thresholds.
𝑇𝑃 𝐹𝑃
TPR = ,FPR =
𝑇𝑃 + 𝐹𝑁 𝐹𝑃 + 𝑇𝑁
• Interpretation: Curve closer to top-left corner means better classification.

• Area Under Curve (AUC): Scalar summarizing ROC curve; 1.0 is perfect model; 0.5 means
random guessing.

Specificity and Sensitivity

• Sensitivity (Recall): Ability to detect actual positives.

• Specificity: Ability to detect actual negatives:

𝑇𝑁
𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 = = 1 − 𝐹𝑃𝑅
𝑇𝑁 + 𝐹𝑃
• Used together to evaluate classifier trade-offs.

Business Application Scenarios

• Risk prediction: Logistic regression/LDA predicting heart disease or loan default.

• Spam/email filtering: Decision trees, Naïve Bayes, logistic regression classify emails.
 • Subscription churn and segmentation: KNN predicts user behavior.

• Classification of wines: LDA applied to chemical attribute data for cultivar identification.

• Pattern recognition and market segmentation: Various classifiers used as necessary.

Practical evaluations often use confusion matrices and cross-validation to ensure generalization
across business datasets.

Visuals and Diagrams

• S-form curve (Sigmoid): Shows logistic function mapping inputs to probabilities between
0 and 1.

• Decision tree structure: Nodes and branches illustrating splits and final leaf outcomes.

• ROC curves and confusion matrix tables: Visual performance evaluation.

• Random forest: Illustrations of bootstrapped samples and ensemble voting.

• LDA outputs: Discriminant function graphs, canonical scatterplots, Box M plots from
SPSS.