LM06 Machine Learning 2026 Level II Notes

LM06 Machine Learning

1. Introduction ...........................................................................................................................................................3
1.1 Machine Learning and Investment Management............................................................................3
2. What is Machine Learning ................................................................................................................................3
2.1 Defining Machine Learning......................................................................................................................3
2.2 Supervised Learning ..................................................................................................................................4
2.3 Unsupervised Learning .............................................................................................................................4
2.4 Deep Learning and Reinforcement Learning....................................................................................5
2.5 Summary of ML Algorithms and How to Choose Among Them ................................................5
3. Evaluating ML Algorithm Performance ......................................................................................................6
3.1 Generalization and Overfitting ...............................................................................................................6
3.2 Errors and Overfitting ...............................................................................................................................7
3.3 Preventing Overfitting in Supervised Machine Learning.............................................................9
4. Supervised ML Algorithms: Penalized Regression .............................................................................. 10
4.1 Penalized Regression .............................................................................................................................. 11
5. Support Vector Machine ................................................................................................................................ 12
6. K-Nearest Neighbor ......................................................................................................................................... 13
7. Classification and Regression Tree ............................................................................................................ 13
8. Ensemble Learning and Random Forest.................................................................................................. 16
9. Unsupervised ML Algorithms and Principal Component Analysis ............................................... 19
9.1 Principal Components Analysis .......................................................................................................... 19
10. Clustering .......................................................................................................................................................... 21
11. K-means clustering ........................................................................................................................................ 21
12. Hierarchical clustering ................................................................................................................................. 22
13. Neural Networks, Deep Learning Nets and Reinforcement Learning ....................................... 25
13.1 Neural Networks .................................................................................................................................... 25
14. Deep Neural Networks ................................................................................................................................. 26
14.1 Reinforcement Learning ..................................................................................................................... 26
15. Choosing an Appropriate ML Algorithm ............................................................................................... 27
Summary................................................................................................................................................................... 29

Required disclaimer: IFT is a CFA Institute Prep Provider. Only CFA Institute Prep Providers are
permitted to make use of CFA Institute copyrighted materials which are the building blocks of the
exam. We are also required to create / use updated materials every year and this is validated by CFA
Institute. Our products and services substantially cover the relevant curriculum and exam and this is
validated by CFA Institute. In our advertising, any statement about the numbers of questions in our
products and services relates to unique, original, proprietary questions. CFA Institute Prep Providers

© IFT. All rights reserved 1

LM06 Machine Learning 2026 Level II Notes

are forbidden from including CFA Institute official mock exam questions or any questions other than
the end of reading questions within their products and services.
CFA Institute does not endorse, promote, review or warrant the accuracy or quality of the product and
services offered by IFT. CFA Institute®, CFA® and “Chartered Financial Analyst®” are trademarks
owned by CFA Institute.
Studying the curriculum issued by CFA Institute is essential to success. Prep Provider courses and
materials are developed to complement the curriculum and to facilitate the learning process, not to
replace it.
© Copyright CFA Institute

Version 1.0

© IFT. All rights reserved 2

LM06 Machine Learning 2026 Level II Notes

1. Introduction
Over the last few decades, the quantity of data that we have access to has gone up
considerably. At the same time, there has been a substantial increase in computing power.
This has led to a growth in technologies that allow us to evaluate data and extract actionable
insights from the data. One such technology is machine learning. This reading provides a
high-level view of machine learning (ML).
In this learning module we will cover:
• What is machine learning
• Overview of Evaluating ML algorithm performance
• Supervised machine learning algorithms
• Unsupervised machine learning algorithms
• Neural networks, deep learning nets, and reinforcement learning
1.1 Machine Learning and Investment Management
Machine learning has become increasingly important in the world of investment
management. In fact, machine learning can help at every step of the investment management
process such as:
• Understanding clients better
• Investment advice
• Security selection
• Optimal portfolio weights
• Trade execution
For example, chatbots can answer basic retirement savings questions, and learn from their
interactions with investors. Machine learning methods can help calculate target portfolio
weights that incorporate client restrictions.
2. What is Machine Learning
2.1 Defining Machine Learning
Machine learning is a set of computer-driven approaches aimed at generating structure or
predictions from data without human intervention by finding a pattern and then applying
the pattern.
The objective of machine learning is to extract knowledge from large amounts of data. A ML
algorithm evaluates data to come up with a structure which can then be used to make
predictions without human intervention.
ML programs have several advantages over other approaches:
• They do not rely on restrictive assumptions such as linear relationships, normally
distributed population data, etc.
• They can handle problems with high dimensionality (a large number of variables).

© IFT. All rights reserved 3

LM06 Machine Learning 2026 Level II Notes

• They can handle problems with high degree of non-linearity.

There are three classes of ML techniques:
1. Supervised learning
2. Unsupervised learning
3. Deep learning
We will cover these techniques over the next few sections.
2.2 Supervised Learning
Supervised ML algorithms infer patterns between a set of inputs (X’s) and desired output
(Y). It makes use of labelled training data, i.e., a set of inputs are matched with associated
outputs. Training the algorithm involves finding a pattern between the inputs and output.
This pattern can then be used to predict output values for new inputs (out of sample data).
An example of supervised learning is using ML algorithms to predict whether credit card
transactions are fraudulent or legitimate. The ML program is given training data, which may
consist of several hundred transactions of different amounts, different origins, etc. These
transactions are labelled as ‘fraudulent’ or ‘not fraudulent’. The ML program learns from this
labelled training data and can predict whether new transactions are fraudulent.
Machine learning terminology: In multiple regression, we call our X variables the
‘independent variables’ and the Y variable the ‘dependent variable’. In machine learning, the
Y variable is called the ‘target’ variable and the X variables are called ‘features’.
Supervised learning can be divided into two categories:
• Regression: making predictions of continuous target variables.
• Classification: sorting observations into distinct categories.
Learning from experience: Continuing with the credit card example. Let’s say that for the
first 1 million transactions the performance measure was 80%, which means that the
program was 80% accurate.
However, for the next 1 million transactions, when the experience level has reached a total of
2 million transactions, we observe that the performance has improved to 85%. This means
that the program has learnt through experience and improved its performance. This is an
example of machine learning.
2.3 Unsupervised Learning
Unsupervised machine learning does not make use of labeled data. Several input variables
are used for analysis but no output (or target variable) is provided. Because labeled data is
not provided, the ML program has to discover structure within the data on its own.
Unsupervised learning is useful for large complex data set which is hard to visualize. Two
important types of problems well suited to unsupervised ML are:
• Dimension reduction – reducing the number of features (X variables)

© IFT. All rights reserved 4

LM06 Machine Learning 2026 Level II Notes

• Clustering – sorting observations into groups.

2.4 Deep Learning and Reinforcement Learning
Deep learning refers to sophisticated algorithms which are used for highly complex tasks
such as:
• Image classification
• Face recognition
• Speech recognition
• Natural language processing
In reinforcement learning, a computer learns from interacting with itself.
Deep learning and reinforcement learning are based on neural networks. These algorithms
can be supervised or unsupervised and they work well for tasks characterized by non-
linearities and interactions among features.
2.5 Summary of ML Algorithms and How to Choose Among Them
Exhibit 2 of the curriculum provides a summary of various ML algorithms. These algorithms
are covered in detail in later sections.

Example: Machine Learning Overview

(This is Example 1 from the curriculum.)
1. Which of the following best describes machine learning? Machine learning:
A. is a type of computer algorithm.
B. is a set of computer-driven approaches aimed at generating structure or predictions from
data without human intervention by finding a pattern and then applying the pattern.

© IFT. All rights reserved 5

LM06 Machine Learning 2026 Level II Notes

C. is a set of computer-driven approaches adapted to extracting information from linear,

labeled data sets.
2. Which of the following statements is most accurate? When attempting to discover
groupings of data without any target (Y) variable:
A. an unsupervised ML algorithm is used.
B. an ML algorithm that is given labeled training data is used.
C. a supervised ML algorithm is used.
3. Which of the following statements concerning supervised learning best distinguishes it
from unsupervised learning? Supervised learning involves:
A. training on labeled data to infer a pattern-based prediction rule.
B. training on unlabeled data to infer a pattern-based prediction rule.
C. learning from unlabeled data by discovering underlying structure in the data themselves.
4. Which of the following best describes dimension reduction? Dimension reduction:
A. focuses on classifying observations in a data set into known groups using labeled training
data.
B. focuses on clustering observations in a data set into unknown groups using unlabeled
data.
C. focuses on reducing the number of features in a data set while retaining variation across
observations to preserve the information in that variation.
Solution to 1: B is correct.
Solution to 2: A is correct.
Solution to 3: A is correct.
Solution to 4: C is correct.
3. Evaluating ML Algorithm Performance
3.1 Generalization and Overfitting
When creating a model, the data set is typically divided into three non-overlapping samples:
1. Training sample: used to train the model.
2. Validation sample: used to validate and tune the model.
3. Test sample: used to test the model’s ability to predict well on new data.
The training sample is also called ‘in-sample’ data, while the validation and test samples are
also called ‘out-of-sample’ data.
Overfitting refers to an issue where the model fits training data perfectly but does not work
well with out-of-sample data.

© IFT. All rights reserved 6

LM06 Machine Learning 2026 Level II Notes

Generalization refers to the degree to which a model retains its explanatory power when
predicting out-of-sample. A model that generalizes well will perform well with both in-
sample and out-of-sample data.
The curriculum provides the analogy of a tailoring a custom suit to help understand the
concepts of underfitting, overfitting and good fitting.
• Overfitting: creating a custom suit that fits only one person and no one else.
• Underfitting: creating a baggy suit that fits no one.
• Good fitting: creating a universal suit that fits all similar people.
These concepts are further illustrated in Exhibit 3.

• Underfit: This model does not capture the relationships in the data well. It has
misclassified three circles and one triangle.
• Overfit: This model has no errors, but it also appears complex and may not work well
with new data.
• Good fit: This model is simple and also has just one error, a misclassified circle. It is
likely to generalize well with acceptable degrees of error.
Complexity refers to the number of features, terms, or branches in the model, and to
whether the model is linear or non-linear (non-linear is more complex). A model with higher
complexity has higher overfitting risk.
3.2 Errors and Overfitting
The total out-of-sample error can be decomposed into:
• Bias error: refers to the degree to which a model fits the training data. Underfitted
models have high bias errors.
• Variance error: refers to how much the model’s result change in response to new
data. Overfitted models have high variance errors.
• Base error: refers to errors due to randomness in the data.
To understand these errors, we look at learning curves (Exhibit 4)which plot accuracy rate
vs. training sample size.

© IFT. All rights reserved 7

LM06 Machine Learning 2026 Level II Notes

• Due to randomness in data, we cannot achieve a 100% accuracy rate. Therefore, the
desired accuracy rate (measured by base error) is slightly below 100%.
• A model is robust if out-of-sample accuracy increases as the training sample size
increases. This implies that for a robust model, the training accuracy rate and the
validation accuracy rate will converge towards each other at the desired accuracy
rate.
• Panel A shows an underfitted model with a high bias error. Here the training accuracy
rate and validation accuracy rate converge below the desired accuracy rate. Adding
more training sample will not improve this model.
• Panel B shows an overfitted model with a high variance error. Here the training
accuracy rate and validation accuracy rate fail to converge.
• Panel C shows a good fitted model with low bias as well as variance errors. Here the
training accuracy rate and the validation accuracy rate converge at the desired
accuracy rate.
Out-of-sample error rates are also a function of model complexity. As model complexity
increases:
• in-sample error rates (Ein) fall and bias error shrinks.
• out-of-sample error rates (Eout) rise and variance error rises.
This is depicted in a fitting curve (Exhibit 5) which shows in- and out-of-sample error rates
(Ein and Eout) on the y-axis plotted against model complexity on the x-axis.

© IFT. All rights reserved 8

LM06 Machine Learning 2026 Level II Notes

As shown in the figure, an optimal point of model complexity exists where the variance and
bias error curve intersect and the total error (Ein + Eout) is minimized.
3.3 Preventing Overfitting in Supervised Machine Learning
There are two methods to reduce overfitting:
• Preventing the algorithm from getting too complex: This is based on the principle that
the simplest solution often tends to be the correct one.
• Cross-validation: This is based on the principle of avoiding sampling bias. A
commonly used technique is k-fold cross-validation. Here the data is shuffled
randomly and then divided into k equal sub-samples, with k-1 samples used as
training samples and one sample, the kth, used as a validation sample. This technique
helps mitigate the holdout sample problem (reducing the training data set too much
to keep a large validation sample).
Example: Evaluating ML Algorithm Performance
(This is based on Example 2 from the curriculum.)
A portfolio manager runs a high-dividend yield fund for wealthy clients. She is interested in
classifying companies in the NIFTY 200 Index—an index of large- and mid-cap companies
listed on the National Stock Exchange of India—into two categories: dividend increase and
no dividend increase. She assembles data for training, validating, and testing an ML-based
model that consists of 1,000 observations of NIFTY 200 companies, each consisting of 25
features (fundamental and technical) and the labeled target (dividend increase or no
dividend increase).
After training her model, the portfolio manager discovers that while it is good at correctly
classifying using the training sample, it does not perform well using new data. In consulting
her colleagues about this issue, she hears conflicting explanations about what constitutes
good generalization in an ML model:

© IFT. All rights reserved 9

LM06 Machine Learning 2026 Level II Notes

Statement 1 The model retains its explanatory power when predicting using new data (i.e.,
out-of-sample).
Statement 2 The model shows low explanatory power after training using in-sample data
(i.e., training data).
Statement 3 The model loses its explanatory power when predicting using new data (i.e.,
out-of-sample).
1. Which statement made to the portfolio manager is most accurate?
A. Statement 1.
B. Statement 2.
C. Statement 3.
2. The model is most likely being impaired by which of the following:
A. Underfitting and bias error.
B. Overfitting and variance error.
C. Overfitting and bias error.
3. By implementing which one of the following actions can the portfolio manager address
the problem?
A. Estimate and incorporate into the model a penalty that decreases in size with the number
of included features.
B. Use the k-fold cross-validation technique to estimate the model’s out-of-sample error,
and then adjust the model accordingly.
C. Use an unsupervised learning model.
Solution to 1:
A (Statement 1) is correct.
Solution to 2:
B is correct. Anand’s model is good at correctly classifying using the training sample, but it
does not perform well using new data. The model is overfitted, so it has high variance error.
Solution to 3:
B is correct. A is incorrect because the penalty should increase in size with the number of
included features. C is incorrect because Anand is using labeled data for classification, and
unsupervised learning models do not use labeled data.
4. Supervised ML Algorithms: Penalized Regression
In supervised machine learning, we make use of labelled training data. Depending on the
nature of the target variable (Y), supervised ML algorithms can be divided into two types:
• Regression – the target variable is continuous.
• Classification – the target variable is categorical or ordinal.

© IFT. All rights reserved 10

LM06 Machine Learning 2026 Level II Notes

As shown in Exhibits 2, under regression, we will cover penalized regression and LASSO.
Under classification, we will cover support vector machine (SVM) and k-nearest neighbor
(KNN). Classification and regression tree (CART) as the name implies can be used for both
classification and regression problems. Finally, in ensemble learning and random forest we
will learn how to combine multiple algorithms.
4.1 Penalized Regression
Penalized regression is a computationally efficient technique used in prediction problems
where the target variable is continuous. Here the regression coefficients are chosen to
minimize sum of squared residuals plus a penalty term that increases with the number of
included variables. So, in a penalized regression, a feature must make a sufficient
contribution to model fit to offset the penalty from including it. Because of this penalty, the
model remains parsimonious and only the most important variables for explaining Y remain
in the model.
A popular type of penalized regression is LASSO (least absolute shrinkage and selection
operator). Under LASSO, the penalty term is expressed as:
K

Penalty term = λ ∑|b̂k |

k=1
In addition to minimizing the sum of squared residuals, LASSO also involves minimizing this
penalty term, i.e., it tries to minimize the following expression:
𝑛 K

∑(𝑌𝑖 − 𝑌̂𝑖 ) + λ ∑|b̂k |

2

𝑖=1 k=1
The penalty term increases as more features are added. Therefore, this method ensures that
a feature will be added only if SSE declines by more than the amount by which the penalty
term increases.
In the above expression, lambda (λ) is a hyperparameter – a parameter whose value must
be set by the researcher before learning begins. If λ is set to 0, then the second term is 0 and
the LASSO penalized regression is equivalent to an OLS (ordinary least squared) regression.
Penalized regression addresses the overfitting problem through a process called
regularization. Regularization refers to methods that reduce statistical variability. It helps
avoid complex models and reduces the risk of overfitting. These methods can also be applied
to non-linear models.
Penalized regression models are useful for:
• prediction problems where the target variable is continuous
• large data sets
• when the features are correlated
• reducing a large number of features to a manageable set

© IFT. All rights reserved 11

LM06 Machine Learning 2026 Level II Notes

5. Support Vector Machine

Support vector machine (SVM) is a linear classifier that aims to seek the optimal hyperplane
that separates the observations into two sets of data points. This is illustrated in Exhibit 6
and 7 from the curriculum.

• The left panel presents a simple data set with two features labeled in two groups –
triangles and crosses.
• These two regions can be separated by many straight lines. The middle panel shows
three such lines. These lines are called linear classifiers.
• An SVM algorithm selects a line that is furthest away from all the observations. In the
right panel we see that from the three line, the middle line is selected, because it
separates the data by the maximum margin.
• The margin is determined by the observations closest to the line in each set (circled
points). These observations are called support vectors.
• With two features we are working in a two-dimensional space and the linear
classifiers can be represented as straight lines. With n features we are working in a n-
dimensional space and the linear classifiers are represented by a n-dimensional
hyperplane.
In the above example, the data was perfectly linearly separable. However, many real-world
data sets are not perfectly linearly separable. There are two methods to deal with this
problem:
• Soft margin classification: A penalty term is added for observations that are
misclassified. The algorithm will select a linear classifier that optimizes the trade-off
between a wider margin and a lower total error penalty.
• Non-linear SVM algorithm: The algorithm will produce non-linear separation
boundaries that reduce the number of misclassifications, but will have more features,
thus adding to the model’s complexity.
SVM applications include:
• It can be used for classification, regression and outlier detection, but it is typically
used for classification problems.
• It is well suited for small-size to medium-size complex high-dimensional data sets.

© IFT. All rights reserved 12

LM06 Machine Learning 2026 Level II Notes

• For example, SVM can be used to predict company failures such as bankruptcies, or to
classify text from document into useful categories such as positive sentiment and
negative sentiment.
6. K-Nearest Neighbor
K-nearest neighbor (KNN) classifies a new observation by finding similarities (“nearness”)
between it and its k-nearest neighbors in the existing data set. This is illustrated in Exhibit 8
of the curriculum.

The diamond represents the new observation. It has to be classified as either a triangle or a
cross. If k=1, the algorithm will look at the nearest neighbor which is a triangle and classify
the new observation as a triangle. If k=5, the algorithm will look at the 5 nearest neighbors
which are 3 triangles and 2 crosses. It will again classify the new observation as a triangle.
The benefits of KNN are that it is a straightforward and intuitive method. It is powerful
because it is non-parametric and makes no assumptions about the distribution of the data. It
can be used directly for multi-class classification.
However, a major challenge of KNN is defining what similar or near means. Also, k, the
hyperparameter of the model, must be chosen carefully as different values of k can lead to
different results.
KNN is most often used for classification and sometimes for regression. KNN application
includes:
• corporate bond credit rating assignment
• bankruptcy prediction
• stock price prediction
• customized equity and bond index creation
7. Classification and Regression Tree
Classification and regression tree (CART) can be applied to predict a categorical variable or a
continuous target variable.

© IFT. All rights reserved 13

LM06 Machine Learning 2026 Level II Notes

To understand CART, consider a simple example where we are trying to classify companies
by whether or not they increase dividend payments. We are working with two features, X1:
Investment opportunities growth (IOG) and X2: Free cash flow growth (FCFG).
Such a classification will require a binary tree: a combination of an initial root node, decision
nodes, and terminal nodes. The root node and each decision node represent a single feature
(f) and a cutoff value (c) for that feature. The CART algorithm iteratively partitions the data
into sub-groups until terminal nodes are formed that contain the predicted label.

We now look at how the CART algorithm selects features and cutoff values for the features.
To do this, the algorithm makes use of labelled training data. In our hypothetical example,
there are 10 companies with dividend increase (the crosses) and 10 companies with no
dividend increase (the dashes).

© IFT. All rights reserved 14

LM06 Machine Learning 2026 Level II Notes

The algorithm selects the feature and cutoff value at each note that generates the widest
separation of labeled data to minimize classification error. It keeps splitting as long as the
classification error is decreasing.
If this is a classification problem, then the prediction of the algorithm at each terminal node
will be the category with the majority of data points. If this is a regression problem, then the
prediction at each terminal node will be the mean of the labeled values.
To prevent a CART algorithm from becoming too complex, we can use the following
regularization techniques:
• Use regularization parameters such as maximum depth of the tree, minimum
population at a node, maximum number of decision nodes.
• Pruning: remove sections of the tree that provide little classification power.
A major advantage of CART is that it can uncover complex non-linear dependencies between
features. As shown in Exhibit 10, high profitability is an important feature for predicting if a
stock is an attractive investment or a value trap. But this feature is only relevant if the stock
is cheap, leverage is high and sales are expanding. Multiple linear regression does not work
in situations where there are non-linear relationships between features.

© IFT. All rights reserved 15

LM06 Machine Learning 2026 Level II Notes

A CART tree provides a visual explanation for the prediction. It is not a ‘black box’ algorithm.
Some applications of CART include:
• Fraud detection in financial statements
• Generating consistent decision process in equity and fixed-income selection
• Simplifying communication of investment strategies to clients
8. Ensemble Learning and Random Forest
In ensemble learning, we combine predictions from a collection of models. This method
typically produces more accurate and more stable predictions than the best single model.
Different categories of ensemble learning include:
Majority-vote classifier:
• Here we use different algorithms and select the result with the most votes. It is also
called heterogeneous learning.
• For example, consider a project where we have used three algorithms – SVM, KNN
and CART. If the SVM and KNN models are both predicting the category “stock
outperformance” and the CART model is predicting the category “stock
underperformance,” then the majority-vote classifier will choose ‘“stock
outperformance.”
• This method is based on the fundamental idea that diversity is good.
• The method assumes that model predictions are independent of each other.

© IFT. All rights reserved 16

LM06 Machine Learning 2026 Level II Notes

Bootstrap aggregating (or bagging):

• Here we use a single algorithm but with different training data. It is also called
homogeneous learning.
• From the original training data set, we generate n new training data sets or bags of
data.
• We then train algorithms on independent data sets to generate n new models.
• This method protects against overfitting and helps generate stable predictions.
Random forest classifier:
• Here we create a collection of many decision trees generated by a bagging method or
by randomly reducing the number of features available during training.
• This method protects against overfitting on the training data and reduces the noise to
signal ratio because errors cancel out across slightly different classification trees.
• However, this is a black-box algorithm and lacks the interpretability of individual
trees.
• Some applications include: factor-based investment strategies, predicting whether an
IPO will be successful.
Example: Support Vector Machine and K-Nearest Neighbor
(This is Example 3 from the curriculum.)
Rachel Lee is a fixed-income portfolio manager analyst with Zeta Investment Management
Company. Zeta manages an investment-grade bond portfolio for small, conservative
institutions and a non-investment-grade (i.e., high-yield) bond portfolio for yield-seeking,
high-net-worth individuals. Both portfolios can hold unrated bonds if the characteristics of
the unrated bonds closely match those of the respective portfolio’s average holding.
Lee is discussing an upcoming straight, 10-year fixed coupon bond issue with senior credit
analyst Marc Watson. Watson comments that although the bond’s issuer, Biotron
Corporation, has not had this issue rated, his analysis of the company’s profitability, cash
flow, leverage, and coverage ratios places the issue near the borderline between low
investment-grade (Baa3/BBB–) and high non-investment-grade (Ba1/BB+) bonds.
Lee decides to use machine learning methods to confirm the implied credit rating of Biotron
Corporation.
1 State the type of problem being addressed by Lee.
2 State two ML algorithms that Lee could use to explore the implied credit rating of Biotron
Corporation, and then describe how each algorithm could be applied.
Lee decides to apply the two identified ML algorithms. Both algorithms clearly support a
high non-investment-grade rating. Watson states that because both ML algorithms agree on
the rating, he has confidence in relying on the rating.
3 State one argument in support of Watson’s viewpoint.

© IFT. All rights reserved 17

LM06 Machine Learning 2026 Level II Notes

Solution to 1:
Lee is addressing a supervised learning classification problem.
Solution to 2:
One suitable ML algorithm is the SVM. The SVM algorithm is a linear classifier that aims to
seek the optimal hyperplane—the one that separates observations into two distinct sets by
the maximum margin.
The KNN algorithm is also well suited for classification problems because it classifies a new
observation by finding similarities (or nearness) between the new observation and the
existing data.
Solution to 3:
Since both algorithms agree on the rating, the resulting classification will likely be correct.

Example: CART and Ensemble Learning

(This is Example 4 from the curriculum.)
Laurie Kim is a portfolio manager at Hilux LLC, a high-yield bond investment firm. The
economy has been in recession for several months, and high-yield bond prices have declined
precipitously as credit spreads have widened in response to the weak macroeconomic
environment. Kim, however, believes this is a good time to buy as she expects to profit as
credit spreads narrow and high-yield bond prices rise in anticipation of economic recovery.
Based on her analysis, Kim believes that corporate high-yield bonds in the credit quality
range of B/B2 to CCC/Caa2 are the most attractive. However, she must carefully select which
bonds to buy and which bonds to avoid because of the elevated default risk caused by the
currently weak economy.
To help with her bond selection, Kim turns to Hilux’s data analytics team. Kim has supplied
them with historical data consisting of 19 fundamental and 5 technical factors for several
thousand high-yield bond issuers and issues labeled to indicate default or no default. Kim
requests that the team develop an ML-based model that will make accurate classifications in
two categories: default and no default. Exploratory data analysis suggests considerable non-
linearities among the feature set.
1. State the type of problem being addressed by Kim.
2. Describe the dimensionality of the model that Kim requests her analytics team to
develop.
3. Evaluate whether a CART model is appropriate for addressing her problem.
4. Describe how a CART model operates at each node of the tree.
5. Describe how the team might avoid overfitting and improve the predictive power of a
CART model.

© IFT. All rights reserved 18

LM06 Machine Learning 2026 Level II Notes

6. Describe how ensemble learning might be used by the team to develop even better
predictions for Kim’s selection of corporate high-yield bonds.
Solution to 1:
Kim is addressing a classification problem
Solution to 2:
With 19 fundamental and 5 technical factors (i.e., the features) the dimensionality of the
model is 24.
Solution to 3:
The CART model is an available algorithm for addressing classification problems. Its ability
to handle complex, non-linear relationships makes it a good choice to address the modelling
problem at hand.
Solution to 4:
At each node in the decision tree, the algorithm will choose the feature and the cutoff value
for the selected feature that generates the widest separation of the labeled data to minimize
classification error.
Solution to 5:
The team can avoid overfitting and improve the predictive power of the CART model by
adding regularization parameters. Alternatively, a pruning technique can be used afterward
to remove parts of the CART model that provide little power to correctly classify instances
into default or no default categories.
Solution to 6:
The analytics team might use ensemble learning to combine the predictions from a collection
of models, where the average result of many predictions leads to a reduction in noise and
thus more accurate predictions.
9. Unsupervised ML Algorithms and Principal Component Analysis
In unsupervised machine learning, we do not use labeled training data; therefore, the
algorithms find patterns within the data themselves. The two main types of unsupervised ML
algorithms are:
• Dimension reduction, using principal components analysis
• Clustering, which includes k-means clustering and hierarchical clustering
9.1 Principal Components Analysis
In dimension reduction, we reduce the set of features to a manageable size while retaining
as much of the variation in data as possible.

© IFT. All rights reserved 19

LM06 Machine Learning 2026 Level II Notes

A popular method of dimension reduction is principal component analysis (PCA). PCA is

used to reduce highly correlated features into a few uncorrelated composite variables. A
composite variable is a variable that combines two or more variables that are statistically
strongly related to each other.
For example, consider a data set with 5 features: X1, X2, X3, X4 and X5. The features are
correlated to each other. Using PCA, these features can be reduced to 2 principal
components, PC1 and PC2, such that there is no correlation between PC1 and PC2. The first
principal component PC1 explains the largest proportion of variance in the data set, say 50%.
The next largest portion of variance is explained by PC2, say 40%.
PCA involves transforming the covariance matrix of the features and involves two key
concepts:
• Eigenvectors: mutually uncorrelated composite variables that are linear
combinations of the original features. PC1 and PC2 in the above example.
• Eigenvalues: proportion of total variance in the initial data that is explained by each
eigenvector. The values 50% and 40% in the above example.
The PCA algorithm finds PC1 such that the sum of projection errors for all data points is
minimized and sum of spread between all the data is maximized. This is illustrated in Exhibit
18. Note that PC1 and PC2 are at right angles, which means that they are uncorrelated.

Scree plots can help us decide how many PCs to keep. The lower the number of principal
components, the better the model is. Scree plots show the proportion of total variance in the
data explained by each principal component. A sample scree plot from the curriculum is
shown below.

© IFT. All rights reserved 20

LM06 Machine Learning 2026 Level II Notes

This plot shows that about 90% of the total variance is explained by the first three principal
components. Therefore, a model should be constructed using these three principal
components only.
The main drawback of PCA is that PCs are difficult to interpret, making it a ‘black box’
algorithm.
The main advantage of PCA is that by reducing the number of features to two or three PCs,
we can easily visualize the data in a 2D or 3D space. Dimension reduction is often performed
before training another supervised or unsupervised learning model.
10. Clustering
A cluster contains a subset of observations that are ‘similar’. Good clustering means that
observations inside each cluster are similar, i.e., close to each other (cohesion) and
observations in two different clusters are as far away from one another as possible
(separation). Exhibit 19 from the curriculum illustrates this concept.

Two popular clustering approaches are:

• K-means clustering
• Hierarchical clustering
11. K-means clustering
This algorithm repeatedly partitions observations into k non-overlapping clusters. The
number of clusters k, is a hyperparameter whose value must be set by the researcher before

© IFT. All rights reserved 21

LM06 Machine Learning 2026 Level II Notes

learning begins. Each cluster is characterized by its centroid and each observation is
assigned to the cluster with the centroid to which that observation is closest.
The k-means algorithm follows an iterative process illustrated in Exhibit 20.

These steps are repeated until no observation can be reassigned to a new cluster and we get
the final clusters.
A limitation of this method is that the hyperparameter k must be decided beforehand, which
can be difficult to do.
Applications of k-means clustering include:
• Deriving alternatives to static industry classifications.
• Data exploration for discovering patterns in highly dimensional data that may not be
otherwise obvious.
12. Hierarchical clustering
Hierarchical clustering algorithms create intermediate rounds of clusters in increasing or
decreasing size until a final clustering is reached.
Agglomerative clustering (or bottom-up) hierarchical clustering begins with each
observation being treated as its own cluster. Then, the algorithm finds the two closest
clusters defined by some measure of distance (similarity), and combines them into one new
larger cluster. This process is repeated iteratively until all observations are combined into a
single cluster.

© IFT. All rights reserved 22

LM06 Machine Learning 2026 Level II Notes

Divisive clustering (or top-down) hierarchical clustering starts with all the observations
belonging to a single cluster. This cluster is then progressively partitioned into smaller
clusters until each cluster contains only 1 observation.

Hierarchical clustering is more computationally intensive as compared to the k-means

clustering algorithm. However, the advantage of hierarchical clustering is that it allows the
analyst to examine alternative segmentations of data of different granularity before deciding
which one to use. Also, this method does not rely on a hyperparameter.
A dendrogram is a type of tree diagram that highlights the hierarchical relationships among
the clusters. Exhibit 22 shows a dendrogram representation of the clustering shown above.

© IFT. All rights reserved 23

LM06 Machine Learning 2026 Level II Notes

The x-axis shows the clusters, and the y-axis indicates some distance measure. Clusters are
represented by a horizontal line, the arch, which connects two vertical lines, called
dendrites. The height of each arch represents the distance between the two clusters being
considered.
General applications of clustering include:
• Portfolio diversification
• Uncovering important underlying structure in complex data sets
• Discovering patterns in high dimensional data
• Deriving alternatives to static industry classifications
Example: Investment Uses of Clustering Algorithms
(This is Example 6 from the curriculum.)
István Perényi is a portfolio manager of the Europe Diversified Equity Fund (“the Fund”)
within the Diversified Investment Management Company (DIMCO) fund family. The Fund is
benchmarked to the STOXX Europe 600 Index, which spans 17 countries, 19 industry
sectors, and 3 market capitalization groupings (large-, mid-, and small-cap).
Examining the Fund’s most recent performance, Perényi is concerned that the Fund’s
holdings, although approximately aligned with the STOXX Europe 600 Index’s country
weights, may have unrecognized risk biases and concentrations. Perényi requests Elsa Lund,
DIMCO’s chief risk officer, to investigate the Fund’s diversification. Lund asks her analysts
for ideas on how Perényi’s request can be addressed and receives three suggestions:

© IFT. All rights reserved 24

LM06 Machine Learning 2026 Level II Notes

Suggestion 1 Estimate the country, industry, and market-cap exposures of each Fund
holding, aggregate them, and compare the aggregate exposures to the benchmark’s
exposures. Then, examine mismatches for evidence of unexpected biases or concentrations.
Suggestion 2 Identify inherent groupings among fund holdings based on a broad set of eight
numerical (operating and financial) measures related to the holdings’ characteristics. Then,
examine the groupings for evidence of unexpected biases or concentrations.
Suggestion 3 Regress the return of the Fund on a set of country equity market indexes and
sector indexes based on the Fund’s benchmark. Then, examine the regression coefficients for
evidence of unexpected biases or concentrations.
Lund has several questions for analyst Greg Kane about using one or more clustering
machine learning algorithms in relation to addressing Perényi’s request.
Lund asks if any information needs to be specified for ML clustering algorithms no matter
which one is used. Kane replies that only the distance measure that the algorithm will use
and the hyperparameter, k, for k-means clustering need to be specified.
Lund further asks whether there would be an advantage to using k-means clustering as
opposed to hierarchical clustering. Kane replies that in his opinion, hierarchical clustering is
the more appropriate algorithm.
1. Which analyst suggestion is most likely to be implemented using machine learning?
A. Suggestion 1.
B. Suggestion 2.
C. Suggestion 3.
2. Kane’s reply to Lund’s first question is:
A. correct.
B. not correct, because other hyperparameters must also be specified.
C. not correct, because the feature set for describing the holdings measure must also be
specified.
3. In stating a preference for hierarchical clustering in his reply to Lund’s second question,
Kane most likely is giving consideration to:
A. the speed of the algorithms.
B. the dimensionality of the data set.
C. the need to specify the hyperparameter, k, in using a k-means algorithm.
Solution to 1:
B is correct. A machine learning clustering algorithm could be used to implement Suggestion
2.
Solution to 2:
C is correct.

© IFT. All rights reserved 25

LM06 Machine Learning 2026 Level II Notes

Solution to 3:
C is correct. The value of the hyperparameter, k, the number of distinct groups into which
the STOXX Europe 600 Index can be segmented, is not known. Using a hierarchical
algorithm, the sorting of observations into clusters will occur without any prior input on the
analyst’s part.
13. Neural Networks, Deep Learning Nets and Reinforcement Learning
Neural networks, deep learning nets and reinforcement learning represent sophisticated
algorithms which can address complex machine learning tasks involving:
• Non-linearities
• Interactions between a large number of features
Applications of these algorithms include:
• Image classification
• Face recognition
• Speech recognition
• Natural language processing
13.1 Neural Networks
Neural networks have layers of nodes connected by links. The three types of layers are:
• Input layer node nodes that correspond to features
• Hidden layer(s) that feed the output node
• Output node that generates the predicted value
This is illustrated in Exhibit 29 of the curriculum.

Each hidden node has two functional parts:

© IFT. All rights reserved 26

LM06 Machine Learning 2026 Level II Notes

• Summation operator which multiplies each input value by a weight and sums the
weighted values to form the total net input.
• Activation function which acts like a dimmer switch. It can increase or decrease the
strength of the input.
In neural networks, learning takes place in the hidden layer through improvements in the
weights applied to nodes. These improvements are undertaken with the aim of reducing the
total error.
14. Deep Neural Networks
Neural networks with many hidden layers (at least 3 but often more than 20 hidden layers)
are known as deep learning nets (DLNs). DLNs are the backbone of the AI revolution. They
are used in complex activities such as image, pattern, and speech recognition.
14.1 Reinforcement Learning
Reinforcement learning (RL) algorithm involves an agent that should perform actions that
will maximize its reward over time, taking into consideration the constraints of its
environment. A RL algorithm observes its environment, learns by testing new actions, and
reuses its pervious experiences. Learning occurs over time through millions of trials and
errors.
Example: Summing Up the Major Types of Machine Learning
(This is Example 10 from the curriculum.)
1. As used in supervised machine learning, regression problems involve:
A. binary target variables.
B. continuous target variables.
C. categorical target variables.
2. Which of the following best describes penalized regression? Penalized regression:
A. is unrelated to multiple linear regression.
B. involves a penalty term that is added to the predicted target variable.
C. is a category of general linear models used when the number of features and overfitting
are concerns.
3. CART is best described as:
A. an unsupervised ML algorithm.
B. a clustering algorithm based on decision trees.
C. a supervised ML algorithm that accounts for non-linear relationships among the features.
4. A neural network is best described as a technique for machine learning that is:
A. exactly modeled on the human nervous system.
B. based on layers of nodes connected by links when the relationships among the features
are usually non-linear.

© IFT. All rights reserved 27

LM06 Machine Learning 2026 Level II Notes

C. based on a tree structure of nodes when the relationships among the features are linear.
5. Hierarchical clustering is best described as a technique in which:
A. the grouping of observations is unsupervised.
B. features are grouped into a pre-specified number, k, of clusters.
C. observations are classified according to predetermined labels.
6. Dimension reduction techniques are best described as a means to reduce a set of
features:
A. to a manageable size without regard for the variation in the data.
B. to a manageable size while increasing the variation in the data.
C. to a manageable size while retaining as much of the variation in the data as possible.

Solution to 1: B is correct.

Solution to 2: C is correct.

Solution to 3: C is correct.

Solution to 4: B is correct.

Solution to 5: A is correct.

Solution to 6: C is correct.

15. Choosing an Appropriate ML Algorithm

Exhibit 37 of the curriculum shows a flowchart for choosing among the different machine
learning algorithms discussed so far.

© IFT. All rights reserved 28

LM06 Machine Learning 2026 Level II Notes

© IFT. All rights reserved 29

LM06 Machine Learning 2026 Level II Notes

Summary
LO: Describe supervised machine learning, unsupervised machine learning, and deep
learning.
Supervised machine learning makes use of labelled training data. It can be divided into two
categories:
• Regression: making predictions of continuous target variables.
• Classification: sorting observations into distinct categories.
Unsupervised machine learning does not make use of labelled training data. The ML program
has to discover structure within the data on its own. Two important types of problems well
suited to unsupervised ML are:
• Dimension reduction – reducing the number of features (X variables)
• Clustering – sorting observations into groups
Deep learning refers to sophisticated algorithms which are used for highly complex tasks. It
is based on neural networks. These algorithms can be supervised or unsupervised and they
work well for tasks characterized by non-linearities and interactions among features.
LO: Describe overfitting and identify methods of addressing it.
Overfitting refers to an issue where the model fits training data perfectly but does not work
well with out-of-sample data. Two methods used to address overfitting are:
• Preventing the algorithm from getting too complex.
• Cross-validation, a commonly used technique is k-fold cross validation.
LO: Describe supervised machine learning algorithms—including penalized
regression, support vector machine, k-nearest neighbor, classification and regression
tree, ensemble learning, and random forest—and determine the problems for which
they are best suited.
In penalized regression, the regression coefficients are chosen to minimize sum of squared
residuals plus a penalty term that increases with the number of included variables. A
popular type of penalized regression is LASSO.
Support vector machine (SVM) is a linear classifier that aims to seek the optimal hyperplane
that separates the observations into two sets of data points.
K-nearest neighbor (KNN) classifies a new observation by finding similarities (“nearness”)
between it and its k-nearest neighbors in the existing data set.
Classification and regression tree (CART) can be applied to predict a categorical variable or a
continuous target variable. A binary CART tree is a combination of an initial root node,
decision nodes, and terminal nodes. The root node and each decision node represent a single
feature (f) and a cutoff value (c) for that feature. The CART algorithm iteratively partitions
the data into sub-groups until terminal nodes are formed that contain the predicted label.

© IFT. All rights reserved 30

LM06 Machine Learning 2026 Level II Notes

In ensemble learning, we combine predictions from a collection of models. This method

typically produces more accurate and more stable predictions than the best single model.
A random forest classifier is a collection of many decision trees generated by a bagging
method or by randomly reducing the number of features available during training.
LO: Describe unsupervised machine learning algorithms—including principal
components analysis, k-means clustering, and hierarchical clustering—and determine
the problems for which they are best suited.
PCA is used to reduce highly correlated features into a few uncorrelated composite variables.
A composite variable is a variable that combines two or more variables that are statistically
strongly related to each other.
K-means algorithm repeatedly partitions observations into k non-overlapping clusters. The
number of clusters, k, is a hyperparameter whose value must be set by the researcher before
learning begins. Each cluster is characterized by its centroid and each observation is
assigned to the cluster with the centroid to which that observation is closest.
Hierarchical clustering algorithms create intermediate rounds of clusters in increasing or
decreasing size until a final clustering is reached. Agglomerative clustering (or bottom-up)
hierarchical clustering begins with each observation being treated as its own cluster.
Divisive clustering (or top-down) hierarchical clustering starts with all the observations
belonging to a single cluster.
LO: Describe neural networks, deep learning nets, and reinforcement learning.
Neural networks have layers of nodes connected by links. The three types of layers are: input
layer, hidden layer and output layer. Learning takes place in the hidden layer through
improvements in the weights applied to nodes.
Neural networks with many hidden layers (at least 3 but often more than 20 hidden layers)
are known as deep learning nets (DLNs).
Reinforcement learning (RL) algorithm involves an agent that should perform actions that
will maximize its reward over time, taking into consideration the constraints of its
environment.

© IFT. All rights reserved 31