AI for Management
Machine Learning
Saji K Mathew, PhD
Professor, Management Studies
INDIAN INSTITUTE OF TECHNOLOGY MADRAS
Learning algorithms
If a machine could engage in conversation
indistinguishable from a human, it could be considered
intelligent—Alan Turing
Turing test was designed to provide a practical method
for evaluating machine intelligence
Explanatory vs predictive modeling
Machine Learning (ML)
In supervised learning, for each observation of the
predictor measurement(s) xi, i = 1, . . . , n there is an
associated response measurement yi. It uses labelled
data
Eg.: regression,
In unsupervised learning, for every observation i = 1, .
. . , n, we observe a vector of measurements xi but no
associated response yi.
Eg.: clustering
Statistical vs algorithmic
Two different knowledge communities
Black box vs white box models
Model interpretability and explainability
Model building
Suppose that we observe a quantitative response Y and p
different predictors, X1,X2, . . .,Xp. We assume that there is
some relationship between Y and X = (X1,X2, . . .,Xp), which
can be written in the very general form
Y = f(X) +
Statistical learning refers to a set of approaches for
estimating f
Training: Residual Sum of Squares,
Choice of models: prediction error-
flexibility (bias-variance) trade off
LHS: Different fits (linear regression (orange)
RHS: Red Line: Test MSE, Grey line: Training MSE
Cognitive neuroscience: Understanding
the brain
Deals with mental abilities and brain functioning
Invention of microscope, and neurons thereafter
Neurons as fundamental elements for brain’s information
processing
Neuron-neuron communication through release of
neurotransmitters (chemicals) into junctions between
neurons (synapse)
An average brain has ~100bn neurons, each one
connected to 15,000 other neurons
When one neuron fires another, it becomes a Hebbian
circuit (neurons that fire together, wire together)
the concept of Hebbian learning (1949)
The neuron
Artificial neural networks are inspired by the architecture of
the human brain.
The idea of artificial neuron
[Link]
to-redeem-the-world-with-logic-
235253/
The Perceptron (Rosenblatt, 1957)
Towards back propagation
Perceptron training may not converge if the linear unit
is not linearly separable
Marvin Minsky vs Frank Rosenblatt debate
([Link]
debates/minsky-vs-rosenblatt
Solution: Back propagation algorithm, a method for
training neural nets devised by researchers in the 60s,
was revisited by Rumelhart, Hinton, and Williams;
they published a paper, which outlined a clear and
concise formulation for the technique and it paved its
way into the mainstream of machine learning research
(Rumelhart et al. 1986).
BP algorithm
is half the squared difference between the target output td and the linear
unit output od, summed over all training tuples.
Here E is a function of weight vector, it is dependent on the linear unit output
(od).
Gradient descent search determines
a weight vector (partial differentials)
that minimizes E
Starts with an initial weight vector and
modifies it in steps
At each step the weight vector is altered
In the direction that produces steepest
descent along the error surface
A case: feedforward ANN training for
prediction
Target
Compare
Feed-Forward Neural N/w topologies
ANN learning models
Feedforward, back propagation
Limitation: vanishing gradient problem, 2-3 hidden layers
Applications: forecasting, classification
Convolutional Neural Networks (CNN)
Convolution”—working and reworking the original input, by
pooling data across layers; can have thousands of layers,
computationally expensive
Applications: image recognition
Recurrent Neural Networks (RNN)
Prior inputs to influence the current input and output
(feedback), effective in speech recognition and timeseries’
reinforcement learning takes human/machine feedback
Deep learning with transformer architecture
State of the art, thousands of layers, used in GenAI
[Link]
The cusp: Large Language Models (LLM)
Foundation of Generative AI, which generates new content
ChatGPT, DALL-E
Stochastic parrots? (lacks in connection, composition)
Hallucination/confabulation
Evolution of LLM
Bag of words to semantic meaning through Word2Vec
(Mikolov et al. 2013)
Transformer architecture and deep learning (“Attention is all
you need”: Vaswani et al. 2017; Wei et al. 2022)
Overcomes the limitation of BP algorithm for ANN training
Word embedding
Distance between words based on semantics
The transformer
[Link]
[Link]
LLM gallop