RV Institute of Technology and

Management®

Machine Learning: Techniques in AI

Artificial Intelligence (AI) operates by combining massive data with fast, iterative processing and
intelligent algorithms, enabling software to automatically learn and deduce patterns in data. Building an AI
system involves reverse-engineering human traits and capabilities into a machine. AI is a broad field with
many theories, methods, and technologies, all processing large amounts of data to learn from patterns.
The sub-domains of AI include:
• Neural Networks: These mimic human brain cells, consisting of nodes (neurons) that process information
by responding to inputs and relaying information. Learning involves multiple passes at data to derive meaning.
A basic neural network includes an input layer, at least one hidden layer where algorithms process inputs, and
an output layer for the final result.
• Machine Learning (ML): A branch of computer science that analyses data and identifies patterns to teach a
machine to deduce results and make decisions without human intervention. ML algorithms learn from
experiences rather than instructions, automatically improving by learning from their output without explicit
programming.
• Deep Learning (DL): An ML technique that teaches a machine to process inputs through layers for accurate
classification, inference, and prediction. DL creates large neural networks with multiple processing units
(hidden layers) to learn complex patterns from vast data. Applications include image and speech recognition.
DL models use deep neural networks where temporary outputs are processed through hidden layers (forward
propagation). If results are unsatisfactory, errors are identified, weights are updated, and pushed back to
previous layers (backward propagation) to refine the model. Deep learning works with both labelled and
unlabelled data, supporting supervised and unsupervised learning.
• Natural Language Processing (NLP): A science that enables machines to read, understand, interpret, and
respond to human language, facilitating human-machine communication.
• Computer Vision (CV): A branch of AI focused on understanding images by breaking them into parts,
allowing machines to classify and learn from images to make decisions. CV is used in facial recognition,
autonomous vehicles, and medical diagnostics.
• Cognitive Computing: Algorithms that mimic the human brain by analysing text, speech, images, or objects
to produce desired outputs, aiming for natural, human-like interaction with machines.
Additional technologies supporting AI include:
• Graphical Processing Units (GPUs): Provide the heavy computing power needed for iterative processing
and training neural networks.
• Internet of Things (IoT): Generates massive amounts of data from connected devices, which AI models can
analyse to extract useful information and predict rare events or optimize complex systems.
Machine Learning Model
Professor Mitchell defined learning in computers as: "A computer program is said to learn from experience E
with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as
measured by P, improves with experience E".
Based on this definition, the Machine Learning Model consists of:
• Task (T): The real-world problem to be solved, such as predicting product sales, classifying spam emails,
classification, regression, clustering, or recognition.
• Experience (E): The knowledge gained from data provided to the algorithm, where the model iteratively runs
to learn inherent patterns. Machines learn from experience by analysing situations and relationships, similar to
humans. Supervised, unsupervised, and reinforcement learning are ways to gain experience.
• Performance (P): A measure indicating how well an ML algorithm performed task T using experience E. It is
analysed using metrics like accuracy, F1-score, confusion matrix, precision, recall, and sensitivity.
 RV Institute of Technology and
Management®

Regression Analysis in Machine Learning

Regression analysis is a statistical method used to study the relationship between a dependent (target)
variable and one or more independent (predictor) variables. It helps understand how the dependent
variable's value changes with respect to independent variables. The values predicted are typically continuous
or real, such as temperature, age, salary, or price. For example, it can predict rainfall using temperature and
other factors, determine market trends, or predict road accidents.
How it works:
• Regression analysis creates a mathematical equation that defines the dependent variable (y) as a function of
the predictor variables (x). This equation then predicts 'y' based on the 'x' values.
• Linear regression is the simplest and most widely used technique for continuous variable prediction,
assuming a linear relationship between outcome and predictors. It can work with both continuous and
categorical predictor variables.
• When the relationship is non-linear, non-linear regression models like polynomial and spline regression
are preferred.
• Different regression models can be applied and compared using statistical metrics to select the best one that
explains the data and predicts new test data.
Types of Regression:
• Linear regression: Simplest technique for predicting values where a linear relationship exists between
response and predictors.
• Polynomial regression: Used when the relationship between the outcome and predictor variables is not
linear.
• Stepwise regression: Builds predictive models by naturally adding or subtracting variables from a group of
descriptive variables at each step. This can involve forward selection, backward elimination, or bidirectional
elimination.
• ElasticNet regression: A blend of ridge and lasso regression, producing a grouping effect for highly
correlated predictors. It's used when there are many predictors relative to observations, often in SVM, metric
training, and document optimization.
K-fold cross-validation is a technique to assess a model's performance on unseen data by splitting the
dataset into 'k' subsets, using one as test data and the rest for training, and repeating this 'k' times to average
the prediction error.
Classification Techniques
Classification algorithms predict the probability that data will fall into one of predetermined categories.
1. K-Nearest Algorithm (k-NN):
◦ A non-parametric supervised learning algorithm where the output value of data is known, but the
method to get it is unknown.
◦ It stores all available cases and classifies new cases based on a majority vote of its 'k' nearest
neighbours.
◦ The term "non-parametric" means it makes no assumptions about the underlying data distribution, and
model parameters grow with the training data set.
◦ The value for a new instance is predicted by finding the 'k' most similar cases in the training set and
summarizing their output values.
◦ Choosing K Value: A large 'k' reduces noisy data but may ignore smaller, useful patterns. The accuracy
depends heavily on the 'k' value, with small 'k' leading to large variance and large 'k' to large model bias.
◦ Applications: Extensively used in the pharmaceutical industry to detect cancer cells or diseases.
◦ Disadvantages: Computationally expensive, sensitive to data scale (requires standardization),
 RV Institute of Technology and
Management®

performance deteriorates with multiple independent variables, works poorly if the target variable is skewed,
and accuracy depends on 'k' value.
2. Decision Trees:
◦ An intuitive and popular data mining technique that provides explicit rules for classification. It works well
with heterogeneous data and predicts the target value by mapping observations.
◦ Represents choices and their results in a tree structure, where nodes represent events or choices, and
edges represent decision rules or conditions.
◦ Basic Introduction: Used to model sequential decision problems.
◦ Structure: Composed of nodes and edges. Key terms include:
▪ Root node: The topmost node, representing the entire sample.
▪ Splitting: Dividing a node into two or more sub-nodes.
▪ Branch/Sub-tree: A subsection of the entire tree.
▪ Internal node: Nodes with incoming and outgoing edges.
▪ Leaf node (Terminal node): Nodes that predict the outcome and do not split further. Each terminal
node ideally consists of individuals of a single class.
▪ Parent node: A node that splits into sub-nodes.
◦ Advantages: Easy to implement and understand, results can be easily interpreted without statistical
knowledge, programmers can code the model, faster execution on new data, less data cleaning required, not
influenced by missing values, handles numerical and categorical variables, and employs non-parametric
methods.
3. Random Forests:
◦ A versatile machine learning technique for both regression and classification.
◦ Offers better performance than decision trees by de-correlating trees.
◦ Creates multiple decision trees on training datasets. Each split considers a random sample of 'm'
predictors (where m < p, p being the full set of predictors) from the full set, using only one of these 'm'
predictors.
◦ Disadvantages: Slow in generating predictions due to multiple tree constructions and voting, and more
difficult to interpret than a single decision tree model.
Clustering Techniques
Clustering is a set of techniques used to partition data into groups called clusters. Objects within the same
cluster are similar to each other, and dissimilar to objects in other clusters. It helps data scientists identify
meaningfulness (enhancing domain knowledge, e.g., grouping patients by treatment response) and
usefulness (serving as an intermediate step, e.g., customer segmentation for targeted ads) in data.
Clustering is often preferred over classification because it is adaptable to changes and identifies distinguishing
features between groups.
Overview of Clustering Techniques: The selection of an appropriate clustering algorithm depends on cluster
characteristics, dataset features, number of outliers, and data objects.
1. Partitional Clustering: Divides data objects into non-overlapping groups where each object belongs to
one and only one cluster, and each cluster has at least one object. Users can specify the number of clusters
('k'). These algorithms iteratively assign data points into 'k' clusters. Examples include k-means and k-
medoids. They are nondeterministic, producing different results in separate runs even with the same data.
2. Hierarchical Clustering: Specifies the hierarchy of clusters and depicts relationships between them.
3. Density-Based Clustering: Defines clusters based on the density of datasets.
K-Means Algorithm:
• An unsupervised learning technique for finding subgroups of observations in a dataset. Observations in
the same group are similar, and those in different groups are dissimilar. It finds relationships between 'n'
 RV Institute of Technology and
Management®

observations without being trained by a response variable.

• How it Works:
1. Specify the number of clusters ('k') to be generated.
2. Randomly select 'k' observations (centroids) as initial cluster centres.
3. Assign every remaining observation to its closest cluster, calculated using Euclidean distance between
the cluster's centroid and the observation (cluster assignment step).
4. Centroids are recalculated as the mean of all observations assigned to that cluster.
5. Steps 3 and 4 are repeated until cluster assignments no longer change or a maximum number of
iterations is reached.
• Applications: Customer segmentation, document clustering, image segmentation, and recommendation
engines.
• Pros: Simple and fast, efficiently handles very large datasets.
• Cons: Number of clusters must be pre-specified, sensitive to outliers, changing data order gives different
results.
Density-Based Spatial Clustering of Applications with Noise (DBSCAN):
• An unsupervised learning technique that identifies distinct clusters as contiguous regions of high point
density, separated by low-density regions.
• Discovers clusters of various shapes and sizes from large datasets, handling outlier values.
• Parameters:
◦ minPoint (minimum number of points)
◦ ε (epsilon, radius)
• Types of Points:
◦ Core: A point with at least minPoint points within distance ε from itself.
◦ Border: A point with at least one Core point at distance ε.
◦ Noise: A point not falling into the above categories, meaning it has fewer than minPoint points within
distance ε.
• Algorithmic Steps:
1. Arbitrarily pick a point.
2. If there are at least minPoint points within radius ε, they form part of the same cluster.
3. Expand clusters by recursively computing the neighbourhood for each point.
4. Repeat until all points are visited.
Naïve Bayes Classification
Naïve Bayes is a probabilistic machine learning algorithm based on the Bayes Theorem. It is simple yet
powerful, often outperforming complex algorithms for very large datasets.
• The term "naïve" implies that features used in the model are assumed to be independent of each other; a
change in one feature does not affect others.
• Conditional Probability: The possibility of an event based on the existence of a previous event.
• Bayes' Theorem: Determines the probability of an event with uncertain knowledge, relating conditional and
marginal probabilities of two random events. It updates an event's probability prediction when new information
is added.
• Applications:
◦ Text classification: Classifying text documents into categories.
◦ Spam filtration: Distinguishing spam from legitimate emails (e.g., in Gmail, DSPAM, SpamBayes).
◦ Sentiment analysis: Analysing tweets, comments, and reviews to classify sentiment (negative, positive,
or neutral).
 RV Institute of Technology and
Management®

◦ Recommendation Systems: Predicting user preference or purchase likelihood, often combined with
collaborative filtering.
• Limitations: With a 0 frequency for a class and feature, conditional probability becomes 0 (Zero Conditional
Probability Problem), which can be avoided using Laplace Correction.
Neural Network (NN)
A Neural Network (NN), or Artificial Neural Network (ANN), is a machine learning algorithm inspired by the
biological neuron system, learning by examples. It consists of many highly interconnected processing
elements called neurons. It processes information in parallel and can change its internal structure by adjusting
input weights. NNs were developed to solve problems easy for humans but difficult for machines, such as
pattern recognition (e.g., optical character recognition, object detection).
• Working Principle:
◦ Similar to how human brains process information: dendrites receive signals, and axons transmit outputs.
NNs use connected input/output units, where each connection has an associated weight.
◦ In the learning phase, the network adjusts weights to predict the correct class label for given inputs.
◦ NNs take several inputs, process them through multiple neurons in multiple hidden layers (forward
propagation), and return the final result via an output layer.
◦ Inputs and Weights: Inputs (x1, x2, x3) are multiplied by their respective weights (w1, w2, w3), which
indicate the importance of an input.
◦ Bias: Each perceptron (neuron) has a bias indicating its flexibility.
• Activation Function: Takes the sum of weighted inputs and bias as an argument and returns the neuron's
output. It makes a non-linear transformation, allowing for fitting non-linear hypotheses and estimating complex
functions (e.g., Sigmoid, Tanh, ReLu).
• Gradient Descent: An algorithm used to update the weights of neurons. Variants include Full Batch
Gradient Descent (uses all training data) and Stochastic Gradient Descent (SGD, uses one or more data
points, not the entire set).
• Relationship with Deep Learning: Deep learning is based on deep neural networks (NNs with multiple
hidden layers) and is a subset of machine learning, which is a sub-domain of AI. Deep learning flourishes with
large amounts of diverse, unstructured, and interconnected data, and strong computing power.
• Pros: Very fast due to parallel calculations.
• Cons:
◦ Require more development time and computational power.
◦ Need more data than other ML algorithms.
◦ Can only be performed on numerical inputs and non-missing value datasets.
◦ "Black box" models, difficult to understand due to complexity.
• Applications:
◦ Pattern recognition: Facial recognition, object detection, fingerprint recognition, anomaly detection.
◦ Time series prediction: Stock prices, weather forecasting.
◦ Natural language processing: Text classification, Named Entity Recognition (NER), Speech
Recognition, Spell Checking.
◦ Computer Vision: Classifying objects in images, driverless cars, medical image analysis (cancer
detection), identifying objects from satellite imagery.
◦ Industrial automation: Improving worker safety.
◦ Digital assistants: Siri, Cortana, Alexa, Google Now for natural language processing and speech
recognition.
◦ Language translation: Skype and Google Translate.
◦ Spam detection: Email systems like Gmail.
 RV Institute of Technology and
Management®

◦ Fraud detection: PayPal.

◦ Image analysis: CamFind for mobile visual search, image colourisation.
◦ Customer service: Chatbots and service bots.
◦ Personalized shopping and entertainment.
Support Vector Machine (SVM)
A Support Vector Machine (SVM) is a supervised machine learning algorithm primarily used for
classification, though it can also be applied to regression challenges. It classifies data based on its features.
• How it Works:
◦ Input data is supplied, and the SVM algorithm automatically extracts features. This knowledge is used to
segregate and classify data to generate desired output.
◦ Each data item is plotted as a point in n-dimensional space (where 'n' is the number of features). The
coordinates of individual observations are called support vectors.
◦ For classification, the goal is to identify the best hyperplane/line or class that clearly separates the data
points into two classes.
◦ Margin: The distance between the hyperplane and the closest data point. Maximizing this margin leads to
better classification.
◦ Handling Non-linear Data: SVM can handle cases where linear separation is not possible by adding new
features (e.g., z=x^2+y^2) or using a kernel function. The kernel function transforms data into a higher-
dimensional feature space to enable linear separation (e.g., Gaussian RBF, Sigmoid, Polynomial kernels).
• Tuning Parameters: Can be divided into linear kernel SVM and non-linear kernel SVM.
• Advantages:
◦ Efficient for text and image classification.
◦ Free from multicollinearity problems.
◦ Memory efficient, as only a subset of training points (support vectors) is needed for decision-making.
◦ Flexible, classifying both linear and non-linear data.
• Disadvantages:
◦ Takes more time with large datasets.
◦ Does not directly return probability estimates.
◦ With linear separable data, a linear kernel is similar to logistic regression.
• Applications in Real World:
◦ Face Detection: Classifying image parts as faces or non-faces (e.g., Facebook photo tagging). Can also
classify emotions.
◦ Text and Hypertext Categorization: Differentiating and classifying documents (news articles, emails,
web pages).
◦ Bioinformatics: Classifying proteins, cancer, genes, and other biological problems.
◦ Handwriting Recognition: Recognizing handwritten characters for data entry and signature validation.
◦ Geological Structure Analysis: Determining layered planetary structures.
◦ Seismic Liquefaction Potential: Checking for liquefaction occurrence using SPT and CPT data.
◦ Mathematics: Smooth SVM (SSVM) variant used for various math problems.
◦ Texture Classification: Classifying images as smooth or rough by analysing texture.
◦ Speech Recognition: Applied to acoustic data for speech recognition interfaces.
◦ Steganography Detection: Finding hidden messages in digital images for security