Machine Learning Algorithms for

Big Data Analytics

Manjunath G.S.
Asst. Professor
Dept. of ISE, BNMIT
Introduction
 Analytics uses the mathematical equations, formulae, and models.
 Analytics also uses AI, ML, and DL to predict the behavior of entities, objects
and events.
 AI refers to the science and engineering – reduces human tasks.
 Two concepts of AI are:
 ML-Machine Learning
 DL-Deep Learning
Contd,…
 Machine Learning: It is a field of computer science based on AI which deals
with learning from data in three phases
 Collect
 Analyze
 Predict
 Advanced ML techniques use unsupervised, semi-supervised or supervised
learning.
Contd,…
 Deep Learning: It is a machine learning technique that teaches computers to
do what comes naturally to humans. It refers to structured learning or
hierarchical learning.
 DL methods are advanced methods, such as artificial neural networks (ANN)
or neural nets, deep neural networks, deep belief networks, and recurrent
neural networks.
 Estimating the Relationships,
Outliers, Variances, Probability
Distributions and Correlations
1. Introduction
There are different methods to study about the relationship using variables.
Types of variables are:
 Independent variable
 Predictor variable
 Outcome variable
 Explanatory variable
 Response variable
 Feature variable
 Categorial variable
2. Relationships - Using Graphs, Scatter Plots, and Charts
1. Linear and Non-linear Relationships
 A linear relationship exists between two variables x and y.
2. Relationships - Using Graphs, Scatter Plots, and Charts
2. Estimating the Relationships
 Estimating the relationships means finding a mathematical expression, which
gives the value of the variable according to its relationships with other
variables.
 Predictions are up to a certain degree of certainty.
2. Relationships - Using Graphs, Scatter Plots, and Charts
3. Outliers
 Outliers are data, which appear as they do not belong to the dataset.
 Outliers are data points that are numerically far distant from the rest of the
points in the dataset.
 There are several reasons for the presence of outliers in the relationships.
 Anomalous situation
 Presence of a previously unknown fact
 Human error
 Participants intentionally reporting incorrect data
 Sampling error
2. Relationships - Using Graphs, Scatter Plots, and Charts
4. Variance
 A random variable is a variable whose possible values are outcomes of a
random phenomenon.
 A random variable is a function that maps the outcomes of unpredictable
processes to numerical quantities.
 A random variable is also called stochastic variable or random quality.
 Variance measures by the sum of squares of the difference in values of a
variable with respect to the expected value.
 A variance is also a measure of dispersion with respect to expected value.
 No variance is sometimes hard to understand in real datasets.
2. Relationships - Using Graphs, Scatter Plots, and Charts
5. Probabilistic Distribution of Variables, Items or Entities
 The probability distribution for a random variable describes how the
probabilities are distributed over the values of the random variable.
 For a discrete random variable, x, the probability distribution is defined by
a probability mass function, denoted by f(x).
 This function provides the probability for each value of the random variable.
 In the development of the probability function for a discrete random variable,
two conditions must be satisfied:
 f(x) must be nonnegative for each value of the random variable.
 The sum of the probabilities for each value of the random variable must equal one.
2. Relationships - Using Graphs, Scatter Plots, and Charts
5. Probabilistic Distribution of Variables, Items or Entities
 The probability distribution is the distribution of P values as a function of all
possible independent values, variables, situations, distances or variables.

 Kernel Functions
 Moments
 Unequal Variance
 Analysis of Variance (ANOVA)
 No Relationship Case
2. Relationships - Using Graphs, Scatter Plots, and Charts
6. Correlation
 Correlation means analysis which lets us find the association or the absence
of the relationship between two variables x, and y.
 Correlation is a statistical term describing the degree to which two variables
move in coordination with one another.
 If the two variables move in the same direction, then those variables are said
to have a positive correlation. If they move in opposite directions, then they
have a negative correlation.
 R-Square
 Interpretation of R-squared
3. Regression Analysis
 Correlation and regression are two analyses based on multivariate
distribution.
 It is a set of statistical processes for estimating the relationships between a
dependent variable (often called the 'outcome' or 'response' variable) and
one or more independent variables.
 It majorly helps in finding how a dependent variable changes when variation
in an independent variable.
 It uses both linear and non-linear regression models.
 Non-linear regression equation is as follows:
y = a0 + a1x + a2x2 + a3x3
3. Regression Analysis
1. Simple Linear Regression
 Linear regression is a simple and widely used algorithm.
 It is a supervised ML algorithm for predictive analysis.
 It models a relationship between the independent predictor or explanatory,
and the dependent outcome or variable, y using a linearity equation.
 Simple linear regression is performed when the requirement is prediction of
values of one variable, with given values of another variable.
3. Regression Analysis
2. Least Square Estimation
 Assume n data-points, i = 1, 2, ….….. , n.
 If your data shows a linear relationship between the X and Y variables, you
will want to find the line that best fits that relationship. That line is called a
Regression Line and has the equation ŷ= a + b x.
 The Least Squares Regression Line is the line that makes the vertical distance
from the data points to the regression line as small as possible.
 It’s called a “least squares” because the best line of fit is one that minimizes
the variance (the sum of squares of the errors).
3. Regression Analysis
3. Multiple Regressions
 Multiple regression is a statistical technique that can be used to analyze the
relationship between a single dependent variable and several independent
variables.
 The objective of multiple regression analysis is to use the independent
variables whose values are known to predict the value of the single dependent
value.
 Each predictor value is weighed, the weights denoting their relative
contribution to the overall prediction.
 It is often referred as regression analysis, examines the effects of multiple
independent variables on the value of a dependent variable or outcome.
3. Regression Analysis
4. Modelling Possibilities using Regression
 Regression ranges from simple models to highly complex equations. Two
primary uses for regression are forecasting and optimization.
 Examples:
 Sales data
 Investing funds
 Business promotion
 Insurance / Financial company
 Characteristics of a child
 Prices of houses, Higher bioactivity
3. Regression Analysis
5. Predictions using Regression Analysis
 Regression analysis is a powerful technique used for predicting the unknown
value of a variable from the known value of another variable.
 One or more variables whose values are hypothesized are called independent
variables.
 The prediction for the dependent variable can be made by accurate selection
of independent variables to estimate a dependent variable.
 Two steps for predicting the dependent variable:
 Estimation step
 Prediction step
3. Regression Analysis
6. K-Nearest-Neighbour Regression Analysis
 K nearest neighbors is a simple algorithm that stores all available cases and
predict the numerical target based on a similarity measure (ex: distance
functions).
 KNN has been used in statistical estimation and pattern recognition already
in the beginning of 1970’s as a non-parametric technique.
 KNN algorithm has easily been the simplest to pick up. Despite its simplicity, it
has proven to be incredibly effective at certain tasks.
 It can be used for both classification and regression problems! KNN algorithm
is by far more popularly used for classification problems, however.
3. Regression Analysis Contd
6. K-Nearest-Neighbour Regression Analysis Contd,…

To Understand KNN we follow these methods:

 A simple example to understand the intuition behind KNN algorithm
 How does the KNN algorithm work?
 Methods of calculating the distance between points
 How to choose the k factor?
 Working on a dataset
4. Finding Similar Items, Sets and
Collaborative Filtering
1. Finding Similar Items
An analysis requires may items to find similar items:
 Application of Near Neighbour Search
 Similar items can be found using NNS.
 NSS algorithm is as follows: consider a set $ having points in a space M. the
queried point q € M, which means q is member of M.
4. Finding Similar Items, Sets and
Collaborative Filtering
2. Jaccard Similarity of Sets
Let A and B be two sets. Jaccard similarity coefficient of two sets measures using
notations in set theory:
4. Finding Similar Items, Sets and
Collaborative Filtering
2.1. Similarity of Documents
An application of Jaccard similarity coefficient is in NLP and TP:
 Find bag of words and remove words such as is, are, does, at, in
 Assign weighting factor is the Term Frequency and Inverse Document
Frequency (TF-IDF).
 Find k-shingles.
 Find n-grams.

Many other methods also exist for computing similarity of documents.

4. Finding Similar Items, Sets and
Collaborative Filtering
3. Collaborative Filtering
 Collaborative filtering refers to a filtering algorithm, which filters the items
sets that have similarities with different items in a dataset.
 CF finds the sets with items having the same or close similarity coefficients.

 Following are some examples of applications of CF:

 Find the set of students opted for an elective
 Find the set who showed excellent performance in a subject

 Algorithm finds the similarities between the sets of the CF.

4. Finding Similar Items, Sets and
Collaborative Filtering
4. Distance Measures for Finding Similar Items or Users
 Distance can be measured in number of ways.
 Distance measures compute the dissimilarities. Complement of dissimilarity
gives similarity.
 Distances between all members in a set of points can be computed in metrics
space using a mathematical equation.
 Distance can also be defined as dissimilarity coefficient in v-dimensional
space.
 An algorithm computes the distances or any other distance metric.
5. Frequent Itemsets and Association
Rule Mining
1. Frequent Itemset Mining
 Data mining
 Frequent itemset mining
 Frequent subsequence
 Frequent substructure
 The analysis results in:
 Improvement of arrangement of products
 Marketing and sales promotion
 Planning of products
 Support cross-selling and product bundling
5. Frequent Itemsets and Association
Rule Mining
2. Association Rule
 An important method of data mining is association rule mining or association
analysis.
 The objective is to find uncovered relationships using some strong rules.
 The method analyzes the items in a group and then identifies which items
typically appear together.
5. Frequent Itemsets and Association
Rule Mining
3. Apriori Algorithm
 Apriori algorithm simply follows a basis that any subset of a large itemset
must be a large itemset.
 The apriori principle can reduce the number of itemsets needed to be
examined.
 Example: {A, B, C} is a frequent itemset, then all of its subset {A}, {B}, {C}, {A,
B}, {A, C}, and {B, C} must be frequent.
 The frequent itemset algorithm uses candidate generation process.
 This algorithm uses BFS method and hash tree to count candidate itemsets.
5. Frequent Itemsets and Association
Rule Mining
4. Evaluation of Candidate Rules
 Candidate itemsets are generated using only large itemsets of the previous
iteration.
 The large itemset of the previous iteration is joined with itself to generate all
itemsets having size higher by 1.
 Each generated itemset that does not have a large subset is discarded.
5. Frequent Itemsets and Association
Rule Mining
5. Applications of Association Rules
5.1. Market Basket Model
 Market basket analysis is a tool for knowledge discovery about co-occurrence
of items.
 If people tend to buy two products (say A and B) together, then the buyer of
the product A is a potential customer for an advertisement of product B.
 A set of items that is present in many baskets is termed as a frequent itemset.
 Market basket analysis can be applied to many areas.
5. Frequent Itemsets and Association
Rule Mining
5.1. Market Basket Model (Applications)
 Medical analytics
 Web usage analytics
 Fraud detection and technical dependence analysis
 Click stream analysis
 Telecommunication services analysis
 Plagiarism detection
Text, Web Content, Link, and Social
Network Analysis
Introduction
 Text analysis is often termed as “text mining”.
 It refers to analyzing and extracting the meanings, patterns, correlation and
structure hidden in unstructured or semi-structured data.
 Text data stores consist of strong temporal dimensions, have modularity over
time and sources, such as topics and sentiments.
 Methods of machine-learning are prevalent in text analysis also.
Introduction Contd,…
The key terms are:
 Vector
 Feature
 Category
 Label
 Dimensions
 Graph data model
 Graph data network organization
Introduction Contd,…
 Web content mining
 Hyperlinks
 Link analytics
 Count triangles Algorithm
 Graph node centrality
 Betweenness centrality
 Closeness centrality
Text Mining
Textual data presents the challenges for computing and storage requirements.
 Definitions:
 Process of deriving high-quality information from text.
 Process of discovering and extracting knowledge from unstructured data.
 Process of analyzing collections of textual contents in order to capture key
concept themes.
 It is a technique which helps in revealing the patterns and relationships in
large volumes of textual content that are not visible to naked eye.
Text Mining
1. Text Mining Overview
Text Mining
2. Areas and Applications of Text Mining
 Natural Language Processing
 Information Retrieval
 Information Extraction
 Document Clustering
 Document Classification
 Web Mining
 Concept Extraction
Text Mining
3. Text Mining Process
Text Mining
4. Text Mining Process Phases

Phase-1: Text pre-processing

 Text cleanup
 Tokenization
 Part of speech tagging
 Word sense disambiguation
 Parsing
Text Mining
4. Text Mining Process Phases

Phase-2: Feature Generation

 Bag of words
 Stemming
 Removing stop words
 Vector space model
Text Mining
4. Text Mining Process Phases

Phase-3: Features Selection

 Dimensionality reduction
 N-gram evaluation
 Noise detection and evaluation of outliers
Text Mining
4. Text Mining Process Phases

Phase-4: Data Mining Techniques

 Unsupervised learning
 Supervised learning
 Identifying evolutionary patterns
Text Mining
4. Text Mining Process Phases

Phase-5: Analyzing Results

 Evaluate the outcome of te complete process.
 Interpretation of result
 Visualization
 Use the results for further improvement
Text Mining
5. Text Mining Challenges
 NLP issues
 Mining techniques
 Variety of data
 Information visualization
 Efficiency when processing real-time text stream
 Scalability
Text Mining
6. Supervised Text Classification
 KNN Method
 Support vector machine
 Naïve Bayes Method
 Decision Tree
 Decision Rule
Naïve Bayes Analysis
 Naïve Bayes classifier is simple, probabilistic and statistical classifier.
 It is also known as multivariate Bernoulli method.
 Naïve Bayes classifiers have good results in complex real-world situations.
 An advantage of naive Bayes is that it only requires a small amount of
training data to estimate the parameters necessary for classification and that
the classifier can be trained incrementally.
 Naïve Bayes is a conditional probability model: given a problem instance to
be classified, represented by a vector x = (x1, …, xn) representing some n
features (independent variables), it assigns to this instance probabilities for
each of K possible outcomes or classes.
Support Vector Machines (SVM)
 SVM is a set of related supervised learning methods that analyze data,
recognize patterns, classify text, recognize hand-written characters, classify
images as well as bioinformatics and bio sequence analysis.
 A vector has in general n components. A datapoint represents by (X1, X2, …… ,
Xn) in n-dimensional space.
 If a space is 3D then it is hyperplanes are 2D planes, if space is 2D, this
hyperplanes are 1D which means lines.
SVM Contd,…
 The below figure shows the concept of support vectors, separating hyperplane
and margins when using B as a classifier.
Binary Classification
Web Mining, Web Content and Web
Usage Analysis
 Web is a collection of interrelated files at web servers.
 Features of web data are:
 Volume of information and its ready availability.
 Heterogeneity
 Variety and diversity
 Mostly semi-structured
 Hyperlinks
 Redundant or similar information
 Webpage has multiple sections
 Website contents are dynamic in nature
Web Mining, Web Content and Web
Usage Analysis
1. Web Mining
 Data mining is a process of discovering patterns in large datasets to gain
knowledge. Web data mining is the mining of web data.
 Definition
 Extracting the fragment from a web document that represents the full web
document.
 Identifying interesting graph patterns or pre-processing the whole web graph
to produce metrics, such as PageRank.
 User identification, session creation, malicious activity detection and filtering,
and extracting usage path patterns.
Web Mining Taxonomy
Web Mining, Web Content and Web
Usage Analysis
2. Web Content Mining
 Web content mining is the process of information or resource discovery from
the content of web documents across WWW.
 Web content mining relates to both data mining as well as text mining.
 Applications:
 Classifying the web documents into categories.
 Identifying topics of web documents.
 Finding similar web pages across the different web servers.
 Applications related to relevance: Recommendations, Filters, Queries
Web Mining, Web Content and Web
Usage Analysis
2.1. Common Web Content Mining Techniques
 Pre-processing of contents:
 Extracting of text from HTML
 Data cleaning by filling missed data
 Tokenizing
 Stemming
 Removing the stop words
 Calculate collection wide-word frequencies (CF)
 Calculate per Document Term Frequencies (TF)
 Bag of words
Web Mining, Web Content and Web
Usage Analysis
2.1. Common Web Content Mining Techniques
 Mining tasks for Web Content Analysis:
 Classification
 Clustering
 Identifying the association between web documents
 Other significant mining tasks:
 Topic identification, tracking and drift analysis
 Concept hierarchy creation
 Relevance of content
Web Mining, Web Content and Web
Usage Analysis
3. Web Usage Mining
 Web usage mining discovers and analyzes the patterns in click streams.

 The phases are:

 Pre-processing
 Pattern discovery
 Pattern analysis
Web Mining, Web Content and Web
Usage Analysis
Process of Web Usage Mining
Page Rank, Structure of Web and
Analyzing Web Graph
4.1. Page Rank Definition
 The in-degree of a link is the measure of number of in-links from other links.
 The out-degree of a link is number of other links to which that link points.

4.2. Web Structure

 Web structure models as directed-graphs network-organization.
Page Rank, Structure of Web and
Analyzing Web Graph
4.2.1. Dead Ends
 Dead end web pages refer to the pages with no out-links.
 The web structure of the service pages may have pages with a dead end.
 The end causes no further flows for further action and no internal links.
 A good website always have a actions towards the next step.
Page Rank, Structure of Web and
Analyzing Web Graph
4.2.2. Analyzing and Implementing a System with Web Graph Mining
 In-degrees and out-degrees.
 Closeness is centrally metric.
 Betweenness
 PageRank and LineRank
 Hubs and authorities
 Communities parameters, triangle count, clustering coefficient, KNN
 Top K-shortest paths.
Page Rank, Structure of Web and
Analyzing Web Graph
4.2.3. Computation of PageRank and PageRank Iteration
Page Rank, Structure of Web and
Analyzing Web Graph
4.2.4. Topic Sensitive PageRank and Link Spam
Page Rank, Structure of Web and
Analyzing Web Graph
4.2.5. Hubs and Authorities
Page Rank, Structure of Web and
Analyzing Web Graph
4.2.6. Web Communities
Page Rank, Structure of Web and
Analyzing Web Graph
4.2.7. Limitations of Link, Rank and Web Graph Analysis
 Search engines rely on metatags or metadata of the documents.
 Search engines themselves may introduce bias while ranking.
 A top authority may be a hub of pages.
 Topic drift and content evolution can affect the rank.
 Mutually reinforcing affiliates or affiliated pages/sites can enhance each
other’s rank and authorities.
 The ranks may be unstable as adding additional nodes may have greater
influence in rank changes.
Social Networks as Graphs and Social
Network Analytics
5.1. Social Network as Graphs
 It provide several metrics for analysis.
 Network topological analysis tools compute many parameters.
 Social network characteristics from observations in the organizations are:
 3 step neighborhoods show positive correlation between a person and high
performance.
 Social network of a person shows high performance outcome when the
network exhibits structural diversity.
Social Networks as Graphs and Social
Network Analytics
5.2. Social Graph Network Topological analysis using Centralities and
PageRank
 Degree
 Closeness
 Effective closeness
 Betweenness
 PageRank
 Contacts Size
 Indirect Contacts
 Structure Diversity
Social Networks as Graphs and Social
Network Analytics
5.3. Social Graph Network analysis using K-core and Neighbourhood
Metrics
Social Networks as Graphs and Social
Network Analytics
5.4. Clustering in Social Network Graphs and SimRank
Social Networks as Graphs and Social
Network Analytics
5.5. Counting Triangles and Graph Matches
 Degree
 Closeness
 Effective closeness
 Betweenness
 PageRank
 Contacts Size
 Indirect Contacts
 Structure Diversity
Social Networks as Graphs and Social
Network Analytics
5.6. Using SparkGraph (MapReduce) for Network Graphs
 There are three different topics to cover when we discuss graph data related
technologies:
 Graph Databases
 Graph Data Analytics
 Graph Data Visualization
Social Networks as Graphs and Social
Network Analytics
5.7. Direct Discovery of Communities
 Three metrics identify groups and communities from a social graph:
 Cliques
 Structurally cohesive blocks
 Social circles from connections and neighbourhoods