BUSINESS ANALYTICS

Module 2

Descriptive Analytics I -Nature of Data, Big Data, and

Statistical Modeling
2.1 The Nature of Data in Analytics

2.1.1 Role of Data

• Data is the main ingredient for Business Intelligence (BI), Data Science, and Business
Analytics.

• It is the raw material that produces:

o Information

o Insight

o Knowledge

• Without data, these technologies cannot exist.

• Earlier: analytics used expert knowledge + little/no data.

• Today: data is essential → considered one of the most valuable assets of

organizations.

• Benefits: helps organizations understand customers, competitors, and business

processes.

2.1.2. Nature & Types of Data

• Size: can be small or very large.

• Structure:

o Structured data → organized, computer-friendly (e.g., databases).

o Unstructured data → human-created, not easily readable by machines (e.g.,

text, social media posts).
[AUTHOR NAME] 1
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

• Flow:

o Arrives continuously in smaller batches.

o Or all at once as large batch.

• This complexity = Big Data.

• Though challenging, Big Data is more valuable → enables discovery of new

knowledge.

Data collection evolution:

o Past: manual (surveys, human-entered records).

o Present: automated via Internet, sensors, RFID, networks.

• Automated collection:

o Higher volume of data.

o Better quality & integrity.

2.1.3. Analytics Continuum

Data → Analytics → Actionable Information

Figure 2.1 illustrates a typical analytics continuum—data to analytics to actionable information

[AUTHOR NAME] 2
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

2.1.4. Importance of Data Preparation

• Ignoring data tasks = risk of wrong answers to the right problem.

• Poorly handled data → misleading insights & bad decisions.

2.1.5. Metrics for Analytics-Ready Data

➢ Data Source Reliability

• Refers to originality & appropriateness of data source.
• Question: Do we trust this source?
• Original source is preferred → reduces risk of misrepresentation during
transfers.
• Every move = chance of errors (data drop/reformat).
➢ Data Content Accuracy
• Refers to whether data values are correct and suitable for the problem.
• Example: customer phone number in database must match what customer
provided.
➢ Data Accessibility
• Data should be easily obtainable when needed.
• Challenge: data stored in multiple locations/storage systems.

[AUTHOR NAME] 3
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

• Requires merging/transformation.
• With new tech (data lakes, Hadoop), accessibility is more critical.
➢ Data Security & Privacy
• Only authorized people should access data.
• Prevents unauthorized access & misuse.
• Very critical in sensitive areas (e.g., healthcare).
• Example: HIPAA law ensures patient health records are secure and accessible
only to authorized users.
➢ Data Richness (Comprehensiveness)
• Data should contain all required elements for meaningful analysis.
• Provides dimensionality for deeper insights.
• Should be complete enough to support predictive/prescriptive models.
➢ Data Consistency
• Data should be accurately collected & merged.
• Problem: wrong merging can mix up records (e.g., merging two patient
records).
• Consistency ensures data from multiple sources aligns correctly.
➢ Data Currency / Timeliness
• Data should be up-to-date and recent.
• Recorded close to the event → avoids memory/recall errors.
• Accurate & timely data = reliable analytics.
➢ Data Granularity
• Data should have the required level of detail.
• Example:
o Lab test results must have correct decimal precision.
o Demographic data should be detailed enough to differentiate
subpopulations.
• Rule: Aggregated data cannot be disaggregated, but granular data can be
aggregated.
➢ Data Validity
• Actual data values must match expected values/ranges.
• Example: gender variable → valid entries = male, female, other.

[AUTHOR NAME] 4
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

➢ Data Relevancy
• Only include variables relevant to the study.
• Relevancy = spectrum (least relevant → most relevant).
• Avoid irrelevant data → can mislead algorithms and reduce accuracy.

2.2 A Simple Taxonomy Of Data

• Data (datum in singular form) refers to a collection of facts usually obtained as the
result of experiments, observations, transactions, or experiences.
• Data may consist of numbers, letters, words, images, voice recordings, and so on,
as measurements of a set of variables (characteristics of the subject or event that we
are interested in studying).
• Data are often viewed as the lowest level of abstraction from which information and
then knowledge is derived. At the highest level of abstraction, one can classify data as
structured and unstructured (or semi structured).
• Unstructured data/semi structured data is composed of any combi nation of textual,
imagery, voice, and Web content..
• Structured data is what data mining algorithms use and can be classified as categorical
or numeric.

Figure 2.2 shows a simple data taxonomy.

The categorical data can be subdivided into nominal or ordinal data, whereas numeric data can be
subdivided into intervals or ratios.

➢ Categorical Data

[AUTHOR NAME] 5
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

• Labels used to divide variables into groups (e.g., race, age group, education).

• Age and education can be numeric but are often better expressed as categories (e.g.,
“teen,” “adult”).

• Also called discrete data → finite values, no continuum.

• Numbers used as codes are only symbols, not for calculations like fractions.

➢ Nominal Data (Subtype of Categorical Data)

• Codes/labels assigned to objects, not true measurements.

• Example: Marital status → (1) Single, (2) Married, (3) Divorced.

• Types:

o Binomial → Two categories (Yes/No, True/False, Good/Bad).

o Multinomial → Three or more categories (Eye color: Brown/Green/Blue;

Race: White/Black/Latinx/Asian; Marital status variations).

➢ Ordinal Data

• Codes/labels assigned to objects/events that also represent rank order.

• Examples:

o Credit score: (1) Low, (2) Medium, (3) High

o Age group: Child, Young, Middle-aged, Older adults

o Education level: High school, College, Graduate school

• Use in analytics: Algorithms like ordinal multiple logistic regression use rank-order
info to improve classification.

➢ Numeric Data (Continuous Data)

• Represents numeric values of variables.

• Examples:

o Age

[AUTHOR NAME] 6
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

o Number of children

o Household income (USD)

o Travel distance (miles)

o Temperature (°F)

• Can be:

o Integer → whole numbers only

o Real → can include fractional values

• Also called continuous data

• Infinite possible fractional values

• Unlike discrete variables, which have finite, countable values

➢ Interval Data

• Variables measured on interval scales.

• Example: Temperature (Celsius)

• Key points:

o Unit of measurement: 1/100 difference between water’s melting & boiling

points at atmospheric pressure

o No absolute zero → zero is arbitrary

o Only differences between values are meaningful; ratios are not meaningful

➢ Ratio Data

• Measurement variables common in physical sciences & engineering.

• Examples:

o Mass, Length, Time, Plane angle, Energy, Electric charge

• Key points:

[AUTHOR NAME] 7
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

o Ratio scale = measurement of ratio between magnitude & unit magnitude

o Non-arbitrary zero → zero represents absence of quantity

• Example: Kelvin temperature

o Absolute zero = –273.15°C

o At this point, particles have zero kinetic energy

• Unlike interval data, ratios are meaningful

Other Data Types: Textual, spatial, imagery, video, audio/voice.

o Must be converted into categorical or numeric forms for analytics.

o Can also be static (fixed) or dynamic/time series (changing over time).

Algorithm Compatibility:

o Some predictive analytics or machine-learning algorithms require specific data

types.

o Using incompatible data may produce wrong models or stop model

development.

Examples of Requirements:

Numeric-only algorithms (e.g., neural networks, SVM, logistic regression)

o Nominal/ordinal variables converted via 1-of-N (one-hot) encoding.

o Caution: may increase variable count for high-cardinality categories.

Categorical-only algorithms (e.g., ID3, Rough Sets)

o Numeric variables may need discretization.

o Modern tools often handle mixed data automatically.

Data Transformation:

[AUTHOR NAME] 8
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

o Analytics tools help convert and represent data so algorithms can process it
properly.

2.3 The Art And Science Of Data Preprocessing

Raw Data Characteristics:

• Data in its original form (real-world data) is usually not ready for analytics tasks.

• Common issues:

o Dirty

o Misaligned

o Overly complex

o Inaccurate

Need for Data Preprocessing:

• A tedious and time-demanding process.

• Converts raw real-world data into a well-refined form suitable for analytics
algorithms.

• Reference: Kotsiantis, Kanellopoulos, & Pintelas, 2006.

• Time factor:

o Often takes longer than the rest of the analytics tasks (model building and
assessment).

o Considered the least enjoyable phase by analytics professionals.

• Figure 2.3 illustrates the main steps in data preprocessing.

[AUTHOR NAME] 9
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

First Phase of Data Preprocessing:

1. Data Collection

o Gather relevant data from identified sources.

2. Record & Variable Selection

o Select necessary records and variables.

o Filter out unnecessary information.

o Requires intimate understanding of the data.

3. Data Integration / Merging

o Combine records from multiple data sources.

o Handle synonyms and homonyms properly.

o Modern term for this process: Data Blending

Data Blending

• Importance:

[AUTHOR NAME] 10
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

o Data blending is a critical part of data science, the most popular job of the 21st
century.

o Types of data blending:

▪ Data source blending

▪ Data type blending

▪ Database blending

▪ Time blending

▪ Tool blending

Second phase of data preprocessing

Cleaning (Data Scrubbing)

• Data in raw/real-world form is usually dirty, misaligned, overly complex, and

inaccurate.

• Tasks in this phase:

o Identify and deal with missing values:

▪ Impute (fill with probable value) or ignore if anomaly

▪ Recognize natural missing values (e.g., high-income people often leave

household income blank)

o Identify noisy values/outliers and smooth them

o Handle inconsistencies using domain knowledge or expert opinion

Third phase of data preprocessing

• Prepares data for better processing and modelling.

• Tasks in this phase:

o Normalization: Scale numeric variables to a common range to prevent bias

▪ Example: Household income vs. number of dependents

o Discretization / Aggregation:

[AUTHOR NAME] 11
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

▪ Convert numeric to categorical (e.g., low/medium/high)

▪ Reduce nominal variable range using concept hierarchies (e.g., states

→ regions)

o Feature creation / simplification:

▪ Create new variables to magnify information

▪ Example: Blood-type match (1 = match, 0 = no match) instead of

separate donor/recipient types

The final phase of data preprocessing is data reduction, which addresses two main dimensions:

1. Variables (columns / dimensionality reduction):

o Large numbers of variables can complicate analysis.

o Dimensional reduction (variable selection) helps retain only the most relevant
features.

o Techniques include consulting domain experts, statistical tests (e.g., PCA,

ICA), and combining methods.

2. Cases (rows / sampling):

o Data sets may contain millions/billions of records, making full analysis

impractical.

o Sampling is used, but the sample must fairly represent the whole dataset.

o Random sampling is preferred, while stratified sampling is needed for skewed

data.

o Class imbalance can be addressed by oversampling minority classes or under

sampling majority classes, as balanced datasets yield better prediction models.

[AUTHOR NAME] 12
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

Table 2.1 A summary of Data Preprocessing Tasks and Potential Methods

2.4 Definition of Big Data

Using data to understand customers and business operations is increasingly difficult due to the
explosive growth of data, commonly referred to as Big Data. While often overhyped, Big
Data generally refers to data sets too large or complex for traditional tools to handle
effectively within a reasonable time.

• Meaning of Big Data:

o Traditionally linked to massive datasets analyzed by large organizations (e.g.,

Google, NASA).

o For most businesses, "big" is relative—what matters is extracting value, not

just size.

o Often expanded to Big Data Analytics, emphasizing its use for generating
insights.

[AUTHOR NAME] 13
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

• Sources of Big Data:

o Everywhere: web logs, sensors, GPS, RFID, social media, search indexes, call
records, scientific research, medical records, e-commerce, multimedia
archives, etc.

o Data once ignored due to technical limitations is now considered valuable.

• Evolution:

o Not new—large-scale data analysis dates back to 1990s data warehouses.

o The scale has grown from terabytes → exabytes, driven by demand for deeper
insights.

• Characteristics (beyond just size):

o Volume (scale of data)

o Variety (structured/unstructured data)

o Velocity (speed of data generation/processing)

o Veracity (data quality/trustworthiness)

o Variability (inconsistencies/changes)

o Value (business benefit from insights)

The “V”s That Define Big Data

Big Data is typically described by three core “V”s—Volume, Variety, Velocity—but other
solution providers have added Veracity, Variability, and Value Proposition.

1. Volume

o Refers to the massive size of data, growing from petabytes (PB) to zettabytes
(ZB).

o Driven by transactions, social media, sensors, RFID, GPS, IoT, etc.

o Storage is less of an issue today; the challenge is identifying relevant and

valuable data.

2. Variety

[AUTHOR NAME] 14
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

o Data exists in many formats: structured (databases), semi-structured (XML,

emails), and unstructured (text, video, audio, sensor data).

o About 80–85% of organizational data is semi/unstructured, making analysis

complex but necessary for decision-making.

3. Velocity

o Refers to both the speed of data generation and the speed of data processing
required.

o Real-time data streams (RFID, sensors, GPS, smart meters) demand fast
analytics.

o Data stream analytics (in-motion analytics) is emerging as critical, sometimes

more valuable than traditional “at-rest analytics.”

4. Veracity (IBM)

o Concerns the trustworthiness, quality, and accuracy of data.

o Requires cleaning, validation, and transformation to ensure reliable insights.

5. Variability (SAS)

o Refers to inconsistent data flows with sudden peaks (e.g., trending social
media events, seasonal surges).

o Managing unpredictable data loads is a challenge, especially in social-driven

environments.

6. Value Proposition

o The ultimate purpose of Big Data.

o Large, rich datasets provide more patterns, anomalies, and insights than small
data.

o Leads to better business decisions, innovation, and competitive advantage.

[AUTHOR NAME] 15
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

2.5 Fundamentals Of Big Data Analytics

• Big Data itself is useless unless it is analyzed to deliver business value.

• Traditional data warehouses and BI (reports/dashboards) are limited:

o Tools are too complex for average users.

o Data repositories often lack all relevant data.

• Big Data analytics enables deeper, on-demand exploration with new data sources and
technologies.

When Organizations Should Consider Big Data Analytics

You likely need Big Data analytics if:

1. Current systems cannot process the required data volume.

2. You want to integrate new data types (social media, RFID, sensors, web, GPS, text,
etc.) that don’t fit traditional schemas.

3. You need real-time or near-real-time integration to keep analysis current.

4. You prefer schema-on-demand instead of rigid predefined RDBMS schemas (due to

unknown or fast-changing data).

5. Data arrives too fast for traditional analytics platforms to handle.

As is the case with any other large IT investment, the success in Big Data analytics depends on a
number of factors. Figure 2.4 shows a graphical depiction of the most critical success factors.

Fig 2.4: critical success factors.

[AUTHOR NAME] 16
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

Critical Success Factors for Big Data Analytics

1. Clear Business Need – Big Data investments must align with business vision and
strategy (strategic, tactical, or operational).

2. Strong, Committed Sponsorship – Requires executive champions; departmental-

level for small projects, enterprise-wide for transformation.

3. Business–IT Alignment – Analytics must support business strategy, not dictate it.

4. Fact-Based Decision-Making Culture – Decisions driven by data, not intuition.

o Senior leadership should support analytics, discontinue outdated methods, ask

for data-backed decisions, and link incentives to analytics-driven outcomes.

5. Strong Data Infrastructure – Combine traditional data warehouses with new Big
Data technologies.

High-Performance Computing (Techniques for Big Data)

• In-memory analytics – Fast, real-time insights by processing in-memory across

nodes.

• In-database analytics – Process within the database to reduce data movement.

• Grid computing – Shared pool of IT resources for efficiency and lower costs.

• Appliances – Scalable integrated hardware/software units.

Challenges in Implementing Big Data Analytics

• Data Volume – Capturing, storing, and processing massive amounts at speed.

• Data Integration – Combining diverse, unstructured, or semi-structured data quickly

and affordably.

• Processing Capabilities – Real-time/stream analytics needed; batch methods often

insufficient.

[AUTHOR NAME] 17
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

• Data Governance – Ensuring security, privacy, ownership, and quality as volume,

variety, and velocity increase.

• Skills Availability – Shortage of data scientists and Big Data experts.

• Solution Cost – High experimentation costs; need cost-effective solutions for ROI.

Business Problems Addressed by Big Data Analytics

• General Priorities: Process efficiency, cost reduction, and enhanced customer

experience.

• Industry-Specific Priorities:

o Manufacturing, government, energy/utilities, transport, healthcare:

Process efficiency & cost reduction.

o Insurance, retail: Customer experience.

o Banking, education: Risk management.

Partial List of Problems Solved:

• Process efficiency & cost reduction

• Brand management

• Revenue maximization, cross-/up-selling

• Enhanced customer experience & service

• Customer churn identification & recruiting

• New product & market discovery

• Risk management & regulatory compliance

• Enhanced security

[AUTHOR NAME] 18
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

2.6 Big Data Technologies

Several technologies share common traits:

• Use commodity hardware for scalability and parallel processing.

• Support nonrelational storage for unstructured/semi-structured data.

• Employ advanced analytics & visualization for insights.

The three key technologies shaping Big Data are Hadoop, MapReduce, and
NoSQL.

Hadoop

• Definition: An open-source framework for storing, processing, and analyzing massive

distributed datasets.

• Origin: Created by Doug Cutting at Yahoo!, inspired by Google’s MapReduce.

• Key Features:

o Runs on commodity hardware (low cost, scalable).

o Handles petabytes–exabytes of data.

o Breaks Big Data into smaller parts, processed in parallel.

o Uses HDFS (Hadoop Distributed File System) for storage.

How Hadoop Works

1. Data Input: Structured, semi-structured, and unstructured data (e.g., logs, social
media, internal stores).

2. Storage (HDFS):

o Splits data into parts across nodes.

o Replicates data for fault tolerance.

o Name Node: Manages metadata (which nodes hold what data, failures, etc.).

[AUTHOR NAME] 19
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

3. Processing (MapReduce):

o Map Phase: Client submits a “Map” job (query, often Java). Job Tracker
assigns tasks to nodes holding relevant data. Nodes process their share in
parallel.

o Reduce Phase: Results from map tasks are aggregated across nodes to
produce a final answer.

4. Output: Results are returned to the client and can be further analyzed in analytic
tools.

After MapReduce Phase

Once the MapReduce phase is complete, the processed data is ready for advanced analysis by data
scientists and analytics professionals. They can:

• Search for hidden insights and patterns.

• Build user-facing analytic applications.

• Transfer and model the data in relational databases, warehouses, or IT systems for
further analysis or transactional support.

MapReduce

• A programming model (not a programming language) popularized by Google for

distributing processing of very large multistructured data files across clusters.

• Programs written in this style are automatically parallelized and executed on

clusters of commodity machines.

• Enables programmers without distributed systems expertise to leverage parallel

computing.

[AUTHOR NAME] 20
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

Fig 2.5 : A Graphical Depiction of the MapReduce Process.

How MapReduce Works (Illustrated by Figure 2.5)

1. Input: Coloured squares (as example dataset).

2. Splitting: Input file is split into multiple parts.

3. Map Phase: Each map program groups data (e.g., by color).

4. Shuffle/Sort: System merges map outputs and prepares them for reduction.

5. Reduce Phase: Reduce program aggregates counts (e.g., total number of squares by
colour).

6. Output: Results ready for further analysis.

Programmers can optimize performance by providing custom shuffle/sort logic or adding combiners
to minimize remote file transfers.

Why Use MapReduce?

• Handles large multistructured datasets.

• Application areas: search, graph analysis, text analysis, ML, data transformation.

• Advantages:

o Simple procedural model for programmers.

o Automatic handling of parallelism/distribution.

o Accessible to non-programmers via pre-built applications & libraries (e.g.,

Apache Mahout for ML).
[AUTHOR NAME] 21
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

Hadoop Technical Components

• HDFS (Hadoop Distributed File System): Default storage layer.

• Name Node: Manages metadata (data location, node failures).

• Secondary Node: Backup for Name Node.

• Job Tracker: Coordinates MapReduce jobs.

• Worker Nodes: Store data and perform computations.

Hadoop Ecosystem Tools

Beyond the core of HDFS and MapReduce, Hadoop includes a rich set of ecosystem tools that
extend its functionality for data warehousing, querying, integration, monitoring, and
machine learning.

1. Hive

• Hadoop-based data warehousing framework, developed by Facebook.

• Uses HiveQL (SQL-like language) → queries are automatically converted into

MapReduce jobs.

• Enables SQL programmers to use Hadoop without needing MapReduce skills.

• Easily integrates with BI and visualization tools (e.g., Tableau, MicroStrategy,

Revolution Analytics).

2. Pig

• High-level dataflow language developed by Yahoo!.

• Easier to learn than Java MapReduce.

• Suitable for complex, long data pipelines, where SQL may fall short.

3. HBase

• A NoSQL database on Hadoop for low-latency lookups.

• Supports transactions (updates, inserts, deletes).

[AUTHOR NAME] 22
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

• Widely used by eBay and Facebook.

4. Flume

• Framework for ingesting streaming data into Hadoop.

• Uses distributed agents (e.g., on web servers, apps, mobile devices) to collect and
push data into HDFS.

5. Oozie

• A workflow scheduler for Hadoop.

• Allows defining and chaining jobs written in MapReduce, Hive, Pig, etc.

• Supports dependencies (e.g., a job starts only after required previous jobs complete).

6. Ambari

• Web-based tool for deploying, administering, and monitoring Hadoop clusters.

• Developed by Hortonworks.

• Provides easy UI and REST APIs for management.

7. Avro

• Data serialization framework for Hadoop.

• Supports schema definition and encoding.

• Useful for parsing, data exchange, and remote procedure calls (RPCs).

8. Mahout

• A machine learning library for Hadoop.

• Implements popular algorithms for clustering, classification, regression, and

collaborative filtering.

• Uses MapReduce for large-scale execution.

9. Sqoop

• A connectivity tool for transferring data between Hadoop and traditional data stores.

[AUTHOR NAME] 23
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

• Moves data from relational databases (Oracle, Teradata, MySQL, etc.) into Hadoop
(and vice versa).

10. HCatalog

• A metadata and table management service for Hadoop.

• Provides a centralized schema repository.

• Allows tools like Pig and Hive to access data without knowing its physical storage
location.

2.7. Statistical Modeling for Business Analytics

Statistical modeling is used in business analytics to support decision-making. It helps
managers and analysts understand data patterns and make predictions. Statistics includes
methods for collecting, analyzing, and interpreting data.

Descriptive Statistics

Descriptive statistics describe the basic features of data. It helps summarize and present data in a
simple, understandable way using numbers, tables, or graphs. It doesn’t make conclusions
about a larger population—only about the sample data available.

Measures of Central Tendency

These show the center or average value in data. Common measures include Mean, Median, and
Mode.

Mean (Average)
Mean = (Sum of all values) ÷ (Number of values)
It is most commonly used, but it can be affected by extreme values (outliers).

Median
The middle value when data is arranged in order. It is not affected by outliers.

Mode
The value that appears most often in a data set. It is useful for categorical or nominal data.

Measures of Dispersion

These show how spread out or varied the data is. Common measures include Range, Variance,
Standard Deviation, Mean Absolute Deviation, and Quartiles.
[AUTHOR NAME] 24
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

Range
Range = Maximum value – Minimum value. It shows total spread of data.

Variance
Variance measures how far each data point is from the mean. Higher variance means more spread-out
data.

Standard Deviation
It is the square root of variance. It tells how much data values deviate from the mean.

Mean Absolute Deviation (MAD)

MAD = Average of absolute differences between each value and the mean. It is a simple measure of
spread.

Quartiles and Interquartile Range (IQR)

Quartiles divide data into four equal parts. IQR = Q3 - Q1, which shows the range of the middle 50%
of data.

Box-and-Whiskers Plot
This graphical tool shows median, quartiles, minimum, maximum, and outliers. It helps visualize the
spread and skewness of data.

Shape of Distribution
The shape of data distribution shows how data values are spread. The most common shape is the
normal distribution, which is symmetric around the mean.

Two key shape measures are:

• Skewness – shows if data leans to left or right.

• Kurtosis – shows if data is more peaked or flat compared to normal distribution.

2.8 Regression Modeling for Inferential Statistics

Regression is a statistical method used to find relationships between variables. It helps make
predictions and understand how one variable changes when another changes.

Correlation vs. Regression

• Correlation measures how strongly two variables are related (no direction assumed).
• Regression explains how one variable depends on another (cause–effect).

[AUTHOR NAME] 25
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

Simple and Multiple Regression

• Simple Regression – involves one independent (input) and one dependent (output) variable.
• Multiple Regression – involves one dependent variable and two or more independent
variables.

Linear Regression Model

The relationship between input (x) and output (y) is represented as:
y = β₀ + β₁x
Where β₀ = intercept, β₁ = slope. The line best fits the data points to minimize error.

Evaluating the Model

Model fit tells how well regression explains the data. Common evaluation measures:
• R² (R-squared) – shows how much of the variation is explained by the model.
• F-test – checks overall model significance.
• RMSE – measures prediction error (lower is better).

Assumptions of Linear Regression

1. Linearity – Relationship between variables must be linear.

2. Independence – Errors should not be related to each other.

3. Normality – Errors must be normally distributed.

4. Constant Variance – Errors have equal variance (homoscedasticity).

5. No Multicollinearity – Independent variables shouldn’t be highly correlated.

Logistic Regression

Used when the output is categorical (like Yes/No, Pass/Fail). It predicts the probability of a
class using a sigmoid (S-shaped) curve. It is widely used in business, health, and social
sciences.

Time Series Forecasting

Time series uses past data to predict future values. Data is collected at regular intervals (daily,
monthly, etc.).
Common methods:
• Moving Average – uses average of recent data points.
• Weighted Moving Average – gives more importance to recent data.

[AUTHOR NAME] 26
RLJIT Dept Of CSE (Data science)
 BUSINESS ANALYTICS

• Exponential Smoothing – uses decreasing weights for older data.

• ARIMA – combines autoregression and moving average to model patterns.

Forecast accuracy is measured using:

• Mean Absolute Error (MAE)
• Mean Squared Error (MSE)
• Mean Absolute Percentage Error (MAPE)

[AUTHOR NAME] 27
RLJIT Dept Of CSE (Data science)