CHAPTER-II

DATA

2.1 Introduction of Data:

Data is a collection of information gathered by observations, measurements, research or

analysis. They may consist of facts, numbers, names, figures or even description of things.
Data is organized in the form of graphs, charts or tables. There exist data scientists who does
data mining and with the help of that data analyse our world.

In general, data is a distinct piece of information that is gathered and translated for some
purpose. If data is not formatted in a specific way, it does not valuable to computers or
humans. Data can be available in terms of different forms, such as bits and bytes stored in
electronic memory, numbers or text on pieces of paper, or facts stored in a person's mind.
Since the invention of computers, people have used the word data to mean computer
information, and this information is transmitted or stored. There are different kinds of data;
such are as follows:

o Sound
o Video
o Single character
o Number (integer or floating-point)
o Picture
o Boolean (true or false)
o Text (string)

In a computer's storage, data is stored in the form of a series of binary digits (bits) that
contain the value 1 or 0. The information can be in terms of pictures, text documents,
software programs, audio or video clips, or other kinds of data. The computer data may be
stored in files and folders on the computer's storage, and processed by the computer's CPU,
which utilizes logical operations to generate output (new data) form input data.
As the data is stored on the computer in binary form (zero or one), which can be processed,
with the help of various media devices or a network connection. Furthermore, if you use data
multiple times, it does not deteriorate over time or lose quality.

2.2 Structure Data:

Structured data is data that has a standardized format for efficient access by software and
humans alike. It is typically tabular with rows and columns that clearly define data attributes.
Computers can effectively process structured data for insights due to its quantitative nature.
 For example, a structured customer data table containing columns - name, address, and phone
number- can provide insights like the total number of customers and the locality with the
maximum number of customers. In contrast, unstructured data, like a list of social media
posts, is more challenging to analyse.
2.2.1 Benefits of structured data:
There are several benefits of using structured data.

Ease of use
Anyone can quickly comprehend and access structured data. Operations such as updating and
amending structured data are straightforward. Storage is efficient, as fixed-length storage
units can be allocated for data values.
Scalability
Structured data scales algorithmically. You can add storage and processing power as your
data volume increases. Modern systems that process structured data can scale to several
thousand TB of data.
Analytics
Machine learning algorithms can analyse structured data and identify common patterns for
business intelligence. You can use structured query language (SQL) to generate reports as
well as modify and maintain data. Structured data is also useful for big data analytics.

2.3 Unstructured Data:

Unstructured data is all that information that isn't predefined and searchable on a table, such as
text messages, images, videos, audio files, and emails. Unlike structured data, which is easily
placed into tables like those found in Microsoft Excel, unstructured data can't be quickly
analysed and searched without further processing.
Unstructured data can take various forms, including text documents, emails, social media posts,
images, videos, audio recordings, presentations, and more. It often contains free-form text,
natural language, and multimedia content. In other words, then, unstructured data encompasses
all the different kinds of qualitative data produced by individuals every day that lack clear-cut
quantitive data points.
In turn, data professionals can find unstructured data from a wide variety of different sources.
Some particularly rich sources of unstructured data include:
 Customer reviews
 Social media conversations
 News articles
 Sensor data from Internet of Things (IOT) devices
 2.4 Metadata
Metadata is data about data. It describes attributes of your data that can be important to know
but may not be immediately relevant to your data’s primary function. This could include
information about how data was collected, where it’s stored, and how it’s used.
For example, metadata is often attached to digital files, such as photos, videos, and audio, to
describe aspects of the file, such as its name, size, when it was captured, and where it was
captured.
Metadata can be helpful for verification and tracking purposes. You can refer to metadata to
confirm data is accurate and reliable (data integrity) or to create an organisational system for
your data. Certain software and programs also use metadata to interpret your data. For example,
search engines use metadata to categorise web pages.
2.4.1 Types of metadata
Several types of metadata exist, but three common types are descriptive, structural, and
administrative:
 Descriptive metadata describes identification attributes, such as the file name and author.
 Structural metadata describes how data is organised, such as versions and relationships to
other pieces of data.
 Administrative metadata describes technical attributes, such as the file size and creation date.
2.5 Big Data:
Big Data is a collection of data that is huge in volume, yet growing exponentially with time.
It is a data with so large size and complexity that none of traditional data management tools
can store it or process it efficiently. Big data is also a data but with huge size.

2.5.1 Characteristics of Big Data:

Big data can be described by the following characteristics:

 Volume
 Variety
 Velocity
 Variability

(i) Volume – The name Big Data itself is related to a size which is enormous. Size of data
plays a very crucial role in determining value out of data. Also, whether a particular data can
actually be considered as a Big Data or not, is dependent upon the volume of data.
Hence, ‘Volume’ is one characteristic which needs to be considered while dealing with Big
Data solutions.
(ii) Variety – The next aspect of Big Data is its variety.
Variety refers to heterogeneous sources and the nature of data, both structured and
unstructured. During earlier days, spread sheets and databases were the only sources of data
considered by most of the applications. Nowadays, data in the form of emails, photos, videos,
monitoring devices, PDFs, audio, etc. are also being considered in the analysis applications.
This variety of unstructured data poses certain issues for storage, mining and analysing data.
(iii) Velocity – The term ‘velocity’ refers to the speed of generation of data. How fast the
data is generated and processed to meet the demands, determines real potential in the data.

Big Data Velocity deals with the speed at which data flows in from sources like business
processes, application logs, networks, and social media sites, sensors, Mobile devices, etc.
The flow of data is massive and continuous.
(iv) Variability – This refers to the inconsistency which can be shown by the data at times,
thus hampering the process of being able to handle and manage the data effectively.

2.6 Advantages of Big Data Processing

Ability to process Big Data in DBMS brings in multiple benefits, such as-
 Businesses can utilize outside intelligence while taking decisions
Access to social data from search engines and sites like Facebook, Twitter are enabling
organizations to fine tune their business strategies.
 Improved customer service
Traditional customer feedback systems are getting replaced by new systems designed with
Big Data technologies. In these new systems, Big Data and natural language processing
technologies are being used to read and evaluate consumer responses.

 Early identification of risk to the product/services, if any

 Better operational efficiency
Big Data technologies can be used for creating a staging area or landing zone for new data
before identifying what data should be moved to the data warehouse. In addition, such
integration of Big Data technologies and data warehouse helps an organization to offload
infrequently accessed data.

2.7 Linked Data:

Linked Data is a set of design principles for sharing machine-readable interlinked data on the
Web. When combined with Open Data (data that can be freely used and distributed), it is
called Linked Open Data (LOD). An RDF database such as Ontotext's Graph DB is an
example of LOD. It is able to handle huge datasets coming from disparate sources and link
them to Open Data, which boosts knowledge discovery and efficient data-driven analytics.
Linked Data is one of the core pillars of the Semantic Web, also known as the Web of Data.
The Semantic Web is about making links between datasets that are understandable not only to
humans, but also to machines, and Linked Data provides the best practices for making these
links possible. In other words, Linked Data is a set of design principles for sharing machine-
readable interlinked data on the Web.

2.8 Lab Testing:

Lab tests are conducted by a User Centric Design / Human Factors expert, who is typically
supported by a note taker. Key stake holders connected to the website (or product) participate
as Observers and their job is to get a close understanding of the customer experience. Stake
holders can be business owners, engineers and developers, web analysts), product managers
etc, anyone who has something to do with the website or customer experience. Tests can be
conducted with live version of the website, beta versions, on-screen HTML or PowerPoint
prototypes, or even with paper printouts. These paper prototypes, sometimes called wire-
frames, approximate what a user might otherwise see on a computer screen, but save the
development team from having to produce an on-screen product.
2.9 What is Experimental Design?
Experimental design simply refers to the strategy that is employed in conducting
experiments to test hypotheses and arrive at valid conclusions. The process comprises
firstly, the formulation of research questions, variable selection, specifications of the
conditions for the experiment, and a protocol for data collection and analysis. The
importance of experimental design can be seen through its potential to prevent bias, reduce
variability, and increase the precision of results in an attempt to achieve high internal
validity of studies. By using experimental design, the researchers can generate valid results
which can be generalized in other settings which helps the advancement of knowledge in
various fields.
2.9.1 Types of Experimental Design
Experimental design encompasses various approaches to conducting research studies, each
tailored to address specific research questions and objectives. The primary types of
experimental design include:
 Pre-experimental Research Design
 True Experimental Research Design
 Quasi-Experimental Research Design
 Statistical Experimental Design
Pre-experimental Research Design
A preliminary approach where groups are observed after implementing cause and effect
factors to determine the need for further investigation. It is often employed when limited
information is available or when researchers seek to gain initial insights into a topic. Pre-
experimental designs lack random assignment and control groups, making it difficult to
establish causal relationships.
Classifications:
 One-Shot Case Study
 One-Group Pretest-Posttest Design
 Static-Group Comparison
True-experimental Research Design
The true-experimental research design involves the random assignment of participants to
experimental and control groups to establish cause-and-effect relationships between
variables. It is used to determine the impact of an intervention or treatment on the outcome
of interest. True-experimental designs satisfy the following factors:
Factors to Satisfy:
 Random Assignment
 Control Group
 Experimental Group
 Pretest-Posttest Measures
Quasi-Experimental Design
A quasi-experimental design is an alternative to the true-experimental design when the
random assignment of participants to the groups is not possible or desirable. It allows for
comparisons between groups without random assignment, providing valuable insights into
causal relationships in real-world settings. Quasi-experimental designs are used typically in
conditions wherein the random assignment of the participants cannot be done or it may not
be ethical, for example, an educational or community-based intervention.
Statistical Experimental Design
Statistical experimental design, also known as design of experiments (DOE), is a branch of
statistics that focuses on planning, conducting, analysing, and interpreting controlled tests
to evaluate the factors that may influence a particular outcome or process. The primary goal
is to determine cause-and-effect relationships and to identify the optimal conditions for
achieving desired results.
2.10 Selecting participants for lab testing and experimental design requires careful
consideration to ensure reliability, validity, and ethical integrity. Here's a step-by-step
guide:

2.10.1 Define Research Objectives:

 Purpose: Clarify what you aim to achieve through the experiment (e.g., testing a
hypothesis, evaluating a new process).
 Variables: Identify dependent, independent, and control variables.

2.10.2 Identify Target Population:

 Demographics: Determine the specific characteristics needed (e.g., age, gender,
geographic region, occupation).
 Eligibility Criteria: Set inclusion (who qualifies) and exclusion (who does not
qualify) criteria based on research needs.

2.10.3 Decide on Sampling Method:

 Probability Sampling: Every individual in the population has an equal chance of

selection (e.g., simple random sampling, stratified sampling).
  Non-Probability Sampling: Selection is based on specific criteria or convenience
(e.g., convenience sampling, purposive sampling).

2.10.4 Calculate Sample Size

 Use statistical methods or tools to determine an adequate sample size based on:
o Population size.
o Confidence level (e.g., 95%).
o Margin of error (e.g., ±5%).
o Effect size (magnitude of the expected effect).
2.10.5 Recruitment Strategies
 Methods: Utilize advertisements, flyers, emails, or recruitment platforms to reach
potential participants.
 Incentives: Offer compensation or benefits (monetary, gift cards, certificates) if
appropriate and ethical.

2.10.6 Screen Participants

 Pre-Screening Surveys: Use questionnaires to ensure participants meet the criteria.
 Consent Forms: Explain the study’s purpose, risks, benefits, and rights to
participants.
 Medical Screening: Conduct tests if health conditions might affect results.

2.10.7 Randomization (if applicable)

 Assign participants to different experimental groups randomly to reduce bias.
 Methods: Random number generators, software tools, or stratified randomization.

2.10.8 Ethical Considerations

 IRB Approval: Obtain clearance from an Institutional Review Board (IRB) or Ethics
Committee.
 Informed Consent: Ensure participants voluntarily agree after being informed of all
relevant details.
 Privacy: Protect participant data and maintain confidentiality.

2.10.9 Plan the Testing Environment

 Standardization: Create consistent conditions across all tests.

 Training: Train researchers and staff to ensure uniform protocols.
2.10.10 Conduct Pilot Testing
 Test the experimental design with a small sample to identify and fix issues before the
main study.
2.11 When deciding between a within-subject or between-subject :
1. Understand the Two Study Designs
Within-Subject Design
 Definition: Each participant is exposed to all experimental conditions or treatments.
 Key Advantages:
o Requires fewer participants.
o Eliminates individual differences as a confounding variable.
o Provides more statistical power.
 Key Challenges:
o Risk of carryover effects (e.g., learning, fatigue).
o May require counterbalancing to mitigate order effects.
Between-Subject Design
 Definition: Different groups of participants are exposed to different conditions or
treatments.
 Key Advantages:
o No carryover or order effects.
o Easier to implement when testing interventions that cannot be undone or
reversed.
 Key Challenges:
o Requires more participants to achieve statistical power.
o Potential for group differences to introduce variability.
2. Key Considerations for Selection
1. Nature of the Experiment:
o Use a within-subject design when studying relative effects (e.g., comparing
reactions to different stimuli in the same individual).
o Opt for a between-subject design when the intervention is irreversible (e.g.,
long-term behaviour change).
2. Practical Constraints:
o Within-Subject: Suitable when participant recruitment is difficult.
o Between-Subject: Preferred when there's concern about fatigue or learning
effects in repeated testing.
3. Ethical Implications:
o Ensure that participants in a between-subject design are not disadvantaged by
being in a control group.
 o Minimize discomfort or confusion in within-subject designs due to repeated
exposures.
4. Statistical Power:
o Within-subject designs generally require fewer participants and have higher
sensitivity to detecting differences.
o Between-subject designs rely on more participants to average out individual
differences.

3. Steps for Experimental Design

1. Define the Research Question:
Clearly state the objective and determine if the study requires comparing conditions
within the same individuals or across different groups.
2. Recruit Participants:
o Ensure a diverse and representative sample.
o Calculate the required sample size based on the study design and statistical
goals.
3. Randomization and Counterbalancing:
o For within-subject designs, use counterbalancing to address order effects
(e.g., Latin Square design).
o For between-subject designs, randomize participants to groups to reduce bias.
4. Control Variables:
o In between-subject designs, ensure groups are equivalent by controlling for
confounding variables.
o In within-subject designs, minimize environmental factors that could
influence results between conditions.
5. Pilot Testing:
Conduct a small-scale trial to identify potential issues such as carryover effects, task
complexity, or participant fatigue.

2.12 Counterbalancing:
Counterbalancing is a technique used in within-subject experimental designs to reduce the
impact of order effects, such as learning, fatigue, or carryover effects, that arise when
participants experience multiple conditions.

2.12.1 Why Use Counterbalancing?

1. Order Effects:
 o Participants' responses may be influenced by the sequence in which conditions
are presented.
 Examples: Improved performance due to practice (learning effect) or
reduced performance due to exhaustion (fatigue effect).
2. Carryover Effects:
o Experiencing one condition might influence responses to subsequent
conditions.
 Example: A strong emotional stimulus in the first condition could alter
responses in later conditions.
3. Fair Representation:
o Ensures that no single condition benefits or suffers disproportionately due to
its position in the sequence.
2.12.2 Types of Counterbalancing

1. Complete Counterbalancing:
o All possible orders of conditions are presented.
o Best suited for studies with a small number of conditions.
o Example:
For three conditions (A, B, C), all six orders are tested:
 ABC, ACB, BAC, BCA, CAB, CBA.
o Challenges:
 Number of orders grows factorially (n!, where n is the number of
conditions).
 Difficult to implement with many conditions.
2. Partial Counterbalancing:
o A subset of all possible orders is used.
o Often employs techniques like a Latin Square design to ensure each
condition appears in every position at least once.
o Example:
For three conditions (A, B, C):
 Order 1: ABC
 Order 2: BCA
 Order 3: CAB.
o Benefits:
 Reduces complexity while still controlling order effects.
3. Randomized Counterbalancing:
 o The order of conditions is randomized for each participant.
o Works well for large sample sizes where randomization averages out biases.
4. Block Randomization:
o Conditions are divided into blocks, and the order within each block is
randomized.
o Useful for longer studies to prevent prolonged exposure to a single condition.

2.13 Independent and Dependent Variables

In experimental research, the independent and dependent variables are crucial for establishing
cause-and-effect relationships. Here's an overview:

1. Independent Variable (IV)

 Definition:
The variable that the researcher manipulates or controls to observe its impact on the
dependent variable.
 Purpose:
To determine how changes in the independent variable influence the outcome.
 Examples:
o In a study on teaching methods: Teaching method (e.g., traditional vs. online)
is the IV.
o In a drug efficacy test: Dosage of the drug is the IV.
 Key Characteristics:
o It is the presumed cause.
o Typically has different levels or conditions (e.g., high vs. low dosage).

2. Dependent Variable (DV)

 Definition:
The variable that is measured or observed to assess the effect of changes in the
independent variable.
 Purpose:
To evaluate the outcome or result influenced by the IV.
 Examples:
o In a study on teaching methods: Students’ test scores are the DV.
o In a drug efficacy test: Improvement in symptoms is the DV.
 Key Characteristics:
o It is the presumed effect.
o Depends on the manipulation of the IV.
2.14 A/B Testing: An Overview
Definition:
A/B testing is a method of comparing two versions of a product, webpage, or experience
(Version A and Version B) to determine which performs better based on a defined metric,
such as conversion rate, click-through rate, or user engagement.
2.14.1 Key Components of A/B Testing
1. Version A (Control):
o The original version that serves as the baseline.
2. Version B (Variant):
o A modified version of the control, incorporating a change designed to improve
performance.
3. Hypothesis:
o A testable prediction about how the change in Version B will affect the
outcome.
o Example: "Changing the button color from blue to green will increase click-
through rates."
4. Metrics:
o The measurable outcome used to evaluate success (e.g., conversions, time
spent on a page, purchases).
5. Target Audience:
o The segment of users randomly divided into groups to receive either Version
A or Version B.
2.14.2 How A/B Testing Works: Step-by-Step

1. Define the Goal:

o Identify the key performance indicator (KPI) you want to optimize (e.g.,
newsletter sign-ups, purchases).
2. Develop Hypotheses:
o Predict how a specific change will impact the KPI.
o Example: "Adding customer testimonials to the homepage will increase trust
and lead to higher sales."
3. Create Variants:
o Develop the control (Version A) and the variant (Version B).
o Ensure that the change is isolated to one element to accurately measure its
impact.
4. Split the Audience:
 o Randomly divide the audience into two groups:
 Group 1 sees Version A (Control).
 Group 2 sees Version B (Variant).
5. Run the Test:
o Expose the versions to users simultaneously under identical conditions.
6. Measure Results:
o Collect data on the predefined metric (e.g., clicks, purchases).
7. Analyze and Interpret Data:
o Use statistical analysis to determine if the difference between Version A and
Version B is significant.
8. Implement the Winning Version:
o If the variant performs better, implement it for all users.
2.15 Multivariate Analysis (MVA)
Multivariate analysis refers to a set of statistical techniques used to analyze data that involves
multiple variables simultaneously. It explores relationships, patterns, and influences between
variables, often with the goal of understanding complex phenomena or making predictions.
2.15.1 Why Use Multivariate Analysis?
 To identify relationships between variables.
 To determine the influence of one or more independent variables on one or more
dependent variables.
 To reduce the dimensionality of data while preserving key information.
 To test hypotheses involving multiple variables.
2.15.2 Key Types of Multivariate Analysis:
1. Descriptive Multivariate Techniques
Used to explore and summarize relationships between variables without focusing on
causation.
 Principal Component Analysis (PCA):
Reduces data dimensionality by identifying key components that explain the most
variance.
o Example: Identifying main factors influencing customer satisfaction.
 Factor Analysis:
Groups related variables into factors to uncover latent constructs.
o Example: Grouping survey items into broader dimensions like "customer
loyalty" or "brand trust."
  Cluster Analysis:
Groups similar observations into clusters.
o Example: Segmenting customers into groups based on purchasing behaviour.

2. Inferential Multivariate Techniques

Used to test hypotheses and examine cause-and-effect relationships.
 Multiple Regression Analysis:
Explores the relationship between one dependent variable and multiple independent
variables.
o Example: Predicting house prices based on size, location, and number of
bedrooms.
 Multivariate Analysis of Variance (MANOVA):
Extends ANOVA to multiple dependent variables.
o Example: Studying the effect of teaching methods on test scores and student
motivation.
 Discriminant Analysis:
Determines which variables best differentiate between predefined groups.
o Example: Identifying factors that distinguish between high- and low-
performing employees.
3. Predictive Multivariate Techniques
Used for forecasting or predicting outcomes based on input variables.
 Logistic Regression:
Predicts a categorical dependent variable using one or more independent variables.
o Example: Predicting customer churn (yes/no) based on usage data.
 Time Series Multivariate Analysis:
Examines multiple time-dependent variables to predict future trends.
o Example: Forecasting sales based on advertising spend and seasonality.
 Canonical Correlation Analysis (CCA):
Assesses relationships between two sets of variables.
o Example: Linking financial performance indicators with market conditio ns.

2.16 Controlled Experiments

A controlled experiment is a scientific study where one or more
independent variables are manipulated while all other variables are kept constant to observe
their effect on the dependent variable. This method is fundamental in establishing cause-and-
effect relationships.
2.16.1 Key Elements of Controlled Experiments:
1. Independent Variable: The factor that is intentionally changed or manipulated to
observe its effect.
2. Dependent Variable: The outcome or response that is measured to assess the effect
of the independent variable.
3. Control Group: A group that does not receive the experimental treatment or is
subjected to standard conditions. This group serves as a baseline for comparison.
4. Experimental Group: A group that receives the treatment or condition being tested.
5. Controlled Variables: All other factors are kept constant to prevent them from
influencing the results.
2.16.2 Steps in Conducting Controlled Experiments:
1. Define the Hypothesis: Clearly state what you aim to test.
2. Identify Variables: Determine the independent, dependent, and controlled variables.
3. Set Up Groups: Divide subjects into control and experimental groups randomly.
4. Apply Treatment: Administer the experimental condition only to the experimental
group.
5. Measure Outcomes: Collect data on the dependent variable for both groups.
6. Analyze Results: Compare the outcomes between the control and experimental
groups to draw conclusions.

2.17 Basics of Data Analysis:

Data analysis involves examining, organizing, and interpreting data to uncover patterns,
draw conclusions, and inform decisions. Whether qualitative or quantitative, it is a structured
approach to making sense of raw data.
Steps in Data Analysis
1. Define Objectives
o Identify what you want to analyse and why.
o Formulate clear questions or hypotheses.
2. Collect Data
o Gather data through surveys, experiments, observations, or secondary sources.
o Ensure the data is relevant, accurate, and reliable.
3. Clean Data
o Handle missing, inconsistent, or irrelevant data.
o Standardize formats and remove outliers if necessary.
4. Organize Data
o Use tools like spread sheets or databases to sort and structure data.
 o Categorize or segment data into meaningful groups.
5. Analyze Data
o Apply statistical methods, tools, or qualitative techniques.
o Use exploratory data analysis (EDA) to identify patterns or trends.
6. Interpret Results
o Relate findings back to the objectives or hypotheses.
o Identify insights and implications of the results.
7. Present Findings
o Use visualizations (charts, graphs, tables) to convey insights clearly.
o Write reports or summaries to communicate results effectively.

2.18 Types of Data

Data can be broadly categorized into two main types: qualitative and quantitative. Each has
subtypes that describe its characteristics and applications.
1. Qualitative Data

Qualitative data is descriptive and non-numerical. It captures qualities, characteristics, or

attributes.

Key Features
 Cannot be measured numerically but can be categorized.

 Often collected through interviews, surveys, or observations.

 Helps in understanding underlying opinions, motivations, or trends.

Subtypes of Qualitative Data

 Nominal Data:
o Categories without a logical order.
o Example: Gender (male, female), Colors (red, blue, green).
 Ordinal Data:
o Categories with a logical order but no consistent difference between ranks.
o Example: Customer satisfaction (poor, average, good, excellent)

2. Quantitative Data

Quantitative data is numerical and can be measured or counted.

Key Features
 Used for statistical analysis.

 Can show trends, correlations, and cause-effect relationships.

Subtypes of Quantitative Data
 Discrete Data:
o Countable, finite values.

o Example: Number of employees, number of cars in a parking lot.

 Continuous Data:
o Measurable and can take any value within a range.
o Example: Height, weight, temperature.

2.19 Metrics Data:

Metrics data refers to quantifiable measurements used to assess, track, and analyze the
performance, progress, or quality of specific processes, systems, or objectives. It is essential
in fields like business, research, IT, healthcare, and education to make informed decisions
based on evidence.
2.19.1 Types of Metrics Data
1. Key Performance Indicators (KPIs)
 High-level metrics that reflect critical success factors.
 Example:
o Business: Revenue growth rate, customer retention rate.
o Education: Graduation rates, student satisfaction scores.
2. Operational Metrics
 Day-to-day performance measures for operational processes.
 Example: Average response time, production efficiency.

3. Financial Metrics
 Metrics focusing on the financial performance of an organization.
 Example: Profit margins, return on investment (ROI), cash flow.
4. Productivity Metrics
 Measures of efficiency and output.
 Example: Employee output per hour, sales per employee.

5. Quality Metrics
 Used to measure the quality of outputs or processes.
 Example: Customer complaint rate, defect rate.
6. Customer Metrics
 Metrics to understand customer behaviour and satisfaction.
 Example: Net Promoter Score (NPS), customer lifetime value (CLV).
7. Website/Software Metrics
  Data about user engagement, system performance, or functionality.
 Example: Page views, bounce rate, server uptime.

2.20 Comparing Means:

Comparing means is a statistical process used to evaluate whether the average values of two
or more groups differ significantly. It is commonly applied in fields like research, business,
and education to test hypotheses and make data-driven decisions.
Key Methods for Comparing Means
1. Two-Sample Comparison
 Independent Samples t-Test:
o Compares the means of two independent groups.
o Example: Examining whether two different teaching methods result in
different average test scores.
o Assumptions:
 Data in each group is normally distributed.
 Equal variance between groups (use Welch’s t-test if this assumption is
violated).
 Observations are independent.
 Mann-Whitney U Test:
o A non-parametric alternative to the t-test, used when data is not normally
distributed.
o Example: Comparing customer satisfaction scores between two products.

2. Paired Comparison
 Paired Samples t-Test:

o Compares means from the same group at two different time points or under
two conditions.
o Example: Testing the effect of a training program by comparing employee
performance scores before and after the program.
o Assumptions:
 Data is normally distributed.
 Paired observations (e.g., same subjects measured twice).
 Wilcoxon Signed-Rank Test:

3. Multiple Group Comparison

 One-Way ANOVA (Analysis of Variance):
o Compares the means of three or more independent groups.
 o Example: Evaluating whether average customer ratings differ across three
store locations.
o Assumptions:
 Data in each group is normally distributed.
 Homogeneity of variances (use Welch’s ANOVA if violated).
 Groups are independent.
 Kruskal-Wallis Test:
o A non-parametric alternative to one-way ANOVA for non-normally
distributed data.
o A non-parametric alternative to the paired t-test for non-normally distributed
data.

2.21 Correlations

Correlation measures the strength and direction of a relationship between two variables. It
does not imply causation but helps understand how variables move together.
Types of Correlation
1. Positive Correlation:
o Both variables increase or decrease together.
o Example: As income increases, spending increases.
2. Negative Correlation:
o One variable increases while the other decreases.
o Example: As exercise time increases, weight decreases.
3. No Correlation:
o No discernible relationship between the variables.
o Example: Hair color and exam scores.

2.22 Non-Parametric Tests

Non-parametric tests are statistical tests that do not assume a specific distribution for the
data (e.g., normal distribution). These tests are ideal for analysing ordinal data, non-normal
distributions, or when sample sizes are small.

2.22.1 Common Non-Parametric Tests

1. Comparing Two Groups
 Mann-Whitney U Test:
o Compares medians of two independent groups.
 o Example: Comparing customer satisfaction scores between two companies.
o Parametric Equivalent: Independent t-test.
 Wilcoxon Signed-Rank Test:
o Compares medians of two related groups.
o Example: Comparing pre- and post-test scores of students.
o Parametric Equivalent: Paired t-test.
2. Comparing Multiple Groups
 Kruskal-Wallis Test:
o Compares medians of three or more independent groups.
o Example: Comparing average ratings for three products.
o Parametric Equivalent: One-way ANOVA.
 Friedman Test:
o Compares medians of three or more related groups.
o Example: Comparing performance under three different teaching methods.
o Parametric Equivalent: Repeated Measures ANOVA.
3. Correlation
 Spearman’s Rank Correlation:
o Measures the monotonic relationship between two variables.
o Example: Relationship between ranks in sales and advertising spend.
o Parametric Equivalent: Pearson’s correlation.
 Kendall’s Tau:
o Measures association between two ordinal variables.
o Example: Examining the association between ranks given by two judges.
4. Goodness of Fit
 Chi-Square Goodness of Fit Test:
o Tests whether observed categorical data matches an expected distribution.
o Example: Testing if dice rolls are fair.
o Parametric Equivalent: None (used for categorical data).
5. Test of Association
 Chi-Square Test of Independence:
o Tests the association between two categorical variables.
o Example: Analysing if gender is related to product preference.
 Fisher’s Exact Test:
o Similar to the Chi-Square test but used for small sample sizes.
o Example: Testing association between two binary variables.
2.22 Non –Parametric Test:
Non-Parametric Tests: Overview
Non-parametric tests are statistical tests that do not rely on specific assumptions about the
underlying data distribution. These tests are versatile and often used when:

1. The data is ordinal, nominal, or non-normal.

2. The sample size is small.
3. The assumptions of parametric tests are violated.

Categories of Non-Parametric Tests

1. Tests for One Sample

 Chi-Square Goodness-of-Fit Test:
o Purpose: Tests if the observed data fits an expected distribution.
o Example: Checking if the distribution of votes matches predicted proportions.
 Kolmogorov-Smirnov Test:
o Purpose: Compares a sample to a reference probability distribution.
o Example: Testing if data follows a uniform distribution.
 Runs Test:
o Purpose: Tests for randomness in a sequence.
o Example: Analyzing if coin toss outcomes are random.

2. Tests for Two Independent Samples

 Mann-Whitney U Test:
o Purpose: Compares the medians of two independent groups.
o Example: Comparing customer satisfaction scores between two brands.
o Parametric Equivalent: Independent t-test.
 Kolmogorov-Smirnov Test for Two Samples:
o Purpose: Compares the distribution of two independent samples.
o Example: Testing if two groups have the same income distribution.