Introduction to Data Science

Saturday, July 5, 2025 9:53 PM

Data science is a field of study that investigates how to collect,

manage, and analyze data of all types in order to retrieve meaning
information. You can consider 'data' to be any pieces of evidence or
observations that can be analyzed to provide some insights.

The Data Science Lifecycle (DSLC)

Data science tasks follow a process, called the data science
lifecycle, which includes problem definition, then data collection,
preparation, analysis, and reporting, which describes the data
science lifecycle in much the same way.

Problem Definition, Data Collection, and Data

Preparation
The first step in the Data Science lifecycle is a precise definition
of the problem statement to establish clear objectivies for the goal
and scope of the data analysis project. Once the problem has been
well defined, the data must be generated and collected.

Data Collection is the systematic process of gathering information

on variables of interest. A lot of data is around our daily life is
simply a by-product of our activity. A good example is our web
search history, We use google which stores our search history in
some google server. Google employees can utilize the records of a
number of users to analyze common search patterns, present accurate
search results, and potentially, to display relatable advertisements
back to the searcher.

Collected data is not in an optimal form for analysis. It needs to

be processed somehow so it can be analyzed, in a phase known as data
prepartion or data processing.

Let's assume you work at google, you want to find out 'what is the
most search food' people around the globe search about at during the
night. You have the user data but you can't use it. Reasons could
for user's keywords will probably be in different languages, and
users live in different parts of the world so the timezone is
different. Some keywords will have typos, be blank spave, or not
make sense. Note all these scenarios are possible, therefore these
issues should be addressed so that the actual data analysis is more
accurate in its results.

Data Analysis
Once the data is collected and prepared, it must be analyzed in
order to discover meaningful insights, a process called data
analysis. There a varient of data analysis methods to choose from,
such as simple ones like checking minimum and maximum values, to
more advanced ones such as modeling a dependet variable. Most of the

Data Science Page 1

more advanced ones such as modeling a dependet variable. Most of the
time data scientists start with the simple methods and then move to
more advanced methods based on what they want to investigate
further. Certain methods might contain 'Descriptive Statistics:
Statistical Measurements and Probability Distributions','Inferential
Statistics and Regression Analysis', 'Time series and Forecasting',
and 'Decision-Making using machine learning'.

Data Reporting
Data reporting involves the presentation of data in a way that will
best convey the information learned from data analysis. The
importance of data reporting cannot be overemphasized. Data
Scientist works with many domain experts in different fields. It is
the Data Scientist responsibility to communicate the results of
their analysis in a way that the domain experts can understand.

Data visualization is a graphical way of presenting and reporting

data to point out patterns, trends, and hidden insight; it involves
the use of visual elements such as charts, graphs, and maps to
present data in a way that is easy to understand and analyze. The
goal of data visualization to show information effectively and
facilitate better decision-making.

Visualization Techniques:

Histograms & box plots -> understand distributions and spread

Scatter plots -> reveal correlations and relationships between

features

Heatmaps -> visulize correlation matrices across all variables

Time series plots -> identify trends, seasonality, and drift

Key Objectives:

Validate hypotheses -> test initial assumptions about the data

Identify feature importance -> determine which variables carry the

most signal

Sport data quality issues -> catch problems missed during

preparation

Refine the dataset -> engineer new features based on discovered

patterns

Domain Expert(Subject Matter Expert) is a specialist who possesses

deep knowledge, skill, and experience in a specific non-data-science
field. They are grouped with data related to their field because
their expertise is the key to understanding and correctly
interpreting that data.

Data Science Page 2

Modeling
The modeling phase in a machine learning workflow is the stage where
algorithms are applied to prepared data in order to learn patterns,
relationships, or structures that can answer a specific business or
research problem. During this phase, data scientists select
appropriate algorithms, train models using historical data, and
adjust model parameters to improve predictive performance. The goal
is to create a model that can generalize well to new, unseen data
rather than just memorizing the training dataset.

1. Algorithm selection

Algorithm selection is the process of choosing the most appropriate

machine learning method based on the type of problem and the
structure of the dataset. Different algorithms are suited for
different tasks such as prediction, categorization, or pattern
discovery.

Regression

Regression is a type of supervised learning algorithm used when the

target variable is continuous (numeric).

The objective is to model the relationship between independent

variables (features) and a dependent variable (target) so the model
can predict numerical values.

Example:

• Predicting house prices based on size, location, and number of

rooms
• Predicting monthly sales revenue

Typical regression algorithms:

• Linear Regression
• Polynomial Regression
• Ridge/Lasso Regression

Classification

Classification is also a supervised learning technique, but it is

used when the target variable is categorical (discrete classes).
The model learns from labeled data to assign new observations to
predefined categories or classes.

Example:

• Email spam detection (Spam / Not Spam)

• Fraud detection (Fraud / Legitimate)
• Medical diagnosis (Disease / No Disease)

Typical classification algorithms:

• Logistic Regression

Data Science Page 3

 • Logistic Regression
• Decision Trees
• Random Forest
• Support Vector Machines (SVM)

Clustering

Clustering is an unsupervised learning technique used to group

similar data points together based on patterns in the data without
using labeled outcomes.

The goal is to discover hidden structures or natural groupings

within the dataset.

Example:

• Customer segmentation in marketing

• Grouping similar products based on purchase patterns
• Social network community detection

Typical clustering algorithms:

• K-Means Clustering
• Hierarchical Clustering
• DBSCAN

2. Hyperparameter Tuning

Hyperparameters are configuration settings that control how a

machine learning algorithm learns from data but are not learned
automatically during training.

Hyperparameter tuning involves systematically searching for the best

combination of these parameters to maximize model accuracy or
performance.

Common optimization techniques include:

• Grid Search – tests all possible parameter combinations within a

defined set.
• Random Search – randomly samples parameter combinations from a
defined range.
• Bayesian Optimization – uses probability models to intelligently
search for optimal parameters.

3. Cross-Validation

Cross-validation is a statistical technique used to evaluate how

well a machine learning model generalizes to unseen data.

A common approach is k-fold cross-validation, where the dataset is

divided into k equal subsets (folds):

1. The model is trained on k-1 folds.

Data Science Page 4

1. The model is trained on k-1 folds.

2. It is tested on the remaining fold.

3. This process repeats k times, each time using a different fold

as the test set.

The final model performance is calculated as the average of the

results across all folds, which helps reduce overfitting and biased
evaluation.

Evaluation
A model is only as good as its ability to generalize to unseen data
and meet the business objectives defined in the problem definition.
Evaluation is where you rigorously validate performance, interpret
results, and decide whether the model is ready for the real world.

Performance Metrics -> Measure accuracy, precision, recall, F1-

score, AUC-ROC, or RMSE depending on the problem type. Compare
against baseline and business thresholds.

Generalization Testing -> Validate on held-out test data the model

has never seen. Check for overfitting by comparing training vs. test
performance gaps.

Business Alignment -> Translate statistical metrics into business

impact. Does a 2% accuracy improvement justify deployment costs?
Stakeholders need answers in their language.

Iterate or Proceed -> If the model falls short, loop back to earlier
phases — refine features, collect more data, or try different
algorithms. Iteration is expected, not failure.

Deployment & Maintenance

The final step bridges the gap between a promising model and real-
world business value. Deployment means integrating the model into
production systems where it can serve predictions at scale - but the
work doesn't stop there.

1. Integration

Embed the model into applications, dashboards, or operational

workflows via APIs or microservices.

2. Monitoring

Track model performance, data drift, and prediction quality in real

time to catch degradation early.

3. Retraining

Data Science Page 5

Schedule periodic retraining with fresh data to keep the model
accurate as business conditions evolve.

4. Pipeline Maintenance

Maintain data ingestion pipelines, feature stores, and

infrastructure to ensure reliable, continuous operation.

Lifecycle Comparison: Effort vs Impact

Understanding where teams spend their time - and where the greatest
business impact is generated - helps organizations allocate
resources wisely across the lifecycle.

Core Components of Data Science

Extracting -> Gathering raw data from multiple sources

Preparing -> Cleaning and organizing data for analysis

Analyzing -> Discovering patterns and insights

Visualizing -> Presenting findings in clear formats

Maintaining -> Managing and updating information systems

Analysis Techniques

Exploratory Analysis -> Discovering patterns and relationships in

data

Predictive Analysis -> Forecasting future trends and outcomes

Regression -> Modeling relationships between variables

Text Mining -> Extracting insights from textual data

Roles in Data Science

The field of data science encompasses a wide array of specialized
roles, each contributing unique skills to transform raw data into
valuable insights and strategic decisions.

Data Analyst -> Interprets data to identify trends and create

reports.

Data Engineer -> Builds and maintains robust data infrastructure and
pipelines.

Data Administrator -> Manages, secures, and optimizes database

systems.

Data Science Page 6

ML Engineer -> Designs, builds, and deploys machine learning models.

Data Scientist -> Analyzes complex data to derive actionable

insights and predictions.

Data Architect -> Designs and oversees an organization's data

strategy and systems.

Statistician -> Applies statistical methods to collect, analyze, and

interpret data.

Business Analyst -> Translates business needs into data requirements

and solutions.

Data & Analytics Manager -> Leads data teams and strategic data
initiatives

Data Engineer
Core Skills

• Information architecture

• Build data pipelines and storage

• Maintain data access

Essential Tools

• SQL: Store and organize data

• Java, Scala, Python: Process data
• Shell: Automate tasks
• Cloud: AWS, Azure, Google Cloud

Key Contributions

Data Engineers are the backbone of data infrastructure, ensuring

data is accessible, reliable, and optimized for analysis.

Core Competencies

• Data Architecture Design

• ETL/ELT Development & Optimization
• Database Management & Performance
• Cloud Infrastructure Expertise
• Data Governance & Security

Key Activities

• Building scalable data pipelines

• Maintaining data warehouses/lakes
• Implementing data integration solutions
• Monitoring data quality and flow
• Collaborating with Data Scientists/Analysts

Data Science Page 7

• Collaborating with Data Scientists/Analysts

Data Analyst
Key Responsibilities

• Perform descriptive analyses

• Create reports and dashboards
• Clean data for analysis

Primary Tools

• SQL: Retrieve and aggregate data

• Spreadsheets: Excel, Google Sheets
• BI Tools: Tableau, Power BI, Looker
• Optional: R and Python for cleaning

Core Contributions

Data Analysts bridge the gap between raw data and actionable
business decisions, providing clarity and direction through their
insights.

Core Competencies

• Data Extraction & Transformation

• Statistical Analysis & Interpretation
• Data Visualization Techniques
• Business Domain Knowledge
• Communication & Presentation Skills

Key Activities

• Gathering & Cleaning Datasets

• Performing Exploratory Data Analysis (EDA) • Developing Reports &
Dashboards
• Identifying Trends & Patterns
• Translating Data into Business Recommendations

Data Scientist
Expertise Areas

• Versed in statistical methods

• Run experiments for insights
• Traditional machine learning

Technical Stack

• SQL: Retrieve and aggregate data

• Python and/or R: Data science libraries

Role of Data Scientist

Data Science Page 8

Data Handling -> Works with structured and unstructured data in raw
format

Data Processing -> Extensive preprocessing, cleaning, and organizing

datasets

Statistical Analysis -> Describes, visualizes, and hypothesizes

using statistical methods

Predictive Modeling -> Uses advanced ML algorithms to predict events

and drive decisions

The Function of Data Science

Exploratory Analysis -> Discover insights from data patterns

Predictive Analytics -> Identify future event occurrences

Decision Science -> Make data-driven predictions and choices

Data Science combines predictive causal analytics, prescriptive

analytics, and machine learning to enable informed decision-making.

Skills Needed for Data Science

1. Application Domain

Quickly learn how data is used in specific contexts

2. Communication

Translate between technical terms and business vocabulary

3. Data Presentation

Understand data storage, linking, and metadata

4. Transformation & Analysis

Transform, summarize, and make inferences from data

5. Visualization

Create effective displays like charts to communicate results

6. Quality Assurance

Know data limitations, quantify accuracy, suggest improvements

Machine Learning Scientist

Specialized Tasks

• Predictions and extrapolations

Data Science Page 9

• Predictions and extrapolations
• Classifications
• Deep learning applications
• Image processing
• Natural language processing

Technical Tools

R and/or Python with machine learning libraries like TensorFlow or

Spark

Core Contributions

Machine Learning Scientists are at the forefront of AI development,

leveraging advanced algorithms to build intelligent systems that
drive innovation.

Core Competencies

• Advanced Statistical Modeling

• Machine Learning Algorithms (Supervised, Unsupervised)
• Deep Learning Architectures (CNNs, RNNs, Transformers)
• Feature Engineering & Selection
• Model Evaluation & Validation

Key Activities

• Develop & Train Predictive Models

• Design & Optimize Neural Networks
• Deploy & Monitor AI Solutions in Production • Conduct Research on
New ML Techniques
• Integrate ML into Product Features

Data Administrator
Responsible for defining data elements, data names, and their
relationship with the database analyst. Controls the data of the
database and determines what data to be stored based on
organizational requirements.

The Data Administrator works on requirements gathering, analysis,

and design phases, ensuring data is properly stored and organized to
meet business needs.

Data Architect
Technology Selection -> Chooses optimal data storage solutions

Query Optimization -> Creates and optimizes database queries

Design & Planning -> Develops technical specifications and plans

Security Management -> Monitors and ensures database security

Data Science Page 10

Data/Analytics Manager
Leadership Role

Provides strategic direction for analytics teams

Key Responsibilities

• Oversee analytics department work

• Ensure accuracy of data analysis
• Develop effective analysis processes
• Lead reporting initiatives

Business Analyst
Business Analysts act as a critical bridge between business needs
and data solutions. They leverage data to inform strategic decisions
and optimize operations, translating complex data findings into
actionable insights for stakeholders.

Core Competencies

• Proficient in statistical analysis and data interpretation

• Skilled in exploratory and predictive analytics techniques
• Strong understanding of business processes and objectives

Key Activities

• Deep-dive into business decisions and their outcomes

• Conduct thorough exploratory data analysis
• Design and create compelling data visualizations (charts, graphs)
• Communicate data-driven insights to business teams

Business Intelligence Manager

The Business Intelligence Manager leads the charge in transforming
raw data into actionable insights, driving strategic decision-making
across the organization.

Core Competencies

• Data Warehousing & ETL Processes

• Advanced Dashboarding & Reporting
• Data Modeling & SQL Expertise
• Proficiency in BI Tools (e.g., Tableau, Power BI)

Key Activities

• Develop BI Strategy & Roadmaps

• Lead BI Team & Mentor Analysts
• Communicate Insights to Stakeholders
• Monitor & Optimize BI Performance

Data Science Page 11

• Monitor & Optimize BI Performance
• Evaluate New BI Technologies

Data Security Issues

Essential Insights for Data Science -Understanding risks,
protections, and best practices in the modern data landscape.

What is Data Security?

Data Security is the process of protecting digital data from
unauthorized access, theft, or corruption. It ensures integrity,
confidentiality, and availability of data throughout the entire Data
Science Process.

Real-World Example: Healthcare organizations secure patient records

using encryption and access controls to comply with HIPAA
regulations - safeguarding sensitive health data from breaches.

Data Security vs. Data Privacy

Two essential but distinct concepts - often confused, but each
playing a unique role in protecting data and people.

Data Security -> Technical safeguards that protect data from

breaches - including encryption, firewalls, and access controls.

Focus: How data is protected

Data Privacy -> Policies and practices governing how data is

collected, shared, and used ethically - ensuring individual rights.

Focus: How data is governed

Example: Encryption protects a database from hackers (Security),

while GDPR governs how companies may collect and process personal
data (Privacy). Both are essential - neither alone is sufficient.

"Security protects data. Privacy governs data use."

Together, they form the twin pillars of responsible data

management - one technical, one ethical. Effective data science
demands both.

Top Data Security

Risks Understanding the most prevalent threats is the first step in
effective data protection. These risks evolve rapidly, demanding
vigilance and proactive strategies.

Accidental Exposure

Data inadvertently becomes accessible to unauthorized individuals

due to misconfigurations, human error, or oversight.

Data Science Page 12

due to misconfigurations, human error, or oversight.

Example: A cloud storage bucket misconfigured to be public, exposing

millions of customer records.

Phishing & Social Engineering

Cybercriminals manipulate individuals into divulging confidential

information or performing actions that compromise systems.

Example: An employee falls for an email impersonating a senior

executive, granting attackers access to critical systems.

Insider Threats

Security breaches originating from within an organization, either

maliciously or unintentionally, by current or former personnel.

Example: A departing data scientist intentionally copies proprietary

algorithms and client databases for personal use.

Beyond the common threats, these advanced risks demand specific

attention and robust defense mechanisms.

Ransomware Attacks

Malicious software encrypts data, holding it hostage until a ransom

is paid, often disrupting critical operations. For example, a
hospital's patient records are encrypted, demanding cryptocurrency
for decryption keys:

Attackers encrypt critical datasets and demand payment. A 2025

hospital attack delayed patient care for days.

Data Loss in the Cloud

Despite advanced cloud security, misconfigurations, accidental

deletion, or vendor issues can lead to irrecoverable data loss. For
instance, a company accidentally deletes a critical S3 bucket due to
incorrect permissions.

SQL Injection Vulnerabilities

Attackers insert malicious SQL code into input fields to manipulate

database queries, gaining unauthorized access or altering sensitive
data. A common example is bypassing website authentication to access
customer details.

Beyond finances, breaches cause lasting reputational damage —

eroding customer trust, triggering regulatory scrutiny, and reducing
market share. The Equifax breach remains a defining example of
systemic failure

Common Data Security Solutions & Techniques

Data Science Page 13

Access Controls

Role-based permissions and multi-factor authentication (MFA) limit

who can access sensitive data.

Encryption

Strong algorithms protect data at rest and in transit - rendering

stolen data unreadable.

Data Loss Prevention

DLP tools monitor and block unauthorized data transfers across

networks and endpoints.

Continuous Monitoring

Real-time auditing detects anomalies and compliance gaps before they

become breaches.

Example: Netflix deploys encryption and DLP across its global

infrastructure to protect millions of user profiles and viewing
histories from unauthorized access.

Data Science Page 14