AST 101

Introduction to Applied Statistics and Data Science

—Lecture 3: Introduction to Data Science—

Syed Shahadat Hossain, Ph.D.

shahadat@[Link]

Professor
Institute of Statistical Research and Training
University of Dhaka, Bangladesh

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Basic Definitions

Data Science
Dictionary meanings of the word Data Science:
1 UNCOUNTABLE NOUN Data science is the scientific analysis of
large amounts of information held on computers, done for example in
order to learn about people’s shopping or voting behaviour. The
social network has its own in-house team working on data science.
[Collins Languages]

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Basic Definition

Data Science
Data science is the study of data to extract meaningful insights from data
generated perpetually in diverse domains of life. It is a multidisciplinary
approach that combines principles and practices from the fields of
mathematics, statistics, artificial intelligence and computer engineering to
analyze large amounts of data.

Mainly evolved to support the understanding and interpretation of the

large amounts of data which has been being amassed from the
processes of reporting, registering, recording
(manual/digital/automated) over time.
This helps data scientists to ask and answer questions like what
happened, why it happened, what will happen, and what can be done
with the results.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Historical Background (Cont.)

The term Data Science was first used in the early 1960s to describe a
new profession that would support analyzing the then hugely amassed
data particularly from businesses. (At the time, there was no way of
predicting the truly massive amounts of data over the next sixty
years.)
In 1962, John Tukey wrote a paper titled The Future of Data Analysis
and described a shift in the world of statistics, saying, “. . . as I have
watched mathematical statistics evolve, I have had cause to wonder
and to doubt. . . I have come to feel that my central interest is in data
analysis. . . ”
In 1974, Peter Naur authored the Concise Survey of Computer
Methods where he used the term Data Science repeatedly.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Historical Background (Cont.)

In 1977, The International Association for Statistical Computing

(IASC) was formed with a mission statement emphasizing linking
traditional statistical methodology, modern computer technology, and
the knowledge of domain experts.
In 1994, Business Week ran the cover story, Database Marketing,
revealing the ominous news companies had started gathering large
amounts of personal information, with plans to start strange new
marketing campaigns. The flood of data was, at best, confusing to
many company managers, who were trying to decide what to do with
so much disconnected information.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Historical Background (Cont.)

In 1999, in Mining Data for Nuggets of Knowledge, Jacob Zahavi

pointed out the need for new tools beyond conventional statistical
methods
In 2001, Software-as-a-Service (SaaS) was created. This was the
pre-cursor to using cloud-based applications.
In 2001, William S. Cleveland laid out plans for training data
scientists to meet the needs of the future.
In 2002, the International Council for Science: Committee on Data for
Science and Technology began publishing the Data Science Journal

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Historical Background (Cont.)

In 2006, Hadoop 0.1.0, an open-source, non-relational database, was

released.
In 2008, the title, “data scientist” became a buzzword,
In 2009, the term NoSQL was reintroduced (a variation had been
used since 1998)
In 2011, job listings for data scientists increased by 15,000%.
Since 2011, there was also an increase in seminars and conferences
devoted specifically to Data Science and big data.
In 2015, declared as a a landmark year for AI, using Deep Learning
techniques, Google’s speech recognition, Google Voice, experienced a
dramatic performance jump of 49 percent.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Historical Background (Cont.)

2023 experiences the magic of ChatGPT.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Historical Background (Cont.)

Modern stalwarts in the development of the subject of Data Science

is
John Tukey (1915-2000) contributed greatly to statistical practice and
data analysis in general. In fact, some regard John Tukey as the father
of Data Science. At the very least, he pioneered many of the key
foundations of what came later to be known as Data Science. He was
credited with coining two terms used in the computer industry: the
first was “software,” or the programs used to run a computer, and the
second was “bit,” a contraction for binary digit.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Statistics, Applied Statistics and Data Science

Subject matter: Data

Statistics/Applied Statistics Data Science
Data Size Small, Moderate Huge
Data Manual, Calculator Computer
Handling Computer
Data Designed way Automatic
generation Needs Scientific Rigour Needs Digital
Systems in
operation

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Statistics, Applied Statistics and Data Science

Subject matter: Data

Statistics Data Science
/Applied Statistics
Data Paper, Pen-drive, Central Server
Storage laptop/desktop drive /Cloud
Data Manually Customized, Data base prepared
Management Spread Sheet using dedicated
Data Management
Software (Python, Java..)
Data Manual, Direct Retrieval from
Input entry into multiple data bases
Calculator/Computer, Using designated
retrieval from programming languages
spreadsheet like R, Hadoop, Spark, SQL...

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Statistics, Applied Statistics and Data Science

Subject matter: Analysis

Statistics Data Science
/Applied Statistics
Analysis Visualization Visualization
Types Description Pattern discovery
Prediction Prediction
Scientific
Generalization
Analysis Summary measures Summary measures
Tools Graphs and charts Info-graphics
Predictive Models Predictive Models
Inferential tools like
Estimation and
Test of Hypotheses

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Statistics, Applied Statistics and Data Science

Subject matter: Analysis

Statistics Data Science
/Applied Statistics
Computation Scientific Statistical Softwares
Tools Calculators, Excel, Python, R, SQL,
Statistical mostly by retrieving
Softwares like C++, algorithms and pakages
R, Stata, SPSS, SAS available on internet/cloud

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Statistics, Applied Statistics and Data Science

Subject matter: Analysis

Statistics Data Science
/Applied Statistics
Decision Probabilistic, Mostly deterministic,
Making States level of Rarely states level
Confidence of Credibility
and validity

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Careers in Data Science

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Skills needed for Careers in Data Science

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Skills used for Careers in Data Science

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Careers in Data Science (Cont.)

Role of different Levels of Data Scientist

Data scientist: Data scientists build models using programming
languages such as Python. They then transform these models into
applications. Often working as part of a team, for example, with a
business analyst, a data engineer, and a data (or IT) architect, they
help solve complex problems by analyzing data and making
predictions about the future. This role is typically considered an
advanced version of a data analyst.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Careers in Data Science (Cont.)

Role of different Levels of Data Scientist (cont.)

Data analyst: Data analysts, unlike data scientists, use structured
data to solve business problems. Using tools such as SQL, Python,
and R, statistical analysis, and data visualization, data analysts
acquire, clean, and reorganize data for analysis to spot trends that
can be turned into business insights. They tend to bridge the gap
between data scientists and business analysts.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Careers in Data Science (Cont.)

Role of different Levels of Data Scientist (cont.)

Data architect: Data architects create the blueprints for data
management systems, designing plans to integrate and maintain all
types of data sources. They oversee the underlying processes and
infrastructure. Their main goal is to enable employees to gain access
to information when they need it.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Careers in Data Science (Cont.)

Role of different Levels of Data Scientist (cont.)

Data engineer: Data engineers prepare and manage large amounts of
data. They also develop and optimize data pipelines and
infrastructure, getting the data ready for data scientists and business
analysts to work with. Data Engineers make the data accessible so
businesses can optimize their performance.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

 Careers in Data Science (Cont.)

Role of different Levels of Data Scientist (cont.)

Machine learning engineer: This role is not an entry-level position,
but one you can build toward as a data scientist or engineer. Machine
learning uses algorithms that replicate how humans learn and act, to
interpret data and build accuracy over time. As part of a data science
team, machine learning engineers research, build, and design artificial
intelligence that facilitates machine learning. They also serve as a
liaison between data scientists, data architects, and more.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]