Challenges of Data Science Types of Data What is Statistical Data Modeling?

1. Preparation of Data for Smart 1. Structured Data:Has a predefined format Statistical data modeling is the process of
Enterprise AI-Finding and cleaning proper and is machine-readable.-Organized in applying statistical analysis to datasets to
data is the top priority.-Around 80% of time tables (rows and columns).-Easy to store identify relationships between variables
is spent on cleaning, organizing, and and [Link]: Names, phone and make predictions.
gathering data.-Data exists in different numbers, dates, prices Statistical Modeling Techniques
formats and platforms, making it complex. 2. Unstructured Data:Does not follow any 1. Supervised Learning:
2. Generation of Data from Multiple predefined format.-Requires human Uses labeled data (with known outputs
Sources-Data is collected from various interpretation.-Difficult to analyze directly Helps in prediction and classification
programs, software, and tools.-Managing Examples: Images, videos, audio files, Types:-Regression Model: Finds
large volumes of data is difficult.-Manual social media posts. relationship between variable
data entry and compilation are time- 3. Semi-Structured Data: Classification Model: Classifies data into
consuming.3. Identification of Business -Combination of structured and categories.2. Unsupervised Learning;
Issues-Identifying the correct problem is unstructured data.-Has some organization -Uses unlabeled data-Finds patterns and
very important.-Data scientists must focus but not fully structured/ Example: Email relationships automatically
on enterprise-critical challenges.-Need to (header = structured, body = unstructured) Types:Clustering (e.g., K-means): Groups
understand the root cause before analysis. similar data -Association Rules: Finds
[Link] of Results to Non- Types of Data Models relationships between data
Technical Stakeholders-Difficult to explain Data models are divided into three main
technical results to [Link] types based on the level of abstraction: Computational modeling is the use of
may not understand tools and 1. Conceptual Data Model: computers to simulate and study complex
technologies.-Requires proper explanation -Provides a high-level (big picture) view of systems using mathematics, physics, and
and visualization.5. Data Security: the system.-Shows what data is needed computer science.A computational model
-Data stored in cloud is vulnerable to and how it is organized.-Includes entities, contains many variables that represent the
cyberattacks.-Sensitive information may be relationships, and basic rules system. By changing these variables,
exposed.-Requires strong security 2. Logical Data Model:-More detailed than different outcomes can be observed.
measures and encryption.6. Efficient conceptual model.-Defines data attributes, Uses :[Link] forecasting: Predicts
Collaboration-Data scientists and data types, and relationships.-Does not weather conditions using atmospheric
engineers work together. include technical details.3. Physical Data [Link] simulators: Helps in pilot
-Poor communication can lead to conflicts. Model:.-Lowest level of abstraction training and aircraft design.
-Requires clear communication channels. Shows how data is physically stored in the [Link] simulations: Predicts impact
database.-Includes tables, primary keys, on buildings and infrastructure
What is data science used for. foreign keys, and DBMS details
1. Descriptive Analysis Bonferroni's principle is an informal
-Helps understand what happened Data modelling is the process of creating a presentation of a statistical theorem that
-Uses charts, graphs, and reports visual representation of a whole information states if your method of finding significant
Example: Finding sales trends or booking system or its parts to show the items returns significantly more items that
patterns2. Diagnostic Analysis: relationships between data and how it is you would expect in the actualpopulation,
-Helps understand why something [Link] helps to:-Illustrate the types of you can assume most of the items you find
happened.-Uses data mining, drill-down, data used and stored-Show relationships with it are bogus. This essentially means
and correlations Example: Finding reasons among data-Organize data into proper that an algorithm or method we think is
for a sudden increase in sales formats and structures useful for finding a particular set of data
3. Predictive Analysis:-Helps predict what actually returns more false positives as it
will happen in the future Data modelling process: returns larger portions of the data than
-Uses machine learning and forecasting 1. Identify the Entities:-Determine the main should be within that category.
[Link]: Predicting future objects or concepts (e.g., customer,
demand or trends4. Prescriptive Analysis: order).-Each entity should be unique and An attribute is a data field representing a
-Suggests what action should be taken clearly defined2. Identify Key Properties characteristic or feature of a data object.
-Recommends the best decision using (Attributes):-Define the characteristics of It is also called dimension, feature, or
advanced [Link]: Deciding best each entity.-Example: Customer → name, [Link] of Data Attributes:
marketing strategy phone number.3. Identify Relationships: 1. Nominal Attribute:
Determine how entities are related to each -Values are names or categories
What is Data Science Process? [Link]: Customer “lives at” an -No meaningful order.-Also called
The data science process begins with a address.-4. Assign Keys and Normalize: categorical [Link]: hair color,
business problem. A data scientist works Assign primary keys to uniquely identify marital status.2. Binary Attribute:
with stakeholders to understand the records.-Use normalization to reduce -Has only two values (0 or 1)0 = absence, 1
problem and solve it using the OSEMN redundancy and improve efficiency = presence.-Can be symmetric or
[Link] Data Science Process 5. Finalize and Validate the Model: asymmetric;Example: smoker (yes/no)
1. O – Obtain Data:Collect data from -Review and refine the mode 3. Ordinal Attribute:
various sources-Sources include -Ensure it meets business requirements -Values have a meaningful order or rankin
databases, CRM software, web logs, social - It update as needed -Difference between values is not known
media, or third-party data -Example: small, medium, large
2. S – Scrub Data (Data Cleaning) Benefits of Data Modeling 4. Numeric Attribute:Represents
-Clean and standardize the data 1. Better Understanding of Data: quantitative values.-Can be measured
-Handle missing values, errors, and outliers Helps developers and stakeholders numerically:a) Equal intervals between
Example: fixing formats, removing understand relationships among data [Link]: temperature
duplicates3. E – Explore Data 2. Reduces Error:Minimizes mistakes in b) Ratio-Scaled:
-Perform initial analysis using statistics and database and software development -Has a true zero point
visualization.-Identify patterns and 3. Improves Consistenc:Ensures uniform -Ratios are meaningful
relationships in data4. M – Model Data design and documentation across the Example: height, weight, age
-Apply machine learning algorithms organization4. Enhances Performance:
-Build models for prediction and analysis Improves application and database
Test and improve accuracy performance.5. Eases Data Mapping
5. N – Interpret Results Makes it easier to integrate and manage
-Convert results into useful insights data across systems
-Use graphs, charts, and reports
-Help stakeholders make decisions