DATA VISUALIZATION

Solved Internal Assessment - I

Q1. Importance of Data Visualization and Analytics

Data analytics is the process of collecting, cleaning, examining and interpreting data to
obtain useful information. Data visualization presents the results in the form of graphs,
charts and dashboards. Both help engineers understand complex data and make better
decisions.

Importance:
1. Helps identify trends, patterns and relationships quickly.
2. Makes large and complex datasets easier to understand.
3. Supports evidence-based decision making.
4. Helps detect errors, abnormal values and failures.
5. Improves communication of technical results.

Engineering applications:
• In manufacturing, sensor data can be analysed to predict machine failures and plan
maintenance.
• In software engineering, application logs can be analysed to identify modules producing
more errors.
• In transportation, traffic data can be visualized to identify peak hours and improve
route planning.
• In energy systems, electricity-consumption data can be analysed to estimate demand and
reduce wastage.

Example: A company can analyse previous sales data and visualize monthly demand to
estimate future demand. Engineers and managers can then plan production and inventory
accordingly.
Q2. Data Sources, Data Types, Observations, Variables and Measurement Scales
Data sources are the places from which data is collected.

Types of data sources:

1. Primary sources – data collected directly through surveys, experiments, sensors or
interviews.
2. Secondary sources – existing data obtained from databases, reports, websites or public
datasets.
3. Internal sources – data generated inside an organization, such as sales, production and
employee records.
4. External sources – government datasets, research repositories, social media and other
public sources.

Data types:
• Qualitative/categorical data: department, gender, product type.
• Quantitative/numerical data: age, marks, temperature and salary.
• Discrete data: number of students or machines.
• Continuous data: height, weight or voltage.

An observation is one recorded instance in a dataset. A variable is a characteristic

measured for each observation.

Measurement scales:
1. Nominal – categories without order, e.g., blood group or department.
2. Ordinal – ordered categories, e.g., poor, average, good and excellent.
3. Interval – equal intervals but no true zero, e.g., temperature in Celsius.
4. Ratio – equal intervals with a true zero, e.g., weight, height and income.
Q3. Data Quality and Its Dimensions
Data quality means the degree to which data is suitable, accurate and reliable for its
intended use. Good-quality data is essential because incorrect or incomplete data can lead
to wrong analysis and decisions.

Major dimensions of data quality:

1. Accuracy – data correctly represents the real-world value.
2. Completeness – required values are present and not missing.
3. Consistency – the same information follows the same format across sources.
4. Timeliness – data is available when it is needed and is sufficiently up to date.
5. Validity – values follow defined rules and acceptable formats.
6. Uniqueness – duplicate records are avoided.

Example: In a student database, marks should be correct, attendance should not be missing,
student IDs should be unique, and department names should use a consistent format.

Maintaining good data quality reduces errors, improves the reliability of statistical
analysis and machine-learning models, and supports better engineering and business
decisions.
Q4. Data Wrangling
Data wrangling is the process of converting raw, messy data into a clean and organized
form suitable for analysis.

Major steps:
1. Data collection – obtain data from files, databases, APIs, sensors or other sources.
2. Data cleaning – handle missing values, incorrect values, outliers and duplicate
records.
3. Data transformation – change formats, scale numerical values, convert data types and
create derived fields.
4. Data integration – combine data from multiple sources.
5. Data organization – arrange columns, records and categories in a useful structure.
6. Validation – check whether the processed data satisfies expected rules.

Example: Suppose student data contains names in different cases, missing marks and
duplicate student IDs. We can standardize names, fill or remove missing values according
to the analysis requirement, remove duplicates and convert marks into a numerical format.
The cleaned dataset can then be used for visualization and analysis.
Q5. Population, Sample and Sampling Techniques
Population is the complete group about which we want to draw conclusions. A sample is a
smaller subset selected from the population.

Example: If a college has 5,000 students, all 5,000 students form the population. If 500
students are selected for a survey, those 500 form the sample.

Sampling is used because studying the entire population may be expensive, time-consuming
or impractical.

Sampling techniques:
1. Simple random sampling – every member has an equal chance of selection. Example:
randomly selecting student IDs.
2. Systematic sampling – select every kth member after a random starting point.
3. Stratified sampling – divide the population into groups and sample from each group.
Example: selecting students from each department.
4. Cluster sampling – divide the population into clusters and select some complete
clusters.
5. Convenience sampling – select easily available participants; it is simple but may
introduce bias.

A properly selected sample should represent the population as closely as possible.

Q6. Data Transformation
Data transformation is the process of converting data into a suitable form for analysis or
modelling.

1. Scaling: Changes numerical variables to a comparable range.

Example: converting marks from 0–100 to 0–1.

2. Normalization: A common min-max transformation is:

x' = (x - xmin) / (xmax - xmin)
It converts values approximately to the range 0 to 1.

3. Standardization: Converts values using:

z = (x - mean) / standard deviation
The transformed variable has mean approximately 0 and standard deviation 1.

4. Type conversion: Changes data types, such as converting a date stored as text into a
date object or converting marks from strings to numbers.

5. Derived attributes: New attributes are created from existing attributes.

Example: Total_Marks = Maths + Science + English.
Another example is Percentage = Total_Marks / Maximum_Marks × 100.

Data transformation improves consistency and makes data more suitable for statistical
analysis, visualization and machine-learning algorithms.
Q7. Snowflake and SQL Queries
Snowflake is a cloud-based data platform and data warehouse used to store, process and
analyse large amounts of data. Its architecture separates storage and computing resources,
allowing them to scale independently.

Basic operations include creating databases and tables, loading data, querying tables,
filtering records, grouping data and joining related tables.

Example SQL queries:

CREATE DATABASE CollegeDB;

CREATE TABLE Students (

Student_ID INT,
Name VARCHAR,
Department VARCHAR,
Marks INT
);

SELECT * FROM Students;

SELECT Name, Marks

FROM Students
WHERE Marks > 75;

SELECT Department, AVG(Marks) AS Average_Marks

FROM Students
GROUP BY Department;

SELECT Department, COUNT(*) AS Student_Count

FROM Students
GROUP BY Department
ORDER BY Student_Count DESC;

SQL in Snowflake can therefore be used to retrieve, filter, aggregate and analyse data
efficiently.
Q8. Student Dataset Using Pandas
Assume the dataset contains Student_ID, Name, Department, Marks, Attendance and Grade.
Pandas can be used to perform common data-preparation operations.

1. Import data:
import pandas as pd
df = pd.read_csv("[Link]")

2. Filter records:
df = df[df["Marks"] >= 40]

3. Sort records:
df = df.sort_values("Marks", ascending=False)

4. Handle missing values:

df["Marks"] = df["Marks"].fillna(df["Marks"].mean())
df["Attendance"] = df["Attendance"].fillna(0)

5. Remove duplicates:
df = df.drop_duplicates(subset=["Student_ID"])

6. Create a derived attribute:

df["Percentage"] = df["Marks"] / 100 * 100

If marks are out of 100, Percentage can simply be:

df["Percentage"] = df["Marks"]

7. Create a performance category:

df["Result"] = df["Marks"].apply(lambda x: "Pass" if x >= 40 else "Fail")

These operations make the student dataset clean and ready for analysis and visualization.
Q9. Advanced Visualization Techniques
Advanced visualization techniques represent relationships, distributions and patterns that
may not be clear from simple bar or line charts.

1. Scatter plot: Shows the relationship between two numerical variables. Example: marks
versus attendance.
2. Bubble chart: Similar to a scatter plot, but bubble size represents a third variable.
Example: sales versus profit with bubble size representing market size.
3. Candlestick chart: Commonly used for financial data. It represents opening, closing,
high and low values over a time period.
4. Heatmap: Uses colour intensity to show values in a matrix. Example: correlation between
different variables or attendance across departments.
5. Map chart: Displays geographical information on a map. Example: showing sales or
population by state.

These techniques help identify correlations, clusters, trends, geographical patterns and
unusual observations.
Q10. Visual Encoding
Visual encoding is the method of representing data values using visual properties such as
position, colour, size and shape.

1. Position – placing values along an axis. It is one of the most accurate ways to compare
numerical values.
2. Colour – different colours can represent categories or a continuous range of values.
Example: a heatmap uses colour intensity to represent magnitude.
3. Size – larger or smaller marks can represent quantity. Example: bubble size can
represent sales volume.
4. Shape – different shapes can distinguish categories, such as circles for one product
type and triangles for another.
5. Length and area – bars or symbols can represent numerical quantities.

Good visual encoding should be clear, consistent and appropriate for the data. Too many
colours, shapes or visual elements can make a chart difficult to interpret. Effective
encoding helps the viewer understand patterns and make comparisons quickly.