Data Normalization and SQL Techniques
Data Normalization and SQL Techniques
DATA
Normalisation
Dr Mohammed Bahja
School of Computer Science
[Link]@[Link]
2024/2025 - Week 4
Aims of the Session
This session aims to help you:
Apply the principles of database design to produce to produce relations that are
normalised to the 3NF (third normal form)
Outline
Normalisation
Data Anomolies
Functional Dependence
Data Wrangling and reporting with
SQL(last week)
Raw data is often messy and unstructured, making it challenging to
extract valuable insights.
Data wrangling is the process of cleaning and transforming this data into
a usable format.
It involves cleaning, transforming, and enriching raw data
into a more usable format for analysis.
Data wrangling is the process of cleaning and
transforming this data into a usable format.
SQL is the standard language for interacting with relational databases
and is widely used for both data wrangling and reporting.
Data Wrangling with SQL (last week)
A 7 C 9
GROUP BY and HAVING Clauses (last week)
The HAVING Clause – Restrict Group output
SELECT col_1, COUNT(*), SUM(col_2)
FROM table
GROUP BY col_1
HAVING SUM(col_2) < 10;
FROM DEPARTMENT
WHERE [Link] = [Link]) AS DepartmentName
FROM
EMPLOYEE;
Multiple Table Queries (last week)
Subquery
Performance: Subqueries can be slow, especially on large datasets, because
the subquery runs once for each row in the outer query.
Readability: As queries grow in complexity, using multiple subqueries can
make the query harder to read and maintain.
Subqueries are more limited in terms of the operations they can perform.
As queries grow more complex, nested subqueries can become unwieldy.
In some scenarios we use it in Nested Select Statements
SELECT
[Link],
[Link],
[Link],
(SELECT [Link]
FROM DEPARTMENT
WHERE [Link] = [Link]) AS DepartmentName
FROM
EMPLOYEE;
SQL JOIN clauses (last week)
In the realm of relational databases, the ability to combine data from multiple
tables is crucial for effective data analysis, reporting, and preparation.
SQL provides powerful tools to accomplish this, primarily through the use of JOIN
statements and nested queries.
SQL JOIN clauses are used to combine rows from two or more tables based on a
related column between them.
By using JOINs, you can retrieve data that is spread across multiple tables,
making it possible to analyse and report on complex datasets.
Nested Queries (Subqueries)
Definition: A query within another SQL query, usually enclosed within
parentheses.
When to Use Nested Queries Over Joins
Complex Filtering: When the filtering condition depends on an aggregated
result from another table.
Non-Relational Subsets: When you need to compare a value against a set of
values.
Existence Checks: Using EXISTS or NOT EXISTS with subqueries to check for
the presence or absence of rows.
SELECT Name
FROM Customers
WHERE CustomerID NOT IN (
SELECT CustomerID FROM Orders
);
Nested Queries (Subqueries) Examples
Retrieve customers along with the total number of orders they have placed.
SELECT Name, (
SELECT COUNT(*)
FROM Orders
WHERE [Link] =
[Link]
) AS OrderCount
FROM Customers;
Problem-Solving Approach for SQL
Queries
1. Understand the Problem
• Read the question carefully and identify what is being asked.
• Highlight keywords such as "find," "list," "calculate," etc.
2. Identify the Required Data
• Determine which tables contain the necessary information.
• List out the specific columns needed for the result.
3. Determine Table Relationships
• Identify primary and foreign keys to understand how tables are related.
• Decide on the type of join (INNER, LEFT, RIGHT, FULL) based on the
requirement.
4. Choose Between Join or Subquery
• Use joins when you need to combine rows from two or more tables based on
related columns.
• Use subqueries when you need to use the result of one query in another.
Problem-Solving Approach for SQL
Queries
5. Plan the Query Structure
• Write down the basic SQL clauses: SELECT, FROM, WHERE, GROUP BY, HAVING, ORDER BY.
• Order your clauses logically to build the query step by step.
6 .Write and Test the Query Incrementally
1. Start with a simple query and add complexity gradually.
2. Test each part of the query to ensure it's working as expected.
Problem-Solving Approach for SQL
Queries
Example : Using Joins
Problem: "List the names of customers and the orders they've placed."
Step 1: Understand the Problem
We need customer names and their corresponding orders.
Step 2: Identify the Required Data
Tables needed: Customers, Orders
Columns needed: [Link], [Link]
Step 3: Determine Table Relationships
[Link] (primary key) relates to [Link] (foreign key).
Step 4: Choose Between Join or Subquery
A join is appropriate because we are combining data from two tables based on a relationship.
Problem-Solving Approach for SQL
Queries
Step 5: Plan the Query Structure
Get names from customers
OrderId from order
Maker the join based on the PK and FK
Consider the activities listed and determine how changing the data could be
undertaken more efficiently by reorganizing the data.
Rule:
All attributes in a table must be atomic (no repeating groups).
A Primary Key is defined from all Candidate Keys
Normalisation st
(1 NF)
Rule:
All attributes in a table must be atomic (no repeating groups.
A Primary Key is defined from all Candidate Keys
Student ID Name Date of Subject Grade
Birth
960100 Smith, J 14/11/1977 Databases C
Software Development A
ISDE D
960105 White, A 10/05/1975 Software Development B
ISDE B
960120 Moore, T 11/03/1970 Databases A
Software Development B
Workshop C
960145 Smith, J 09/01/1972 Databases B
960150 Black, D 21/08/1973 Databases B
Software Development D
ISDE C
Workshop D
Non Atomic Fields
Normalisation st
(1 NF)
Rule:
All attributes in a table must be atomic (no repeating groups.
A Primary Key is defined from all Candidate Keys
Student ID Name Date of Subject Grade
Birth
960100 Smith, J 14/11/1977 Databases C
960100 Smith, J 14/11/1977 Software Development A
Which Column(s) should act 960100 Smith, J 14/11/1977 ISDE D
as Primary Key – Candidate 960105 White, A 10/05/1975 Software Development B
Key 960105 White, A 10/05/1975 ISDE B
960120 Moore, T 11/03/1970 Databases A
960120 Moore, T 11/03/1970 Software Development B
960120 Moore, T 11/03/1970 Workshop C
960145 Smith, J 09/01/1972 Databases B
960150 Black, D 21/08/1973 Databases B
960150 Black, D 21/08/1973 Software Development D
960150 Black, D 21/08/1973 ISDE C
960150 Black, D 21/08/1973 Workshop D
Normalisation st
(1 NF)
Rule:
All attributes in a table must be atomic (no repeating groups.
A Primary Key is defined from all Candidate Keys
Rule:
Table must be in First Normal Form
Every non-key attribute must be fully functionally dependent on
the Primary Key.
Define uniqueness
Determination of non-key attributes
Key also can be called a Determinant
Functional Dependence
Student ID Name
DoB
Rule:
Table must be in First Normal Form
Every non-key attribute must be fully functionally dependent on
the Primary Key.
Student ID Name Date of Birth Subject Grade
Yes
960145 Databases B
960150 Databases B
960150 Software Development D
960150 ISDE C
960150 Workshop D
What about this scenario?
Project Project Title Project Project Employee No Employee Name Department No Department Hourly
Code Manager Budget Name Rate
PC010
PC010 Pensions
Pensions System
System M.
M. Phillips
Phillips 24500
24500 510001
510001 A.
A. Smith
Smith L004
L004 IT
IT 22.00
22.00
PC010
PC010 Pensions System
Pensions System M.
M. Phillips
Phillips 24500
24500 510030
510030 L. Jones
L. Jones L023
L023 Pensions
Pensions 18.50
18.50
PC010
PC010 Pensions
Pensions System
System M. Phillips
M. Phillips 24500
24500 521010
521010 P.
P. Gold
Gold L004
L004 IT
IT 22.00
22.00
PC045
PC045 Salaries
Salaries System
System H.
H. Martin
Martin 17400
17400 510010
510010 B.
B. Jones
Jones L004
L004 IT
IT 21.75
21.75
PC045
PC045 Salaries
Salaries System
System H. Martin
H. Martin 17400
17400 510001
510001 A.
A. Smith
Smith L004
L004 IT
IT 18.00
18.00
PC045
PC045 Salaries System
Salaries System H.
H. Martin
Martin 17400
17400 531002
531002 T. Gilbert
T. Gilbert L028
L028 Database
Database 25.50
25.50
PC045
PC045 Salaries
Salaries System
System H.
H. Martin
Martin 17400
17400 513210
513210 W.
W. Richards
Richards L008
L008 Salary
Salary 17.00
17.00
PC064
PC064 HR
HR System
System K.
K. Lewis
Lewis 12250
12250 531002
531002 T.
T. Gilbert
Gilbert L028
L028 Database
Database 23.25
23.25
PC064
PC064 HR
HR System
System K.
K. Lewis
Lewis 12250
12250 521010
521010 P.
P. Gold
Gold L004
L004 IT
IT 17.50
17.50
PC064 HR System K. Lewis 12250 510034 B. Jeffries L009 HR 16.50
Is it in First
SecondNormal Form
Normal (1NF)?
Form (2NF)? No
What about this scenario?
Project ( ProjectCode, ProjectTitle, ProjectManager, ProjectBudget,
EmployeeNo, EmployeeName, DepartmentNo, DepartmentName, HourlyRate )
ProjectCode ProjectTitle
Project Manager
Project Budget
EmployeeNo EmployeeName
DepartmentNo
DepartmentName
Department No Department
Name
L004 IT
L023 Pensions
L028 Database
L008 Salary
L009 HR
Normalisation Notation - UNF
Project Project Title Project Project Employee No Employee Name Department No Department Hourly
Code Manager Budget Name Rate
PC010 Pensions M. Phillips 24500 510001 A. Smith L004 IT 22.00
System
PC010 Pensions M. Phillips 24500 510030 L. Jones L023 Pensions 18.50
System
PC010 Pensions M. Phillips 24500 521010 P. Gold L004 IT 21.00
System
PC045 Salaries H. Martin 17400 510010 B. Jones L004 IT 21.75
System
PC045 Salaries H. Martin 17400 510001 A. Smith L004 IT 18.00
System
PC045 Salaries H. Martin 17400 531002 T. Gilbert L028 Database 25.50
System
PC045 Salaries H. Martin 17400 513210 W. Richards L008 Salary 17.00
System
PC064 HR System K. Lewis 12250 531002 T. Gilbert L028 Database 23.25
PC064 HR System K. Lewis 12250 521010 P. Gold L004 IT 17.50
PC064 HR System K. Lewis 12250 510034 B. Jeffries L009 HR 16.50
Project Project Title Project Project Employee No Employee Name Department No Department Hourly
Code Manager Budget Name Rate
PC010 Pensions M. Phillips 24500 510001 A. Smith L004 IT 22.00
System
PC010 Pensions M. Phillips 24500 510030 L. Jones L023 Pensions 18.50
System
PC010 Pensions M. Phillips 24500 521010 P. Gold L004 IT 21.00
System
PC045 Salaries H. Martin 17400 510010 B. Jones L004 IT 21.75
System
PC045 Salaries H. Martin 17400 510001 A. Smith L004 IT 18.00
System
PC045 Salaries H. Martin 17400 531002 T. Gilbert L028 Database 25.50
System
PC045 Salaries H. Martin 17400 513210 W. Richards L008 Salary 17.00
System
PC064 HR System K. Lewis 12250 531002 T. Gilbert L028 Database 23.25
PC064 HR System K. Lewis 12250 521010 P. Gold L004 IT 17.50
PC064 HR System K. Lewis 12250 510034 B. Jeffries L009 HR 16.50