Orig Python Data Science Textbook
Orig Python Data Science Textbook
```bash
Pip install numpy
```
```python
Import numpy as np
```
```python
Import numpy as np
Print(np.__version__)
Executing this program returns the
version of NumPy currently installed
on the computer, confirming that
the library is available for use.
Verifying the version is particularly
useful when following tutorials or
collaborating on projects, as some
functions and features may differ
slightly between releases. Keeping
NumPy up to date ensures
compatibility with other Python
libraries and provides access to
performance improvements, new
functionality, and important bug
fixes.
Review Questions
1. 1. Explain an important concept from this chapter. 2. 2.
Explain an important concept from this chapter. 3. 3.
Explain an important concept from this chapter. 4. 4.
Explain an important concept from this chapter.
5. 5. Explain an important concept from this chapter.
Understanding the Relational
Database Model
Unit 3
•
Table Characteristics
• Each row must be unique
• Columns must have distinct names and types
• Cells hold a single value (atomic) Order of
rows/columns doesn't matter
•
Each table row represents a single
entity occurrence within the entity
Keys
• Primary key (PK) is an attribute that uniquely
identifies any given row
• Composite key is a key composed of more than
one attribute eg in a Dependent table,
Employee number and Department number
might be a composite key
• Foreign key (FK) is an attribute whose values
match primary key values in the related table
Database Systems: Design,
Implementation, &
Management, International
Edition, Rob, Coronel &
B. Keys - Example
Foreign key
232 and 235
Occur twice
In PRODUCT
table
Primary key in VENDOR table
Database Systems: Design, Implementation, & Management, International Edition, Rob, Coronel & Crockett
B. Keys - Composite
Database Systems: Design, Implementation, & Management, International Edition, Rob, Coronel & Crockett Primary
key in VENDOR table
Integrity Rules
• Entity Integrity: No duplicate PKs, no NULL PK
• Referential Integrity: FK must match a PK or be
NULL
• Ensures reliable and valid accounting data
Relational schema
• A textual representation of database tables,
where each table is described by its name
followed by a list of its attributes in parenthesis
– R = {a1 , a2 , a3 …. an}
– Student (Stud_Num, FirstName, Surname, Age)
Student
Index
• An index is composed of an index key and a set of pointers
– Index key is an index’s reference point. Eg the primary key
– Pointers are identifiers, eg numbers that point to the data location
identified by the key
Indexes in Databases
• Help speed up data searches
• Use on frequently accessed columns like
InvoiceDate
• Like a table of contents for a database
More Key Concepts
• A surrogate key is a system-generated unique
ID used when no natural key exists.
• • Natural keys are real-world identifiers (e.g.,
ID number).
• • Surrogate keys are useful for simplifying
relationships.
Functional Dependency in Relational
Models
• If you know the InvoiceID, you can determine
the Invoice Date, Amount, and ClientID.
• This is called functional dependency: A → B (A
determines B).
• Understanding dependencies helps with
normalization.
Normalization
• Normalization removes data duplication and
improves data integrity.
– 1NF: No repeating groups, atomic values.
– 2NF: No partial dependencies (only applies to
composite keys).
– 3NF: No transitive dependencies (non-key
attributes depend only on PK).
Quiz – Test Your Knowledge
• 1. What is a primary key?
• 2. Why use foreign keys?
• 3. What is referential integrity?
• 4. How does indexing help?
• 5. Why avoid NULLs in PKs?
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Normalization
1
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
B. The need for normalization
C. The normalization process
D. Improving the design
E. Surrogate key considerations
2
DATABASE SYSTEMS: Design Implementation and Management (Rob,
3
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
• 2NF is better than 1NF; 3NF is better than 2NF
• For most business database design purposes, 3NF is as high as we need to go in
normalization process
• Highest level of normalization is not always most desirable
B. The Need for Normalization
• Example: Company that manages building projects
• Charges its clients by billing hours spent on each contract
• Hourly billing rate is dependent on employee’s position
• Periodically, report is generated that contains information displayed in Table 7.1
4
DATABASE SYSTEMS: Design Implementation and Management (Rob,
5
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
B. The Need for Normalization
Coronel & Crockett 9781844807321)
6
DATABASE SYSTEMS: Design Implementation and Management (Rob,
7
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Characteristics of a
relational table
• All values in a column must conform to the same data format
• Each column has a specific range of values known as the attribute
domain
• The order of the rows and columns is immaterial to the DBMS
• Each table must have an attribute that uniquely identifies each row
8
DATABASE SYSTEMS: Design Implementation and
Normalization
Coronel & Crockett 9781844807321)
Easiest approach
• You might decide to set up a table with all the information in the
format you want in your report
9
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
• This is often done in spreadsheets
1
0
1
1
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
1
2
B. The Need for Normalization
• The table structure appears to work because the report can be generated
with ease
• Unfortunately, report may yield different results depending on what data
anomaly has occurred
• Unnecessary repetition of data (redundancy)
• Ease of making spelling mistakes in multiple occurrences of the same data
1
3
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
1
4
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
1
5
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Normal Forms
Normal Form Characteristics Section
First normal form (1NF) Table format: no repeating groups and PK identified 7.3.1
Boyce-Codd normal form (BCNF) Every determinant is a candidate key (special case of 7.6.1
3NF)
Fourth normal form (4NF) 3NF and no independent multivalued dependencies 7.6.2
1
6
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Dependencies
• Partial dependency
• Attributes which are only dependent on part of the composite primary key
• Transitive dependency
• Attribute is dependent on any other attribute except the primary key
1
7
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
1
8
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
•T
hi
s
is
1
9
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
• Present data in tabular format, where each cell has a single value and there are no
repeating groups
• Try to eliminate nulls by making sure that each repeating group attribute contains
an appropriate data value.
• If you can’t eliminate nulls at least reduce them as much as possible
2
0
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
2
1
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
2
2
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
2
3
DATABASE SYSTEMS: Design Implementation and Management (Rob,
2
4
DATABASE SYSTEMS: Design Implementation and Management (Rob,
2
5
DATABASE SYSTEMS: Design Implementation and Management (Rob,
2
6
DATABASE SYSTEMS: Design Implementation and Management (Rob,
• Still subject to data redundancies
• Transitive dependency is a dependency of one non-primary key
attribute on another non-primary key attribute
Coronel & Crockett 9781844807321)
Normal Form Characteristics Section
First normal form (1NF) Table format: no repeating groups and PK identified 7.3.1
Boyce-Codd normal form (BCNF) Every determinant is a candidate key (special case of 7.6.1
3NF)
Fourth normal form (4NF) 3NF and no independent multivalued dependencies 7.6.2
2
7
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Normal Forms
2
8
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
2
9
DATABASE SYSTEMS: Design Implementation and Management (Rob,
3
0
DATABASE SYSTEMS: Design Implementation and Management (Rob,
3
1
DATABASE SYSTEMS: Design Implementation and Management (Rob,
3
2
DATABASE SYSTEMS: Design Implementation and Management (Rob,
3
3
DATABASE SYSTEMS: Design Implementation and Management (Rob,
3
4
DATABASE SYSTEMS: Design Implementation and Management (Rob,
3
5
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Normal Forms
Normal Form Characteristics Section
First normal form (1NF) Table format: no repeating groups and PK identified 7.3.1
Boyce-Codd normal form (BCNF) Every determinant is a candidate key (special case of 7.6.1
3NF)
Fourth normal form (4NF) 3NF and no independent multivalued dependencies 7.6.2
3
6
DATABASE SYSTEMS: Design Implementation and Management (Rob,
3
7
DATABASE SYSTEMS: Design Implementation and Management (Rob,
• Identify attributes dependent on each
determinant identified in step 1 and identify the dependency
• Name table to reflect its contents and function
C3. Conversion to Third Normal FormCoronel & Crockett
9781844807321)
3
8
DATABASE SYSTEMS: Design Implementation and Management (Rob,
• Draw new dependency diagram to show
all tables defined in steps
1–3
• Check new tables as well as tables modified in step 3 to make sure that each
table has a determinant and that no table contains inappropriate
dependencies
3
9
DATABASE SYSTEMS: Design Implementation and Management (Rob,
4
0
DATABASE SYSTEMS: Design Implementation and Management (Rob,
4
1
DATABASE SYSTEMS: Design Implementation and Management (Rob,
FK
4
2
DATABASE SYSTEMS: Design Implementation and Management (Rob,
4
3
DATABASE SYSTEMS: Design Implementation and
44
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Attribute Name Sample Value Sample Value Sample Value Sample Value Sample Value
SALE_DATE 1 Sep 2011 26 Aug 2011 30 Aug 2011 3 Sep 2011 25 Aug 2011
0.25-cm. drill
PROD_DESC Rotary sander Band saw Rotary sander Power drill
bit
VEND_CODE 211 211 309 211 157
VEND_NAME NeverFail, Inc. NeverFail, Inc. Saws, Inc. NeverFail, Inc. ToughGo, Inc.
NUM_SOLD 1 8 1 2 1
PROD_PRICE
€34.46 €2.73 €31.59 €34.46 €69.32
4
5
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Problem 1 Solution
Problem Question 2
4
6
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
2. Using the initial dependency diagram drawn in Problem 1,
remove all partial dependencies, draw the new dependency diagrams, and identify the normal
4
7
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
forms for each table structure you created.
4
8
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Problem Question 3
3. Using the table structures you created in Problem 2, remove all transitive dependencies, and
draw the new dependency diagrams. Also identify the normal forms for each table structure you
created.
4
9
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
5
0
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Problem Question 4
4. Using the results of Problem 3, draw the ERD using UML notation.
5
1
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
5
2
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
5
3
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
5
4
DATABASE SYSTEMS: Design Implementation and Management (Rob,
FK
5
5
DATABASE SYSTEMS: Design Implementation and Management (Rob,
5
6
DATABASE SYSTEMS: Design Implementation and Management (Rob,
E. Surrogate Key
ConsiderationsCoronel & Crockett 9781844807321)
• When primary key is considered to be unsuitable, designers use
surrogate keys
Different
Job_Code
but same
details
5
7
DATABASE SYSTEMS: Design Implementation and Management (Rob,
• Data entries in Table 7.3 are
inappropriate because they duplicate existing records
• Yet there has been no violation of either entity integrity or referential integrity
• Enforce unique values for JOB_DESCRIPTION with a unique index
5
8
DATABASE SYSTEMS: Design Implementation and Management (Rob,
5
9
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Crockett 9781844807321)
–2
6
0
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
H. Denormalization
• Creation of normalized relations is important database design goal
• Processing requirements should also be a goal
• If tables decomposed to conform to normalization requirements:
• Number of database tables expands
6
1
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
H. Denormalization
• Joining the larger number of tables takes additional input/output
(I/O) operations and processing logic, thereby reducing system speed
• Conflicts between design efficiency, information requirements, and
processing speed are often resolved through compromises that may
include denormalization
6
2
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
H. Denormalization
• Unnormalized tables in production database tend to suffer from these
defects:
• Data updates are less efficient because programs that read and update tables
must deal with larger tables
• Indexing is more cumbersome
• Unnormalized tables yield no simple strategies for creating virtual tables known as
views
• Use denormalization cautiously
• Understand why—under some circumstances—unnormalized tables
are better choice
6
3
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Summary
6
4
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
• when it is in 1NF and
• contains no partial dependencies
•Table is in 3NF
• when it is in 2NF and
• contains no transitive dependencies
•Table that is not in 3NF may be split into new tables
until all of the tables meet 3NF requirements
•Normalization is important part—but only part—of
design process
6
5
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Summary
6
6
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Summary
6
7
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
6
8
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Summary
6
9
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Summary
7
1
DATABASE SYSTEMS: Design Implementation and
Attribute Name Sample Value Sample Value Sample Value Sample Value Sample Value
SALE_DATE 1 Sep 2011 26 Aug 2011 30 Aug 2011 3 Sep 2011 25 Aug 2011
0.25-cm. drill
PROD_LABEL Rotary sander Band saw Rotary sander Power drill
bit
VEND_CODE 211 211 309 211 157
VEND_NAME NeverFail, Inc. NeverFail, Inc. Saws, Inc. NeverFail, Inc. ToughGo, Inc.
QUANT_SOLD 1 8 1 2 1
PROD_PRICE
€34.46 €2.73 €31.59 €34.46 €69.32
72
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
dependencies). You can assume that the table does
not contain repeating groups and that any invoice number may reference more than one
product.
(Hint: This table uses a composite primary key.)
7
3
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Problem 1 Solution
Problem Question 2
2. Using the initial dependency diagram drawn in Problem 1, remove all partial dependencies,
draw the new dependency diagrams, and identify the normal forms for each table structure
you created.
7
4
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
7
5
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Problem Question 3
3. Using the table structures you created in Problem 2, remove all transitive dependencies, and
draw the new dependency diagrams. Also identify the normal forms for each table structure
you created.
7
6
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
7
7
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
Problem Question 4
4. Using the results of Problem 3, draw the ERD using UML notation.
7
8
DATABASE SYSTEMS: Design Implementation and Management (Rob,
Coronel & Crockett 9781844807321)
7
9
Data Modelling
In this chapter, you will learn:
A. Data model building blocks
B. Business rules
C. Data abstraction
Database Systems: Design,
Implementation, &
Management, International
Edition, Rob, Coronel &
In-class exercise
Consider a university course offered to students and involving a lecturer. Give
an example of a business rule showing both directions of the relationship
(remember the 3 types of relationships).
• Fort Hare has many departments. (Hint: University + department)
• Fort Hare has different departments that teaches a variety of specific
courses. (Hint: Department + course)
• The lectures are taught in a classroom allocated specifically to that course
(Hint: Classroom + lecture)
Database Systems: Design,
Implementation, &
Management, International
Edition, Rob, Coronel &
Crockett
• The students attend classes taught by a lecturer. (Hint: class+ student)
In-class exercise
• Consider a university course offered to students
and involving a lecturer. Give an example of a
business rule showing both directions of the
relationship.
– Example 1: One lecturer teaches many students.
Students are taught by one lecturer.
– Example 2: One department offers many courses. A
course is offered by one department.
Database Systems: Design,
Implementation, &
Management, International
Edition, Rob, Coronel &
Crockett
– Example 3: A classroom is used for many lectures.
A lecture takes place in one classroom.
E2
E3
E4
Database Systems: Design, Implementation, & Management, International Edition, Rob, Coronel & Crockett