Building Database Application

BITS CS Transcripts

2026-05-10

Contents
1 Module 1: Course Introduction 4
1.1 Course Introductory Video . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Meet Your Instructor - C. Rakesh Prasanna . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Meet Your Instructor - Dr. Saikishor Jangiti . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Module 2: SQL Primer 9

2.1 Altering Table Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Creating and Dropping Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Data Control Language (DCL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.4 Data Definition Language (DDL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.5 Data Manipulation Language (DML) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.6 Data Query Language (DQL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.7 Inserting Data into Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.8 Introduction to SQL and Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.9 Real-world SQL Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.10 Recording of Building Database Applications Week 1 - Live Session on 26-03-13 . . . . . . . . . 43
2.11 SQL Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.12 String Functions in SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.13 Summary and Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.14 Table Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.15 Transaction Control Language (TCL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

3 Module 3: Intermediate SQL and Indexing 73

3.1 Aggregation – MIN, MAX with GROUPBY, AVG, SUM . . . . . . . . . . . . . . . . . . . . . . 73
3.2 Auto – Increment Columns, Replace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.3 B+ Tree Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3.4 B-Tree Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.5 Filtering Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.6 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.7 Hashing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.8 Ordered Indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.9 Recording of Building Database Applications Week 2 - Live Session on 26-03-20 . . . . . . . . . 108
3.10 Stored Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
3.11 Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

4 Module 4: Advanced SQL Relationships and Normalisation 125

4.1 Advanced Join Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

1
 4.2 Cross Join . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
4.3 Database Normalisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
4.4 Inner Join . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.5 Introduction to Window Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.6 Left Join & Right Join . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
4.7 One-to-Many Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
4.8 Practical Applications of Window Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
4.9 Recording of Building Database Applications Week 3 - Live Session on 26-03-27 . . . . . . . . . 161
4.10 Using PARTITION BY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

5 Module 5: Database Design 173

5.1 Data Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
5.2 Designing for Scalability and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
5.3 Entity-Relationship Diagrams (ERDs) - Advanced . . . . . . . . . . . . . . . . . . . . . . . . . 183
5.4 Entity-Relationship Diagrams (ERDs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
5.5 Extended ER Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
5.6 Introduction to Domain Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
5.7 Normalisation and Denormalisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
5.8 Primary Terminologies Used in Database Design . . . . . . . . . . . . . . . . . . . . . . . . . . 201
5.9 Recording of Building Database Applications Week 4 - Live Session on 26-04-03 . . . . . . . . . 206
5.10 Why Database Design is Important . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

6 Module 6: Monitoring and Maintaining Database Applications 217

6.1 Backup Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
6.2 Bulk Uploads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
6.3 Ensuring Data Integrity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
6.4 Handling Large Data Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
6.5 Importance of Backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
6.6 Importing and Exporting Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
6.7 Monitoring Database Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
6.8 Optimising Database Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
6.9 Performing Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
6.10 Rebuilding Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
6.11 Recording of Building Database Applications Week 5 - Live Session on 26-04-10 . . . . . . . . . 267
6.12 Restoring Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

7 Module 7: Introduction and Project Setup 277

7.1 Creating Initial Database Tables and Testing Connections . . . . . . . . . . . . . . . . . . . . . . 277
7.2 Designing the Database Schema for the Library Application . . . . . . . . . . . . . . . . . . . . 283
7.3 Initialising a Spring Boot Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
7.4 Integrating MySQL with Spring Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
7.5 Introduction to the Library Management Application . . . . . . . . . . . . . . . . . . . . . . . . 300
7.6 ORM Setup and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
7.7 Recording of Building Database Applications Week 6 - Live Session on 26-04-17 . . . . . . . . . 308
7.8 Setting Up MySQL Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
7.9 Setting Up the Development Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
7.10 Tools and Technologies Needed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
7.11 Understanding the MVC Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330

8 Module 8: Implementing the Model Layer 336

2
 8.1 Creating a Repository Interface for Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
8.2 Creating the Book Entity Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
8.3 Defining Relationships Between Entities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
8.4 Implementing Custom Query Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
8.5 Introduction to DAOs and Repositories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
8.6 Introduction to JPA and Hibernate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
8.7 Recording of Building Database Applications Week 7 - Live Session on 26-04-24 . . . . . . . . . 364
8.8 Service Layer Introduction and Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
8.9 Testing Entity Classes with a Simple Main Method . . . . . . . . . . . . . . . . . . . . . . . . . 376
8.10 Using Annotations for Entity Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
8.11 Writing Unit Tests for DAOs and Service Methods . . . . . . . . . . . . . . . . . . . . . . . . . 386

9 Module 9: Implementing the View Layer 393

9.1 Adding JavaScript to JSP Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
9.2 AJAX Calls with JQuery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
9.3 Binding Data to JSP Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
9.4 Creating and Processing HTML Forms in JSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406
9.5 Creating Basic HTML Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
9.6 Enhancing Interactivity with JQuery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
9.7 Integrating CSS for Styling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420
9.8 Introduction to JavaScript and JQuery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
9.9 Introduction to JSP (JavaServer Pages) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
9.10 Validating Forms with JavaScript and JQuery . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439

10 Module 10: Implementing the Controller Layer 445

10.1 Creating a Controller for the Home Page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
10.2 Creating a Controller Method to Add a Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
10.3 Creating a Controller Method to Delete a Book . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
10.4 Creating a Controller Method to View Books . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
10.5 Displaying Validation Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
10.6 Handling Exceptions in Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
10.7 Handling Form Submissions (Add Book) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473
10.8 Introduction to Spring MVC Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477
10.9 Redirecting and Forwarding Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
10.10Updating Book Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486

11 Module 11: Testing, Debugging, and Deployment 492

11.1 Configuring Application Properties for Different Environments . . . . . . . . . . . . . . . . . . . 492
11.2 Debugging Common Issues in Database Applications . . . . . . . . . . . . . . . . . . . . . . . . 496
11.3 Deploying the Application to a Web Server (Tomcat) . . . . . . . . . . . . . . . . . . . . . . . . 500
11.4 Final Project Review and Next Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
11.5 Introduction to Testing Spring Boot Applications . . . . . . . . . . . . . . . . . . . . . . . . . . 508
11.6 Monitoring and Maintaining the Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513
11.7 Packaging the Application (JARWAR) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515
11.8 Using Postman to Test API Endpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519
11.9 Writing Integration Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
11.10Writing Unit Tests for Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528

3
1 Module 1: Course Introduction
1.1 Course Introductory Video
1.1.1 1. Course Overview
• Instructor: Professor Rakesh Prasanna Chennupati (Modules 1–5)
• Course Title: Building Database Applications
• Structure: Divided into 10 modules, covering foundational to advanced topics in database design, SQL,
application development, and maintenance.

1.1.2 2. Module Breakdown

[Link] 2.1. Modules Taught by Professor Rakesh Prasanna Chennupati (Modules 1–5)

[Link].1 Module 1: SQL Primer Key Topics: - Database Creation: - Commands to create and drop
databases. - Table management (creation, deletion). - Data Insertion: - Single-row insertion (inserting one
record at a time). - Multiple-row insertion (inserting multiple records in a single command). - Table Constraints:
- UNIQUE Constraint: Ensures all values in a column are distinct. - CHECK Constraint: Validates data against
a specified condition. - Column Constraints: Rules applied to individual columns (e.g., NOT NULL, DEFAULT). -
Multiple Constraints: Combining constraints (e.g., UNIQUE + CHECK). - Table Alteration: - Adding/Dropping
Columns: Modifying table structure post-creation. - Renaming Columns/Tables: Changing identifiers without
losing data. - Modifying Data Types: Adjusting column attributes (e.g., VARCHAR(50) → VARCHAR(100)).

[Link].2 Module 2: Intermediate SQL Queries Key Topics: - SQL Functions: - Built-in functions (e.g.,
string manipulation, mathematical operations). - Stored Procedures: - Precompiled SQL code stored in the
database for reuse. - Triggers: - Automated actions executed in response to database events (e.g., BEFORE INSERT,
AFTER UPDATE). - Indexing: - Improving query performance via indexed columns. - Data Manipulation Tech-
niques: - Replacement: Updating existing values (e.g., REPLACE() function). - Filtering: Restricting query re-
sults using: - DISTINCT (eliminating duplicates). - ORDER BY (sorting results). - Aggregators (MIN, MAX, COUNT, AVG,
SUM) with GROUP BY. - Auto-Incrementing Values: Automatically generating unique IDs (e.g., AUTO_INCREMENT
in MySQL).

[Link].3 Module 3: Advanced SQL Queries Key Topics: - Database Relationships: - One-to-Many (1:N):
A single record in one table relates to multiple records in another (e.g., Customer → Orders). - Normalization: -
Structuring data to minimize redundancy using primary keys (PK) and foreign keys (FK). - Types of Joins (with
Examples): - Cross Join: Cartesian product (all possible combinations of rows). - Inner Join: Returns matching
rows from both tables. - Left Join: All rows from the left table + matching rows from the right. - Right Join: All
rows from the right table + matching rows from the left.

[Link].4 Module 4: Database Design Key Topics: - Domain Model vs. Data Model: - Domain Model:
Conceptual representation of real-world entities (e.g., classes in OOP). - Data Model: Database-specific structure
(tables, relationships). - Documentation Tools: - Entity-Relationship Diagrams (ERD): Visual representation
of tables, attributes, and relationships. - Extended ERD (EERD): Includes additional concepts (e.g., inheritance,
weak entities).

[Link].5 Module 5: Monitoring and Maintaining Database Applications Key Topics: - Backup and Re-
store: - Creating database backups to prevent data loss. - Restoring data from backups in case of failure. - Index
Rebuilding: - Reorganizing indexes to maintain query performance (also covered in Module 2). - Data Loading:
- Import/Export: Moving data between databases or files (e.g., CSV, SQL dumps). - Bulk Uploads: Efficiently
inserting large datasets (e.g., BULK INSERT in SQL Server).

4
[Link] 2.2. Modules Taught by Professor Saikishor (Modules 6–10)

[Link].1 Module 6: Foundations of Database Application Development Key Topics: - Initiating Devel-
opment: - Setting up the project structure. - Configuring database connections. - Defining initial schemas and
entities.

[Link].2 Module 7: Managing Entities and Relationships in Spring Boot Key Topics: - Entity Manage-
ment: - Defining Java classes as database entities (e.g., @Entity annotation). - Mapping relationships (e.g., @One-
ToMany, @ManyToOne). - Data Access Optimization: - Using Spring Data JPA for efficient CRUD operations. -
Query optimization (e.g., @Query annotations, derived queries).

[Link].3 Module 8: Dynamic Web Pages with JSP, HTML, CSS, and JavaScript Key Topics: - Frontend
Development: - JSP (JavaServer Pages): Embedding Java code in HTML for dynamic content. - HTML Forms:
Collecting user input. - CSS Styling: Enhancing UI/UX. - JavaScript/jQuery: Adding interactivity (e.g., form
validation, AJAX calls).

[Link].4 Module 9: Spring MVC Controllers and Thread Operations Key Topics: - Controller Layer: -
Handling HTTP requests (e.g., @GetMapping, @PostMapping). - Integrating with Service Layer (business logic)
and Repository Layer (database operations). - Thread Management: - Handling concurrent requests efficiently.
- Exception Handling: - Validating user input (e.g., @Valid annotations). - Custom error responses (e.g., @Excep-
tionHandler).

[Link].5 Module 10: Testing, Deployment, and Monitoring Key Topics: - Testing Spring Boot Ap-
plications: - Unit testing (e.g., @SpringBootTest, Mockito). - Integration testing (e.g., @DataJpaTest). -
Deployment: - Configuring Tomcat for production. - Environment-specific settings (e.g., application-
[Link], [Link]). - Monitoring and Debugging: - Logging (e.g., Logback,
SLF4J). - Performance tuning (e.g., JVM optimization, database profiling).

1.1.3 3. Course Objectives

By the end of the course, students will be able to: 1. Design and implement databases using SQL (from basic to
advanced queries). 2. Develop database applications with Spring Boot, integrating frontend (JSP/HTML/CSS/JS)
and backend (MVC controllers). 3. Optimize performance through indexing, normalization, and efficient query-
ing. 4. Maintain and monitor database applications, including backups, restores, and debugging. 5. Deploy
applications in real-world environments (e.g., Tomcat) with proper testing and configuration.
Final Note: “Wish you all good luck, and to develop great database applications.” — Professors Rakesh and
Saikishor.

1.2 Meet Your Instructor - C. Rakesh Prasanna

1.2.1 Introduction to the Instructor
• The instructor for the course “Building Database Applications” is Professor C. Rakesh Prasanna Chana-
pati.
• He is a faculty member at Birla Institute of Technology and Science (BITS) Pilani.

1.2.2 Academic Background

• Educational Qualification:
– Holds a Master’s Degree in Computer Science.

5
1.2.3 Professional Affiliation
• Institutional Association:
– Has been associated with BITS Pilani since 2010 (as of the lecture recording).
– Currently serves as a Faculty in the Computer Science Department under the Work Integrated
Learning Programmes (WLP) Division of BITS Pilani.

1.2.4 Areas of Specialization and Teaching Interests

• Core Expertise:
– Algorithms
– Algorithm Design
– Data Structures
– Database Management Systems (DBMS)
• Relevance to the Course:
– His specialization in DBMS aligns directly with the course “Building Database Applications”, ensur-
ing expert guidance in database concepts, design, and implementation.

1.2.5 Closing Remarks

• Encouragement to Learners:
– The instructor extends best wishes to all students for their learning journey in the course.
– Emphasizes a supportive and engaging approach to teaching database applications.

1.3 Meet Your Instructor - Dr. Saikishor Jangiti

1.3.1 Introduction to Dr. Saikishor Jangiti
• Professional Identity:
– A passionate advocate for simplicity and effective teaching in computer science.
– Enthusiastic about solving complex problems through programming.
– Actively engaged with evolving trends in the field of computer science.

1.3.2 Academic Background

[Link] Undergraduate Education
• Degree: Bachelor of Technology ([Link]) in Computer Science and Information Technology.
• Institution: Jawaharlal Nehru Technological University (JNTU), Hyderabad.
• Year of Completion: 2005.

[Link] Postgraduate Education

• Degree: Master of Technology ([Link]) in Computer Science and Engineering.
• Institution: Sri Venkateswara University, College of Engineering, Tirupati.
• Year of Completion: 2007.
• Significance: Marked the beginning of his teaching career.

[Link] Doctoral Education

• Degree: Doctor of Philosophy (Ph.D.) in Computer Science and Engineering.
• Research Focus:
– Heuristic search for optimizing virtual machine (VM) initial placement in cloud data centers.

6
 • Institution: Pursued during his ten-year tenure at SASTRA Deemed University, Thanjavur.

1.3.3 Professional Career

[Link] Teaching Experience
• Duration: 10 years at SASTRA Deemed University, Thanjavur.
– Engaged in academic research and teaching.
– Laid the foundation for his Ph.D. research.

[Link] Current Role (as of 2020)

• Position: Assistant Professor (Off-Campus).
• Institution: BITS Pilani, Hyderabad Campus.
• Department: Computer Science and Information Systems (CSIS) Group.

1.3.4 Academic Interests & Expertise

Dr. Jangiti’s research and teaching span a diverse range of advanced topics in computer science, including:
1. System Programming
• Low-level programming, interaction with hardware, and operating system development.
2. Parallel and Distributed Programming
• Techniques for concurrent execution across multiple processors or machines.
• Frameworks for distributed systems (e.g., MapReduce, MPI, Hadoop).
3. Cloud Computing & Data Centers
• Virtualization, resource allocation, and scalability in cloud environments.
• Optimization techniques for data center management (e.g., VM placement, energy efficiency).
4. Compiler Design
• Study of programming language translation (source code → machine code).
• Techniques for code optimization, parsing, and semantic analysis.
5. Formal Languages and Automata Theory
• Mathematical models of computation (e.g., finite automata, pushdown automata, Turing machines).
• Grammar hierarchies (regular, context-free, context-sensitive, recursively enumerable languages).
6. Search-Based Artificial Intelligence (AI)
• Heuristic search algorithms (e.g., A*, genetic algorithms, simulated annealing).
• Applications in optimization problems, including cloud resource management.

1.3.5 Key Research Contribution

• Focus Area: Heuristic Search for Virtual Machine Placement in Cloud Data Centers.
– Problem: Efficiently allocating virtual machines (VMs) to physical servers to optimize:
* Resource utilization (CPU, memory, storage).
* Energy consumption (reducing operational costs).
* Performance (minimizing latency, maximizing throughput).
– Methodology: Employed search-based AI techniques to develop optimal placement strategies.

1.3.6 Teaching Philosophy

• Core Principles:
– Simplicity: Breaking down complex concepts into digestible, intuitive explanations.

7
 – Effective Teaching: Ensuring student comprehension through practical examples and hands-on
learning.
– Problem-Solving: Emphasizing programming as a tool to address real-world challenges.

1.3.7 Conclusion
Dr. Saikishor Jangiti brings extensive academic and industry-relevant expertise to the course “Building
Database Applications”, with a strong foundation in: - Theoretical computer science (automata, compilers,
formal languages). - Applied computing (cloud systems, parallel programming, AI-driven optimization). -
Pedagogical excellence (simplified, effective teaching methodologies).
His research in heuristic search and cloud optimization provides a unique perspective on scalable, efficient
database systems, aligning with the course’s focus on practical database application development.

8
2 Module 2: SQL Primer
2.1 Altering Table Structures
2.1.1 Introduction to ALTER TABLE
• Purpose: The ALTER TABLE command in SQL is used to modify the structure of an existing table.
• Key Operations: This command allows for:
– Adding new columns.
– Dropping existing columns.
– Renaming columns.
– Modifying column data types.
• Objective: By the end of this lecture, you will be able to:
– Use the ALTER TABLE command to perform the above operations.
– Understand practical examples of altering table structures.

2.1.2 Adding a New Column

ALTER TABLE table_name ADD column_name data_type;

[Link] Syntax
• Components:
– ALTER TABLE: Command to modify an existing table.
– table_name: Name of the table to be altered.
– ADD: Keyword to add a new column.
– column_name: Name of the new column.
– data_type: Data type of the new column (e.g., varchar, int, date).

ALTER TABLE Students ADD email VARCHAR(255);

[Link] Example
• Explanation:
– Adds a new column named email to the Students table.
– The data type of the email column is VARCHAR(255), allowing up to 255 characters.
• Use Case:
– Adding new columns helps store additional information without creating a new table or relationship.

2.1.3 Dropping an Existing Column

ALTER TABLE table_name DROP COLUMN column_name;

[Link] Syntax
• Components:
– DROP COLUMN: Keyword to remove an existing column.
– column_name: Name of the column to be dropped.

9
ALTER TABLE Students DROP COLUMN Age;

[Link] Example
• Explanation:
– Removes the Age column from the Students table.
– Warning: Dropping a column permanently deletes all data in that column.
• Use Case:
– Useful for removing unnecessary or redundant data to streamline the database.

2.1.4 Renaming an Existing Column

ALTER TABLE table_name RENAME COLUMN old_column_name TO new_column_name;

[Link] Syntax
• Components:
– RENAME COLUMN: Keyword to rename an existing column.
– old_column_name: Current name of the column.
– new_column_name: New name for the column.

ALTER TABLE Students RENAME COLUMN Name TO FullName;

[Link] Example
• Explanation:
– Renames the Name column to FullName in the Students table.
– Note: Renaming a column does not affect the data stored in it.
• Use Case:
– Improves clarity and consistency in the database schema.

2.1.5 Modifying a Column’s Data Type

ALTER TABLE table_name MODIFY COLUMN column_name new_data_type;

[Link] Syntax
• Components:
– MODIFY COLUMN: Keyword to change the data type of an existing column.
– column_name: Name of the column to be modified.
– new_data_type: New data type for the column (e.g., smallint, datetime).

ALTER TABLE Students MODIFY COLUMN Age SMALLINT;

[Link] Example
• Explanation:
– Changes the data type of the Age column from int to smallint in the Students table.

10
 – Consideration: Modifying data types is typically easier when the table is empty or contains compatible
data.
• Use Case:
– Optimizes storage and ensures data is stored in the appropriate format.

[Link] Self-Learning Topic: Difference Between smallint and int

• smallint:
– Typically stores integers in the range of -32,768 to 32,767.
– Occupies 2 bytes of storage.
• int:
– Typically stores integers in the range of -2,147,483,648 to 2,147,483,647.
– Occupies 4 bytes of storage.
• Use Case:
– Use smallint for smaller ranges to save storage space.
– Use int for larger ranges or when the exact range is unknown.

2.1.6 Practical Exercises

[Link] Exercise 1: Adding a Column Task: Add a new column to store phone numbers in the Students table.
Solution:
ALTER TABLE Students ADD phone_number VARCHAR(15);

• Analysis:
– Adds a phone_number column with a maximum length of 15 characters.
– Improvement: Using VARCHAR may not be ideal for phone numbers. Consider using a numeric type or
a formatted string type depending on the database system.
– Best Practice: For numerical data like phone numbers, VARCHAR is often used to preserve leading zeros
or special characters (e.g., + or -).

[Link] Exercise 2: Dropping a Column Task: Drop the MiddleName column from the Employee table. So-
lution:
ALTER TABLE Employee DROP COLUMN MiddleName;

• Explanation:
– Removes the MiddleName column from the Employee table.
– Use Case: Streamlines the database by removing redundant or unnecessary columns.

[Link] Exercise 3: Renaming a Column Task: Rename the Instructor column to Teacher in the Course
table. Solution:
ALTER TABLE Course RENAME COLUMN Instructor TO Teacher;

• Explanation:
– Renames the Instructor column to Teacher for better clarity.
– Use Case: Enhances the intuitiveness and readability of the database schema.

[Link] Exercise 4: Modifying a Column’s Data Type Task: Change the data type of the OrderDate column
to datetime in the Orders table. Solution:

11
ALTER TABLE Orders MODIFY COLUMN OrderDate DATETIME;

• Explanation:
– Converts the OrderDate column to the DATETIME data type.
– Use Case: Ensures the column can store both date and time information accurately.

2.1.7 Key Takeaways

• Versatility of ALTER TABLE:
– The ALTER TABLE command is essential for dynamically adjusting database schemas as requirements
evolve.
• Core Operations:
– Add Column: Introduce new data fields without restructuring the entire table.
– Drop Column: Remove unnecessary or obsolete data fields.
– Rename Column: Improve schema clarity and consistency.
– Modify Column: Optimize storage and data integrity by adjusting data types.
• Best Practices:
– Always ensure compatibility when modifying data types to avoid data loss.
– Use appropriate data types to balance storage efficiency and functionality.
– Document schema changes to maintain clarity for future database administrators.

2.1.8 Conclusion
• The ALTER TABLE command is a powerful tool for maintaining and optimizing database structures.
• Mastery of these operations ensures databases remain flexible, efficient, and aligned with evolving applica-
tion needs.

2.2 Creating and Dropping Tables

2.2.1 Introduction to Table Management in SQL
• This lecture focuses on defining table structures and managing database schemas using SQL commands.
• Key objectives:
– Understand how to create and drop tables using SQL.
– Comprehend the structure and schema of a table.
– Learn essential commands for table management in database applications.

2.2.2 1. Creating Tables with CREATE TABLE

[Link] 1.1 Purpose of CREATE TABLE
• The CREATE TABLE command defines a new table in a database by specifying:
– Column names (attributes).
– Data types for each column.
– Optional constraints (e.g., primary keys).

CREATE TABLE table_name (

column1 datatype [constraints],
column2 datatype [constraints],
...
);

12
[Link] 1.2 Syntax

CREATE TABLE student (

student_id INT,
name VARCHAR(255),
age INT,
major VARCHAR(255)
);

[Link] 1.3 Example: Creating a student Table

[Link].1 Breakdown of Columns:

1. student_id INT
• Column name: student_id
• Data type: INT (stores integer values).
• Purpose: Likely serves as a unique identifier for each student.
2. name VARCHAR(255)
• Column name: name
• Data type: VARCHAR(255) (variable-length character string, max 255 characters).
• Purpose: Stores the student’s name as text.
3. age INT
• Column name: age
• Data type: INT (stores integer values).
• Purpose: Represents the student’s age.
4. major VARCHAR(255)
• Column name: major
• Data type: VARCHAR(255).
• Purpose: Stores the student’s field of study (e.g., “Computer Science”).

[Link].2 Key Observations:

• The parentheses () enclose the list of columns and their data types.
• The command creates a structured table with four columns:
– student_id, name, age, major.
• Each column is designed to store a specific type of data, ensuring organization and readability.

CREATE TABLE courses (

course_id INT PRIMARY KEY,
course_name VARCHAR(255),
instructor VARCHAR(255)
);

[Link] 1.4 Example: Creating a courses Table with a Primary Key

[Link].1 Breakdown of Columns:

1. course_id INT PRIMARY KEY
• Column name: course_id

13
 • Data type: INT.
• Constraint: PRIMARY KEY ensures:
– Each value in this column is unique.
– No NULL values are allowed.
• Purpose: Uniquely identifies each course.
2. course_name VARCHAR(255)
• Column name: course_name
• Data type: VARCHAR(255).
• Purpose: Stores the name of the course.
3. instructor VARCHAR(255)
• Column name: instructor
• Data type: VARCHAR(255).
• Purpose: Stores the instructor’s name.

[Link].2 Key Observations:

• The PRIMARY KEY constraint enforces uniqueness for the course_id column.
• The table has three columns: course_id, course_name, instructor.

2.2.3 2. Viewing Table Structure with DESCRIBE

[Link] 2.1 Purpose of DESCRIBE
• The DESCRIBE command (or DESC in some SQL dialects) displays the structure of a table, including:
– Column names.
– Data types.
– Constraints (e.g., primary keys).
• Does not show the actual data in the table.

DESCRIBE table_name;
-- or
DESC table_name;

[Link] 2.2 Syntax

DESCRIBE student;

[Link] 2.3 Example: Describing the student Table

[Link].1 Expected Output:

Column Type Null Key Default Extra

student_id INT YES NULL
name VARCHAR(255) YES NULL
age INT YES NULL
major VARCHAR(255) YES NULL

14
[Link].2 Key Observations:
• Shows column names, data types, and whether columns allow NULL values.
• If a PRIMARY KEY were defined, it would appear under the Key column.

2.2.4 3. Clearing Table Data with TRUNCATE

[Link] 3.1 Purpose of TRUNCATE
• The TRUNCATE TABLE command deletes all rows from a table without removing the table structure.
• Faster than DELETE for large tables because it does not log individual row deletions.
• Resets auto-increment counters (if applicable).

TRUNCATE TABLE table_name;

[Link] 3.2 Syntax

TRUNCATE TABLE students;

[Link] 3.3 Example: Truncating the students Table

[Link].1 Key Observations:

• Deletes all records (rows) but keeps the table structure (columns, constraints).
• Cannot be rolled back in some database systems (e.g., MySQL).
• Use with caution—data loss is permanent.

2.2.5 4. Deleting Tables with DROP TABLE

[Link] 4.1 Purpose of DROP TABLE
• The DROP TABLE command permanently removes a table and all its data from the database.
• Cannot be undone—use with extreme caution.

DROP TABLE table_name;

[Link] 4.2 Syntax

DROP TABLE student;

[Link] 4.3 Example: Dropping the student Table

[Link].1 Key Observations:

• Deletes the entire table structure and all data.
• If the table is referenced by other objects (e.g., foreign keys), the command may fail unless constraints are
dropped first.
• Best practice: Backup data before using DROP TABLE.

15
2.2.6 5. Understanding Database Schema
[Link] 5.1 Definition of Schema
• A schema is a logical container that defines the structure of a database, including:
– Tables.
– Columns.
– Constraints (e.g., primary keys, foreign keys).
– Other database objects (e.g., views, procedures).

[Link] 5.2 Purpose of Schema

• Organizes related database objects (e.g., tables for a “school” system).
• Provides namespace separation (avoids naming conflicts).
• Simplifies management of large databases.

-- Create a schema named 'school'

CREATE SCHEMA school;

-- Create a table 'student' within the 'school' schema

CREATE TABLE [Link] (
student_id INT PRIMARY KEY,
name VARCHAR(255),
age INT,
major VARCHAR(255)
);

[Link] 5.3 Example: Creating a Schema and Table

[Link].1 Breakdown:
1. CREATE SCHEMA school;
• Creates a logical group named school.
2. CREATE TABLE [Link] (...);
• Defines a table student within the school schema.
• Columns:
– student_id (primary key).
– name, age, major.

[Link].2 Key Observations:

• The schema name (school) acts as a prefix for the table ([Link]).
• Helps group related tables (e.g., all tables for a school database).

2.2.7 6. Summary of Key Commands

Command Purpose Example

CREATE TABLE Defines a new table with columns and data types. CREATE TABLE student
(...);

16
Command Purpose Example
DESCRIBE Displays the structure of a table (columns, data types, DESCRIBE student;
constraints).
TRUNCATE TABLE Deletes all rows from a table but keeps the structure. TRUNCATE TABLE
students;
DROP TABLE Permanently removes a table and all its data. DROP TABLE student;
CREATE SCHEMA Creates a logical container for database objects. CREATE SCHEMA school;

2.2.8 7. Best Practices and Considerations

1. Primary Keys
• Always define a primary key (e.g., student_id) to ensure unique row identification.
• Example: course_id INT PRIMARY KEY.
2. Data Types
• Choose appropriate data types (e.g., INT for numbers, VARCHAR for text).
• Example: name VARCHAR(255) limits text to 255 characters.
3. Caution with DROP and TRUNCATE
• DROP TABLE removes the entire table permanently.
• TRUNCATE TABLE clears data but keeps the structure.
• Always back up data before using these commands.
4. Schema Organization
• Use schemas to group related tables (e.g., [Link], [Link]).
• Improves database manageability and avoids naming conflicts.
5. Viewing Structure
• Use DESCRIBE to inspect table structure before modifying it.

2.2.9 8. Conclusion
• Creating tables (CREATE TABLE) is fundamental for defining database structure.
• Primary keys ensure unique row identification.
• DESCRIBE helps inspect table structure.
• TRUNCATE clears data without deleting the table.
• DROP TABLE permanently removes a table—use with caution.
• Schemas organize related tables and improve database management.
These skills are essential for efficient database design and manipulation in real-world applications.

2.3 Data Control Language (DCL)

2.3.1 Introduction to Data Control Language (DCL)
• Definition: Data Control Language (DCL) is a subset of SQL commands used to control access to data
within a database.
• Purpose: DCL commands manage user permissions and privileges on database objects (e.g., tables, views,
procedures).
• Primary Commands:
– GRANT – Assigns privileges to users.
– REVOKE – Removes privileges from users.

17
2.3.2 GRANT Command
[Link] Definition & Purpose
• The GRANT command is used to give users specific privileges on database objects.
• Ensures controlled access by allowing administrators to define what actions a user can perform.

GRANT privilege ON object TO user;

[Link] Syntax
• privilege: The type of permission being granted (e.g., SELECT, INSERT, UPDATE, DELETE, ALL PRIVI-
LEGES).
• object: The database object (e.g., table, view) on which the privilege is granted.
• user: The user or role receiving the privilege.

[Link] Examples
1. Granting a Single Privilege
GRANT SELECT ON student TO user1;

• Effect: user1 can retrieve data from the student table but cannot modify it.
2. Granting Multiple Privileges
GRANT SELECT, INSERT, UPDATE ON student TO user2;

• Effect: user2 can retrieve, insert, and update data in the student table.
3. Granting All Privileges
GRANT ALL PRIVILEGES ON student TO admin;

• Effect: The admin user has full control over the student table (all possible SQL operations).

[Link] Key Observations

• Privileges are object-specific (e.g., a user may have SELECT on one table but not another).
• Restricts users to only necessary actions, enhancing security.

2.3.3 REVOKE Command

[Link] Definition & Purpose
• The REVOKE command is used to remove privileges from users on database objects.
• Essential for maintaining security by ensuring users retain only necessary access.

REVOKE privilege ON object FROM user;

[Link] Syntax
• privilege: The permission being revoked.
• object: The database object from which the privilege is removed.
• user: The user or role losing the privilege.

18
[Link] Examples
1. Revoking a Single Privilege
REVOKE SELECT ON student FROM user1;

• Effect: user1 loses the ability to retrieve data from the student table.
2. Revoking Multiple Privileges
REVOKE SELECT, INSERT, UPDATE ON student FROM user2;

• Effect: user2 can no longer retrieve, insert, or update data in the student table.
3. Revoking All Privileges
REVOKE ALL PRIVILEGES ON student FROM admin;

• Effect: The admin user loses all permissions on the student table.

[Link] When to Use REVOKE

• Access No Longer Needed: When a user’s role changes and they no longer require certain privileges.
• Security Management: Ensures only authorized users retain access.
• Compliance: Enforces security policies by removing unnecessary permissions.

2.3.4 Importance of DCL in Database Security

[Link] Security & Access Control
• Granular Permissions: DCL allows fine-grained control over what each user can do.
• Prevents Unauthorized Access: Ensures users cannot perform actions beyond their role.
• Data Integrity: Limits modifications to authorized personnel only, reducing risks of accidental or mali-
cious changes.

[Link] Best Practices

1. Principle of Least Privilege: Grant only the minimum necessary permissions to users.
2. Regular Audits: Periodically review and revoke unused privileges.
3. Role-Based Access Control (RBAC): Assign privileges to roles rather than individual users for easier man-
agement.

2.3.5 Summary of Key Concepts

Command Purpose Syntax Example

GRANT Assigns privileges to a GRANT privilege ON GRANT SELECT ON
user. object TO user; student TO user1;
REVOKE Removes privileges from REVOKE privilege ON REVOKE INSERT ON
a user. object FROM user; student FROM user2;

[Link] Key Takeaways

• DCL is critical for database security and access management.
• GRANT and REVOKE are the core commands for controlling user permissions.
• Proper use of DCL ensures data integrity, security, and compliance with organizational policies.

19
2.4 Data Definition Language (DDL)
2.4.1 1. Introduction to SQL Command Categories
[Link] 1.1 Overview of SQL Command Types SQL (Structured Query Language) commands are categorized
into five primary types based on their functionality:
1. Data Definition Language (DDL)
• Defines and manages database structure (e.g., tables, schemas).
• Commands: CREATE, ALTER, DROP.
• Permanent changes: DDL commands auto-commit (cannot be rolled back).
2. Data Manipulation Language (DML)
• Manages data within existing structures (e.g., inserting, updating, deleting records).
• Commands: INSERT, UPDATE, DELETE.
• Changes are not auto-committed (can be rolled back).
3. Data Query Language (DQL)
• Retrieves data from the database.
• Primary command: SELECT.
4. Data Control Language (DCL)
• Manages access permissions and user privileges.
• Commands: GRANT, REVOKE.
5. Transaction Control Language (TCL)
• Manages transactions (groups of operations treated as a single unit).
• Commands:
– COMMIT: Permanently saves changes.
– ROLLBACK: Reverts changes to a previous state (e.g., after an error).
– SAVEPOINT: Sets a point to roll back to.

2.4.2 2. Data Definition Language (DDL) Deep Dive

[Link] 2.1 Definition and Purpose
• DDL consists of SQL commands that define, modify, and delete database objects (e.g., tables, indexes,
schemas).
• Key Functions:
– Create new database structures.
– Modify existing structures.
– Delete structures when no longer needed.
• Auto-commit: DDL operations are immediately permanent (unlike DML, which requires explicit COMMIT).

[Link] 2.2 Primary DDL Commands

Command Purpose Example Syntax

CREATE Creates a new database object (e.g., table). CREATE TABLE table_name (...);
ALTER Modifies an existing object. ALTER TABLE table_name ...;
DROP Deletes an existing object. DROP TABLE table_name;

2.4.3 3. CREATE TABLE Command

20
CREATE TABLE table_name (
column1 datatype [constraints],
column2 datatype [constraints],
...
);

[Link] 3.1 Syntax

• Components:
– table_name: Name of the table to be created.
– column1, column2, ...: Column names with their data types and constraints (e.g., PRIMARY KEY).

CREATE TABLE students (

studentID INT PRIMARY KEY,
name VARCHAR(255)
);

[Link] 3.2 Example: Creating a students Table

• Breakdown:
– studentID: Column of type INT with a PRIMARY KEY constraint (ensures uniqueness and non-null
values).
– name: Column of type VARCHAR(255) (variable-length string, max 255 characters).

[Link] 3.3 Key Notes

• Constraints (e.g., PRIMARY KEY, NOT NULL) enforce data integrity rules.
• Data Types must be specified for each column (e.g., INT, VARCHAR, DATE).

2.4.4 4. DROP TABLE Command

DROP TABLE table_name;

[Link] 4.1 Syntax

• Purpose: Permanently deletes a table and all its data.
• Irreversible: Once executed, the table and its data are lost (no UNDO).

DROP TABLE students;

[Link] 4.2 Example: Dropping the students Table

• Effect: The students table and all its records are deleted from the database.

[Link] 4.3 Warnings

• Use with caution: DROP TABLE cannot be rolled back.
• Dependencies: If other objects (e.g., views, foreign keys) reference the table, the command may fail or cause
cascading deletions.

21
2.4.5 5. ALTER TABLE Command
[Link] 5.1 Purpose Modifies the structure of an existing table without deleting it. Common operations: - Add
a new column. - Delete a column. - Rename a column. - Modify a column’s data type.

ALTER TABLE table_name

[ADD column_name datatype]
[DROP COLUMN column_name]
[RENAME COLUMN old_name TO new_name]
[MODIFY COLUMN column_name new_datatype];

[Link] **5.2 Syntax

• Keywords:
– ADD: Inserts a new column.
– DROP COLUMN: Removes a column.
– RENAME COLUMN: Changes a column’s name.
– MODIFY COLUMN: Alters a column’s data type.

[Link] 5.3 Examples

ALTER TABLE students

ADD email VARCHAR(255);

[Link].1 5.3.1 Adding a Column

• Effect: Adds an email column of type VARCHAR(255) to the students table.

ALTER TABLE students

DROP COLUMN age;

[Link].2 5.3.2 Dropping a Column

• Effect: Removes the age column from the students table.

ALTER TABLE students

RENAME COLUMN name TO fullName;

[Link].3 5.3.3 Renaming a Column

• Effect: Renames the name column to fullName.

ALTER TABLE students

MODIFY COLUMN age SMALLINT;

[Link].4 5.3.4 Modifying a Column’s Data Type

• Effect: Changes the age column’s data type from its original type to SMALLINT.

22
[Link] 5.4 Best Practices
• Perform ALTER operations before inserting data to avoid integrity constraints (e.g., data loss or type
conflicts).
• Backup data before altering tables, especially when modifying columns with existing data.

2.4.6 6. Practical Examples

CREATE TABLE courses (

CourseID INT PRIMARY KEY,
CourseName VARCHAR(25),
credits INT
);

[Link] 6.1 Creating a courses Table

• Result: A new table courses with columns:
– CourseID (primary key, INT).
– CourseName (VARCHAR(25)).
– credits (INT).

[Link] 6.2 Modifying the courses Table

ALTER TABLE courses

ADD instructor VARCHAR(255);

[Link].1 6.2.1 Adding an instructor Column

• Effect: Adds an instructor column to store the name of the course instructor.

ALTER TABLE courses

DROP COLUMN credits;

[Link].2 6.2.2 Dropping the credits Column

• Effect: Removes the credits column from the courses table.

2.4.7 7. Summary of Key Concepts

[Link] 7.1 DDL Commands Recap

Command Action Example

CREATE TABLE Creates a new table. CREATE TABLE students (...);
ALTER TABLE Modifies an existing table. ALTER TABLE students ADD email
VARCHAR;
DROP TABLE Deletes a table permanently. DROP TABLE students;

23
[Link] 7.2 When to Use DDL
• Database Design: Define schemas, tables, and constraints.
• Schema Evolution: Adapt the database structure to new requirements (e.g., adding columns).
• Cleanup: Remove obsolete tables or columns.

[Link] 7.3 Critical Considerations

• Auto-commit: DDL changes are immediate and permanent.
• Data Integrity: Altering tables with existing data may require migrations or backups.
• Dependencies: Ensure no other objects (e.g., views, foreign keys) rely on the table before dropping it.

2.4.8 8. Hands-On Exercise

Task: Practice the following in a database system (e.g., MySQL, PostgreSQL): 1. Create a table employees with
columns: - employeeID (INT, primary key). - firstName (VARCHAR(50)). - lastName (VARCHAR(50)). - salary
(DECIMAL(10, 2)). 2. Add a new column department (VARCHAR(50)). 3. Rename the salary column to
annualSalary. 4. Drop the lastName column. 5. Modify the firstName column to VARCHAR(100).

Expected Output: - A dynamically modified employees table reflecting all changes.

2.5 Data Manipulation Language (DML)

2.5.1 Introduction to Data Manipulation Language (DML)
• Definition: Data Manipulation Language (DML) consists of SQL commands used to manipulate data
within an existing database structure.
• Purpose: DML commands enable the insertion, modification, and deletion of data in database tables.
• Primary DML Commands:
1. INSERT INTO – Adds new data to a table.
2. UPDATE – Modifies existing data in a table.
3. DELETE – Removes data from a table.

2.5.2 1. INSERT INTO Command

[Link] Purpose
• Used to insert new rows (records) into a database table.
• Essential for populating tables with initial or additional data.

INSERT INTO table_name (column1, column2, column3, ...)

VALUES (value1, value2, value3, ...);

[Link] Syntax
• table_name: The name of the table where data is being inserted.
• column1, column2, ...: The columns into which data will be inserted.
• value1, value2, ...: The corresponding values for each column, in the same order as the columns
listed.

INSERT INTO Student (StudentID, Name, Age, Major)

VALUES (1, 'John', 20, 'Computer Science');

24
[Link] Example
• Explanation:
– Inserts a new row into the Student table.
– Assigns:
* StudentID = 1
* Name = 'John'
* Age = 20
* Major = 'Computer Science'
• Use Case: Adding a single new student record to the database.

[Link] Inserting Multiple Rows

• Multiple rows can be inserted in a single INSERT INTO statement by listing multiple value sets.
• Example:
INSERT INTO Course (CourseID, CourseName, Credits)
VALUES
(101, 'Math', 3),
(102, 'Physics', 4),
(103, 'Chemistry', 3);

– Explanation:
* Inserts three new records into the Course table in one command.
* Each set of values corresponds to a new row.
2.5.3 2. UPDATE Command
[Link] Purpose
• Used to modify existing data in a table.
• Allows changes to one or more columns based on specified conditions.

UPDATE table_name
SET column1 = value1, column2 = value2, ...
WHERE condition;

[Link] Syntax
• table_name: The table containing the data to be updated.
• SET column1 = value1: Specifies the column(s) to update and their new values.
• WHERE condition: Defines which rows should be updated (critical to avoid unintended updates).

UPDATE Student
SET Age = 21
WHERE StudentID = 1;

[Link] Example (Single Column Update)

• Explanation:

25
 – Updates the Age of the student with StudentID = 1 to 21.
– The WHERE clause ensures only the intended row is modified.
– If StudentID is a primary key, only one row is affected.

UPDATE Course
SET Credits = 5, CourseName = 'Advanced Physics'
WHERE CourseID = 102;

[Link] Example (Multiple Column Update)

• Explanation:
– Updates both Credits and CourseName for the course with CourseID = 102.
– Changes:
* Credits from (e.g., 4) to 5.
* CourseName from (e.g., ‘Physics’) to ‘Advanced Physics’.
[Link] Importance of the WHERE Clause
• Without WHERE, the UPDATE command modifies all rows in the table.
• Example of Dangerous Update (No WHERE Clause):
UPDATE Student SET Age = 22;

– Result: Every student’s age in the table is set to 22 (likely unintended).

2.5.4 3. DELETE Command

[Link] Purpose
• Used to remove rows (records) from a table.
• Operates at the row level (deletes entire rows, not individual column values).

DELETE FROM table_name

WHERE condition;

[Link] Syntax
• table_name: The table from which rows will be deleted.
• WHERE condition: Specifies which rows to delete (critical to avoid accidental data loss).

DELETE FROM Student

WHERE StudentID = 1;

[Link] Example (Single Row Deletion)

• Explanation:
– Deletes the row where StudentID = 1.
– If StudentID is a primary key, only one row is deleted.

26
DELETE FROM Course
WHERE CourseID = 103 AND Credits < 4;

[Link] Example (Deletion with Multiple Conditions)

• Explanation:
– Deletes rows where:
* CourseID = 103 and
* Credits < 4.
– Ensures only specific records meeting both conditions are removed.

[Link] Warning: DELETE Without WHERE

• Example of Dangerous Deletion:
DELETE FROM Course;

– Result: All rows in the Course table are deleted (irreversible unless backed up).

2.5.5 Practical Applications of DML Commands

[Link] 1. Populating a Database (INSERT INTO)
• Used when:
– Initially filling a table with data.
– Adding new records (e.g., new students, courses, or transactions).

[Link] 2. Maintaining Data Accuracy (UPDATE)

• Used when:
– Correcting errors (e.g., typos in names, incorrect ages).
– Updating records (e.g., changing a student’s major, adjusting course credits).

[Link] 3. Managing Data Integrity (DELETE)

• Used when:
– Removing obsolete records (e.g., deleted courses, inactive users).
– Cleaning up data (e.g., removing test entries).

2.5.6 Key Takeaways

1. DML Commands Are Essential for Data Management:
• INSERT INTO → Adds new data.
• UPDATE → Modifies existing data.
• DELETE → Removes data.
2. Syntax Precision Matters:
• Column names and values must match in order.
• WHERE clauses are critical to avoid unintended updates/deletions.
3. Best Practices:
• Always test DML commands in a safe environment before executing in production.
• Backup data before running DELETE or UPDATE operations.
• Use transactions (not covered here) to ensure data integrity during bulk operations.

27
 4. Real-World Impact:
• DML enables dynamic and flexible data management.
• Ensures databases remain accurate and up-to-date.
End of Notes

2.6 Data Query Language (DQL)

2.6.1 Introduction to Data Query Language (DQL)
• Definition: Data Query Language (DQL) is a subset of SQL commands specifically used to query and
retrieve data from a database.
• Primary Purpose: DQL enables users to extract meaningful information from one or more database tables.
• Key Command: The SELECT statement is the fundamental DQL command for data retrieval.

2.6.2 The SELECT Command

[Link] Definition & Purpose
• The SELECT command retrieves data from one or more tables in a database.
• It allows users to specify which columns and which rows should be returned in the result set.

SELECT column1, column2, ..., columnN

FROM table_name;

[Link] Basic Syntax

• column1, column2, ..., columnN: Names of the columns to retrieve.
• table_name: The table from which data is being fetched.

SELECT StudentID, Name, Age

FROM Student;

[Link] Example
• Explanation:
– Retrieves only the StudentID, Name, and Age columns from the Student table.
– If the table contains additional columns (e.g., Address, Email), they will not be included in the result.
– Returns all rows where these columns have data.

[Link] Retrieving All Columns

• Instead of listing every column, the wildcard * can be used to select all columns in a table.
• Syntax:
SELECT * FROM table_name;

• Example:
SELECT * FROM Courses;

– Retrieves every column (e.g., CourseID, CourseName, Credits, Instructor) from the Courses ta-
ble.

28
 – Also returns all rows in the table.

2.6.3 The WHERE Clause

[Link] Definition & Purpose
• The WHERE clause filters records based on a specified condition.
• Only rows that meet the condition are included in the result set.

SELECT column1, column2, ..., columnN

FROM table_name
WHERE condition;

[Link] Basic Syntax

• condition: A logical expression that evaluates to true or false for each row.

SELECT StudentID, Name, Age

FROM Student
WHERE Age > 20;

[Link] Example
• Explanation:
– Retrieves StudentID, Name, and Age only for students older than 20.
– Rows where Age <= 20 are excluded from the result.

SELECT CourseID, CourseName, Credits

FROM Courses
WHERE Credits > 3;

[Link] Practical Application

• Explanation:
– Retrieves CourseID, CourseName, and Credits only for courses with more than 3 credits.
– Courses with Credits <= 3 are not displayed.

2.6.4 The ORDER BY Clause

[Link] Definition & Purpose
• The ORDER BY clause sorts the result set in ascending (ASC) or descending (DESC) order based on one
or more columns.
• Default sorting is ascending (ASC) if not specified.

SELECT column1, column2, ..., columnN

FROM table_name
ORDER BY column_name [ASC | DESC];

[Link] Basic Syntax

29
 • column_name: The column used for sorting (must be one of the selected columns).
• ASC: Ascending order (default).
• DESC: Descending order.

SELECT StudentID, Name, Age

FROM Student
ORDER BY Age DESC;

[Link] Example (Descending Order)

• Explanation:
– Retrieves all students but sorts them by Age in descending order (oldest first).
– If two students have the same age, their order is not guaranteed unless a secondary sort column is
specified.

SELECT CourseID, CourseName, Credits

FROM Courses
ORDER BY CourseName ASC;

[Link] Example (Ascending Order)

• Explanation:
– Retrieves all courses but sorts them alphabetically by CourseName (A-Z).
– ASC is optional (default behavior).

SELECT CourseID, CourseName, Credits

FROM Courses
WHERE Credits > 3
ORDER BY CourseName ASC;

[Link] Combining with WHERE

• Explanation:
– Filters courses with Credits > 3.
– Sorts the filtered results by CourseName in ascending order.

2.6.5 Combining DQL Commands

SELECT *
FROM Courses
WHERE Credits > 3
ORDER BY CourseName ASC;

[Link] Full Example with SELECT, WHERE, and ORDER BY

• Explanation:
– Retrieves all columns from the Courses table.
– Filters to include only courses with Credits > 3.
– Sorts the results by CourseName in ascending order (A-Z).

30
[Link] Key Observations
• The ORDER BY clause must refer to a column included in the SELECT statement (or the table).
• The WHERE clause is applied before sorting (ORDER BY).
• Using SELECT * retrieves all columns, which may impact performance on large tables.

2.6.6 Summary of Key DQL Commands

Command Purpose Example

SELECT Retrieves data from one or more tables. SELECT Name, Age FROM Student;
WHERE Filters rows based on a condition. SELECT * FROM Courses WHERE Credits > 3;
ORDER Sorts the result set in ascending (ASC) or SELECT * FROM Student ORDER BY Age DESC;
BY descending (DESC) order.

2.6.7 Importance of DQL in Database Applications

• Dynamic Data Retrieval: DQL allows users to fetch specific data without modifying the database structure.
• Flexibility: Commands can be combined (SELECT + WHERE + ORDER BY) to create complex queries.
• Efficiency: Proper use of WHERE reduces unnecessary data transfer, improving performance.
• Data Analysis: Sorting (ORDER BY) and filtering (WHERE) enable meaningful data interpretation.

2.6.8 Best Practices

1. Avoid SELECT * in Production:
• Explicitly list required columns to improve performance and reduce network load.
2. Use WHERE for Filtering:
• Minimize result sets by applying conditions early.
3. Specify Sort Order:
• Always define ASC or DESC for clarity, even though ASC is default.
4. Test Queries:
• Verify results with small datasets before running on large tables.

2.6.9 Conclusion
• DQL is essential for interacting with databases, enabling data retrieval, filtering, and sorting.
• Mastery of SELECT, WHERE, and ORDER BY allows for efficient and precise data querying.
• These commands form the foundation for more advanced SQL operations in database applications.

2.7 Inserting Data into Tables

2.7.1 Introduction to Inserting Data
• This lecture covers SQL data insertion operations, focusing on:
– Single-row insertion
– Multi-row insertion
– Handling NULL values
– Setting default values in tables

[Link] Learning Objectives By the end of this lecture, students will be able to: 1. Perform single and multiple
row insertions using SQL commands. 2. Understand how to handle NULL values in table columns. 3. Define
and use default values for columns during data insertion.

31
2.7.2 1. Single Row Insertion
[Link] Syntax The basic SQL command for inserting a single row into a table is:
INSERT INTO table_name (column1, column2, column3, ...)
VALUES (value1, value2, value3, ...);

• table_name: The target table where data will be inserted.

• column1, column2, ...: The columns into which values will be inserted.
• value1, value2, ...: The corresponding values for each column, ordered correctly.

[Link] Key Rules

1. Column-Value Order Matching:
• The order of values must match the order of columns specified.
• Mismatches can lead to data integrity errors (e.g., inserting a string into an integer column).
2. String Data Types:
• Values for VARCHAR, CHAR, or other string-based columns must be enclosed in single quotes (').
• Example: 'Computer Science' (correct) vs. Computer Science (incorrect, may cause syntax errors).

INSERT INTO students (StudentID, name, age, major)

VALUES (1, 'John', 20, 'Computer Science');

[Link] Example: Inserting a Single Row

• Effect:
– A new row is added to the students table with:
* StudentID = 1
* name = 'John'
* age = 20
* major = 'Computer Science'

2.7.3 2. Multiple Row Insertion

[Link] Syntax To insert multiple rows in a single command, extend the VALUES clause with additional tuples:
INSERT INTO table_name (column1, column2, ...)
VALUES
(value1_row1, value2_row1, ...),
(value1_row2, value2_row2, ...),
...
(value1_rowN, value2_rowN, ...);

• Each set of values must be separated by a comma.

• The command ends with a semicolon (;).

[Link] Advantages
• Efficiency: Reduces the number of database calls (better performance than individual INSERT statements).
• Atomicity: All rows are inserted in a single transaction (either all succeed or none do).

32
INSERT INTO students (StudentID, name, age, major)
VALUES
(2, 'Smith', 22, 'Maths'),
(3, 'Brown', 21, 'Engineering'),
(4, 'White', 19, 'Physics');

[Link] Example: Inserting Multiple Rows

• Effect:
– Three new rows are added to the students table with the specified values.

2.7.4 3. Handling NULL Values

[Link] Definition of NULL
• NULL represents:
– Missing data (e.g., age not provided).
– Unknown data (e.g., major not declared).
– Not applicable (e.g., a column irrelevant for a specific row).

[Link] Syntax for Inserting NULL To explicitly insert a NULL value:

INSERT INTO table_name (column1, column2, ...)
VALUES (value1, NULL, ...);

• Key Point: NULL is a keyword, not a string (do not enclose it in quotes).

INSERT INTO students (StudentID, name, age, major)

VALUES (5, 'Tom', NULL, 'Biology');

[Link] Example: Inserting NULL Values

• Interpretation:
– age is set to NULL (e.g., Tom did not provide his age).
– The database treats this as missing/unknown data.

[Link] Use Cases for NULL

1. Optional Fields: Columns where data may not always be available (e.g., middle_name).
2. Temporary Placeholders: When data will be updated later.
3. Logical Exclusions: When a value does not apply (e.g., spouse_name for unmarried individuals).

2.7.5 4. Setting Default Values

[Link] Definition of Default Values
• A default value is a predefined value assigned to a column if no explicit value is provided during insertion.
• Useful for:
– Avoiding NULLs when a sensible default exists.
– Standardizing data (e.g., default age for new users).

33
[Link] Syntax for Defining Defaults Defaults are set during table creation using the DEFAULT keyword:
CREATE TABLE table_name (
column1 datatype [DEFAULT default_value],
column2 datatype [DEFAULT default_value],
...
);

• If no value is provided for column1 during insertion, default_value is used.

CREATE TABLE students (

StudentID INT,
name VARCHAR(250),
age INT DEFAULT 18,
major VARCHAR(100) DEFAULT 'Undeclared'
);

[Link] Example: Table with Default Values

• Defaults:
– age: Defaults to 18 if not specified.
– major: Defaults to 'Undeclared' if not specified.

[Link] Insertion Behavior with Defaults When inserting a row without specifying all columns, defaults are
applied:
INSERT INTO students (StudentID, name)
VALUES (6, 'Alice');

• Resulting Row:
– StudentID = 6
– name = 'Alice'
– age = 18 (default)
– major = 'Undeclared' (default)

[Link] When to Use Defaults

1. Common Values: E.g., status = 'active' for new users.
2. Business Rules: E.g., account_balance = 0 for new accounts.
3. Data Integrity: Ensures columns are never left NULL unintentionally.

2.7.6 Summary of Key Concepts

Concept Syntax/Example Purpose

Single Row INSERT INTO table (col1, col2) VALUES Add one row to a table.
Insertion (val1, val2);
Multi-Row INSERT INTO table (col1, col2) VALUES Add multiple rows efficiently.
Insertion (val1a, val2a), (val1b, val2b);
NULL Values INSERT INTO table (col1, col2) VALUES Handle missing/unknown data.
(val1, NULL);

34
Concept Syntax/Example Purpose
Default Values CREATE TABLE table (col1 INT DEFAULT Auto-fill columns when no value is
0); + INSERT INTO table (col2) VALUES provided.
(x);

2.7.7 Best Practices

1. Column Order: Always specify columns explicitly to avoid errors from implicit ordering.
⊠ INSERT INTO students VALUES (1, 'John', 20, 'CS'); (risky if table schema changes)
• [OK] INSERT INTO students (StudentID, name, age, major) VALUES (1, 'John', 20,
'CS');
2. NULL vs. Defaults:
• Use NULL for truly unknown/missing data.
• Use defaults for logical placeholders (e.g., status = 'inactive').
3. Performance:
• Prefer multi-row inserts over loops of single-row inserts for bulk data.
4. Data Types:
• Ensure values match column data types (e.g., strings in quotes, numbers unquoted).

2.8 Introduction to SQL and Databases

2.8.1 1. Overview of SQL (Structured Query Language)
[Link] 1.1 Definition and Role of SQL
• SQL (Structured Query Language) is the standard language for interacting with relational databases.
• It enables users to perform operations such as:
– Querying data (retrieving specific information).
– Updating records (modifying existing data).
– Managing database structure (creating, altering, or deleting tables and databases).
• SQL commands are integral to database management and are widely used across platforms and appli-
cations.

[Link] 1.2 Importance of SQL

• Provides a consistent method for interacting with relational data.
• Ensures efficient data retrieval and manipulation, which is critical for:
– Simple tasks (e.g., web browsing, user authentication).
– Complex applications (e.g., data analysis, business intelligence).
• Example Use Case:
– A school database where SQL can quickly retrieve a student’s grade without manual searching.
– A business analyzing sales data to identify trends, calculate totals, and generate reports for decision-
making.

2.8.2 2. Databases and Database Management Systems (DBMS)

[Link] 2.1 Definition of a Database
• A structured collection of data managed by a Database Management System (DBMS).
• The DBMS facilitates:
– Storage (holding large volumes of data).

35
 – Retrieval (accessing data efficiently).
– Manipulation (updating, deleting, or modifying data).

[Link] 2.2 Examples of Popular DBMS

• MySQL (open-source, widely used for web applications).
• PostgreSQL (advanced, supports complex queries).
• Oracle (enterprise-level, high performance).
• SQL Server (Microsoft’s relational database system).

[Link] 2.3 Analogy: Library Catalog as a Database

• A library catalog functions like a database:
– Stores structured information (book titles, authors, subjects).
– Allows efficient searching (e.g., by title, author, or keyword).

2.8.3 3. Applications of Databases in Real-World Scenarios

Databases are used across industries for data-driven operations:

Industry Use Case Example

Enterprise Managing sales, HR, and accounting records. Sales database tracking customer purchases
Info and generating reports.
Manufacturing Monitoring production, inventory, and supply Database tracking inventory levels and order
chain logistics. fulfillment.
Banking & Handling customer accounts, loans, and Tracking account balances and transaction
Finance transactions. histories.
Education Student registration, course enrollment, and University database storing student records
grading. and grades.
Airlines Managing flight schedules, reservations, and Database for flight bookings and passenger
passenger info. details.
Telecom Recording call logs, text data, usage, and Tracking call records and generating customer
billing. bills.
E- Supporting online retail with order tracking Managing product catalogs, customer info, and
Commerce and recommendations. order histories.

2.8.4 4. Relational Databases: Structure and Key Concepts

[Link] 4.1 Tables, Rows, and Columns
• Data in a relational database is stored in tables.
– Rows (Records): Represent individual entries (e.g., a single customer).
– Columns (Attributes): Define the properties of each record (e.g., customer ID, name, contact info).

[Link] 4.2 Example: Customer and Order Tables

Customer Table Order Table

customer_id (Key) order_id (Key)
name customer_id (Foreign Key)
contact_info product

36
 Customer Table Order Table
quantity
price

• Relationships between tables are established using keys:

– Primary Key (customer_id in Customer Table) uniquely identifies a record.
– Foreign Key (customer_id in Order Table) links to the Customer Table, ensuring data integrity.

[Link] 4.3 Benefits of Relational Databases

• Efficient organization of data.
• Data consistency through enforced relationships.
• Scalability for large datasets.

2.8.5 5. SQL Commands for Database Management

[Link] 5.1 Creating a Database
• Command:
CREATE DATABASE database_name;

• Example:
CREATE DATABASE SchoolDB;

– Initializes a new database named SchoolDB.

– Purpose: Sets up the environment for storing and managing data.

[Link] 5.2 Deleting a Database

• Command:
DROP DATABASE database_name;

• Example:
DROP DATABASE SchoolDB;

– Permanently removes the database and all its data.

– Caution: Irreversible action; use only when necessary.

[Link] 5.3 Viewing Existing Databases

• Command:
SHOW DATABASES;

– Displays a list of all databases in the DBMS.

2.8.6 6. Summary of Key Takeaways

1. SQL is the standard language for managing relational databases, enabling data retrieval, updates, and
structural management.
2. Databases are structured collections of data managed by a DBMS (e.g., MySQL, PostgreSQL).

37
 3. Real-world applications span industries (e.g., banking, education, e-commerce).
4. Relational databases use tables, rows, and columns, with keys ensuring data integrity.
5. Essential SQL Commands:
• CREATE DATABASE → Initializes a new database.
• DROP DATABASE → Deletes a database (permanent action).
• SHOW DATABASES → Lists all available databases.

2.9 Real-world SQL Scenarios

2.9.1 Introduction to Real-world SQL Applications
• This lecture explores practical applications of SQL across diverse industries.
• Objective:
– Apply SQL to solve common data problems in real-world scenarios.
– Understand industry-specific use cases through case studies and examples.
• SQL is a versatile tool for data management, retrieval, updates, and analysis in sectors such as:
– E-Commerce
– Banking
– Healthcare
– Education
– Telecom
– Manufacturing

2.9.2 SQL in E-Commerce

[Link] 1. Managing Customer Orders
• Purpose: Retrieve order details for a specific customer.
• SQL Query:
SELECT OrderID, CustomerName, OrderDate, TotalAmount
FROM Orders
WHERE CustomerID = 123;

– Explanation:
* Retrieves OrderID, CustomerName, OrderDate, and TotalAmount from the Orders table.
* Filters results using WHERE CustomerID = 123 to fetch orders for a specific customer.
[Link] 2. Updating Inventory After Purchase
• Purpose: Decrease stock quantity after a product is purchased.
• SQL Query:
UPDATE Product
SET Stock = Stock - 1
WHERE ProductID = 456;

– Explanation:
* Decrements the Stock value by 1 for the product with ProductID = 456.
* Ensures real-time inventory management post-purchase.

38
2.9.3 SQL in Banking
[Link] 1. Retrieving Account Balances
• Purpose: Fetch the balance of a specific customer.
• SQL Query:
SELECT CustomerID, Balance
FROM Accounts
WHERE CustomerID = 789;

– Explanation:
* Retrieves CustomerID and Balance from the Accounts table.
* Filters using WHERE CustomerID = 789 to display the balance for one customer.
[Link] 2. Recording a New Transaction
• Purpose: Insert a new transaction into the database.
• SQL Query:
INSERT INTO Transactions (TransactionID, AccountID, Amount, TransactionDate)
VALUES (101, 789, 500.00, CURRENT_TIMESTAMP);

– Explanation:
* Inserts a new record into the Transactions table with:
ꞏ TransactionID = 101
ꞏ AccountID = 789 (linked to the customer)
ꞏ Amount = 500.00
ꞏ TransactionDate set to the current timestamp (automatically records exact time).
* Ensures auditability and real-time tracking of financial transactions.

2.9.4 SQL in Healthcare

[Link] 1. Retrieving Patient Information
• Purpose: Fetch medical details for a specific patient.
• SQL Query (Specific Columns):
SELECT PatientID, Name, DateOfBirth, MedicalHistory
FROM Patients
WHERE PatientID = 101;

– Explanation:
* Retrieves only specified columns (PatientID, Name, DateOfBirth, MedicalHistory) for Pati-
entID = 101.
* Avoids fetching unnecessary data (unlike SELECT *).
• SQL Query (All Columns):
SELECT * FROM Patients;

– Explanation:
* Retrieves all columns for all patients (less efficient for large datasets).

39
[Link] 2. Scheduling a New Appointment
• Purpose: Add a new appointment to the database.
• SQL Query:
INSERT INTO Appointments (AppointmentID, PatientID, DoctorID, AppointmentDate, Reason)
VALUES (202, 101, 303, '2023-12-15', 'Routine Checkup');

– Explanation:
* Inserts a new appointment with:
ꞏ AppointmentID = 202
ꞏ PatientID = 101 (must exist in the Patients table)
ꞏ DoctorID = 303 (must exist in the Doctors table)
ꞏ AppointmentDate = '2023-12-15'
ꞏ Reason = 'Routine Checkup'

[Link] 3. Enforcing Data Integrity with Foreign Keys

• Purpose: Ensure referential integrity (e.g., no appointments for non-existent patients/doctors).
• Implementation:
– PatientID and DoctorID in the Appointments table should be foreign keys referencing the Patients
and Doctors tables.
– Primary Key: AppointmentID (uniquely identifies each appointment).
– Benefits:
* Prevents orphaned records (e.g., appointments for deleted patients).
* Maintains database consistency.
2.9.5 SQL in Education
[Link] 1. Retrieving Student Grades
• Purpose: Fetch grades for a specific student.
• SQL Query (Specific Student):
SELECT StudentID, CourseID, Grade
FROM Grades
WHERE StudentID = 202;

– Explanation:
* Retrieves StudentID, CourseID, and Grade for StudentID = 202.
• SQL Query (All Grades):
SELECT * FROM Grades;

– Explanation:
* Retrieves all grades for all students (useful for broad analysis but inefficient for large datasets).
[Link] 2. Enrolling a Student in a Course
• Purpose: Add a new enrollment record.
• SQL Query:

40
 INSERT INTO Enrollments (EnrollmentID, StudentID, CourseID, EnrollmentDate)
VALUES (301, 202, 404, '2023-09-01');

– Explanation:
* Inserts a new enrollment with:
ꞏ EnrollmentID = 301
ꞏ StudentID = 202
ꞏ CourseID = 404
ꞏ EnrollmentDate = '2023-09-01'

2.9.6 SQL in Telecom

[Link] 1. Retrieving Caller Details
• Purpose: Fetch call records for a specific caller.
• SQL Query (All Columns):
SELECT * FROM CallerRecords
WHERE CallerID = 505;

– Explanation:
* Retrieves all columns (e.g., CallerID, ReceiverID, CallDuration, CallDate) for CallerID =
505.

• SQL Query (Specific Columns):

SELECT CallerID, ReceiverID, CallDuration, CallDate
FROM CallerRecords
WHERE CallerID = 505;

– Explanation:
* Retrieves only specified columns for efficiency.
[Link] 2. Generating Monthly Bills
• Purpose: Calculate the total bill for a caller based on call duration.
• SQL Query:
SELECT SUM(CallDuration * RatePerMinute) AS TotalBill
FROM CallerRecords
WHERE CallerID = 505
AND CallDate BETWEEN '2023-01-01' AND '2023-01-31';

– Explanation:
* Uses SUM() to calculate the total bill by multiplying CallDuration by RatePerMinute.
* Filters records for:
ꞏ CallerID = 505
ꞏ Calls made between January 1, 2023, and January 31, 2023.
* Key Functions:
ꞏ SUM(): Aggregates values.
ꞏ BETWEEN: Specifies a date range.

41
2.9.7 SQL in Manufacturing
[Link] 1. Retrieving Production Data
• Purpose: Fetch products manufactured on a specific date.
• SQL Query:
SELECT * FROM Production
WHERE ProductionDate = '2023-10-10';

– Explanation:
* Retrieves all production records for ProductionDate = '2023-10-10'.
[Link] 2. Updating Supply Chain Status
• Purpose: Modify the delivery status of an order.
• SQL Query:
UPDATE SupplyChain
SET Status = 'Delivered'
WHERE OrderID = 701;

– Explanation:
* Updates the Status to 'Delivered' for OrderID = 701.
* Possible status values: 'Undelivered', 'In Transit', 'Dispatched', 'Delivered'.
* Ensures real-time tracking of order fulfillment.
2.9.8 Case Study: E-Commerce Order Management
[Link] Identifying Top-Selling Products
• Purpose: Retrieve the top 5 best-selling products by sales volume.
• SQL Query:
SELECT ProductID, SUM(Quantity) AS TotalSold
FROM OrderItems
GROUP BY ProductID
ORDER BY TotalSold DESC
LIMIT 5;

– Explanation:
* SUM(Quantity): Calculates the total quantity sold for each product.
* GROUP BY ProductID: Groups results by product.
* ORDER BY TotalSold DESC: Sorts products by sales volume (highest to lowest).
* LIMIT 5: Returns only the top 5 results.
– Key Functions:
* Aggregation (SUM) : Computes totals.
* Grouping (GROUP BY) : Organizes data by categories.
* Sorting (ORDER BY) : Ranks results.
* Limiting (LIMIT) : Restricts output size.

42
2.9.9 Summary of SQL Applications Across Industries

Industry Key SQL Use Cases

E-Commerce Order management, inventory updates, sales analytics.
Banking Account balance retrieval, transaction recording, fraud detection.
Healthcare Patient record management, appointment scheduling, medical history tracking.
Education Student enrollment, grade retrieval, course management.
Telecom Call detail records, billing generation, usage analytics.
Manufacturing Production tracking, supply chain updates, order status management.

[Link] Key Takeaways

1. SQL is industry-agnostic: Applicable in any sector requiring data management.
2. Core SQL Operations:
• SELECT: Retrieve data.
• INSERT: Add new records.
• UPDATE: Modify existing records.
• DELETE: Remove records.
• Aggregations (SUM, AVG, COUNT): Compute metrics.
• Grouping (GROUP BY): Organize data.
• Filtering (WHERE, BETWEEN): Narrow down results.
3. Data Integrity: Use foreign keys and constraints to maintain consistency.
4. Efficiency: Prefer specific column selection (SELECT col1, col2) over SELECT * for performance.

[Link] Best Practices

• Use meaningful column names for clarity.
• Leverage indexes for faster queries on large datasets.
• Validate data before insertion/updates (e.g., check for existing patient/doctor IDs).
• Document queries for maintainability.

2.10 Recording of Building Database Applications Week 1 - Live Session on 26-03-13

2.10.1 1. Course Overview
[Link] 1.1 Introduction to Database Applications
• The course focuses on building database-driven web applications (e.g., a college library system).
• Key components:
– Frontend (Browser/Users): Users interact via web pages (JSP, HTML).
– Backend (Java/Spring Boot): Handles business logic, communicates with the database.
– Database (MySQL): Stores and retrieves data.

[Link] 1.2 System Architecture (MVC Model)

• Model-View-Controller (MVC) Architecture:
– View (JSP): Displays data to users (converts database data to HTML).
– Controller (Java/Spring Boot): Processes user requests, interacts with the database.
– Model (Database): Stores data (e.g., books, students, courses).

43
[Link] 1.3 Course Objectives
• Development Environment Setup: Configuring tools (MySQL, VS Code, Java).
• Database Design: Creating tables, defining relationships.
• Backend Development: Using Spring Boot for database interactions.
• Scalable Web Applications: Building efficient, maintainable systems.

2.10.2 2. SQL Primer: Database Basics

[Link] 2.1 Database Fundamentals
• Database: A collection of related entities (tables).
– Example: A university database may include:
* Students, Faculty, Courses, Library, Admissions, Placements.
• Entity (Table): Represents a real-world object (e.g., Student, Course).
• Attributes (Columns): Properties of an entity (e.g., student_id, name, age).
• Records (Rows): Individual entries in a table.

[Link] 2.2 SQL Commands Overview SQL commands are categorized into: 1. Data Definition Language
(DDL): - Defines database structure (e.g., CREATE, ALTER, DROP). 2. Data Manipulation Language (DML): -
Manages data (e.g., INSERT, UPDATE, DELETE, SELECT). 3. Data Control Language (DCL): - Controls access (e.g.,
GRANT, REVOKE). 4. Transaction Control Language (TCL): - Manages transactions (e.g., COMMIT, ROLLBACK).

2.10.3 3. Data Definition Language (DDL)

[Link] 3.1 Creating a Database
• Command:
CREATE DATABASE database_name;

• Usage:
CREATE DATABASE university;

• Selecting a Database:
USE database_name;

[Link] 3.2 Creating Tables

• Syntax:
CREATE TABLE table_name (
column1 datatype constraints,
column2 datatype constraints,
...
);

• Example: Student Table:

CREATE TABLE Student (
student_id INT AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(50) NOT NULL,
age INT,

44
 grade VARCHAR(3),
city VARCHAR(20)
);

– AUTO_INCREMENT: Automatically assigns a unique ID.

– PRIMARY KEY: Uniquely identifies each record (cannot be NULL).
– NOT NULL: Ensures the column must have a value.

[Link] 3.3 Constraints

Constraint Description
PRIMARY KEY Uniquely identifies a record (only one per table).
NOT NULL Ensures a column cannot have a NULL value.
UNIQUE Ensures all values in a column are unique (but can be NULL).
DEFAULT Sets a default value if none is provided.
AUTO_INCREMENT Automatically generates a unique number (typically for PRIMARY KEY).

[Link] 3.4 Candidate Keys vs. Primary Key

• Candidate Key:
– A column (or set of columns) that can uniquely identify a record.
– Example: student_id, aadhaar_number, pan_number.
• Primary Key:
– One candidate key chosen as the main identifier.
– Example: student_id is selected as the PRIMARY KEY.
• Composite Key:
– A primary key made of multiple columns (e.g., student_id + course_id).

[Link] 3.5 Altering Tables (ALTER)

• Add a Column:
ALTER TABLE Student ADD COLUMN email VARCHAR(100);

• Modify a Column:
ALTER TABLE Student MODIFY COLUMN age VARCHAR(3);

• Drop a Column:
ALTER TABLE Student DROP COLUMN email;

[Link] 3.6 Dropping Tables (DROP)

• Command:
DROP TABLE table_name;

• Effect:
– Permanently deletes the table and all its data.
– Cannot be undone.

45
2.10.4 4. Data Manipulation Language (DML)
[Link] 4.1 Inserting Data (INSERT)
• Basic Syntax:
INSERT INTO table_name (column1, column2, ...)
VALUES (value1, value2, ...);

• Example:
INSERT INTO Student (name, age, grade, city)
VALUES ('Alice', 20, 'A', 'Hyderabad');

• Rules:
– String values must be enclosed in single quotes ('Alice').
– Numeric values are written without quotes (20).
– Order matters: Values must match the column order.
– Omitting columns: If a column is AUTO_INCREMENT or has a DEFAULT, it can be omitted.
– Omitting column list: If inserting values for all columns in order, the column list can be omitted:
INSERT INTO Student VALUES (1, 'Bob', 22, 'B', 'Bangalore');

[Link] 4.2 Retrieving Data (SELECT)

• Basic Query:
SELECT * FROM Student;

– * retrieves all columns.

• Filtering with WHERE:
SELECT * FROM Student WHERE city = 'Hyderabad';

• Sorting with ORDER BY:

SELECT * FROM Student ORDER BY age DESC;

– DESC: Descending order (default is ASC ascending).

• Grouping with GROUP BY:
SELECT city, COUNT(*) AS total_students
FROM Student
GROUP BY city;

– Groups records by city and counts students per city.

• Filtering Groups with HAVING:
SELECT grade, COUNT(*) AS total
FROM Student
GROUP BY grade
HAVING COUNT(*) >= 2;

46
 – Only includes groups with 2 or more students.

[Link] 4.3 Updating Data (UPDATE)

• Syntax:
UPDATE table_name
SET column1 = value1, column2 = value2, ...
WHERE condition;

• Example:
UPDATE Student
SET grade = 'A+'
WHERE name = 'Bob';

• Warning:
– Without WHERE, all records will be updated.

[Link] 4.4 Deleting Data (DELETE)

• Syntax:
DELETE FROM table_name WHERE condition;

• Example:
DELETE FROM Student WHERE name = 'Ashoka';

• Warning:
– Without WHERE, all records are deleted (but the table structure remains).
– Difference from DROP:
* DELETE removes rows but keeps the table.
* DROP removes the entire table.
2.10.5 5. Joins (Combining Tables)
[Link] 5.1 Foreign Keys
• A foreign key links two tables by referencing a PRIMARY KEY in another table.
• Example:
CREATE TABLE Course (
course_id INT AUTO_INCREMENT PRIMARY KEY,
course_name VARCHAR(50),
student_id INT,
FOREIGN KEY (student_id) REFERENCES Student(student_id)
);

– student_id in Course references student_id in Student.

– Referential Integrity: A foreign key value must exist in the referenced table.

[Link] 5.2 Types of Joins

47
Join Type Description Example
INNER Returns matching records from both tables. SELECT [Link], C.course_name FROM
JOIN Student S INNER JOIN Course C ON
S.student_id = C.student_id;
LEFT Returns all records from the left table and SELECT [Link], C.course_name FROM
JOIN matched records from the right. Student S LEFT JOIN Course C ON
S.student_id = C.student_id;
RIGHT Returns all records from the right table and SELECT [Link], C.course_name FROM
JOIN matched records from the left. Student S RIGHT JOIN Course C ON
S.student_id = C.student_id;
FULL Returns all records when there is a match in Not directly supported in MySQL (use LEFT
JOIN either table. JOIN + RIGHT JOIN + UNION).

[Link] 5.3 Aliases in Joins

• Purpose: Shortens table names for readability.
• Example:
SELECT [Link], C.course_name
FROM Student AS S
JOIN Course AS C ON S.student_id = C.student_id;

– S = Student, C = Course.

2.10.6 6. Practical Examples

-- Create table
CREATE TABLE Student (
student_id INT AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(30) NOT NULL,
age INT,
grade VARCHAR(3),
city VARCHAR(20)
);

-- Insert records
INSERT INTO Student (name, age, grade, city)
VALUES
('Arjun', 28, 'A', 'Delhi'),
('Bhanu', 21, 'B', 'Chennai'),
('Chandini', 18, 'A', 'Delhi'),
('Isha', 25, 'B', 'Hyderabad');

[Link] 6.1 Creating and Populating a Table

-- Select all students

SELECT * FROM Student;

48
-- Count students per city
SELECT city, COUNT(*) AS total_students
FROM Student
GROUP BY city;

-- Update a student's grade

UPDATE Student
SET grade = 'A+'
WHERE name = 'Arjun';

-- Delete a student
DELETE FROM Student
WHERE name = 'Ashoka';

[Link] 6.2 Querying Data

-- Copy selected columns into a new table

CREATE TABLE Std AS
SELECT name, city
FROM Student;

[Link] 6.3 Creating a New Table from an Existing One

2.10.7 7. Key Takeaways

1. SQL Commands:
• CREATE, ALTER, DROP (DDL).
• INSERT, SELECT, UPDATE, DELETE (DML).
2. Constraints:
• PRIMARY KEY, NOT NULL, UNIQUE, AUTO_INCREMENT.
3. Joins:
• INNER JOIN, LEFT JOIN, RIGHT JOIN for combining tables.
4. Case Sensitivity:
• SQL keywords (e.g., SELECT, WHERE) are not case-sensitive.
• Data values (e.g., 'Alice' vs 'alice') are case-sensitive.
5. Best Practices:
• Always use WHERE in UPDATE/DELETE to avoid accidental data loss.
• Use GROUP BY with HAVING for filtered aggregations.
• Prefer VARCHAR over CHAR for variable-length strings to save space.

2.11 SQL Data Types

2.11.1 Introduction to SQL Data Types
• Objective: Understand different SQL data types, their use cases, and effective implementation.
• Importance: Choosing the correct data type ensures:
– Efficient storage (minimizing wasted space).
– Data accuracy (preventing invalid entries).
– Optimal performance (faster queries and operations).

49
2.11.2 1. Numeric Data Types
Used for storing numerical values. Three primary types:

[Link] 1.1 INT (Integer)

• Purpose: Stores whole numbers (no fractional/decimal parts).
• Use Cases:
– Counting (e.g., number of items).
– Indexing (e.g., primary keys).
– Any scenario where decimal precision is unnecessary.
• Example:
age INT

– Stores 25 as a whole number (no decimals).

[Link] 1.2 FLOAT (Floating-Point Number)

• Purpose: Stores numbers with fractional parts (approximate values).
• Syntax: FLOAT(size, d)
– size: Total number of digits.
– d: Number of digits after the decimal point.
• Use Cases:
– Scientific data (e.g., measurements).
– Any value requiring fractional precision (but not exact decimal accuracy).
• Example:
height FLOAT(5, 2)

– Stores 175.50 (5 total digits, 2 after decimal).

[Link] 1.3 DECIMAL (Fixed-Precision Decimal)

• Purpose: Stores exact decimal values (high precision).
• Syntax: DECIMAL(total_digits, decimal_places)
– total_digits: Maximum total digits (including decimals).
– decimal_places: Number of digits after the decimal.
• Use Cases:
– Financial data (e.g., currency, where precision is critical).
– Any scenario requiring exact decimal representation (no rounding errors).
• Example:
price DECIMAL(10, 2)

– Stores 199.99 (10 total digits, 2 after decimal).

50
CREATE TABLE example (
age INT,
height FLOAT(5, 2),
price DECIMAL(10, 2)
);

[Link] 1.4 Example Table with Numeric Data Types

• Columns:
– age: Whole number (e.g., 30).
– height: Fractional (e.g., 180.75).
– price: Precise decimal (e.g., 299.99).

2.11.3 2. Text Data Types

Used for storing character strings. Three primary types:

[Link] 2.1 CHAR (Fixed-Length String)

• Purpose: Stores fixed-length strings (padded with spaces if shorter).
• Syntax: CHAR(length)
– length: Exact number of characters (e.g., CHAR(10) stores 10 chars).
• Use Cases:
– Data with known, fixed lengths (e.g., state abbreviations like “CA”).
– Efficient for static-length fields (no variable storage overhead).
• Example:
gender CHAR(6) -- Stores "Male" or "Female" (padded to 6 chars if needed).

[Link] 2.2 VARCHAR (Variable-Length String)

• Purpose: Stores variable-length strings (only uses space for actual content).
• Syntax: VARCHAR(max_length)
– max_length: Maximum characters allowed (e.g., VARCHAR(255)).
• Use Cases:
– Dynamic-length data (e.g., names, addresses).
– More flexible and space-efficient than CHAR for varying lengths.
• Example:
first_name VARCHAR(50) -- Stores "John" (uses only 4 chars, not 50).

[Link] 2.3 TEXT (Large Text Data)

• Purpose: Stores large blocks of text (e.g., documents, descriptions).
• Use Cases:

51
 – Long-form content (e.g., product descriptions, articles).
– When VARCHAR max length (often 255 or 65,535 chars) is insufficient.
• Example:
bio TEXT -- Stores a lengthy employee biography.

CREATE TABLE employee (

first_name VARCHAR(50),
gender CHAR(6),
bio TEXT
);

[Link] 2.4 Example Table with Text Data Types

• Columns:
– first_name: Variable-length (e.g., “Alice”).
– gender: Fixed-length (e.g., “Female”).
– bio: Large text (e.g., a paragraph about the employee).

2.11.4 3. Date and Time Data Types

Used for storing temporal data. Four primary types:

[Link] 3.1 DATE

• Purpose: Stores date values (year, month, day).
• Format: YYYY-MM-DD.
• Use Cases:
– Birthdays, event dates, or any date-only field.
• Example:
birth_date DATE -- Stores "1990-05-15".

[Link] 3.2 TIME

• Purpose: Stores time values (hours, minutes, seconds).
• Format: HH:MM:SS.
• Use Cases:
– Appointment times, schedules.
• Example:
appointment_time TIME -- Stores "14:30:00".

[Link] 3.3 DATETIME

• Purpose: Stores combined date and time.
• Format: YYYY-MM-DD HH:MM:SS.

52
 • Use Cases:
– Timestamps for events (e.g., “2023-10-25 09:45:00”).
• Example:
last_updated DATETIME -- Stores "2023-10-25 10:15:30".

[Link] 3.4 TIMESTAMP

• Purpose: Stores date and time with timezone awareness (often auto-updated).
• Key Feature: Can default to current timestamp on row creation/update.
• Use Cases:
– Auditing (e.g., tracking when a record was created/modified).
• Example:
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP

– Automatically sets to the current time when a row is inserted.

CREATE TABLE appointment (

patient_id INT,
doctor_id INT,
birth_date DATE,
appointment_date DATETIME,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

[Link] 3.5 Example Table with Date/Time Data Types

• Columns:
– birth_date: Date only (e.g., “1985-07-20”).
– appointment_date: Date + time (e.g., “2023-11-10 15:00:00”).
– created_at: Auto-populated timestamp (e.g., current system time).

2.11.5 4. Boolean Data Type

Used for storing true/false values.

[Link] 4.1 BOOLEAN

• Purpose: Stores binary states (TRUE/FALSE).
• Use Cases:
– Flags (e.g., “Is active?”, “Is verified?”).
– Toggleable attributes.
• Example:
is_active BOOLEAN -- Stores `TRUE` (active) or `FALSE` (inactive).

53
CREATE TABLE user (
user_id INT PRIMARY KEY,
username VARCHAR(50),
is_active BOOLEAN
);

[Link] 4.2 Example Table with Boolean Data Type

• Columns:
– is_active: TRUE (user is active) or FALSE (user is inactive).

2.11.6 5. Summary of Key Data Types

Category Data Types Use Cases Example

Numeric INT, FLOAT, DECIMAL Whole numbers, fractions, precise age INT, price
decimals DECIMAL(10,2)
Text CHAR, VARCHAR, TEXT Fixed/variable strings, large text gender CHAR(6), bio
blocks TEXT
Date/Time DATE, TIME, DATETIME, Dates, times, timestamps birth_date DATE,
TIMESTAMP created_at TIMESTAMP
Boolean BOOLEAN True/false flags is_active BOOLEAN

2.11.7 6. Best Practices for Choosing Data Types

1. Use INT for whole numbers (e.g., IDs, counts).
2. Use DECIMAL for financial data (avoid FLOAT for exact precision).
3. Prefer VARCHAR over CHAR for variable-length text (saves space).
4. Use TEXT for large content (e.g., articles, descriptions).
5. Leverage TIMESTAMP for auditing (auto-populate creation/modification times).
6. Use BOOLEAN for binary states (e.g., active/inactive flags).
7. Match data types to real-world constraints (e.g., DATE for birthdays, not strings).

2.11.8 7. Conclusion
• Correct data types ensure:
– Efficiency: Optimal storage and performance.
– Accuracy: Prevent invalid or inconsistent data.
– Reliability: Databases function as intended.
• Key Takeaway: Always select the most specific data type that fits the data’s nature and constraints.

2.12 String Functions in SQL

2.12.1 Introduction to String Functions in SQL
• Purpose: String functions in SQL are used to manipulate text data within database queries.
• Key Objectives:
– Understand how to use string functions to manipulate text data.
– Apply CONCAT and SUBSTRING functions in SQL queries.
– Explore additional string functions (UPPER, LOWER, LENGTH) for text processing.

54
2.12.2 1. The CONCAT Function
[Link] Definition
• Function: Combines two or more strings into a single string.
• Syntax:
CONCAT(string1, string2, ...)

• Use Case: Combines multiple text fields (e.g., first name + last name) for readability or processing.

SELECT CONCAT(FirstName, ' ', LastName) AS FullName FROM Students;

[Link] Example
• Explanation:
– Retrieves FirstName and LastName from the Students table.
– Combines them with a space (' ') in between.
– Displays the result as a new column named FullName.
• Output: A single column (FullName) containing concatenated first and last names.

[Link] Applications
• Creating full names, addresses, or combined text outputs.
• Simplifying data presentation in reports or queries.

2.12.3 2. The SUBSTRING Function

[Link] Definition
• Function: Extracts a portion (substring) of a string based on specified start and length parameters.
• Syntax:
SUBSTRING(string, start_position, length)

– string: The original text to extract from.

– start_position: The 1-based index where extraction begins.
– length: The number of characters to extract.

SELECT SUBSTRING(FirstName, 1, 3) AS Initials FROM Students;

[Link] Example
• Explanation:
– Extracts the first 3 characters from the FirstName column.
– Displays the result as Initials.
• Output: A column showing the first 3 letters of each student’s first name.

[Link] Applications
• Extracting initials, prefixes, or specific parts of strings (e.g., email domains, IDs).
• Data cleaning or formatting (e.g., truncating long strings).

55
2.12.4 3. Practical Examples

SELECT CONCAT(FirstName, ' ', LastName, ', Age: ', Age) AS StudentInfo
FROM Students;

[Link] Example 1: Combining Columns with CONCAT

• Explanation:
– Combines FirstName, LastName, and Age into a single formatted string.
– Adds descriptive text (e.g., ", Age: ") for clarity.
• Output: A column (StudentInfo) with formatted student details (e.g., “John Doe, Age: 20”).

SELECT SUBSTRING(Email, 1, 5) AS EmailStart FROM Students;

[Link] Example 2: Extracting Substrings with SUBSTRING

• Explanation:
– Extracts the first 5 characters from the Email column.
– Useful for analyzing email prefixes or domains.
• Output: A column (EmailStart) showing the first 5 characters of each email.

SELECT CONCAT(SUBSTRING(FirstName, 1, 1), '. ', LastName) AS InitialLastName

FROM Students;

[Link] Example 3: Combining CONCAT and SUBSTRING

• Explanation:
– Extracts the first character of FirstName (e.g., "J" from "John").
– Combines it with LastName (e.g., "Doe") to form an initial-style name (e.g., “J. Doe”).
• Output: A column (InitialLastName) with abbreviated names.

2.12.5 4. Additional String Functions

[Link] 1. UPPER and LOWER

[Link].1 UPPER Function

• Definition: Converts a string to uppercase.

• Syntax:
UPPER(string)

• Example:
SELECT UPPER(FirstName) AS UpperCaseName FROM Students;

– Output: Converts all characters in FirstName to uppercase (e.g., “JOHN”).

[Link].2 LOWER Function

• Definition: Converts a string to lowercase.

56
 • Syntax:
LOWER(string)

• Example:
SELECT LOWER(FirstName) AS LowerCaseName FROM Students;

– Output: Converts all characters in FirstName to lowercase (e.g., “john”).

[Link] 2. LENGTH Function

• Definition: Returns the number of characters in a string.
• Syntax:
LENGTH(string)

• Example:
SELECT LENGTH(FirstName) AS NameLength FROM Students;

– Output: A column (NameLength) showing the character count for each FirstName.

2.12.6 5. Summary of Key String Functions

Function Description Example

CONCAT Combines multiple strings. CONCAT(FirstName, ' ', LastName)
SUBSTRING Extracts a substring. SUBSTRING(Email, 1, 5)
UPPER Converts text to uppercase. UPPER(FirstName)
LOWER Converts text to lowercase. LOWER(FirstName)
LENGTH Returns the length of a string. LENGTH(FirstName)

2.12.7 6. Importance of String Functions

• Flexibility: Enable dynamic text manipulation within queries.
• Data Formatting: Standardize or transform text for reports, exports, or analysis.
• Efficiency: Reduce the need for post-processing in application code.
• Common Use Cases:
– Generating full names or addresses.
– Extracting parts of strings (e.g., initials, domains).
– Case normalization (e.g., for comparisons or displays).

2.12.8 7. Conclusion
• String functions are essential tools for working with text data in SQL.
• Mastery of CONCAT, SUBSTRING, UPPER, LOWER, and LENGTH enables:
– Data combination (e.g., merging columns).
– Data extraction (e.g., isolating substrings).
– Data transformation (e.g., case conversion, length calculation).
• These functions enhance query efficiency and data presentation in database applications.

57
2.13 Summary and Best Practices
2.13.1 1. Introduction
• This lecture serves as the final session in the SQL Primer module.
• Focuses on:
– Recapping key concepts covered in Module 1.
– Best practices for writing efficient and maintainable SQL.
• Learning Objectives:
– Recap core SQL concepts.
– Understand best practices for SQL efficiency and maintainability.

2.13.2 2. Recap of Key SQL Concepts

[Link] 2.1 SQL Basics
• SQL Syntax and Structure:
– Understanding the fundamental syntax of SQL (e.g., clauses, keywords, statements).
– Proper use of semicolons (;) to terminate statements.
– Case sensitivity rules (varies by DBMS, but keywords are typically uppercase for readability).

[Link] 2.2 Database and Table Operations

• Database Operations:
– Creating databases: CREATE DATABASE database_name;
– Modifying databases: ALTER DATABASE database_name ...;
– Deleting databases: DROP DATABASE database_name;
• Table Operations:
– Creating tables:
CREATE TABLE table_name (
column1 datatype constraints,
column2 datatype constraints,
...
);
– Modifying tables:
* Adding columns: ALTER TABLE table_name ADD column_name datatype;
* Dropping columns: ALTER TABLE table_name DROP COLUMN column_name;
* Modifying constraints: ALTER TABLE table_name MODIFY COLUMN column_name datatype
NEW_CONSTRAINT;
– Deleting tables: DROP TABLE table_name;

[Link] 2.3 Data Manipulation

• Inserting Data:
INSERT INTO table_name (column1, column2, ...)
VALUES (value1, value2, ...);

• Updating Data:
UPDATE table_name
SET column1 = value1, column2 = value2, ...
WHERE condition;

58
 • Deleting Data:
DELETE FROM table_name
WHERE condition;

[Link] 2.4 Data Querying

• Retrieving Data:
– Basic SELECT statement:
SELECT column1, column2, ...
FROM table_name;
– Filtering with WHERE:
SELECT column1, column2, ...
FROM table_name
WHERE condition;
– Sorting with ORDER BY:
SELECT column1, column2, ...
FROM table_name
ORDER BY column1 [ASC|DESC];

[Link] 2.5 String Functions

• Commonly used string functions:
– CONCAT(str1, str2, ...): Combines strings.
SELECT CONCAT('Hello', ' ', 'World') AS greeting; -- Returns "Hello World"
– SUBSTRING(str, start, length): Extracts a substring.
SELECT SUBSTRING('Database', 1, 4); -- Returns "Data"
– LENGTH(str) / LEN(str): Returns string length.
– UPPER(str) / LOWER(str): Converts case.

[Link] 2.6 Transaction Control

• Key Commands:
– COMMIT: Saves changes permanently.
– ROLLBACK: Reverts changes if an error occurs.
– SAVEPOINT: Sets a point to roll back to.
START TRANSACTION;
INSERT INTO accounts (id, balance) VALUES (1, 1000);
SAVEPOINT before_update;
UPDATE accounts SET balance = 500 WHERE id = 1;
-- If error occurs:
ROLLBACK TO before_update;
-- Otherwise:
COMMIT;
• Purpose: Ensures data integrity in multi-step operations.

[Link] 2.7 Access Control

59
 • Granting Privileges:
GRANT privilege_type ON database_name.table_name TO 'username'@'host';

Example:
GRANT SELECT, INSERT ON [Link] TO 'user1'@'localhost';

• Revoking Privileges:
REVOKE privilege_type ON database_name.table_name FROM 'username'@'host';

Example:
REVOKE DELETE ON [Link] FROM 'user2'@'%';

2.13.3 3. Best Practices for Writing Efficient and Maintainable SQL

[Link] 3.1 Consistent Naming Conventions
• Why? Improves readability and maintainability.
• Guidelines:
– Use snake_case for tables and columns (e.g., user_accounts, first_name).
– Avoid reserved keywords (e.g., order, group).
– Prefix tables with contextual names (e.g., app_users instead of users if multiple systems exist).
– Use singular nouns for table names (e.g., employee, not employees).

[Link] 3.2 Commenting Code

• Why? Helps others (and future you) understand complex logic.
• Best Practices:
– Use single-line comments (--) for brief explanations.
-- Retrieve active users with orders > $100
SELECT u.user_id, [Link]
FROM users u
JOIN orders o ON u.user_id = o.user_id
WHERE [Link] > 100 AND [Link] = 'active';
– Use multi-line comments (/* ... */) for detailed explanations.
/*
* This query calculates monthly revenue growth.
* It joins sales data with customer segments
* and applies a 3-month rolling average.
*/
SELECT ...

[Link] 3.3 Modularizing Code

• Why? Reduces complexity and duplication.
• Approaches:
– Break large queries into smaller, reusable views or CTEs (Common Table Expressions).
WITH active_customers AS (
SELECT user_id FROM users WHERE status = 'active'

60
 )
SELECT * FROM active_customers;
– Use stored procedures for repetitive tasks.
CREATE PROCEDURE get_customer_orders(IN customer_id INT)
BEGIN
SELECT * FROM orders WHERE user_id = customer_id;
END;

[Link] 3.4 Using Transactions

• Why? Ensures atomicity (all-or-nothing execution) in multi-step operations.
• Best Practices:
– Wrap related operations in a transaction.
START TRANSACTION;
-- Step 1: Deduct from account A
UPDATE accounts SET balance = balance - 100 WHERE id = 1;
-- Step 2: Add to account B
UPDATE accounts SET balance = balance + 100 WHERE id = 2;
COMMIT;
– Use SAVEPOINT for partial rollbacks.
– Avoid long-running transactions (can lock tables and degrade performance).

[Link] 3.5 Regularly Review and Refactor

• Why? Improves performance, reliability, and maintainability.
• Steps:
– Profile queries using EXPLAIN to identify bottlenecks.
EXPLAIN SELECT * FROM users WHERE status = 'active';
– Refactor based on:
* Index usage (missing or unused indexes).
* Query complexity (simplify nested subqueries).
* Outdated practices (e.g., replacing SELECT * with explicit columns).
2.13.4 4. Common Pitfalls to Avoid
[Link] 4.1 Ignoring Indexes
• Problem: Missing indexes lead to slow queries (full table scans).
• Solution:
– Add indexes on frequently queried columns (e.g., foreign keys, WHERE clauses).
CREATE INDEX idx_user_status ON users(status);
– Avoid over-indexing (slows down INSERT/UPDATE operations).

[Link] 4.2 Overusing Subqueries

• Problem: Nested subqueries can be inefficient and hard to read.
• Solution:
– Replace with JOINs where possible.

61
 -- Instead of:
SELECT name FROM users
WHERE user_id IN (SELECT user_id FROM orders WHERE amount > 100);

-- Use:
SELECT [Link]
FROM users u
JOIN orders o ON u.user_id = o.user_id
WHERE [Link] > 100;

[Link] 4.3 Neglecting Transactions

• Problem: Data inconsistency if operations fail mid-execution.
• Solution:
– Always use transactions for multi-step operations (e.g., bank transfers).

[Link] 4.4 Hardcoding Values

• Problem: Inflexible and error-prone (e.g., magic numbers).
• Solution:
– Use parameters/variables (in stored procedures or application code).
-- Bad: Hardcoded threshold
SELECT * FROM orders WHERE amount > 100;

-- Good: Parameterized
CREATE PROCEDURE get_large_orders(IN min_amount DECIMAL)
BEGIN
SELECT * FROM orders WHERE amount > min_amount;
END;

[Link] 4.5 Lack of Comments

• Problem: Unmaintainable code (others can’t understand logic).
• Solution:
– Comment complex queries, business rules, and edge cases.

2.13.5 5. Final Tips for Mastering SQL

[Link] 5.1 Stay Updated
• Follow SQL standards (e.g., ANSI SQL, DBMS-specific features).
• Resources:
– Official documentation (MySQL, PostgreSQL, SQL Server).
– Blogs (e.g., Use The Index, Luke).

[Link] 5.2 Practice Regularly

• How?
– Solve problems on platforms like LeetCode, HackerRank, or SQLZoo.
– Work on real-world datasets (e.g., Kaggle).

62
[Link] 5.3 Use Tools
• Database Management Tools:
– MySQL Workbench, pgAdmin, DBeaver, SQL Server Management Studio (SSMS).
• Query Optimization Tools:
– EXPLAIN ANALYZE (PostgreSQL), Query Execution Plans (SQL Server).

[Link] 5.4 Learn from Real-World Scenarios

• Study production databases:
– Analyze schema design, indexing strategies, and query patterns.
• Case Studies:
– How companies like Uber, Netflix, or Airbnb optimize SQL for scale.

2.13.6 6. Summary of Key Takeaways

• SQL Fundamentals:
– Database/table operations (CREATE, ALTER, DROP).
– Data manipulation (INSERT, UPDATE, DELETE).
– Querying (SELECT, WHERE, ORDER BY).
– String functions (CONCAT, SUBSTRING).
– Transactions (COMMIT, ROLLBACK, SAVEPOINT).
– Access control (GRANT, REVOKE).
• Best Practices:
– Naming conventions (consistent, readable).
– Commenting (explain complex logic).
– Modularization (CTEs, stored procedures).
– Transactions (ensure data integrity).
– Refactoring (optimize regularly).
• Pitfalls to Avoid:
– Missing indexes.
– Overusing subqueries.
– Neglecting transactions.
– Hardcoding values.
– Lack of comments.
• Final Tips:
– Stay updated, practice, use tools, learn from real-world examples.

2.14 Table Constraints

2.14.1 Introduction to Table Constraints
[Link] Definition
• Table constraints are rules applied to table columns to enforce data integrity.
• They ensure that the data stored in a database adheres to specific rules and conditions.

[Link] Purpose
• Maintain consistency and accuracy of data.
• Prevent invalid or duplicate entries.
• Establish relationships between tables.

63
[Link] Common Types of Constraints
1. UNIQUE – Ensures all values in a column are distinct.
2. CHECK – Ensures all values in a column satisfy a specific condition.
3. PRIMARY KEY – Uniquely identifies each row in a table.
4. FOREIGN KEY – Ensures referential integrity by linking to a primary key in another table.
5. NOT NULL – Ensures a column cannot contain a null value.

2.14.2 UNIQUE Constraint

[Link] Definition
• Ensures that all values in a column are distinct (no duplicates).
• Can be applied to single or multiple columns.

[Link] Use Cases

• Columns requiring unique values:
– Email addresses
– Social Security Numbers (SSN)
– Student IDs
– Employee IDs

CREATE TABLE Student (

StudentID INT UNIQUE,
Name VARCHAR(100)
);

[Link] Syntax Example

• Explanation:
– StudentID must be unique; no two students can have the same ID.
– Name has no uniqueness constraint.

[Link] Key Notes

• Null values are allowed unless explicitly restricted (e.g., UNIQUE NOT NULL).
• Differs from PRIMARY KEY in that UNIQUE allows one null value (unless combined with NOT NULL).

2.14.3 CHECK Constraint

[Link] Definition
• Ensures that all values in a column satisfy a specified condition.
• Used for data validation at the column level.

[Link] Use Cases

• Age restrictions (e.g., students must be >= 18).
• Salary constraints (e.g., salary cannot be negative).
• Valid date ranges (e.g., hire date must be after 2000).

64
[Link] Syntax Examples
1. Age Validation:
CREATE TABLE Student (
StudentID INT,
Name VARCHAR(100),
Age INT CHECK (Age >= 18)
);

• Ensures only students aged 18 or older can be inserted.

2. Salary Validation:
CREATE TABLE Employee (
EmployeeID INT,
Name VARCHAR(100),
Salary DECIMAL(10,2) CHECK (Salary >= 0)
);

• Ensures salary cannot be negative.

[Link] Key Notes

• Custom conditions can be defined using logical expressions (>, <, =, BETWEEN, IN, etc.).
• Applied per row during insertion or updates.

2.14.4 PRIMARY KEY Constraint

[Link] Definition
• Uniquely identifies each row in a table.
• Ensures:
– No two rows have the same primary key value.
– The primary key column cannot contain null values.

[Link] Differences from UNIQUE Constraint

Feature PRIMARY KEY UNIQUE

Null Values Not allowed Allowed (one null)
Purpose Unique identifier Uniqueness only
Default Index Automatically indexed May require explicit indexing

CREATE TABLE Student (

StudentID INT PRIMARY KEY,
Name VARCHAR(100) NOT NULL,
Age INT
);

[Link] Syntax Example

• Explanation:

65
 – StudentID is the primary key (unique and non-null).
– Name cannot be null (NOT NULL constraint).
– Age has no constraints (can be null or duplicate).

[Link] Key Notes

• A table can have only one primary key (but it can be composite, i.e., multiple columns).
• Often used as the foreign key in related tables.

2.14.5 FOREIGN KEY Constraint

[Link] Definition
• Links two tables to enforce referential integrity.
• Ensures that a value in one table matches a primary key in another table.

[Link] Purpose
• Maintains relationships between tables (e.g., student → enrollment).
• Prevents orphaned records (e.g., an enrollment record without a valid student).

CREATE TABLE Enrollment (

EnrollmentID INT PRIMARY KEY,
StudentID INT,
CourseID INT,
FOREIGN KEY (StudentID) REFERENCES Student(StudentID),
FOREIGN KEY (CourseID) REFERENCES Course(CourseID)
);

[Link] Syntax Example

• Explanation:
– StudentID in Enrollment must exist in the Student table’s StudentID (primary key).
– CourseID in Enrollment must exist in the Course table’s CourseID (primary key).

ALTER TABLE Student

ADD CONSTRAINT FK_Department
FOREIGN KEY (DepartmentID) REFERENCES Department(DepartmentID);

[Link] Additional Example (Using ALTER TABLE)

• Adds a foreign key after table creation.

[Link] Key Notes

• The foreign key column must match the data type of the referenced primary key.
• Cascade actions (e.g., ON DELETE CASCADE) can be defined to handle updates/deletions in the parent table.

66
2.14.6 NOT NULL Constraint
[Link] Definition
• Ensures that a column cannot contain a null value.
• Forces the column to always have a value during insertion or updates.

CREATE TABLE Student (

StudentID INT PRIMARY KEY,
Name VARCHAR(100) NOT NULL,
Age INT
);

[Link] Syntax Example

• Explanation:
– Name must be provided (cannot be null).
– Age can be null (no constraint).

[Link] Key Notes

• Often used with PRIMARY KEY (since primary keys cannot be null).
• Helps prevent missing critical data (e.g., a student record without a name).

2.14.7 Applying Column Constraints

[Link] Single Column Constraints
• Constraints applied to individual columns during table creation.

CREATE TABLE Student (

StudentID INT PRIMARY KEY,
Name VARCHAR(100) NOT NULL,
Age INT CHECK (Age >= 18),
Email VARCHAR(100) UNIQUE
);

[Link].1 Example: Multiple Constraints on a Table

• Breakdown:
– StudentID: Primary key (unique + non-null).
– Name: Cannot be null.
– Age: Must be >= 18.
– Email: Must be unique.

[Link] Key Observations

• Multiple constraints can be applied to a single column.
• Constraints are enforced during INSERT/UPDATE operations.

67
2.14.8 Multi-Column Constraints
[Link] Definition
• Constraints that enforce rules across multiple columns.
• Often used for composite uniqueness or composite primary keys.

[Link] Use Cases

• Ensuring a combination of columns is unique (e.g., a student can enroll in a course only once).
• Enforcing business rules that depend on multiple fields.

CREATE TABLE Enrollment (

EnrollmentID INT PRIMARY KEY,
StudentID INT,
CourseID INT,
CONSTRAINT UC_Enrollment UNIQUE (StudentID, CourseID),
FOREIGN KEY (StudentID) REFERENCES Student(StudentID),
FOREIGN KEY (CourseID) REFERENCES Course(CourseID)
);

[Link] Syntax Example (Composite Unique Key)

• Explanation:
– UC_Enrollment ensures that the combination of StudentID and CourseID is unique.
– Prevents duplicate enrollments (e.g., a student cannot enroll in the same course twice).

[Link] Key Notes

• Composite primary keys can also be defined similarly:
CREATE TABLE Enrollment (
StudentID INT,
CourseID INT,
PRIMARY KEY (StudentID, CourseID)
);

• Useful for junction tables in many-to-many relationships.

2.14.9 Summary of Key Constraints

Allows
Constraint Purpose Null? Example Use Case
UNIQUE Ensures all values in a column are distinct. Yes (one) Email, SSN, StudentID
CHECK Ensures values satisfy a condition. Depends Age >= 18, Salary >= 0
PRIMARY Uniquely identifies each row; cannot be null. No StudentID, EmployeeID
KEY
FOREIGN Links to a primary key in another table for referential Yes StudentID in Enrollment
KEY integrity. table
NOT Ensures a column cannot contain null. No Name, Required fields
NULL

68
2.14.10 Importance of Table Constraints
1. Data Integrity:
• Prevents invalid, duplicate, or missing data.
2. Referential Integrity:
• Ensures relationships between tables remain consistent.
3. Data Validation:
• Enforces business rules (e.g., age restrictions, salary limits).
4. Performance Optimization:
• Primary keys and unique constraints often improve query performance via indexing.
5. Reliability:
• Reduces errors by automating data validation.

2.14.11 Best Practices

1. Use PRIMARY KEYs for all tables to ensure unique row identification.
2. Apply NOT NULL to critical fields (e.g., names, IDs).
3. Use CHECK constraints for simple data validation (e.g., age, salary).
4. Leverage FOREIGN KEYs to maintain relationships between tables.
5. Consider composite keys for junction tables in many-to-many relationships.
6. Use ALTER TABLE to add constraints after table creation if needed.
Conclusion: Table constraints are fundamental to database design, ensuring data integrity, consistency, and
reliability. By applying constraints such as UNIQUE, CHECK, PRIMARY KEY, FOREIGN KEY, and NOT NULL, developers
can build robust and accurate database applications.

2.15 Transaction Control Language (TCL)

2.15.1 1. Introduction to Transaction Control Language (TCL)
• Definition: Transaction Control Language (TCL) is a subset of SQL commands used to manage transac-
tions within a database.
• Purpose: Ensures data integrity and consistency by controlling the execution of transactions.
• Key Commands:
– COMMIT
– ROLLBACK
– SAVEPOINT

2.15.2 2. Overview of TCL Commands

[Link] 2.1 COMMIT Command
• Definition: Permanently saves all changes made during the current transaction.
• Syntax:
COMMIT;

• Functionality:
– Once executed, all modifications (INSERT, UPDATE, DELETE) are permanently applied to the
database.
– Ensures that operations within the transaction are finalized and visible to other users.
• Example:

69
 INSERT INTO Students (StudentID, Name, Age, Major)
VALUES (1, 'John', 20, 'Computer Science');
COMMIT;

– Effect: The new record is saved into the Students table.

[Link] 2.2 ROLLBACK Command

• Definition: Undoes all changes made during the current transaction.
• Syntax:
ROLLBACK;

• Functionality:
– Reverts the database to its state before the transaction began.
– Used for error handling—if an error occurs, changes can be discarded.
• Example:
INSERT INTO Students (StudentID, Name, Age, Major)
VALUES (1, 'John', 20, 'Computer Science');
ROLLBACK;

– Effect: The insertion of StudentID = 1 is undone—the record is not added to the table.

[Link] 2.3 SAVEPOINT Command

• Definition: Sets a named point within a transaction to which you can later roll back.
• Syntax:
SAVEPOINT savepoint_name;

• Functionality:
– Allows partial rollback—only changes made after the savepoint are undone.
– Useful in complex transactions where multiple steps require validation.
• Example:
INSERT INTO Students (StudentID, Name, Age, Major)
VALUES (1, 'John', 20, 'Computer Science');
SAVEPOINT sp1;

UPDATE Students SET Age = 21 WHERE StudentID = 1;

ROLLBACK TO sp1;

– Effect:
* The UPDATE operation is undone, but the INSERT remains.
* The database returns to the state at sp1.
2.15.3 3. Practical Use Cases of TCL Commands
[Link] 3.1 COMMIT in Action

70
 • Scenario: Inserting a new student record and saving it permanently.
INSERT INTO Students (StudentID, Name, Age, Major)
VALUES (1, 'John', 20, 'Computer Science');
COMMIT;

– Outcome: The record is persisted in the database.

[Link] 3.2 ROLLBACK for Error Recovery

• Scenario: Accidentally deleting a student record and reverting the change.
DELETE FROM Students WHERE StudentID = 2;
ROLLBACK;

– Outcome: The deletion is undone—the student record remains intact.

[Link] 3.3 SAVEPOINT for Fine-Grained Control

• Scenario: Performing multiple operations and rolling back selectively.
-- Step 1: Insert a record
INSERT INTO Students (StudentID, Name, Age, Major)
VALUES (1, 'John', 20, 'Computer Science');
SAVEPOINT sp1;

-- Step 2: Update the record

UPDATE Students SET Age = 22 WHERE StudentID = 1;

-- Step 3: Delete another record

DELETE FROM Students WHERE StudentID = 2;

-- Step 4: Roll back to sp1 (undoes UPDATE and DELETE)

ROLLBACK TO sp1;
COMMIT;

– Outcome:
* The UPDATE and DELETE operations are reverted.
* Only the initial INSERT (up to sp1) is committed.
2.15.4 4. Key Reasons for Using TCL Commands

CommandPrimary Use Case Example Scenario

COMMITPermanently save changes to ensure data Finalizing a batch of student registrations.
persistence.
Revert changes in case of errors or invalid
ROLLBACK Undoing an accidental deletion of a student
operations. record.
Enable partial rollback in multi-step transactions
SAVEPOINT Reverting only the last few updates while keeping
for better control. earlier changes in a grading system.

71
2.15.5 5. Importance of TCL in Database Management
• Data Integrity: Ensures that transactions are atomic (all operations succeed or fail together).
• Error Handling: Provides mechanisms to recover from failures without corrupting data.
• Transaction Control: Allows fine-grained management of complex operations (e.g., banking transactions,
inventory updates).
• Consistency: Maintains a consistent state of the database even if transactions fail mid-execution.

2.15.6 6. Summary of TCL Commands

Command Description Syntax

COMMIT Saves all changes made during the current transaction permanently. COMMIT;
ROLLBACK Undoes all changes made during the current transaction. ROLLBACK;
SAVEPOINT Sets a named point within a transaction for partial rollback. SAVEPOINT
savepoint_name;
ROLLBACK Reverts changes to a specific savepoint. ROLLBACK TO
TO savepoint_name;

2.15.7 7. Conclusion
• TCL commands (COMMIT, ROLLBACK, SAVEPOINT) are essential for managing database transactions.
• They ensure data consistency, error recovery, and controlled execution of operations.
• Proper use of TCL prevents data corruption and maintains database reliability in real-world applications.

72
3 Module 3: Intermediate SQL and Indexing
3.1 Aggregation – MIN, MAX with GROUPBY, AVG, SUM
3.1.1 Introduction to Aggregate Functions
• Purpose: This lecture covers aggregate functions in SQL, specifically:
– MIN and MAX with GROUP BY
– AVG (average)
– SUM (summation)
• Key Objective: Learn how to compute aggregate values (minimum, maximum, average, sum) across groups
of data using GROUP BY or across entire columns.

3.1.2 Aggregate Functions Overview

• Definition: Aggregate functions perform calculations on sets of values and return a single value per group
or for the entire column.
• Common Aggregate Functions:
– MIN() – Finds the smallest value in a column.
– MAX() – Finds the largest value in a column.
– AVG() – Computes the average of numeric values.
– SUM() – Calculates the total sum of numeric values.
• Data Type Requirement: All aggregate functions (except COUNT) operate only on numeric data types (e.g.,
INT, DECIMAL, FLOAT).
• NULL Handling: Aggregate functions ignore NULL values in calculations (e.g., AVG excludes NULLs; it
does not treat them as zero).

3.1.3 The GROUP BY Clause

• Purpose: Groups rows that have the same values in specified columns into aggregated data.
• Usage with Aggregate Functions:
– When used with MIN, MAX, AVG, or SUM, it computes the aggregate per group rather than for the entire
table.
– Without GROUP BY, aggregate functions return a single result for the entire column.

3.1.4 Example Database Table: agent

The lecture uses a table named agent with the following structure:

Column Data Type Description

agent_code String Unique identifier for each agent (e.g., A007, A003).
agent_name String Full name of the agent.
working_area String Geographic area where the agent operates (e.g., New York, Bangalore).
commission Decimal Commission rate (e.g., 0.12 for 12%).
phone_no String Contact number of the agent.

3.1.5 1. MAX with GROUP BY

[Link] Definition
• MAX() returns the highest value in a column.
• When combined with GROUP BY, it finds the maximum value per group.

73
SELECT working_area, MAX(commission)
FROM agent
GROUP BY working_area;

[Link] Syntax

[Link] Explanation
1. SELECT working_area, MAX(commission):
• Retrieves the working_area and the maximum commission for each area.
2. FROM agent:
• Specifies the source table (agent).
3. GROUP BY working_area:
• Groups rows by working_area before applying MAX.

[Link] Example Output

working_area MAX(commission)
Bangalore 0.15
New York 0.14
London 0.13

• Interpretation: For each working_area, the highest commission rate is displayed (e.g., the highest com-
mission in Bangalore is 0.15).

[Link] Key Points

• Without GROUP BY, MAX(commission) would return one value (the highest commission across all agents).
• GROUP BY ensures the maximum is calculated per distinct group (e.g., per working_area).

3.1.6 2. MIN with GROUP BY

[Link] Definition
• MIN() returns the lowest value in a column.
• With GROUP BY, it finds the minimum value per group.

SELECT working_area, MIN(commission) AS min_commission

FROM agent
GROUP BY working_area;

[Link] Syntax

[Link] Explanation
1. SELECT working_area, MIN(commission) AS min_commission:
• Retrieves working_area and the minimum commission for each area.
• AS min_commission renames the output column for clarity (optional).
2. FROM agent:

74
 • Specifies the table (agent).
3. GROUP BY working_area:
• Groups rows by working_area before applying MIN.

[Link] Example Output

working_area min_commission
Bangalore 0.11
New York 0.10
London 0.12

• Interpretation: For each working_area, the lowest commission rate is displayed (e.g., the lowest commis-
sion in Bangalore is 0.11).

[Link] Key Points

• MIN without GROUP BY returns the single lowest value in the entire column.
• AS is used to alias the output column (e.g., min_commission instead of MIN(commission)).

3.1.7 3. SUM Aggregate Function

[Link] Definition
• SUM() calculates the total sum of a numeric column.
• Can be used with or without GROUP BY.

SELECT SUM(advance_amount) AS total_advances

FROM orders;

[Link] Syntax (Without GROUP BY)

[Link] Explanation
1. SELECT SUM(advance_amount) AS total_advances:
• Computes the sum of all values in advance_amount.
• AS total_advances renames the output column.
2. FROM orders:
• Specifies the table (orders), which contains an advance_amount column.

[Link] Example Output

total_advances
15000.00

• Interpretation: The sum of all advance_amount values in the orders table is 15000.00.

75
[Link] Key Points
• Data Type Requirement: SUM only works on numeric columns (e.g., INT, DECIMAL).
• NULL Handling: Ignores NULL values in the column.
• Use Case: Useful for calculating totals (e.g., total sales, total advances).

3.1.8 4. AVG Aggregate Function

[Link] Definition
• AVG() computes the average (mean) of a numeric column.
• Excludes NULL values from calculations.

SELECT AVG(advance_amount) AS avg_advance

FROM orders;

[Link] Syntax

[Link] Explanation
1. SELECT AVG(advance_amount) AS avg_advance:
• Calculates the average of advance_amount.
• AS avg_advance renames the output column.
2. FROM orders:
• Specifies the table (orders).

[Link] Example Output

avg_advance
3000.50

• Interpretation: The average advance_amount across all records is 3000.50.

[Link] Key Points

• Numeric Data Types Only: Works on INT, DECIMAL, FLOAT, etc.
• NULL Exclusion: NULL values are not treated as zero; they are excluded from the calculation.
• Formula: AVG = (Sum of non-NULL values) / (Number of non-NULL rows).
• Use Case: Useful for analyzing trends (e.g., average salary, average sales).

3.1.9 Comparison of Aggregate Functions with and without GROUP BY

Function Without GROUP BY With GROUP BY

MAX Single maximum value for the entire column. Maximum value per group.
MIN Single minimum value for the entire column. Minimum value per group.
SUM Total sum of the entire column. Sum per group.
AVG Average of the entire column. Average per group.

76
3.1.10 General Rules for Aggregate Functions
1. Column Selection in SELECT:
• Any column in SELECT that is not an aggregate function must appear in the GROUP BY clause.
• Example: Invalid to write SELECT agent_name, MAX(commission) without grouping by agent_name.
2. GROUP BY Order:
• The GROUP BY clause must appear after WHERE but before ORDER BY.
3. Aliasing with AS:
• Use AS to rename output columns for clarity (e.g., AS max_commission).
4. Performance Considerations:
• Aggregate functions can be resource-intensive on large tables. Indexing grouped columns may im-
prove performance.

3.1.11 Practical Examples

[Link] Example 1: MAX and MIN with GROUP BY Query:
SELECT working_area, MAX(commission), MIN(commission)
FROM agent
GROUP BY working_area;

Output: | working_area | MAX(commission) | MIN(commission) | |————–|—————–|—————–| | Ban-

galore | 0.15 | 0.11 | | New York | 0.14 | 0.10 |

[Link] Example 2: SUM and AVG Without GROUP BY Query:

SELECT
SUM(advance_amount) AS total_advance,
AVG(advance_amount) AS avg_advance
FROM orders;

Output: | total_advance | avg_advance | |—————|————-| | 15000.00 | 3000.50 |

[Link] Example 3: AVG with GROUP BY Query:

SELECT working_area, AVG(commission) AS avg_commission
FROM agent
GROUP BY working_area;

Output: | working_area | avg_commission | |————–|—————–| | Bangalore | 0.13 | | New York | 0.12 |

3.1.12 Common Mistakes and Best Practices

1. Forgetting GROUP BY for Non-Aggregate Columns:
⊠ Incorrect: SELECT agent_name, MAX(commission) FROM agent;
• [OK] Correct: SELECT MAX(commission) FROM agent; or SELECT agent_name, commission FROM
agent GROUP BY agent_name, commission;
2. Using Aggregate Functions on Non-Numeric Columns:
⊠ Incorrect: SELECT AVG(agent_name) FROM agent; (strings cannot be averaged).
3. Ignoring NULL Values:
• Aggregate functions exclude NULL values. Use COALESCE or ISNULL to handle them if needed.
4. Overusing GROUP BY:
• Only use GROUP BY when you need per-group aggregates. For column-wide aggregates, omit it.

77
3.1.13 Summary of Key Concepts

Concept Description
Aggregate Functions MIN, MAX, SUM, AVG compute single values from sets of data.
GROUP BY Groups rows by distinct column values before applying aggregates.
Data Types Aggregate functions (except COUNT) require numeric columns.
NULL Handling NULL values are ignored in aggregate calculations.
Aliasing (AS) Renames output columns for readability (e.g., AS max_commission).
Performance Indexing grouped columns can optimize queries with GROUP BY.

3.2 Auto – Increment Columns, Replace

3.2.1 1. Introduction to Auto-Increment in SQL
[Link] 1.1 Definition and Purpose
• Auto-increment is a feature in SQL that automatically generates a unique value for a specified column in a
table.
• Ensures each record has a unique identifier without manual assignment.
• Reduces the risk of duplicate values and human errors in data entry.

[Link] 1.2 Common Use Cases

• Primary Key Columns: Often used for primary keys to ensure uniqueness.
• Unique Constraint Columns: Can be applied to any column requiring unique values, even if not a primary
key.

[Link] 1.3 Implementation in MySQL

• Applied using the keyword AUTO_INCREMENT in MySQL.
• By default, starts at 1 and increments by 1 for each new record.

3.2.2 2. Syntax and Table Structure for Auto-Increment

CREATE TABLE students (

id INT PRIMARY KEY AUTO_INCREMENT,
first_name VARCHAR(25) NOT NULL,
last_name VARCHAR(25),
age INT
);

[Link] 2.1 Example Table Definition

• id: Integer column with AUTO_INCREMENT and PRIMARY KEY.
• first_name: VARCHAR(25) with a NOT NULL constraint.
• last_name: VARCHAR(25) (nullable).
• age: Integer field.

[Link] 2.2 How Auto-Increment Works

• When inserting records, no value is specified for the id column.

78
 • The database automatically assigns the next available number.
Example Insertion:
INSERT INTO students (first_name, last_name, age)
VALUES ('Rahul', 'Sharma', 22);

• The id column is auto-filled (e.g., 1 for the first record).

3.2.3 3. Viewing Auto-Incremented Records

[Link] 3.1 Retrieving Data
• Use SELECT * FROM students to view all records, including auto-generated IDs.
Example Output: | id | first_name | last_name | age | |—-|————|———–|—–| | 1 | Rahul | Sharma | 22 | | 2 |
Aakash | Verma | 23 | | 3 | Nick | Patel | 21 |
• IDs are sequential and automatically assigned.

3.2.4 4. Modifying Auto-Increment Starting Value

[Link] 4.1 Changing the Starting Value
• Use ALTER TABLE to set a new starting value.
Syntax:
ALTER TABLE table_name AUTO_INCREMENT = start_value;

Example:
ALTER TABLE students AUTO_INCREMENT = 100;

• Effect: New records will start from 100 (e.g., next ID = 100).

[Link] 4.2 Key Observations

• Existing records remain unchanged (e.g., IDs 1, 2, 3 stay the same).
• New records use the updated starting value.
Example After Alteration: | id | first_name | last_name | age | |—–|————|———–|—–| | 1 | Rahul | Sharma |
22 | | 2 | Aakash | Verma | 23 | | 3 | Nick | Patel | 21 | | 100 | David | Lee | 24 | | 101 | Hitesh | Kumar | 25 |

3.2.5 5. Introduction to the REPLACE Function

[Link] 5.1 Definition and Purpose
• REPLACE is a built-in SQL function that replaces all occurrences of a substring within a string.
• Useful for data cleaning, standardization, and corrections.

REPLACE(string, old_substring, new_substring)

[Link] 5.2 Syntax

• string: The input text where replacement occurs.
• old_substring: The substring to be replaced.

79
 • new_substring: The replacement text.

3.2.6 6. Components of the REPLACE Function

[Link] 6.1 string (Input Text)
• The column or expression where the replacement is applied.

[Link] 6.2 old_substring (Target Substring)

• The portion of the text to be replaced.

[Link] 6.3 new_substring (Replacement Text)

• The new text that replaces old_substring.

3.2.7 7. Use Cases for REPLACE

• Data Standardization: Correcting inconsistent entries (e.g., “Python” vs. “python”).
• Typo Correction: Fixing misspelled words in a database.
• Data Cleaning: Removing unwanted characters (e.g., replacing " " with "").

3.2.8 8. Example: Using REPLACE with UPDATE

[Link] 8.1 Scenario
• Replace all occurrences of "Python" with "C++" in a courses column.
SQL Query:
UPDATE subject
SET course = REPLACE(course, 'Python', 'C++')
WHERE course LIKE '%Python%';

[Link] 8.2 Breakdown of the Query

1. UPDATE subject: Modifies the subject table.
2. SET course = REPLACE(...): Replaces "Python" with "C++" in the course column.
3. WHERE course LIKE '%Python%': Ensures only rows containing "Python" are updated.

[Link] 8.3 Expected Output

• Before: | course | |—————–| | Python Basics | | Advanced Python | | Java Programming|
• After: | course | |—————–| | C++ Basics | | Advanced C++ | | Java Programming|

3.2.9 9. Summary of Key Concepts

[Link] 9.1 Auto-Increment
• Automatically generates unique sequential IDs.
• Applied using AUTO_INCREMENT in MySQL.
• Can be modified with ALTER TABLE.

80
[Link] 9.2 REPLACE Function
• Replaces substrings in a string.
• Used with UPDATE for bulk data modifications.
• Helps in data cleaning and standardization.

3.3 B+ Tree Indexing

3.3.1 1. Introduction to B+ Trees
[Link] 1.1 Definition and Purpose
• B+ Tree: An advanced, self-balancing tree structure where all data values are stored at the leaf level,
ensuring efficient data access and organization.
• Multi-level Indexing: Creates an index of indices, enabling fast and efficient data retrieval.
• Balanced Structure: All leaf nodes are at the same level, maintaining balance and optimizing search oper-
ations.

[Link] 1.2 Key Properties of B+ Trees

1. Balanced Height: All leaves are at the same level, ensuring uniform access time.
2. Root Node Requirement: The root must have at least two children (unless it is a leaf).
3. Node Capacity:
• Maximum children per node (except root): m
• Minimum children per node (except root): ceil(m/2)
• Maximum keys per node: m-1
• Minimum keys per node (except root): ceil(m/2) - 1
4. Sorted Keys: Keys within a node are stored in ascending order, facilitating efficient search.

3.3.2 2. B+ Tree Structure

[Link] 2.1 Components of a B+ Tree
1. Root Node: Topmost node; contains keys and pointers to child nodes.
2. Internal Nodes: Non-leaf nodes that guide the search process; contain keys and pointers but no data
records.
3. Leaf Nodes:
• Store actual data records (key-value pairs).
• Linked sequentially to enable efficient range queries and sequential access.
• Contain pointers to data records (P_i) and search keys (K_i).

[Link] 2.2 Node Structure

• General Form:
– For a node with m pointers and m-1 keys:
P1, K1, P2, K2, ..., Pm-1, Km-1, Pm
– P_i: Pointers to child nodes (for internal nodes) or data records (for leaf nodes).
– K_i: Search key values used for navigation.
• Key Ordering Rules:
– For internal nodes:
* All keys in the subtree pointed to by P1 are less than K1.
* For 1 <= i < m-1, keys in the subtree pointed to by Pi are >= Ki-1 and < Ki.
* All keys in the subtree pointed to by Pm are >= Km-1.

81
 – For leaf nodes:
* Keys are stored in sorted order.
* Pn (last pointer) points to the next leaf node in sequence (linked list structure).
[Link] 2.3 Example Diagram Analysis
• Root Node: Contains keys (e.g., “Einstein,” “Gold”) pointing to internal nodes.
• Internal Nodes:
– Example: Node with key “Einstein” points to leaf nodes containing “Crick,” “Brandt,” “Califano.”
– Node with key “Gold” points to leaf nodes containing “Einstein,” “El Said,” “Gold.”
• Leaf Nodes:
– Store actual records (e.g., “Srinivasan” with department and salary).
– Linked sequentially (e.g., “Srinivasan” → “Wu” → “Kim”).

3.3.3 3. Comparison: B+ Tree vs. B Tree

Feature B+ Tree B Tree

Data Pointers Only in leaf nodes. In all nodes (internal + leaf).
Leaf Connectivity Leaves are linked (sequential access). Leaves are not linked.
Operational Speed Faster due to structured organization. Slower due to less optimization.
Key Discovery All keys at leaves; no early Keys may be found in internal nodes.
termination.
Insertion/Deletion Simpler algorithms. More complex (keys in all nodes).
Range Queries Efficient (linked leaves). Less efficient.
Space Overhead Higher (due to pointers). Lower.

[Link] 3.1 Advantages of B+ Trees Over B Trees

1. Self-Reorganization:
• Automatically reorganizes with local changes during insertions/deletions.
• No need for full file reorganization.
2. Efficient Range Queries:
• Linked leaves enable sequential scans without additional I/O.
3. Consistent Performance:
• Balanced structure ensures logarithmic time complexity for operations.
4. Simpler Implementation:
• Keys only at leaves reduce complexity in pointer management.

[Link] 3.2 Disadvantages of B+ Trees

1. Space Overhead:
• Additional pointers for leaf linkage increase memory usage.
2. Insertion/Deletion Overhead:
• Splitting/merging nodes may require multiple disk writes.
3. No Early Key Discovery:
• Unlike B trees, all searches must reach a leaf node.

[Link] 3.3 Disadvantages of B Trees

1. Limited Early Key Discovery:

82
 • Only a small fraction of keys can be found before reaching leaves.
2. Complex Operations:
• Insertions/deletions require more sophisticated algorithms to maintain balance.
3. Harder Implementation:
• Managing keys in both internal and leaf nodes increases complexity.

3.3.4 4. Insertion in B+ Trees

[Link] 4.1 Insertion Process Overview
1. Find the Correct Leaf Node:
• Traverse from the root to the appropriate leaf using key comparisons.
2. Insert the Key:
• If the leaf has space, insert the key in sorted order.
• If the leaf is full, split it and promote the middle key to the parent.
3. Handle Splits:
• Splitting may propagate upward if parent nodes are also full.
• Ensures the tree remains balanced.

[Link] 4.2 Step-by-Step Insertion Example Initial Tree Structure: - Root: “Mozart” - Internal Nodes:
“Gold,” “Einstein,” “Srinivasan” - Leaf Nodes: “Adams,” “Brandt,” “Crick,” “Einstein,” “El Said,” “Gold,”
“Kim,” “Srinivasan”
Inserting “Lamport”: 1. Traversal: - Start at root (“Mozart”). - “Lamport” > “Mozart” → move to right subtree.
2. Locate Leaf: - Compare with internal keys (“Gold,” “Einstein,” “Srinivasan”). - “Lamport” > “Gold” but <
“Srinivasan” → target leaf contains “El Said,” “Gold.” 3. Insertion: - Leaf has space → insert “Lamport” in sorted
order. - If leaf were full: - Split into two leaves. - Promote the middle key (e.g., “Gold”) to the parent. - Update
parent pointers.
Post-Insertion Structure: - “Lamport” added to the leaf. - Internal nodes adjusted if splits occurred (e.g., “Kim”
may become an internal key).

[Link] 4.3 Splitting Non-Leaf Nodes

1. Copy Node to Memory:
• Allocate space for m+1 pointers and m keys.
2. Insert New Key-Pointer Pair:
• Maintain sorted order.
3. Split the Node:
• First ceil(m/2) pointers/keys → original node.
• Remaining pointers/keys → new node.
• Promote the middle key to the parent.
4. Update Parent:
• Parent now points to both split nodes.
Example: - Before Split: Node with keys [“Adams,” “Brandt,” “Califano,” “Crick”] (full). - After Split: - Left
node: [“Adams,” “Brandt”] - Right node: [“Califano,” “Crick”] - Promote “Califano” to parent.

3.3.5 5. Deletion in B+ Trees

[Link] 5.1 Deletion Process Overview
1. Locate the Key:

83
 • Traverse to the leaf containing the key.
2. Remove the Key:
• Delete the key-value pair from the leaf.
3. Check for Underflow:
• If the leaf has fewer than ceil(m/2) - 1 keys, it is underfull.
4. Rebalance:
• Borrow a key from a sibling (if possible).
• Merge with a sibling if borrowing is not feasible.
• Update parent pointers to reflect changes.

[Link] **5.2 Deletion Example: “Srinivasan” Initial Structure: - Root: “Mozart” - Internal Nodes: “Cali-
fano,” “Einstein,” “Gold,” “Srinivasan” - Leaf Nodes: “Adams,” “Brandt,” …, “Srinivasan”
Steps: 1. Delete “Srinivasan”: - Remove from its leaf node. 2. Underflow Handling: - Leaf becomes under-
full → merge with sibling. - Combine keys from both leaves. - Update parent to point to the merged node. 3.
Parent Update: - Remove the promoted key (“Srinivasan”) from the parent. - If parent becomes underfull, repeat
merging/borrowing.

[Link] 5.3 Borrowing vs. Merging

Scenario Action Impact

Sibling has extra keys Borrow a key from sibling. Balances nodes without restructuring.
Sibling is minimal Merge nodes. Reduces tree height if root merges.

Example: Deleting “Singh” and “Wu”: 1. Delete Keys: - Remove “Singh” and “Wu” from their respective
leaves. 2. Underflow: - Leaf becomes underfull → borrow from left sibling. - Move a key from sibling to
balance the node. 3. Update Parent: - Adjust parent pointers to maintain order.

[Link] 5.4 Post-Deletion Adjustments

1. Merge Siblings:
• Combine keys from two nodes into one.
• Delete the empty node.
2. Update Parent:
• Remove the separator key from the parent.
3. Redistribution:
• If merging causes parent underflow, redistribute keys among siblings.

3.3.6 6. Updating B+ Trees

[Link] 6.1 Update Operations
• Definition: Modifications to the tree after insertions/deletions to maintain properties.
• Steps:
1. Remove Pr, V (pointer-value pair) from the leaf.
2. Check for Underflow:
– If leaf has too few entries, borrow or merge.
3. Merge Siblings (if necessary):
– Combine keys from both nodes.
– Delete the empty sibling.

84
 4. Update Parent:
– Remove the corresponding K_i, P_i pair.
5. Redistribute (if merging is not possible):
– Balance keys between underfull node and sibling.
– Update parent keys to reflect changes.

[Link] 6.2 Cost of Updates

• Measured in I/O Operations:
– Proportional to the height of the tree (O(logn/2 K) for K entries).
• Worst-Case Complexity:
– Insertion/Deletion: O(logn/2 K) (logarithmic due to balanced height).

3.3.7 7. B+ Tree File Organization

[Link] 7.1 Efficiency in Database Systems
• Indexing: B+ trees are widely used for database indexing due to:
– Balanced structure → consistent performance.
– Sequential access → efficient range queries.
– Self-maintenance → minimal manual reorganization.
• Example Use Case:
– E-commerce Databases: B+ tree index on product_id enables fast search/insert/delete operations
without performance degradation.

[Link] 7.2 Trade-offs

Advantage Disadvantage
Fast search/insert/delete. Higher space overhead.
Supports range queries. Complex split/merge operations.
Self-balancing. No early key termination.

3.3.8 8. Summary of Key Concepts

[Link] 8.1 Properties to Remember
1. All leaves at the same level → balanced access.
2. Keys only in leaves (unlike B trees).
3. Linked leaves → efficient sequential access.
4. Logarithmic time complexity for operations.

[Link] 8.2 Critical Operations

Operation Process Impact

Insertion Find leaf → insert → split if full. May increase tree height.
Deletion Remove key → merge/borrow if underfull. May decrease tree height.
Update Rebalance after changes. Ensures tree properties are maintained.

85
[Link] 8.3 When to Use B+ Trees
• Best for:
– Database indexing (e.g., primary/secondary keys).
– Systems requiring range queries (e.g., “find all products with price between X and Y”).
– Applications needing sequential access (e.g., sorted output).
• Avoid when:
– Memory constraints are critical (due to pointer overhead).
– Early key discovery is a priority (use B trees instead).

3.4 B-Tree Indexing

3.4.1 1. Introduction to B-Tree Indexing
[Link] 1.1 Overview
• Definition: B-Trees are a self-balancing tree data structure optimized for systems that read and write large
blocks of data (e.g., databases, file systems).
• Purpose: Designed to maintain stored data efficiently while allowing fast insertion, deletion, and search
operations.
• Key Feature: Specialized form of an m-way tree, where each node can have multiple keys and child
pointers (unlike binary trees, which have at most two children).

[Link] 1.2 Learning Objectives By the end of this lecture, students should be able to: 1. Understand the
fundamental concept of B-Tree indexing. 2. Identify the advantages and disadvantages of B-Trees. 3. Perform
search, insertion, and deletion operations using B-Tree indices.

3.4.2 2. Properties of B-Trees

[Link] 2.1 Definition of Order (m)
• The order (m) of a B-Tree determines:
– Maximum number of children (pointers) per node: m.
– Maximum number of keys per node: m - 1.
• Example: A B-Tree of order 4 can have:
– Maximum 4 children per node.
– Maximum 3 keys per node.

[Link] 2.2 Structural Properties

1. Maximum Children per Node:
• Every node (except the root) can have at most m children.
• Ensures the tree remains balanced and operations remain efficient.
2. Minimum Children per Node (Non-Root, Non-Leaf):
• Every non-root, non-leaf node must have at least ceil(m/2) children.
• Ensures the tree does not degenerate into a linked list.
3. Key Uniqueness:
• Search keys appear only once in the tree (unlike B+ Trees, where keys may be duplicated in leaves).
• Non-leaf nodes contain only routing keys (not actual data records).
4. Leaf Node Structure:
• Leaves contain actual data entries (or pointers to data).
• All leaves are at the same level (balanced tree).

86
[Link] 2.3 Node Composition A non-leaf node in a B-Tree contains: - Keys (K1, K2, …, Km-1): Used for
routing searches. - Tree Pointers (B1, B2, …, Bm): Point to child nodes. - Data Pointers (P1, P2, …, Pm): Point
to actual data records (if stored in leaves).
General Structure:
[B1 | P1 | K1 | B2 | P2 | K2 | ... | Bm-1 | Pm-1 | Km-1 | Bm]

• Bi: Pointer to a child subtree.

• Pi: Pointer to a data record (if applicable).
• Ki: Key value for comparison.

[Link] 2.4 Visualization of a B-Tree Node

• Keys (K): Used for comparison during searches.
• Values (V): Actual data (stored in leaves).
• Pointer Types:
– Data Pointer (P): Directs to the data storage.
– Tree Pointer (B): Points to other nodes in the B-Tree.
– Leaf Pointer: Points to leaf nodes where data is stored.
Why Useful? - Clear distinction between data, tree, and leaf pointers enables efficient navigation.

3.4.3 3. Advantages and Disadvantages of B-Trees

[Link] 3.1 Advantages
1. Fewer Tree Nodes Compared to B+ Trees:
• Can sometimes locate a search key before reaching a leaf node, making searches potentially faster.
2. Efficient for Certain Workloads:
• Useful when non-leaf nodes can terminate searches early.

[Link] 3.2 Disadvantages

1. Limited Early Termination:
• Only a small fraction of search keys are found before reaching leaves, meaning most searches still
traverse multiple levels.
2. Larger Non-Leaf Nodes:
• Reduces the fan-out (number of children per node), increasing the tree depth compared to B+ Trees.
3. Complex Insertion/Deletion:
• More difficult to implement than B+ Trees due to splitting and merging rules.
4. Less Commonly Used Than B+ Trees:
• The advantages often do not outweigh the disadvantages, making B+ Trees more popular in practice.

3.4.4 4. Example: B-Tree of Order 4

[Link] 4.1 Structure
• Order (m) = 4:
– Max children per node = 4.
– Max keys per node = 3.
• Example Tree:
– Root Node: [60]
– Intermediate Nodes: [29, 32], [90, 98]

87
 – Leaf Nodes: [10, 23, 30, 31], [45, 58, 70], [85, 93, 96], [101, 110]

[Link] 4.2 Key Observations

• Balanced Structure: All leaf nodes are at the same level.
• Efficient Operations: Supports fast insertion, deletion, and search.
• Controlled Height: The order (m) determines the maximum height, impacting performance.

3.4.5 5. Search Operation in B-Trees

[Link] 5.1 Algorithm
1. Start at the root node.
2. Compare the search key with the keys in the current node.
3. If the key is found, terminate search.
4. If not found, traverse to the appropriate child node based on comparison.
5. Repeat until the key is found or a leaf node is reached.

[Link] 5.2 Example: Searching for Key = 49

1. Compare with root (78):
• 49 < 78 → Move to left subtree.
2. Compare with node (40, 56):
• 40 < 49 < 56 → Move to right subtree of 40.
3. Compare with node (45):
• 45 < 49 → Move to right subtree of 45.
4. Find 49 → Search terminates.

[Link] 5.3 Time Complexity

• O(log n): Depends on the height of the tree.
• Efficient for large datasets due to logarithmic search time.

3.4.6 6. Insertion Operation in B-Trees

[Link] **6.1 Algorithm
1. Traverse the tree to find the appropriate leaf node for insertion.
2. Check leaf node capacity:
• If the node has < m-1 keys, insert the key in sorted order.
• If the node is full (m-1 keys), split the node:
a. Insert the new key in sorted order.
b. Split the node at the median (middle key).
c. Push the median key up to the parent node.
d. If the parent is full, repeat the split.

[Link] 6.2 Example: Inserting Key = 8 in a B-Tree of Order 5

1. Initial State: Leaf node contains [5, 10, 15, 20] (4 keys, max allowed = 4).
2. Insert 8:
• New node: [5, 8, 10, 15, 20] (5 keys → overflow).
3. Split the Node:
• Median key = 10.

88
 • Left node: [5, 8].
• Right node: [15, 20].
• Push 10 up to the parent.
4. Final Structure:
• Parent now includes 10.
• Tree remains balanced.

[Link] 6.3 Key Points

• Insertions always occur at leaf nodes.
• Splitting ensures the tree remains balanced.
• Recursive splitting may propagate up to the root.

3.4.7 7. Deletion Operation in B-Trees

[Link] 7.1 Algorithm (Leaf Node Deletion)
1. Locate the leaf node containing the key.
2. Check the number of keys in the leaf:
• If > ceil(m/2) - 1, simply delete the key.
• If = ceil(m/2) - 1, borrow or merge:
a. Borrow from left sibling (if it has extra keys).
b. Borrow from right sibling (if it has extra keys).
c. Merge with a sibling (if borrowing is not possible).

[Link] 7.2 Algorithm (Internal Node Deletion)

1. If the key is in an internal node:
• Replace it with its in-order predecessor (largest key in left subtree) or successor (smallest key in right
subtree).
• Delete the predecessor/successor from the leaf.
2. Recursively rebalance the parent if necessary.

[Link] 7.3 Example: Deleting Key = 53 in a B-Tree of Order 5

1. Locate 53 in a leaf node.
2. Delete 53:
• Node now has insufficient keys (e.g., only [57] remains).
3. Rebalance:
• Borrow from sibling or merge with sibling.
• If merging, update parent keys.
4. Final Structure: Tree remains balanced.

[Link] 7.4 Key Steps Summary

1. Locate the key in the leaf.
2. Delete directly if the node has enough keys.
3. Borrow from a sibling if possible.
4. Merge nodes if borrowing is not possible.
5. Update parent and rebalance recursively.

89
3.4.8 8. Comparison with B+ Trees

Feature B-Tree B+ Tree

Key Storage Keys in all nodes Keys only in leaves (duplicated)
Search Termination Can end at non-leaf nodes Always reaches leaves
Insertion/Deletion More complex Simpler
Fan-Out Lower (due to larger nodes) Higher
Usage Less common Preferred in databases

3.4.9 9. Conclusion
[Link] 9.1 Key Takeaways
• B-Trees are self-balancing m-way trees optimized for disk-based storage.
• Search, insertion, and deletion operations are efficient (O(log n)).
• Advantages: Fewer nodes than B+ Trees, potential early search termination.
• Disadvantages: Complex operations, larger non-leaf nodes, less commonly used than B+ Trees.
• Applications: Useful in file systems and databases where balanced tree structures are required.

[Link] 9.2 Final Notes

• B+ Trees are generally preferred in databases due to simpler operations and better performance for range
queries.
• Understanding B-Tree mechanics is foundational for studying advanced indexing techniques.

3.5 Filtering Features

3.5.1 1. Introduction to Filtering in SQL
• This lecture focuses on intermediate SQL filtering techniques for building database applications.
• Key objectives by the end of the session:
– Use the GROUP BY clause to group rows with identical values in a column.
– Apply the ORDER BY clause to sort query results in ascending or descending order.
– Utilize the DISTINCT keyword to retrieve only unique values.
– Implement the LIMIT clause to restrict the number of rows returned.
– Employ the LIKE operator for pattern-based searches in data.

3.5.2 2. ORDER BY Clause

[Link] 2.1 Definition and Purpose
• The ORDER BY clause sorts the result set of a query by one or more columns.
• Default sorting order: ascending (ASC).
• Explicit descending order can be specified using DESC.

SELECT column1, column2, ...

FROM table_name
ORDER BY column1 [ASC|DESC], column2 [ASC|DESC], ...;

[Link] 2.2 Syntax

90
 • ASC: Ascending order (default if not specified).
• DESC: Descending order.
• Multiple columns can be specified, each with its own sort order.

[Link] 2.3 Example Query:

SELECT name
FROM student_detail
ORDER BY roll_number ASC;

• Purpose: Retrieves student names sorted by their roll numbers in ascending order.
• Benefits:
– Improves readability of query results.
– Essential for organizing large datasets.

[Link] 2.4 Key Notes

• If no sort order is specified, ORDER BY defaults to ASC.
• Useful for presenting data in a structured manner (e.g., reports, dashboards).

3.5.3 3. GROUP BY Clause

[Link] 3.1 Definition and Purpose
• The GROUP BY clause groups rows with identical values in specified columns into summary rows.
• Primarily used with aggregate functions (e.g., COUNT, MAX, MIN, SUM, AVG).
• Position in SQL query:
– Comes after WHERE (if present).
– Comes before ORDER BY.

SELECT column1, aggregate_function(column2)

FROM table_name
WHERE condition
GROUP BY column1
ORDER BY column1;

[Link] 3.2 Syntax

• aggregate_function: Functions like SUM, COUNT, etc., applied to grouped data.

[Link] 3.3 Example Scenario: Calculate the total marks obtained by each student from a student_details
table.
Query:
SELECT name, SUM(marks)
FROM student_details
GROUP BY name;

• Explanation:
– SELECT name, SUM(marks): Retrieves student names and sums their marks.

91
 – GROUP BY name: Groups rows by the name column, ensuring each student appears once with their total
marks.
• Output: A list of students with their aggregated total marks.

[Link] 3.4 Key Notes

• Essential for data aggregation and generating summary reports.
• Often used with HAVING (a filter for grouped data, similar to WHERE but for aggregates).

3.5.4 4. DISTINCT Keyword

[Link] 4.1 Definition and Purpose
• The DISTINCT keyword eliminates duplicate rows from the result set.
• Ensures only unique values are returned for specified columns.
• Critical for avoiding redundancy in queries (e.g., retrieving unique IDs, categories).

SELECT DISTINCT column1, column2, ...

FROM table_name
[ORDER BY column1];

[Link] **4.2 Syntax

[Link] 4.3 Example Scenario: Retrieve unique roll numbers from a student table, sorted by age.
Query:
SELECT DISTINCT roll_number
FROM student
ORDER BY age;

• Output: A list of unique roll numbers, sorted by the students’ ages.

• Use Case: If multiple rows share the same roll_number, DISTINCT ensures it appears only once.

[Link] 4.4 Key Notes

• Performance Impact: Can improve efficiency by reducing the result set size.
• Often used with COUNT to count unique entries (e.g., SELECT COUNT(DISTINCT column) FROM table).

3.5.5 5. LIMIT Clause

[Link] 5.1 Definition and Purpose
• The LIMIT clause restricts the number of rows returned by a query.
• Useful for:
– Pagination (e.g., displaying 10 records per page).
– Performance optimization (avoiding large result sets).
– Sampling data (e.g., retrieving the first 5 rows for testing).

SELECT column1, column2, ...

FROM table_name

92
[WHERE condition]
[ORDER BY column1]
LIMIT [offset,] row_count;

[Link] 5.2 Syntax

• row_count: Maximum number of rows to return.
• offset (optional): Number of rows to skip before returning results (default: 0).

[Link] 5.3 Example Scenario: Retrieve the first 3 records from a student table.
Query:
SELECT *
FROM student
LIMIT 3;

• Output: Only the first 3 rows of the table.

• Use Case: If the table has 100 rows, LIMIT 3 returns rows 1–3.

[Link] 5.4 Restrictions on LIMIT

1. Views:
• Cannot use LIMIT when defining a view (views represent complete result sets).
2. Nested SELECT Statements:
• Generally not allowed, except in subqueries used as table expressions in the FROM clause.
3. Stored Procedures/Embedded SQL:
• Not permitted in embedded SELECT statements where the result must be a single row.

[Link] 5.5 Key Notes

• Offset Usage: LIMIT 5, 10 skips the first 5 rows and returns the next 10.
• Database Variations:
– MySQL/PostgreSQL: LIMIT.
– SQL Server: TOP or FETCH NEXT.
– Oracle: ROWNUM.

3.5.6 6. LIKE Operator

[Link] 6.1 Definition and Purpose
• The LIKE operator performs pattern matching in a WHERE clause.
• Used to search for specific patterns in string columns (e.g., names, addresses).
• Case Insensitivity: Most SQL databases treat LIKE as case-insensitive (e.g., 'apple' matches 'APPLE').

SELECT column1, column2, ...

FROM table_name
WHERE column LIKE pattern;

[Link] 6.2 Syntax

• Wildcards:

93
 – %: Matches any sequence of characters (including none).
– _: Matches a single character.

[Link] 6.3 Example Scenario: Retrieve suppliers whose names start with “CA” from a supplier table.
Query:
SELECT supplier_id, name, address
FROM supplier
WHERE name LIKE 'CA%';

• Explanation:
– CA%: Matches names starting with “CA” (e.g., “California Supplies”, “Cafe Goods”).
– Output: Rows where the name column begins with “CA”.

[Link] **6.4 Common Patterns

Pattern Description Example Matches

LIKE 'A%' Starts with “A” “Apple”, “Ant”
LIKE '%A' Ends with “A” “Banana”, “America”
LIKE '%OR%' Contains “OR” “Color”, “Storage”
LIKE '_A%' Second letter is “A” “Banana”, “Cat”
LIKE 'A___' Starts with “A” and is 4 letters long “Apple”, “Aero”

[Link] 6.5 Key Notes

• Performance: LIKE with leading wildcards (e.g., %term) can be slow (avoids index usage).
• Case Sensitivity: Some databases (e.g., PostgreSQL) allow case-sensitive matching with ILIKE or LIKE
BINARY.
• Escape Character: Use ESCAPE to search for literal % or _ (e.g., LIKE '100\%' ESCAPE '\').

3.5.7 7. Summary of Topics Covered

Feature Purpose Key Syntax Example

ORDER Sort query results by column(s). ORDER BY column DESC
BY
GROUP Group rows by column values for aggregation. GROUP BY column
BY
DISTINCT Return only unique values. SELECT DISTINCT column
LIMIT Restrict the number of rows returned. LIMIT 10 or LIMIT 5, 10
LIKE Search for patterns in string columns. WHERE column LIKE 'A%'

3.5.8 8. Practical Applications

• ORDER BY: Sorting customer orders by date or price.
• GROUP BY: Calculating total sales per region.
• DISTINCT: Listing unique product categories.
• LIMIT: Implementing pagination in web applications.
• LIKE: Searching for users by partial name matches.

94
3.5.9 9. Conclusion
• These filtering features are fundamental for efficient data retrieval and analysis.
• Mastery of these clauses enables complex queries, performance optimization, and data-driven decision-
making.
• Upcoming lectures will cover aggregate functions (e.g., COUNT, SUM) in greater detail.

3.6 Functions
3.6.1 1. Introduction to Functions in PL/SQL
• Functions are a fundamental component of PL/SQL, enabling reusable code.
• A function is a subprogram designed to:
– Perform a specific task.
– Return a value (unlike procedures, which do not necessarily return a value).
• Key distinction from procedures:
– Functions must include a RETURN clause specifying the data type of the returned value.
– Procedures do not require a return value.

3.6.2 2. Types of Functions in PL/SQL

Three primary types of functions are covered: 1. Local Functions 2. Stored Functions 3. Recursive Functions

3.6.3 3. Local Functions

[Link] 3.1 Definition
• Declared within the declarative section of a PL/SQL block.
• Scope: Can only be called from within the same block (execution section).
• Lifetime: Exists only for the duration of the block’s execution.

FUNCTION function_name (
parameter1 IN datatype,
parameter2 IN datatype,
...
) RETURN return_datatype IS
-- Optional: Local variable declarations
BEGIN
-- Function body (logic)
RETURN value;
EXCEPTION
-- Optional: Exception handlers
END function_name;

[Link] 3.2 Syntax

• Components:
– function_name: Identifier for the function.
– parameter1, parameter2, ...: Input parameters with data types.
– RETURN return_datatype: Specifies the data type of the returned value.
– Local declarations (optional): Variables used within the function.
– Function body: Contains the logic to compute the return value.

95
 – Exception handlers (optional): For error management.

[Link] 3.3 Example: Local Addition Function Code:

DECLARE
-- Local function declaration
FUNCTION add (
a IN NUMBER,
b IN NUMBER
) RETURN NUMBER IS
sum NUMBER;
BEGIN
sum := a + b;
RETURN sum;
EXCEPTION
WHEN OTHERS THEN
DBMS_OUTPUT.PUT_LINE('Error in addition: ' || SQLERRM);
RETURN NULL;
END add;

-- Variables to test the function

x NUMBER := 5;
y NUMBER := 10;
result NUMBER;
BEGIN
-- Calling the local function
result := add(x, y);

-- Displaying the result

DBMS_OUTPUT.PUT_LINE('Sum: ' || result);
END;

• Explanation:
1. The function add takes two NUMBER inputs (a and b).
2. Computes the sum and returns it.
3. An exception handler is included for robustness.
4. The function is called within the same block, and the result is printed using DBMS_OUTPUT.PUT_LINE.

3.6.4 4. Stored Functions

[Link] 4.1 Definition
• Permanently stored in the database (Oracle in this context).
• Reusable across multiple PL/SQL blocks and applications.
• Advantages:
– Centralized logic maintenance.
– Improved performance (compiled and stored in the database).

CREATE OR REPLACE FUNCTION function_name (

parameter1 IN datatype,

96
 parameter2 IN datatype,
...
) RETURN return_datatype IS
-- Optional: Local variable declarations
BEGIN
-- Function body (logic)
RETURN value;
EXCEPTION
-- Optional: Exception handlers
END function_name;

[Link] 4.2 Syntax

• Components:
– CREATE OR REPLACE: Creates a new function or replaces an existing one.
– function_name: Name of the stored function.
– RETURN return_datatype: Specifies the return type.
– Function body: Contains the logic.
– Exception handlers (optional): For error handling.

[Link] 4.3 Example: Stored Addition Function Code:

-- Creating the stored function
CREATE OR REPLACE FUNCTION add (
a IN NUMBER,
b IN NUMBER
) RETURN NUMBER IS
BEGIN
RETURN a + b;
END add;

• Explanation:
– The function add is stored in the Oracle database.
– It performs the same task as the local function but is globally accessible.
Calling the Stored Function:
DECLARE
a NUMBER := 5;
b NUMBER := 10;
c NUMBER;
BEGIN
-- Calling the stored function
c := add(a, b);

-- Displaying the result

DBMS_OUTPUT.PUT_LINE('Sum: ' || c);
END;

• Key Points:
– The stored function is called without redefining it in the block.
– The logic is centralized in the database.

97
3.6.5 5. Recursive Functions
[Link] 5.1 Definition
• A function that calls itself within its definition.
• Used for problems that can be broken down into smaller, similar subproblems (e.g., factorial, Fibonacci
sequence).
• Requirements:
– Base case: Terminates recursion to prevent infinite loops.
– Recursive case: Calls the function with a modified input.

FUNCTION function_name (
parameter IN datatype
) RETURN return_datatype IS
BEGIN
IF base_case_condition THEN
RETURN base_case_value;
ELSE
RETURN function_name(modified_parameter);
END IF;
END function_name;

[Link] 5.2 Syntax

[Link] 5.3 Example: Recursive Factorial Function Code:

-- Recursive factorial function
CREATE OR REPLACE FUNCTION fact (
x IN NUMBER
) RETURN NUMBER IS
BEGIN
-- Base case: 0! = 1
IF x = 0 THEN
RETURN 1;
ELSE
-- Recursive case: x! = x * (x-1)!
RETURN x * fact(x - 1);
END IF;
END fact;

• Explanation:
– Base case: When x = 0, return 1 (since 0! = 1).
– Recursive case: For x > 0, return x * fact(x - 1).
– The function calls itself with x - 1 until the base case is reached.
Calling the Recursive Function:
DECLARE
num NUMBER := 6;
factorial_result NUMBER;
BEGIN
-- Calling the recursive function

98
 factorial_result := fact(num);

-- Displaying the result

DBMS_OUTPUT.PUT_LINE('Factorial of ' || num || ' is: ' || factorial_result);
END;

• Output:
– For num = 6, the output will be 720 (since 6! = 720).

[Link] 5.4 Importance of the Base Case

• Purpose: Prevents infinite recursion.
• Example: Without the base case (IF x = 0), the function would recursively call itself indefinitely, leading
to a stack overflow error.

3.6.6 6. Summary of Key Concepts

Type Scope Storage Reusability Example Use Case

Local Function Within a PL/SQL Temporary Block-specific Intermediate
block calculations
Stored Database-wide Permanent Global reuse Shared business logic
Function
Recursive Self-referential Temporary/Stored Problem Factorial, Fibonacci
Function decomposition

3.6.7 7. Best Practices

1. Local Functions:
• Use for short-lived, block-specific logic.
• Include exception handling for robustness.
2. Stored Functions:
• Ideal for reusable logic across applications.
• Test thoroughly before deployment.
3. Recursive Functions:
• Always define a base case to avoid infinite loops.
• Use for problems with natural recursive structure (e.g., tree traversals).

3.7 Hashing
3.7.1 1. Introduction to Hashing in Database Management Systems
• Purpose: Hashing is a fundamental technique in database management systems (DBMS) for efficient data
retrieval.
• Key Objectives:
– Understand static hashing and its application in databases.
– Learn methods for handling bucket overflow.
– Examine practical examples of hash file organization.
– Identify deficiencies in static hashing and explore dynamic hashing as a solution.
– Compare hashing with ordered indexing and determine preferred use cases.

99
3.7.2 2. Static Hashing
[Link] 2.1 Definition and Core Concepts
• Static Hashing: A technique where a hash function maps search-key values to a fixed set of buckets.
– Bucket: A storage unit (typically a disk block) that holds one or more records.
– Hash Function (h): A function that maps a search-key value K to a bucket address B.
* Formula: h(K) → B
– Collision: Occurs when multiple records with different search keys hash to the same bucket, requiring
sequential search within the bucket.

[Link] 2.2 Types of Hash-Based Storage

1. Hash Indices:
• Buckets store entries with pointers to the actual records.
2. Hash File Organization:
• Buckets directly store the records themselves.

3.7.3 3. Bucket Overflow and Solutions

[Link] 3.1 Causes of Bucket Overflow
• Insufficient Buckets: The initial number of buckets is too small for the dataset.
• Skewed Distribution:
– Multiple records share the same search-key value (e.g., duplicate keys).
– The hash function fails to distribute keys evenly (poor hash function design).

[Link] 3.2 Overflow Handling Techniques

[Link].1 3.2.1 Overflow Chaining (Closed Addressing / Closed Hashing / Open Hashing)
• Mechanism:
– When a bucket overflows, additional overflow buckets are allocated.
– Overflow buckets are linked in a chain to the original bucket.
– Ensures all records remain accessible even if they exceed the initial bucket capacity.
• Visualization:
– Original bucket → Overflow bucket 1 → Overflow bucket 2 → …
– Each bucket in the chain holds records that couldn’t fit in the previous bucket.

[Link].2 3.2.2 Open Addressing

• Mechanism:
– No additional buckets are used.
– Overflow entries are placed in other slots within the same table using probing (e.g., linear probing,
quadratic probing).
– Probing: Searching for the next available slot in the table to resolve collisions.

3.7.4 4. Practical Example: Hash File Organization

[Link] 4.1 Scenario Setup
• Dataset: Instructor file with department name as the hash key.
• Hash File Structure:

100
 – 10 buckets (numbered 0–9).
– Each bucket stores records based on the hash of the department name.

[Link] 4.2 Hash Function Design

• Method:
– Convert the department name to its binary representation.
– Sum the binary values of characters.
– Apply modulo 10 to ensure the result fits within the 10 buckets.
– Formula: h(department_name) = (sum of binary values) % 10
• Examples:
– h("Music") → Bucket 1
– h("History") → Bucket 2
– h("Physics") → Bucket 3
– h("Electrical Engineering") → Bucket 3 (collision with Physics)

[Link] 4.3 Hash File Distribution

Bucket Department Instructor Salary

0 Empty - -
1 Music Mozart 40,000
2 History El Said 80,000
History Califieri 60,000
3 Physics Einstein 95,000
Physics Gold 87,000
Electrical Engineering Kim 80,000
4 Finance Wu 90,000
Finance Singh 80,000
5 Biology Crick 72,000
6 Computer Science Srinivasan 65,000
Computer Science Katz 75,000
Computer Science Brandt 92,000
7 Empty - -

[Link] 4.4 Key Observations

• Grouping by Key: Records from the same department (e.g., History, Physics) are stored in the same bucket.
• Empty Buckets: Buckets 0 and 7 are unused, demonstrating uneven distribution.
• Efficiency: Hashing enables quick access to records based on department name.

3.7.5 5. Deficiencies of Static Hashing

Issue Description Impact

Fixed Bucket Set Number of buckets is static; cannot adapt to data Poor scalability.
growth/shrinkage.
Performance Small initial bucket count leads to frequent overflows. Slower retrieval due to
Degradation chaining/probing.
Space Waste Allocating extra buckets for anticipated growth leads to Inefficient storage usage.
underutilized space.

101
Issue Description Impact
Periodic Requires rehashing with a new hash function to Disruptive, costly, and
Reorganization redistribute data. time-consuming.

3.7.6 6. Dynamic Hashing: Solutions to Static Hashing Limitations

[Link] 6.1 Overview
• Goal: Dynamically adjust the number of buckets based on database size.
• Techniques:
1. Periodic Rehashing
2. Linear Hashing
3. Extensible Hashing

[Link] 6.2 Dynamic Hashing Techniques

[Link].1 6.2.1 Periodic Rehashing

• Mechanism:
– When the hash table grows beyond a threshold (e.g., 15x its size), a new larger table is created.
– All entries are rehashed into the new table.
• Drawback: High overhead during reorganization.

[Link].2 6.2.2 Linear Hashing

• Mechanism:
– Rehashing is done incrementally (one bucket at a time) rather than all at once.
– Distributes the load over time, reducing disruption.
• Advantage: Smoother performance during growth.

[Link].3 6.2.3 Extensible Hashing

• Mechanism:
– Designed for disk-based hashing.
– Allows multiple hash values to share buckets.
– Supports doubling the number of entries without doubling buckets.
• Advantage: Efficient for large, disk-based databases.

3.7.7 7. Comparison: Hashing vs. Ordered Indexing

[Link] 7.1 Key Considerations

Factor Hashing Ordered Index (e.g., B-Tree)

Insertion/Deletion High frequency may require Handles dynamic operations more
Frequency reorganization. efficiently.
Access Time Trade-off Optimized for average access time. Balances average and worst-case
access.
Query Type Exact-match queries (e.g., WHERE key = Range queries (e.g., WHERE key
X). BETWEEN X AND Y).
Performance in Practice Fast for key-based lookups. Versatile for mixed query types.

102
[Link] 7.2 Real-World Database Support

Database
System Hashing Support Preferred Index Type Notes
PostgreSQL Supports hash indices but B-Tree B-Tree offers better performance
discourages use. in most cases.
Oracle Supports static hash B-Tree (default) Hashing used in specific
organization. scenarios.
SQL Server No hash indices. B+-Tree B+-Tree is versatile for all query
types.

3.7.8 8. Summary of Key Concepts

• Static Hashing:
– Uses a fixed number of buckets.
– Suffers from overflow, performance degradation, and space inefficiency.
• Dynamic Hashing:
– Adjusts bucket count dynamically (e.g., linear hashing, extensible hashing).
– Mitigates static hashing limitations.
• Hashing vs. Ordered Indexing:
– Hashing: Best for exact-match queries.
– Ordered Indexing (B-Tree): Better for range queries and dynamic operations.
• Practical Implications:
– Most modern databases (e.g., PostgreSQL, SQL Server) favor B-Tree over hashing for general use.
– Oracle supports static hashing for niche cases.

3.8 Ordered Indices

3.8.1 Introduction to Indexing
• Definition: An index is a data structure that improves the speed of data retrieval operations in a database.
• Analogy: Similar to an author’s catalog in a library, which helps quickly locate books.
• Search Key: An attribute or set of attributes used to look up records in a database.
• Index File: Consists of index entries, which are smaller than the original data records.

[Link] Types of Indices

1. Ordered Indices
• Search keys are stored in sorted order, improving efficiency for range queries and ordered retrieval.
2. Hash Indices
• Search keys are distributed across buckets using a hash function, optimizing lookups for unique iden-
tifiers.

[Link] Purpose of Indexing

• Primary Benefit: Accelerates data retrieval operations.
• Applications:
– Libraries (quick book location).
– Databases (efficient record access).
– Any system requiring fast data lookup.

103
3.8.2 Ordered Indices: Core Concepts
• Definition: Index entries are stored in sorted order based on the search key value.
• Advantages:
– Enables efficient range queries (e.g., “find all records with values between X and Y”).
– Supports ordered data retrieval, which is critical for reporting and analytics.

[Link] Types of Ordered Indices

1. Clustering Index (Primary Index)
• Defines the sequential order of the file (typically on the primary key).
• The data file itself is physically ordered by the clustering index’s search key.
• Example: A file sorted by student_id with a clustering index on student_id.
2. Secondary Index (Non-Clustering Index)
• Specifies an alternative order different from the file’s sequential order.
• Does not dictate physical storage order but provides an additional access path.
• Example: An index on department_name in an instructor table sorted by instructor_id.
3. Index-Sequential File
• A sequential file ordered on a search key with a clustering index on that key.
• Combines sequential storage with indexed access for efficiency.

3.8.3 Performance Metrics for Indices

Evaluating indices involves analyzing the following metrics to optimize database performance:

Metric Description
Access Time Speed of data retrieval (e.g., time to locate a record).
Insertion Time Speed of adding new records (affected by index maintenance overhead).
Deletion Time Speed of removing records (requires index updates).
Space Overhead Additional storage required for the index (dense vs. sparse trade-offs).
Access Types Supported operations (e.g., exact-match, range queries, sorted retrieval).

[Link] Optimization Considerations

• Query Patterns: Choose indices based on frequent query types (e.g., range queries favor ordered indices).
• Trade-offs:
– Dense indices offer faster lookups but higher storage overhead.
– Sparse indices save space but may require more I/O for some queries.
• Maintenance Cost: Indices slow down INSERT, UPDATE, and DELETE operations due to required updates.

3.8.4 Dense Indices

[Link] Definition
• A dense index contains an index entry for every search key value in the data file.
• Each entry points directly to the corresponding data record.

[Link] Structure

104
Component Description
Search Key The attribute used for indexing (e.g., ID, department_name).
Index Entities Entries in the index file, each mapping a search key to a data record.
Attributes Columns in the data file (e.g., name, department, salary).
Index Record Pointer from the index entry to the physical location of the data record.

[Link] Example: Dense Index on ID (Instructor Table)

ID (Search Key) Pointer to Record Data Record

10101 → Srinivasan, Comp. Sci., $65,000
12121 → Crick, Biology, $72,000
… … …

[Link] Advantages
• Fast Data Retrieval: Direct access to records without full table scans.
• Efficient Lookups: Ideal for exact-match queries (e.g., WHERE ID = 10101).
• Supports Range Queries: Can quickly locate all records within a key range (e.g., ID BETWEEN 10000 AND
20000).

[Link] Use Cases

• Primary Key Indexing: Ensures unique, fast access to records.
• Frequent Exact-Match Queries: When records are often accessed by a specific key.

[Link] Example: Dense Index on department_name

Department (Search Key) Pointer to Record Instructors

Biology → Crick ($72,000)
Comp. Sci. → Srinivasan, Katz, Brandt
Electrical Eng. → …

[Link].1 Benefits of Department-Wise Dense Index

• Efficient Department Access: Quickly retrieve all instructors in a department.
• Organized Data Retrieval: Supports queries like:
SELECT * FROM instructor WHERE department = 'Comp. Sci.';

3.8.5 Sparse Indices

[Link] Definition
• A sparse index contains index entries for only some search key values (typically the first key in each block).
• Applicable when records are sequentially ordered on the search key.

105
[Link] Advantages

Benefit Description
Storage Efficiency Requires less space than dense indices (fewer entries).
Lower Maintenance Fewer updates during INSERT/DELETE operations.
Optimized for Sequences Ideal for large, sequentially ordered files (e.g., log files, time-series data).

[Link] Use Cases

• Large Sequential Files: When records are stored in sorted order (e.g., by timestamp).
• Space Optimization: When storage savings are prioritized over lookup speed.

[Link] Comparison: Dense vs. Sparse Indices

Feature Dense Index Sparse Index

Entries One per search key value. One per block (or subset of keys).
Storage Higher overhead. Lower overhead.
Lookup Speed Faster (direct access). Slower (may require block scans).
Maintenance Higher (every record affects index). Lower (only block-level changes).
Best For Exact-match and range queries. Sequential access and space savings.

3.8.6 Index Updates: Insertion and Deletion

Indices must be updated during data modifications to maintain accuracy. The process varies for dense and sparse
indices.

[Link] Deletion Operations

[Link].1 General Rules

1. Unique Record Deletion:
• If the deleted record is the only one with its search key, the key is removed from the index.
2. Index Integrity:
• The index must reflect the current state of the data to avoid “dangling pointers.”

[Link].2 Dense Index Deletion

• Process:
1. Locate the index entry for the deleted record.
2. Remove the entry from the index.
• Example:
– Deleting ID = 10101 removes its entry from the dense index.

[Link].3 Sparse Index Deletion

• Process:
1. If the deleted record’s key has an index entry:
– Replace the entry with the next search key in the file (to maintain sparsity).
– If the next key already has an entry, simply remove the deleted key’s entry.

106
 2. If the deleted record’s key has no index entry, no action is needed.
• Example:
– If a sparse index has entries for blocks starting with keys 10000 and 20000, deleting 10000 would
replace it with the next key in its block (e.g., 10001).

[Link] Insertion Operations

[Link].1 General Rules

1. Index Lookup:
• Determine where the new record’s key fits in the index.
2. Maintenance:
• Update the index to include the new key (if applicable).

[Link].2 Dense Index Insertion

• Process:
1. If the search key does not exist in the index, add a new entry.
2. Space Management:
– If the index is stored as a sequential file, new entries may require overflow blocks.
• Example:
– Inserting a new instructor with ID = 30000 adds 30000 to the dense index.

[Link].3 Sparse Index Insertion

• Process:
1. If the index stores one entry per block:
– No update is needed unless a new block is created.
2. If a new block is created:
– Insert the first search key of the new block into the index.
• Example:
– If a new block starts with ID = 30000, add 30000 to the sparse index.

[Link] Multilevel Index Updates

• Definition: Indices with multiple levels (e.g., a B-tree) require recursive updates.
• Process:
– Apply the single-level algorithm at each level.
– Example: Inserting into a B-tree may require splitting nodes and propagating changes upward.
• Benefits:
– Maintains Integrity: Ensures indices reflect the current data state.
– Efficient Queries: Keeps query performance optimal after modifications.

3.8.7 Clustered vs. Non-Clustered Indices

[Link] Clustered Index
• Definition: Dictates the physical order of data in the file.
• Characteristics:
– Typically built on the primary key.
– Only one clustered index per table (since data can’t be sorted in multiple ways simultaneously).
• Example:

107
 – A table sorted by student_id with a clustered index on student_id.

[Link] Non-Clustered Index (Secondary Index)

• Definition: Provides an alternative logical order without affecting physical storage.
• Characteristics:
– Multiple non-clustered indices can exist per table.
– Requires pointers to the physical data (either direct or via the clustered index).
• Example:
– An index on last_name in a table clustered by student_id.

[Link] Trade-offs

Feature Clustered Index Non-Clustered Index

Physical Order Determines data storage order. Does not affect storage order.
Number per Table Only one. Multiple allowed.
Performance Faster for range queries on the key. Slower (requires extra lookup step).
Maintenance Overhead High (reorders data on updates). Lower (only index structure updates).

3.8.8 Summary of Key Concepts

1. Ordered Indices:
• Store keys in sorted order for efficient range queries and sorted retrieval.
• Types: Clustering (primary) and secondary (non-clustering).
2. Dense Indices:
• One entry per search key; fast lookups but high storage overhead.
3. Sparse Indices:
• One entry per block; space-efficient but slower for some queries.
4. Index Updates:
• Deletion: Remove or replace entries to maintain integrity.
• Insertion: Add entries for new keys (dense) or blocks (sparse).
5. Multilevel Indices:
• Extend single-level logic for scalability (e.g., B-trees).
6. Performance Metrics:
• Evaluate indices based on access time, insertion/deletion speed, and space overhead.
7. Clustered vs. Non-Clustered:
• Clustered indices define physical order; non-clustered provide alternative access paths.

3.9 Recording of Building Database Applications Week 2 - Live Session on 26-03-20

3.9.1 1. Introduction to PL/SQL and Functions
[Link] 1.1 Overview of PL/SQL
• PL/SQL (Procedural Language for SQL) is a procedural extension of SQL that allows complex database
logic to be written like a programming language.
• **SQL*Plus** is a command-line tool used to execute SQL and PL/SQL commands.
• PL/SQL enables code reuse, reducing redundancy by allowing functions and procedures to be defined once
and called multiple times.

108
[Link] 1.2 Types of Functions in PL/SQL Three primary types of functions: 1. Local Functions 2. Stored
Functions 3. Recursive Functions

3.9.2 2. Local Functions

[Link] 2.1 Definition and Scope
• Local functions are declared inside a BEGIN-END block of a PL/SQL program.
• Their scope is limited to the block in which they are defined.

CREATE OR REPLACE FUNCTION add_numbers(n1 NUMBER, n2 NUMBER) RETURN NUMBER IS

BEGIN
RETURN n1 + n2;
END;
/

[Link] 2.2 Example: Adding Two Numbers

• CREATE OR REPLACE: Replaces an existing function if one with the same name exists.
• Parameters: n1 and n2 are of type NUMBER.
• Return Type: The function returns a NUMBER.

DECLARE
a NUMBER := 15;
b NUMBER := 25;
c NUMBER;
BEGIN
c := add_numbers(a, b);
DBMS_OUTPUT.PUT_LINE('Addition is: ' || c);
END;
/

[Link] 2.3 Calling a Local Function

• DECLARE: Declares variables a, b, and c.
• DBMS_OUTPUT.PUT_LINE: Prints the result (concatenates strings using ||).
• Output: Addition is: 40 (if a=15 and b=25).

3.9.3 3. Stored Functions

[Link] 3.1 Definition and Storage
• Stored functions are saved in the database and can be reused across multiple sessions.
• Unlike local functions, they persist beyond a single execution block.

CREATE OR REPLACE FUNCTION add_numbers(n1 NUMBER, n2 NUMBER) RETURN NUMBER IS

BEGIN
RETURN n1 + n2;
END;
/

109
[Link] 3.2 Example: Stored Addition Function
• Calling the Function Inside a Block:
DECLARE
a NUMBER := 20;
b NUMBER := 30;
c NUMBER;
BEGIN
c := add_numbers(a, b);
DBMS_OUTPUT.PUT_LINE('Addition is: ' || c);
END;
/

– Output: Addition is: 50.

• Calling the Function Directly (Outside a Block):
SELECT add_numbers(20, 30) FROM dual;

– DUAL: A dummy table in Oracle used when no real table is needed in a SELECT statement.
– Output: 50.

3.9.4 4. Recursive Functions

[Link] 4.1 Definition and Use Case
• A recursive function calls itself until a base condition is met.
• Commonly used for problems like factorials, Fibonacci sequences, and tree traversals.

CREATE OR REPLACE FUNCTION factorial(n NUMBER) RETURN NUMBER IS

BEGIN
IF n = 0 OR n = 1 THEN
RETURN 1; -- Base condition
ELSE
RETURN n * factorial(n - 1); -- Recursive call
END IF;
END;
/

[Link] 4.2 Example: Factorial Calculation

• Base Condition: If n = 0 or n = 1, return 1.
• Recursive Step: factorial(n) = n * factorial(n - 1).

DECLARE
result NUMBER;
BEGIN
result := factorial(5);
DBMS_OUTPUT.PUT_LINE('Factorial is: ' || result);
END;
/

110
[Link] 4.3 Calling the Recursive Function
• Output: Factorial is: 120 (since 5! = 120).

3.9.5 5. Stored Procedures

[Link] 5.1 Definition and Structure
• A stored procedure is a reusable block of PL/SQL code that performs a specific action or logic.
• Consists of two parts:
1. Header: Contains the procedure name and parameter list.
2. Body: Contains executable statements (business logic).

CREATE OR REPLACE PROCEDURE greeting IS

BEGIN
DBMS_OUTPUT.PUT_LINE('Hello, World!');
END;
/

[Link] 5.2 Example: Greeting Procedure

• Calling the Procedure:
EXECUTE greeting;

– Output: Hello, World!

[Link] 5.3 Components of a Procedure

Component Description
Header Contains the procedure name and optional parameters.
Body Contains declarative, executable, and exception-handling statements.
Declarative Part Declares variables, cursors, and exceptions.
Executable Part Contains SQL and PL/SQL statements (e.g., INSERT, UPDATE, IF-THEN-ELSE).
Exception Part Handles errors (e.g., RAISE_APPLICATION_ERROR).

3.9.6 6. Triggers
[Link] 6.1 Definition and Purpose
• A trigger is a special type of stored procedure that automatically executes in response to a database event
(e.g., INSERT, UPDATE, DELETE).
• Used for data validation, auditing, and enforcing business rules.

CREATE OR REPLACE TRIGGER check_salary

BEFORE INSERT OR UPDATE ON emp_demo_trigger
FOR EACH ROW
BEGIN
IF :NEW.emp_salary < 10000 THEN
RAISE_APPLICATION_ERROR(-20001, 'Salary must be at least 10,000');

111
 END IF;
END;
/

[Link] 6.2 Example: Salary Validation Trigger

• Trigger Type: BEFORE INSERT OR UPDATE (fires before the operation).
• Condition: Checks if the new salary (:NEW.emp_salary) is less than 10,000.
• Action: Raises an error if the condition is violated.

[Link] 6.3 Testing the Trigger

1. Create the Table:
CREATE TABLE emp_demo_trigger (
emp_id NUMBER PRIMARY KEY,
emp_name VARCHAR2(50),
emp_salary NUMBER
);

2. Insert Valid Data:

INSERT INTO emp_demo_trigger VALUES (1, 'John', 15000);

• Result: Success (salary >= 10,000).

3. Insert Invalid Data:
INSERT INTO emp_demo_trigger VALUES (2, 'Alice', 5000);

• Result: Error (Salary must be at least 10,000).

3.9.7 7. Aggregate Functions

[Link] 7.1 Definition and Common Functions
• Aggregate functions perform calculations on a set of values and return a single result.
• Common aggregate functions:
– MIN(): Finds the smallest value.
– MAX(): Finds the largest value.
– SUM(): Calculates the total.
– AVG(): Computes the average.
– COUNT(): Counts the number of rows.

[Link] 7.2 Examples

1. Minimum Salary:
SELECT MIN(salary) AS min_salary FROM employees;

• Output: Single value (lowest salary in the table).

2. Maximum Salary:
SELECT MAX(salary) AS max_salary FROM employees;

• Output: Single value (highest salary in the table).

112
 3. Sum of Salaries:
SELECT SUM(salary) AS total_salary FROM employees;

• Output: Total of all salaries.

4. Average Salary by Department:
SELECT department, AVG(salary) AS avg_salary
FROM employees
GROUP BY department;

• GROUP BY: Groups records by department before calculating the average.

5. Count of Employees:
SELECT COUNT(*) AS employee_count FROM employees;

• Output: Total number of employees.

3.9.8 8. Filtering and Pattern Matching

[Link] 8.1 DISTINCT Clause
• Returns unique values from a column.
• Example:
SELECT DISTINCT department FROM employees;

– Output: List of unique departments (e.g., IT, HR, Finance).

[Link] 8.2 ORDER BY Clause

• Sorts results in ascending (ASC) or descending (DESC) order.
• Example:
SELECT employee_name, salary
FROM employees
ORDER BY salary DESC;

– Output: Employees sorted by salary (highest to lowest).

[Link] 8.3 LIKE Operator for Pattern Matching

• Used for string pattern matching.
• Wildcards:
– %: Matches any sequence of characters.
– _: Matches a single character.

[Link].1 Examples:
1. Second Letter is ‘a’:
SELECT employee_name FROM employees
WHERE employee_name LIKE '_a%';

113
 • Output: Names where the second letter is ‘a’ (e.g., “David”).
2. Contains the Letter ‘o’:
SELECT employee_name FROM employees
WHERE employee_name LIKE '%o%';

• Output: Names containing ‘o’ (e.g., “John”, “Bob”).

3.9.9 9. Indexing
[Link] 9.1 Definition and Purpose
• Indexing is a data structure that improves the speed of data retrieval operations.
• Works by creating a key-value mapping for faster searches.
• Types of indexing:
1. Primary Indexing
2. Secondary Indexing
3. Clustering Indexing

[Link] 9.2 Types of Indexes

Type Description
Primary Index Defined on an ordered data file (typically on the primary key).
Secondary Index Defined on a candidate key (unique field not necessarily the primary key).
Clustering Index Defined on a non-key field in an ordered file.

[Link] 9.3 Dense vs. Sparse Indexes

Type Description Pros Cons

Dense Contains an entry for every search key in the Faster search. Requires more
Index database. storage.
Sparse Contains entries only for some search keys (e.g., Less storage. Slower for some
Index first entry in a block). queries.

[Link] 9.4 B-Trees and B+ Trees

[Link].1 B-Tree Properties:

• Balanced Tree: All leaf nodes are at the same level.
• Ordering: Keys in each node are stored in ascending order.
• Minimum Children:
– Non-leaf nodes (except root) must have at least ceil(m/2) children.
– Non-leaf nodes must have at least ceil(m/2) - 1 keys.
• Root Node:
– If the root is a leaf, it has at least 1 key.
– If the root is non-leaf, it has at least 2 children.

114
[Link].2 B+ Tree Properties:
• Internal Nodes: Store only keys (act as pointers).
• Leaf Nodes: Store all actual data records.
• Linked Leaves: Leaf nodes are linked for efficient range queries.

[Link].3 Example B+ Tree Search:

1. Search for value 10:
• Start at the root (e.g., 25).
• Since 10 < 25, move to the left subtree.
• Find 10 in the leaf node and retrieve the record.

3.9.10 10. Practical Execution in PL/SQL

-- Local Function Example

DECLARE
FUNCTION add_numbers(n1 NUMBER, n2 NUMBER) RETURN NUMBER IS
BEGIN
RETURN n1 + n2;
END;

a NUMBER := 15;
b NUMBER := 25;
c NUMBER;
BEGIN
c := add_numbers(a, b);
DBMS_OUTPUT.PUT_LINE('Addition is: ' || c);
END;
/

[Link] 10.1 Creating and Testing Functions

• Output: Addition is: 40.

-- Stored Function Example

CREATE OR REPLACE FUNCTION add_numbers(n1 NUMBER, n2 NUMBER) RETURN NUMBER IS
BEGIN
RETURN n1 + n2;
END;
/

-- Test the Function

SELECT add_numbers(50, 30) FROM dual;

[Link] 10.2 Creating and Testing Stored Functions

• Output: 80.

115
-- Drop Table if Exists
BEGIN
EXECUTE IMMEDIATE 'DROP TABLE emp_demo_trigger';
EXCEPTION
WHEN OTHERS THEN NULL;
END;
/

-- Create Table
CREATE TABLE emp_demo_trigger (
emp_id NUMBER PRIMARY KEY,
emp_name VARCHAR2(50),
emp_salary NUMBER
);

-- Create Trigger
CREATE OR REPLACE TRIGGER check_salary
BEFORE INSERT OR UPDATE ON emp_demo_trigger
FOR EACH ROW
BEGIN
IF :NEW.emp_salary < 10000 THEN
RAISE_APPLICATION_ERROR(-20001, 'Salary must be at least 10,000');
END IF;
END;
/

-- Test Trigger
INSERT INTO emp_demo_trigger VALUES (1, 'John', 15000); -- Success
INSERT INTO emp_demo_trigger VALUES (2, 'Alice', 5000); -- Error

[Link] 10.3 Creating and Testing Triggers

• Output for Invalid Insert: ORA-20001: Salary must be at least 10,000.

3.10 Stored Procedures

3.10.1 1. Introduction to Stored Procedures
[Link] 1.1 Definition and Purpose
• A stored procedure is a subprogram in PL/SQL that performs a specific action.
• Purpose:
– Encapsulates logic into modular, reusable code blocks.
– Improves maintainability by organizing database operations.
– Enhances performance by reducing redundant code execution.

[Link] 1.2 Role in Database Programming

• Stored procedures are crucial in database applications because they:
– Allow reusable SQL and PL/SQL logic.
– Enable efficient execution by storing compiled code in the database.

116
 – Provide security by controlling data access through predefined procedures.

3.10.2 2. Types of Procedures

There are two main types of procedures in PL/SQL:

[Link] 2.1 Local Procedures

• Definition:
– Declared and used within the scope of a PL/SQL block (e.g., anonymous block, package, or another
subprogram).
– Not permanently stored in the database; exists only during the execution of the enclosing block.
• Declaration Location:
– Defined in the declarative section of a PL/SQL module.
• Scope and Usage:
– Can be called only within the execution section of the same block where it is declared.
• Syntax:
PROCEDURE procedure_name [(parameter1 [IN|OUT|IN OUT] datatype, ...)]
IS | AS
BEGIN
-- Executable statements
END [procedure_name];

– Key Components:
* procedure_name: Identifier for calling the procedure.
* Parameters (optional):
ꞏ IN: Passes a value into the procedure (read-only).
ꞏ OUT: Returns a value from the procedure (must be a variable).
ꞏ IN OUT: Passes an initial value into the procedure and returns an updated value.
* IS or AS: Used interchangeably to begin the procedure body.
* Procedure Body: Contains executable PL/SQL statements.
• Parameter Modes and Descriptions: | Mode | Description | Behavior | |——–|———————————
——————————————–|———————————–| | IN | Passes a value to the subprogram. |
Read-only (cannot be modified). | | OUT | Returns a value to the calling program. | Must be a variable (passed
by value). | | IN OUT | Passes an initial value to the subprogram and returns an updated value. | Modifiable
within the procedure. |

[Link] 2.2 Stored Procedures

• Definition:
– Permanently stored in the Oracle Database.
– Can be reused across multiple PL/SQL blocks and applications.
• Advantages Over Local Procedures:
– Persistence: Remains in the database until explicitly dropped.
– Reusability: Accessible by any PL/SQL code block or application.

117
 – Performance: Pre-compiled and optimized for execution.
• Syntax:
CREATE [OR REPLACE] PROCEDURE procedure_name [(parameter1 [IN|OUT|IN OUT] datatype, ...)]
IS | AS
BEGIN
-- Executable statements
END [procedure_name];

– Key Differences from Local Procedures:

* Uses CREATE [OR REPLACE] instead of just PROCEDURE.
* Stored in the database schema (not limited to a single block).
3.10.3 3. Benefits of Stored Procedures
Stored procedures offer seven key advantages:

[Link] 3.1 Efficiency

• Precompiled Code:
– Stored procedures are compiled once and stored in the database.
– Avoids re-parsing and re-compiling SQL statements on each execution.
• Reduced Overhead:
– Executes directly on the database server, minimizing client-side processing.

[Link] 3.2 Reusability

• Centralized Logic:
– Single procedure can be called by multiple applications or PL/SQL blocks.
– Eliminates code duplication.
• Modularity:
– Encapsulates business logic in self-contained units.

[Link] 3.3 Security

• Controlled Data Access:
– Users can interact with data only through procedures, restricting direct table access.
– Enforces role-based permissions (e.g., granting execute rights without table access).
• Data Integrity:
– Ensures operations comply with business rules before execution.

[Link] 3.4 Reduced Network Traffic

• Minimized Data Transmission:
– Instead of sending multiple SQL statements, a single procedure call executes all logic on the server.
– Reduces latency and bandwidth usage.

[Link] 3.5 Enhanced Error Handling

• Robust Exception Management:
– Supports structured error handling (e.g., EXCEPTION blocks in PL/SQL).
– Allows transaction rollback on failures.

118
 • Consistent Behavior:
– Errors are handled uniformly across all procedure calls.

[Link] 3.6 Modularity

• Structured Code Organization:
– Breaks complex operations into smaller, manageable procedures.
– Simplifies debugging and maintenance.

[Link] 3.7 Advanced Database Functions

• Transaction Control:
– Can modify data within a transactional scope (e.g., COMMIT/ROLLBACK).
• Encapsulation:
– Hides complex logic behind simple procedure calls.

[Link] 3.8 Summary of Benefits

Benefit Description
Efficiency Precompiled and stored in the database for fast execution.
Reusability Accessible by multiple applications, reducing code duplication.
Security Controls data access and enforces business rules.
Network Traffic Executes on the server, minimizing data transfer.
Reduction
Error Handling Supports structured exception handling for robust applications.
Modularity Organizes code into reusable, maintainable units.
Advanced Enables complex operations (e.g., transactions, data modifications).
Functions

3.10.4 4. Key Takeaways

• Stored procedures are essential for writing efficient, secure, and maintainable PL/SQL code.
• Local procedures are temporary and scoped to a PL/SQL block.
• Stored procedures are permanent, reusable, and offer performance and security benefits.
• Parameter modes (IN, OUT, IN OUT) define how data is passed to/from procedures.
• Seven core benefits make stored procedures indispensable in database applications:
1. Efficiency
2. Reusability
3. Security
4. Reduced Network Traffic
5. Error Handling
6. Modularity
7. Advanced Database Functions
End of Notes

3.11 Triggers
3.11.1 1. Introduction to Triggers
[Link] 1.1 Definition of Triggers

119
 • A trigger is a special kind of stored procedure that is automatically activated (triggered) in response to
a specific event in a database.
• Unlike regular stored procedures, triggers do not require explicit invocation—they execute automatically
when predefined database events occur.

[Link] 1.2 Purpose of Triggers

• Triggers enable automated execution of procedures in response to:
– Data modification events (e.g., INSERT, UPDATE, DELETE).
– Data Definition Language (DDL) events (e.g., CREATE, ALTER, DROP).
– Logon events (e.g., when a user establishes a session with the database server).

3.11.2 2. Types of Triggers

Triggers are categorized based on the type of event they respond to. The three primary types are:

[Link] 2.1 DML (Data Manipulation Language) Triggers

• Definition: Automatically fired when INSERT, UPDATE, or DELETE operations occur on a table.
• Use Cases:
– Enforcing business rules (e.g., preventing invalid data entries).
– Maintaining audit logs (e.g., tracking changes to sensitive data).
– Ensuring referential integrity (e.g., cascading updates/deletes).

[Link] 2.2 DDL (Data Definition Language) Triggers

• Definition: Automatically invoked when CREATE, ALTER, or DROP statements are executed on database ob-
jects (e.g., tables, views, procedures).
• Use Cases:
– Preventing unauthorized schema modifications (e.g., blocking DROP TABLE commands).
– Logging schema changes for auditing purposes.

[Link] 2.3 Logon Triggers

• Definition: Fired when a user logs into the database server (i.e., when a user session is established).
• Use Cases:
– Enforcing security policies (e.g., restricting logins during maintenance).
– Logging user access for monitoring and compliance.
– Setting session-level configurations (e.g., default schema, resource limits).

3.11.3 3. Subtypes of Triggers (Based on Timing and Behavior)

Triggers can further be classified based on when they fire and how they interact with the triggering event:

[Link] 3.1 FOR (AFTER) Triggers

• Definition: Execute after the triggering SQL statement completes successfully.
• Key Characteristics:
– Can be defined on tables or views.
– Cannot be used on views if they modify data (since views do not store data directly).
– Example: An AFTER INSERT trigger that logs new records in an audit table.

120
[Link] 3.2 INSTEAD OF Triggers
• Definition: Override the default action of an INSERT, UPDATE, or DELETE statement.
• Key Characteristics:
– Can be defined on tables or views.
– Useful for updatable views (where direct modifications are not allowed).
– Example: An INSTEAD OF DELETE trigger that archives records instead of deleting them.

[Link] 3.3 Logon Triggers (Special Case)

• Definition: Execute after authentication but before the user session is fully established.
• Key Characteristics:
– Defined at the server level (not tied to a specific table).
– Multiple logon triggers can be defined on a single server.
– Errors in logon triggers prevent the session from being established.

3.11.4 4. Why Use Triggers?

Triggers serve critical functions in database management:

[Link] 4.1 Enforcing Data Integrity

• Ensure that business rules are followed (e.g., preventing negative salaries).
• Maintain consistency across related tables (e.g., cascading updates).

[Link] 4.2 Automating Complex Business Logic

• Derived column values: Automatically compute fields (e.g., total_price = quantity * unit_price).
• Conditional actions: Execute logic based on data changes (e.g., sending alerts for low stock).

[Link] 4.3 Auditing and Logging

• Track who made changes, when, and what was modified (e.g., logging deletions for compliance).

[Link] 4.4 Performance Optimization

• Reduce client-side processing by handling logic in the database layer.
• Minimize network traffic by executing validations server-side.

[Link] 4.5 Maintainability

• Centralize logic: Avoid scattering business rules across application code.
• Easier updates: Modify triggers without redeploying applications.

3.11.5 5. Advantages of Triggers

Advantage Description
Data Integrity Ensures constraints are met before changes
are committed.
Validation Validates data before insertion/update (e.g.,
checking for valid email formats).

121
Advantage Description
Auditing Maintains a log of all changes for
accountability.
Performance Reduces client-side processing by handling
logic in the database.
Maintainability Encapsulates logic in one place, reducing
code duplication.

3.11.6 6. Disadvantages of Triggers

Disadvantage Description
Invisible Execution Triggers run automatically, making
debugging difficult (users may not realize a
trigger fired).
Overhead Increases database server load due to
additional processing.
Complexity Managing multiple triggers for the same
event can become unwieldy.
Scope Limitations Triggers can only be created in the current
database but can reference objects in other
databases.
Debugging Challenges Errors in triggers may roll back
transactions without clear error messages.

3.11.7 7. Examples of Triggers

[Link] 7.1 DDL Trigger Example Scenario: Prevent unauthorized schema modifications (e.g., blocking
CREATE, ALTER, or DROP TABLE commands).
CREATE TRIGGER safety
ON DATABASE
FOR CREATE_TABLE, ALTER_TABLE, DROP_TABLE
AS
BEGIN
PRINT 'You cannot create, alter, or drop tables in this database.';
ROLLBACK;
END;

• Behavior:
– Fires before the CREATE/ALTER/DROP TABLE operation completes.
– Prints a warning and rolls back the transaction.
– Defined at the database level (applies to all tables).

[Link] 7.2 DML Trigger Example Scenario: Prevent insertion of negative salaries in an employee table.
CREATE TRIGGER check_salary
ON employee
AFTER INSERT, UPDATE
AS

122
BEGIN
IF EXISTS (SELECT * FROM inserted WHERE salary < 0)
BEGIN
RAISERROR('Salary cannot be negative.', 16, 1);
ROLLBACK TRANSACTION;
END
END;

• Behavior:
– Fires after an INSERT or UPDATE on the employee table.
– Checks if any inserted/updated salary is negative.
– If true, raises an error and rolls back the transaction.
– Ensures data integrity by rejecting invalid entries.

[Link] 7.3 Logon Trigger Example Scenario: Log user logins and display a welcome message.
CREATE TRIGGER track_logins
ON ALL SERVER
FOR LOGON
AS
BEGIN
PRINT 'Welcome to the database, ' + SYSTEM_USER + '!';
-- Log the login event to an audit table
INSERT INTO login_audit (user_name, login_time)
VALUES (SYSTEM_USER, GETDATE());
END;

• Behavior:
– Fires after authentication but before the session is fully established.
– Prints a welcome message visible in the SQL Server error log.
– Logs the login event to an audit table for tracking.
– Authentication errors prevent the trigger from executing.

3.11.8 8. Summary of Key Concepts

• Triggers are automated stored procedures tied to database events.
• Three main types:
– DML Triggers (INSERT, UPDATE, DELETE).
– DDL Triggers (CREATE, ALTER, DROP).
– Logon Triggers (user session establishment).
• Subtypes:
– AFTER (FOR) Triggers: Execute post-event.
– INSTEAD OF Triggers: Override default actions.
• Advantages: Data integrity, auditing, performance, maintainability.
• Disadvantages: Invisible execution, overhead, complexity, debugging challenges.
• Examples:
– DDL Trigger: Block schema modifications.
– DML Trigger: Prevent invalid data (e.g., negative salaries).
– Logon Trigger: Track user access and display messages.
Final Note: Triggers are powerful but should be used judiciously—overuse can lead to performance issues and

123
maintenance complexity. Always document triggers thoroughly for easier debugging.

124
4 Module 4: Advanced SQL Relationships and Normalisation
4.1 Advanced Join Operations
4.1.1 1. Introduction to Advanced Join Operations
• Context: Part of Module 4 – Advanced SQL Relationships and Normalisation.
• Objective: Master combining multiple joins in a single query and optimizing join performance using index-
ing and query planning.
• Key Skills Acquired:
– Combine multiple joins in a single SQL query.
– Optimize join performance via indexing and query planning.

4.1.2 2. Combining Multiple Joins

[Link] 2.1 Definition and Purpose
• Definition: Using more than one join operation in a single query to fetch data from multiple related tables.
• Purpose:
– Essential in complex databases where data is distributed across multiple tables.
– Enables retrieval of comprehensive information spanning multiple tables in a single query.
• Example Scenario: Combining data from employee, department, and location tables.

SELECT
table1.column1, table1.column2,
table2.column1, table2.column2,
table3.column1, table3.column2
FROM
table1
LEFT JOIN
table2 ON [Link] = [Link]
INNER JOIN
table3 ON [Link] = [Link];

[Link] 2.2 Syntax for Multiple Joins

• Breakdown:
1. SELECT Clause: Specifies columns to retrieve from each table.
2. FROM Clause: Starts with the primary table (table1).
3. LEFT JOIN: Combines table1 and table2; returns all records from table1 and matched records
from table2 (or NULL if no match).
4. INNER JOIN: Combines table2 and table3; returns only rows with matches in both tables.

[Link] 2.3 Key Considerations

• Interaction of Join Types:
– Misunderstanding join interactions can lead to:
* Large, unintended result sets.
* Performance issues (e.g., slow execution).
• Best Practice: Clearly define the intended relationship between tables to avoid logical errors.

125
4.1.3 3. Practical Examples of Multiple Joins
[Link] 3.1 Example 1: Left Join + Inner Join Objective: Retrieve employee details with their departments
and locations, including employees without a department.
Tables Involved: - employees (EmployeeID, Name, Salary, DepartmentID, LocationID) - departments (Depart-
mentID, DepartmentName) - locations (LocationID, LocationName)
Query:
SELECT
[Link],
[Link],
[Link],
[Link],
[Link]
FROM
employees
LEFT JOIN
departments ON [Link] = [Link]
INNER JOIN
locations ON [Link] = [Link];

• Output:
– EmployeeID, Name, Salary, DepartmentName (or NULL if no department), LocationName.
• Behavior:
– LEFT JOIN: Includes all employees, even those without a department (DepartmentName = NULL).
– INNER JOIN: Includes only employees with a valid LocationID.

[Link] 3.2 Example 2: Multiple Inner Joins Objective: Retrieve only employees with both a department
and a location.
Query:
SELECT
[Link],
[Link],
[Link],
[Link],
[Link]
FROM
employees
INNER JOIN
departments ON [Link] = [Link]
INNER JOIN
locations ON [Link] = [Link];

• Output:
– Only employees with matching records in all three tables.
• Behavior:
– Both INNER JOINs enforce exact matches in departments and locations.

126
[Link] 3.3 Example 3: Left Join + Right Join Objective: Include all departments (even without employees)
and all employees (even without departments).
Query:
SELECT
[Link],
[Link],
[Link],
[Link]
FROM
employees
LEFT JOIN
departments ON [Link] = [Link]
RIGHT JOIN
locations ON [Link] = [Link];

• Output:
– All employees (left join) + all locations (right join).
– NULL values appear for unmatched rows (e.g., departments without employees or employees without
locations).
• Behavior:
– LEFT JOIN: Preserves all records from employees.
– RIGHT JOIN: Preserves all records from locations.

[Link] 3.4 Key Takeaways from Examples

Join Combination Behavior

Inner + Left Join Matches from inner join + all left-table records (with NULL for unmatched right
records).
Multiple Inner Joins Requires matches in all tables; excludes rows with any NULL in join columns.
Left + Right Join Includes all records from both tables; NULL for unmatched rows.

4.1.4 4. Join Performance Optimization

[Link] 4.1 Indexing for Join Optimization
• Definition: An index is a database structure that improves the speed of data retrieval operations.
• Purpose in Joins:
– Accelerates join condition evaluations by reducing the data scanned.
– Critical for large tables where joins can be computationally expensive.

[Link].1 4.1.1 Creating Indexes Syntax:

CREATE INDEX index_name ON table_name (column_name);

Example:
-- Index on DepartmentID in employees table
CREATE INDEX idx_department_id ON employees (DepartmentID);

127
-- Index on LocationID in employees table
CREATE INDEX idx_location_id ON employees (LocationID);

• Impact:
– Enables the database to quickly locate rows matching join conditions.
– Reduces I/O operations by limiting the data scanned.

[Link].2 4.1.2 Benefits of Indexing

• Faster Query Execution: Indices allow the database to bypass full table scans.
• Efficiency in Large Datasets: Particularly beneficial for tables with millions of rows.
• Cost: Indexes consume additional storage and require maintenance during data modifications (IN-
SERT/UPDATE/DELETE).

[Link] 4.2 Query Planning and the EXPLAIN Command

• Definition: Query planning analyzes the steps SQL takes to execute a query.
• Tool: The EXPLAIN command provides an execution plan, revealing how the database processes the query.

[Link].1 4.2.1 EXPLAIN Output Breakdown

Column Description
ID Unique identifier for the query (or subquery). Helps distinguish parts of complex queries (e.g.,
nested queries).
Select Type of SELECT operation:
Type
- SIMPLE: No subqueries or unions.
- PRIMARY: Main query in a complex query with subqueries.
Table Name of the table referenced in the row.
Type Join type or access method:
- ALL: Full table scan (least efficient).
- index: Full index scan.
- NULL: Table not accessed (e.g., derived from another table).
Possible Indices that could be used for row lookup.
Keys
Key Length of the key used; indicates efficiency of the chosen index.
Length
Ref Columns or constants compared to the index.
Rows Estimated number of rows examined; helps assess query cost.

[Link].2 4.2.2 Example EXPLAIN Output Interpretation Query:

EXPLAIN
SELECT [Link], [Link]
FROM employees
INNER JOIN departments ON [Link] = [Link];

Hypothetical Output:

128
+----+-------------+------------+-------+---------------+----------------+---------+-----
-+------+
| ID | Select Type | Table | Type | Possible Keys | Key | Key Len | Ref | Rows |
+----+-------------+------------+-------+---------------+----------------+---------+-----
-+------+
| 1 | SIMPLE | departments| ALL | PRIMARY | NULL | NULL | NULL | 10 |
| 1 | SIMPLE | employees | ref | idx_dept_id | idx_dept_id | 4 | func | 50 |
+----+-------------+------------+-------+---------------+----------------+---------+-----
-+------+

• Analysis:
– departments Table: Full scan (ALL) due to no index usage.
– employees Table: Uses idx_dept_id index (ref type), reducing rows scanned to 50.
– Optimization Opportunity: Add an index on [Link] to avoid full scan.

[Link].3 4.2.3 Query Optimization Strategies

1. Use Indexes on Join Columns:
• Ensure columns used in ON clauses are indexed.
2. Avoid SELECT *:
• Retrieve only necessary columns to reduce data transfer.
3. Analyze Join Order:
• The database may not always choose the optimal join order; use hints if needed.
4. Limit Result Sets:
• Use WHERE, LIMIT, or pagination to reduce output size.
5. Update Statistics:
• Regularly update database statistics for accurate query planning.

4.1.5 5. Summary of Key Concepts

[Link] 5.1 Combining Multiple Joins
• Purpose: Retrieve data from multiple related tables in a single query.
• Types:
– Inner + Left/Right Joins: Mix of exact matches and inclusive joins.
– Multiple Inner Joins: Strict matching across all tables.
– Left + Right Joins: Inclusive of all records from both tables.
• Syntax:
SELECT columns
FROM table1
JOIN_TYPE table2 ON condition1
JOIN_TYPE table3 ON condition2;

[Link] 5.2 Performance Optimization

• Indexing:
– Create indexes on join columns to speed up lookups.
– Syntax: CREATE INDEX index_name ON table (column).
• Query Planning:
– Use EXPLAIN to analyze execution plans.

129
 – Optimize based on join types, index usage, and row estimates.

[Link] 5.3 Best Practices

• Design:
– Normalize tables to minimize redundant joins.
– Use appropriate join types based on data requirements.
• Execution:
– Monitor query performance with EXPLAIN.
– Avoid Cartesian products (unintended cross joins).
• Maintenance:
– Update indexes and statistics regularly.
– Test queries with realistic data volumes.

4.2 Cross Join

4.2.1 1. Introduction to Cross Join
[Link] 1.1 Definition
• A cross join returns the Cartesian product of two tables.
• Cartesian product: Every row from the first table is combined with every row from the second table.
• The result set size is determined by multiplying the number of rows in each table:
– If Table A has 3 rows and Table B has 2 rows, the cross join produces 3 × 2 = 6 rows.

[Link] 1.2 Key Characteristics

• No explicit join condition is required (unlike inner/outer joins).
• Generates all possible combinations of rows between the two tables.
• Can produce very large result sets if tables are sizable.

4.2.2 2. Practical Use Cases of Cross Join

[Link] 2.1 Generating Test Data
• Useful for quickly creating large datasets for testing database performance and scalability.
– Example: Combining a products table with a customers table to generate all possible product-
customer pairs for load testing.

[Link] 2.2 Creating All Possible Combinations of Sets

• Applicable in scenarios requiring exhaustive pairing of two datasets:
– Scheduling (e.g., matching employees to shifts).
– Product bundling (e.g., combining items from two categories).
– Exploratory data analysis (e.g., comparing all pairs in two groups).

[Link] 2.3 Combining Data Without a Direct Relationship

• Used when no explicit relationship exists between tables, but analysis requires comparing every possible
pair.
– Example: Merging a sales table with a promotions table to analyze potential impacts without a prede-
fined link.

130
4.2.3 3. SQL Syntax for Cross Join

SELECT *
FROM table1
CROSS JOIN table2;

[Link] 3.1 Basic Syntax

• Explanation:
– SELECT *: Retrieves all columns from both tables.
– CROSS JOIN: Specifies the Cartesian product operation.

[Link] 3.2 Example with employee and department Tables

[Link].1 Table Structures:

• employee: | employeeID | name | salary | |————|——–|——–| | 1 | Bob | 50000 | | 2 | Alice | 60000 | | 3 |
John | 55000 |
• department: | departmentID | departmentName | |————–|—————-| | 1 | HR | | 2 | Engineering | | 3 |
Marketing |

SELECT *
FROM employee
CROSS JOIN department;

[Link].2 Query:

[Link].3 Result:
• 9 rows (3 employees × 3 departments).
• Each employee is paired with every department (e.g., Bob-HR, Bob-Engineering, Bob-Marketing, Alice-HR,
etc.).

SELECT name, departmentName

FROM employee
CROSS JOIN department;

[Link].4 Selecting Specific Columns:

• Output: Only the name (from employee) and departmentName (from department) columns are displayed.

4.2.4 4. Combining Cross Join with Other SQL Functions

[Link] 4.1 Aggregate Functions
• Cross joins can be paired with aggregate functions (e.g., COUNT, SUM, AVG) to derive insights from the Carte-
sian product.

131
SELECT COUNT(*)
FROM employee
CROSS JOIN department;

[Link].1 Example: Counting Combinations

• Result: 9 (total combinations of 3 employees × 3 departments).
• Use Case: Quickly determine the size of the Cartesian product for performance analysis.

[Link] 4.2 Filtering with WHERE Clause

• Filters can refine cross join results to focus on relevant data.

SELECT *
FROM employee
CROSS JOIN department
WHERE [Link] = 'HR';

[Link].1 Example: Filtering by Department

• Result: Only combinations where the department is HR (e.g., Bob-HR, Alice-HR, John-HR).
• Purpose: Reduces the result set to a subset of interest.

4.2.5 5. Performance Considerations

[Link] 5.1 Potential Performance Impact
• Cartesian products grow exponentially:
– If Table A has 1,000 rows and Table B has 1,000 rows, the cross join produces 1,000,000 rows.
– This can consume significant memory and processing power.

[Link] 5.2 Best Practices for Efficient Use

1. Use Sparingly:
• Reserve cross joins for cases where all combinations are truly needed.
• Avoid using on large tables unless filtered.
2. Apply Filters Early:
• Use WHERE clauses to limit results before processing.
• Example: Filter by a specific department or date range.
3. Combine with Aggregates:
• Use COUNT, SUM, or GROUP BY to summarize data rather than retrieving all rows.
4. Test with Small Datasets First:
• Validate logic on smaller tables before scaling to larger datasets.

4.2.6 6. Summary of Key Points

Topic Details
Definition Returns the Cartesian product (all possible row combinations) of two tables.
Use Cases Test data generation, exhaustive combinations, combining unrelated data.

132
Topic Details
Syntax SELECT * FROM table1 CROSS JOIN table2;
Aggregate Functions Pair with COUNT, SUM, etc., to analyze the Cartesian product.
Filtering Use WHERE to refine results.
Performance Risk of large result sets; filter and aggregate to optimize.

4.3 Database Normalisation

4.3.1 Introduction to Database Normalisation
[Link] Importance of Normalisation Normalisation is a critical process in database management that ensures
efficient, consistent, and reliable database design. Its primary objectives include:
1. Minimising Data Redundancy
• Organises data into tables to ensure each piece of information is stored only once.
• Eliminates unnecessary duplication, saving storage space and improving data management
efficiency.
2. Preventing Data Anomalies
• Anomalies occur during insertion, update, and deletion operations due to poor database design.
• Normalisation mitigates:
– Insertion anomalies: Inability to add data due to missing or required fields.
– Update anomalies: Inconsistencies when multiple instances of the same data are not updated
uniformly.
– Deletion anomalies: Unintended loss of important data when deleting a record.
3. Ensuring Data Integrity & Efficient Organisation
• Structures data to enforce relationships and dependencies.
• Maintains constraints and accuracy across the database.
• Leads to reliable data retrieval and manipulation, making the database robust and user-friendly.

4.3.2 Data Redundancy & Data Anomalies

[Link] Definition of Data Redundancy
• Occurs when the same data is stored in multiple places within a database.
• Problems caused by redundancy:
– Increased storage requirements.
– Inconsistencies if all copies of data are not updated simultaneously.

[Link] Definition of Data Anomalies Anomalies are problems arising from redundancy, classified into three
types:
1. Update Anomalies
• When duplicate data exists, updating one instance requires updating all others.
• Risk: If one instance is missed, data becomes inconsistent.
2. Insertion Anomalies
• Occurs when certain data cannot be inserted without the presence of other data.
• Example: Adding a new course to a database where no student has enrolled yet may fail if the table
design requires a student record.
3. Deletion Anomalies
• Deleting a record inadvertently removes unrelated but important data.

133
 • Example: Deleting a student’s record might also delete the only reference to a course they were enrolled
in.

4.3.3 Example of Data Redundancy & Normalisation

[Link] Unnormalised Table: Customers_Orders

Customer_ID Customer_Name Customer_Email Order_ID Order_Date

1 John john@[Link] 101 2023-10-01
1 John john@[Link] 102 2023-10-05
2 Alice alice@[Link] 103 2023-10-03

Problems: - Redundancy: Customer_Name and Customer_Email repeat for the same Customer_ID. - Anomalies:
- Update: Changing John’s email requires updates in multiple rows. - Insertion: Cannot add a customer without
an order. - Deletion: Deleting an order might lose the only record of a customer.

[Link] Normalised Solution

[Link].1 Table 1: Customers (Eliminates Redundancy)

Customer_ID (PK) Customer_Name Customer_Email

1 John john@[Link]
2 Alice alice@[Link]

[Link].2 Table 2: Orders (Uses Foreign Key)

Order_ID (PK) Order_Date Customer_ID (FK)

101 2023-10-01 1
102 2023-10-05 1
103 2023-10-03 2

Benefits of Normalisation: [OK] Eliminates redundancy (customer details stored once). [OK] Prevents update
anomalies (email changes in one place). [OK] Prevents insertion anomalies (customers can exist without orders).
[OK] Prevents deletion anomalies (deleting an order does not remove customer data).

4.3.4 Normal Forms in Database Design

Normalisation is achieved through progressive normal forms, each addressing specific types of anomalies.

[Link] 1. First Normal Form (1NF) Goal: Eliminate repeating groups in tables. Requirements: - Each
field must contain atomic (indivisible) values. - No multivalued attributes (e.g., comma-separated lists).

[Link].1 Example: Violating 1NF

134
 Student Courses
John Maths, Science

Problem: Courses contains multiple values in one field.

[Link].2 Solution: 1NF-Compliant Tables

Student Course
John Maths
John Science

Key Improvement: - Each Course is now in a separate row. - Atomicity is maintained (no composite values).

[Link] 2. Second Normal Form (2NF) Goal: Remove partial dependencies (non-key attributes depending
on part of a composite primary key). Requirements: - Must already be in 1NF. - All non-key attributes must be
fully functionally dependent on the entire primary key.

[Link].1 Example: Violating 2NF

Student_ID (PK) Course (PK) Instructor

101 Maths Dr. Smith
101 Science Dr. Smith
102 Maths Dr. Smith

Problem: Instructor depends only on Course, not the full primary key (Student_ID + Course).

[Link].2 Solution: 2NF-Compliant Tables Table 1: Student_Courses | Student_ID (PK, FK) | Course
(PK, FK) | |————————-|———————| | 101 | Maths | | 101 | Science | | 102 | Maths |
Table 2: Course_Instructors | Course (PK) | Instructor | |—————–|—————-| | Maths | Dr. Smith | |
Science | Dr. Smith |
Key Improvement: - Partial dependency removed (Instructor now depends only on Course). - Redundancy
reduced (instructor data stored once per course).

[Link] 3. Third Normal Form (3NF) Goal: Eliminate transitive dependencies (non-key attributes depending
on other non-key attributes). Requirements: - Must already be in 2NF. - No transitive dependencies (non-key
attributes must depend only on the primary key).

[Link].1 Example: Violating 3NF

Student_ID (PK) Course (PK) Instructor Instructor_Email

101 Maths Dr. Smith smith@[Link]

Problem: Instructor_Email depends on Instructor, which depends on Course (transitive dependency).

135
[Link].2 Solution: 3NF-Compliant Tables Table 1: Student_Courses | Student_ID (PK, FK) | Course
(PK, FK) | |————————-|———————| | 101 | Maths |
Table 2: Course_Instructors | Course (PK, FK) | Instructor_ID (PK, FK) | |———————|——————
———-| | Maths | 1 |
Table 3: Instructors | Instructor_ID (PK) | Instructor_Name | Instructor_Email | |————————|——
—————|———————-| | 1 | Dr. Smith | smith@[Link] |
Key Improvement: - Transitive dependency removed (Instructor_Email now depends only on Instruc-
tor_ID). - Further redundancy eliminated.

[Link] 4. Boyce-Codd Normal Form (BCNF) Goal: Stricter version of 3NF that handles complex depen-
dencies. Requirements: - For any dependency A → B, A must be a superkey (a candidate key). - Addresses
anomalies not fully resolved by 3NF.
Use Case: - Useful when overlapping candidate keys exist, leading to potential anomalies.
Example: - If a table has two candidate keys (Student_ID + Course and Instructor + Course), BCNF ensures
no anomalies arise from their interaction.

[Link] Higher Normal Forms (4NF & 5NF)

• 4NF: Deals with multivalued dependencies (independent many-to-many relationships).
• 5NF: Addresses join dependencies (ensures tables can be reconstructed without loss of information).
• Note: Typically covered in advanced textbooks.

4.3.5 Role of Primary & Foreign Keys in Normalisation

[Link] Primary Key (PK)
• Definition: A unique identifier for each record in a table.
• Purpose:
– Ensures entity integrity (no duplicate records).
– Acts as a reference point for foreign keys.

[Link] Foreign Key (FK)

• Definition: A field that links to a primary key in another table.
• Purpose:
– Enforces referential integrity (ensures relationships between tables remain consistent).
– Prevents orphaned records (e.g., an order without a valid customer).

[Link] Example: Maintaining Data Integrity Table 1: Students | Student_ID (PK) | Student_Name |
|———————|——————| | 101 | John | | 102 | Alice |
Table 2: Enrollments | Enrollment_ID (PK) | Student_ID (FK) | Course | |————————|——————
—-|————| | 1 | 101 | Maths | | 2 | 102 | Science |
Key Benefits: - Consistency: Student_ID in Enrollments must exist in Students. - Integrity: Prevents invalid
references (e.g., enrolling a non-existent student).

136
4.3.6 Summary of Key Concepts
1. Normalisation Goals:
• Reduce redundancy.
• Prevent anomalies (insertion, update, deletion).
• Ensure data integrity.
2. Normal Forms:
• 1NF: Atomic values, no repeating groups.
• 2NF: No partial dependencies.
• 3NF: No transitive dependencies.
• BCNF: Stricter than 3NF for complex dependencies.
3. Keys in Normalisation:
• Primary Key: Uniquely identifies records.
• Foreign Key: Enforces relationships between tables.
4. Practical Impact:
• Efficient storage (less redundancy).
• Reliable operations (no anomalies).
• Scalable design (easier maintenance).

4.4 Inner Join

4.4.1 1. Introduction to Inner Join
[Link] 1.1 Lecture Objectives
• Understand the definition and use cases of INNER JOIN.
• Learn practical examples of INNER JOIN in SQL.
• Perform INNER JOIN operations and combine them with GROUP BY for advanced queries.

[Link] 1.2 Key Takeaways

• INNER JOIN is fundamental for combining related data from multiple tables.
• It ensures data integrity by returning only matching rows.
• Used extensively in relational databases for reporting and analysis.

4.4.2 2. Definition of Inner Join

[Link] 2.1 Core Concept
• An INNER JOIN combines rows from two or more tables based on a related column (common key).
• Only returns rows where there is a match in both tables.
– If no match exists, the row is excluded from the result set.
• Purpose: Retrieve data that exists in both tables, ensuring complete and related datasets.

[Link] 2.2 Key Characteristics

• Match-based: Requires a common column (e.g., DepartmentID) to establish a relationship.
• Non-inclusive: Rows without a match in either table are omitted.
• Versatile: Used in scenarios requiring comprehensive data integration (e.g., reports, analytics).

[Link] 2.3 Common Use Cases

1. Fetching related data from multiple tables:

137
 • Combines data stored in separate tables (e.g., customers + orders).
2. Generating reports:
• Merges data for a complete view (e.g., sales reports with customer + order details).
3. Analyzing relationships:
• Links customer data with order details to study purchase behavior, order history, or demographics.

4.4.3 3. Syntax of Inner Join

SELECT columns
FROM table1
INNER JOIN table2
ON table1.common_column = table2.common_column;

[Link] 3.1 Basic Structure

• SELECT columns: Specifies the columns to retrieve.
• FROM table1: Primary table for data selection.
• INNER JOIN table2: Secondary table to join.
• ON table1.common_column = table2.common_column:
– Defines the matching condition (common key).
– Columns must exist in both tables.

[Link] 3.2 Critical Notes

• The ON clause is mandatory to specify the join condition.
• The common column does not need to have the same name but must have compatible data types.

4.4.4 4. Practical Examples

[Link] **4.1 Example Tables Assume two tables: 1. employees | EmployeeID | Name | DepartmentID | Salary
| |————|———-|————–|——–| | 1 | Alice | 101 | 50000 | | 2 | Bob | 102 | 60000 | | 3 | Charlie | 101 | 55000 |
2. departments | DepartmentID | DepartmentName | |————–|—————–| | 101 | HR | | 102 | Engineering
|

[Link] 4.2 Basic Inner Join Query Objective: Retrieve employees with their department names.
SELECT [Link], [Link]
FROM employees
INNER JOIN departments
ON [Link] = [Link];

Result: | Name | DepartmentName | |———|—————–| | Alice | HR | | Bob | Engineering | | Charlie | HR |

Key Observations: - Only employees with a matching DepartmentID in both tables are included. - If a department
has no employees (or vice versa), it is excluded.

4.4.5 5. Inner Join with GROUP BY

[Link] 5.1 Use Case
• Aggregate data by groups (e.g., count employees per department, calculate total salary).
• Combines INNER JOIN with aggregate functions (COUNT, SUM, AVG, MIN, MAX).

138
SELECT
[Link],
COUNT([Link]) AS NumberOfEmployees,
SUM([Link]) AS TotalSalary
FROM employees
INNER JOIN departments
ON [Link] = [Link]
GROUP BY [Link];

[Link] 5.2 Example 1: Count Employees and Total Salary per Department Result: | DepartmentName |
NumberOfEmployees | TotalSalary | |—————–|——————–|————-| | HR | 2 | 105000 | | Engineering |
1 | 60000 |
Steps: 1. Join: Combine employees and departments on DepartmentID. 2. Group: Aggregate results by
DepartmentName. 3. Calculate: - COUNT(EmployeeID) → Number of employees per department. - SUM(Salary)
→ Total salary per department.

SELECT
[Link],
AVG([Link]) AS AverageSalary
FROM employees
INNER JOIN departments
ON [Link] = [Link]
GROUP BY [Link];

[Link] 5.3 Example 2: Average Salary per Department Result: | DepartmentName | AverageSalary | |——
———–|—————| | HR | 52500 | | Engineering | 60000 |

SELECT
[Link],
MIN([Link]) AS MinSalary,
MAX([Link]) AS MaxSalary
FROM employees
INNER JOIN departments
ON [Link] = [Link]
GROUP BY [Link];

[Link] 5.4 Example 3: Min and Max Salary per Department Result: | DepartmentName | MinSalary |
MaxSalary | |—————–|———–|———–| | HR | 50000 | 55000 | | Engineering | 60000 | 60000 |

4.4.6 6. Advanced Aggregation with GROUP_CONCAT

[Link] 6.1 Use Case
• Concatenate values from multiple rows into a single string (e.g., list all employee names per department).

139
SELECT
[Link],
GROUP_CONCAT([Link] SEPARATOR ', ') AS Employees
FROM employees
INNER JOIN departments
ON [Link] = [Link]
GROUP BY [Link];

[Link] **6.2 Example: List Employees per Department Result: | DepartmentName | Employees | |———
——–|—————–| | HR | Alice, Charlie | | Engineering | Bob |
Key Points: - GROUP_CONCAT combines values from multiple rows into a comma-separated string. - SEPARATOR
', ' defines the delimiter (comma + space here).

4.4.7 7. Comprehensive Example: Multiple Aggregate Functions

SELECT
[Link],
COUNT([Link]) AS NumberOfEmployees,
SUM([Link]) AS TotalSalary,
AVG([Link]) AS AverageSalary,
MIN([Link]) AS MinSalary,
MAX([Link]) AS MaxSalary,
GROUP_CONCAT([Link] SEPARATOR ', ') AS Employees
FROM employees
INNER JOIN departments
ON [Link] = [Link]
GROUP BY [Link];

[Link] 7.1 Query Combining All Aggregates Result: | DepartmentName | NumberOfEmployees | TotalSalary
| AverageSalary | MinSalary | MaxSalary | Employees | |—————–|——————–|————-|—————|—
——–|———–|—————–| | HR | 2 | 105000 | 52500 | 50000 | 55000 | Alice, Charlie | | Engineering | 1 | 60000 |
60000 | 60000 | 60000 | Bob |
Analysis: - Provides a holistic view of each department: - Employee count, salary metrics, and employee names.

4.4.8 8. Advanced Filtering with HAVING COUNT

[Link] 8.1 Use Case
• Filter groups based on aggregate conditions (e.g., departments with >1 employee).

SELECT
[Link],
COUNT([Link]) AS NumberOfEmployees
FROM employees
INNER JOIN departments
ON [Link] = [Link]
GROUP BY [Link]
HAVING COUNT([Link]) > 1;

140
[Link] **8.2 Example: Departments with More Than 1 Employee Result: | DepartmentName | NumberO-
fEmployees | |—————–|——————–| | HR | 2 |
Key Points: - HAVING filters after aggregation (unlike WHERE, which filters before). - Useful for conditional
group-level analysis.

4.4.9 9. Summary of Key Concepts

[Link] 9.1 Inner Join Fundamentals
• Purpose: Combine rows from multiple tables where a match exists.
• Syntax:
SELECT columns
FROM table1
INNER JOIN table2 ON [Link] = [Link];

• Behavior: Excludes non-matching rows.

[Link] 9.2 Common Aggregate Functions

Function Description Example Use Case

COUNT Counts rows in a group. Number of employees per department.
SUM Sums values in a group. Total salary per department.
AVG Calculates the average. Average salary per department.
MIN Finds the minimum value. Lowest salary in a department.
MAX Finds the maximum value. Highest salary in a department.
GROUP_CONCATConcatenates values into a string. List of employee names per department.

[Link] 9.3 Advanced Techniques

• GROUP BY: Groups rows by a column for aggregation.
• HAVING: Filters groups based on aggregate conditions.
• Comprehensive Queries: Combine multiple aggregates for detailed reporting.

4.4.10 10. Conclusion

• INNER JOIN is essential for relational database operations.
• Enables data integration from multiple tables while ensuring referential integrity.
• When combined with GROUP BY and aggregate functions, it supports advanced analytics (e.g., reporting,
trend analysis).
• Mastery of INNER JOIN is critical for efficient SQL query design in real-world applications.

4.5 Introduction to Window Functions

4.5.1 1. Overview of Window Functions
[Link] 1.1 Definition
• A window function in SQL performs calculations across a set of table rows that are related to the current
row.

141
 • Unlike standard aggregate functions (e.g., SUM(), AVG()), window functions retain individual row details
while computing aggregated results.
• Example: Calculating a running total of sales while preserving each transaction record.

[Link] 1.2 Key Characteristics

1. Enhanced Analysis
• Enables advanced operations such as:
– Ranking (e.g., top-performing employees).
– Running totals (e.g., cumulative sales).
– Moving averages (e.g., trend analysis).
• Essential for in-depth data analysis and reporting.
2. Non-Aggregate Data Retention
• Unlike GROUP BY, window functions do not collapse the result set.
• Allows calculations across rows without losing granularity (e.g., seeing individual sales while com-
puting totals).
3. Versatility
• Applicable in:
– Partitioned analysis (e.g., calculations within departments).
– Data transformation (e.g., normalizing values).
– Reporting (e.g., leaderboards, performance metrics).

4.5.2 2. Common Use Cases for Window Functions

Window functions are particularly useful in the following scenarios:

[Link] 2.1 Ranking

• Assigns ranks to rows based on criteria (e.g., sales performance).
• Useful for:
– Leaderboard creation.
– Performance evaluation (e.g., top 10 sales representatives).

[Link] 2.2 Partitioned Data Analysis

• Performs operations within subsets of data (e.g., by department, region).
• Example: Calculating the highest salary per department without collapsing rows.

[Link] 2.3 Running Totals

• Computes cumulative sums (e.g., financial balances, sales over time).
• Example: Tracking a running balance in a bank transaction table.

[Link] 2.4 Moving Averages

• Calculates averages over a sliding window of rows.
• Useful for:
– Trend analysis (e.g., 3-month moving average of stock prices).
– Smoothing fluctuations in time-series data.

142
[Link] 2.5 Data Segmentation
• Divides data into quantiles (e.g., customer segmentation by spending).
• Example: Splitting customers into high/medium/low-value groups for targeted marketing.

4.5.3 3. The PARTITION BY Clause

[Link] 3.1 Definition
• Divides the result set into partitions (groups) where the window function is applied independently.
• Ensures calculations (e.g., sums, ranks) are reset for each partition.

[Link] 3.2 Purpose

• Enables group-specific operations without using GROUP BY.
• Example:
– Calculating the top 3 salaries per department (rather than globally).

WINDOW_FUNCTION() OVER (
PARTITION BY column_name
ORDER BY column_name
)

[Link] 3.3 Syntax

• PARTITION BY: Defines the grouping column(s).
• ORDER BY: Specifies the sorting within each partition.

4.5.4 4. Key Window Functions

[Link] 4.1 ROW_NUMBER()

[Link].1 Definition
• Assigns a unique sequential integer to each row within a partition.
• No ties allowed (even if values are identical, each row gets a distinct number).

[Link].2 Use Case

• Creating unique identifiers within groups (e.g., ranking employees by salary per department).

SELECT
EmployeeID,
Name,
Salary,
DepartmentID,
ROW_NUMBER() OVER (
PARTITION BY DepartmentID
ORDER BY Salary DESC
) AS RowNum
FROM employees;

143
[Link].3 Example Query
• Output:
– Rows are numbered 1, 2, 3,… per department, ordered by descending salary.
– Highest salary in each department gets RowNum = 1.

[Link].4 Applications
• Identifying the top earner in each department.
• Enumerating latest transactions per customer.

[Link] 4.2 RANK()

[Link].1 Definition
• Assigns a rank to each row within a partition.
• Ties receive the same rank, and subsequent ranks are skipped (e.g., two rank 1s → next rank is 3).

[Link].2 Use Case

• Ranking employees by salary, where ties share ranks.

SELECT
EmployeeID,
Name,
Salary,
DepartmentID,
RANK() OVER (
PARTITION BY DepartmentID
ORDER BY Salary DESC
) AS Rank
FROM employees;

[Link].3 Example Query

• Output:
– Employees with identical salaries get the same rank.
– Example: Two employees with salary = $100,000 → both get Rank = 1; next employee gets Rank =
3.

[Link].4 Applications
• Performance leaderboards (e.g., sales rankings).
• Competitive analysis (e.g., product popularity).

[Link] 4.3 NTILE(n)

[Link].1 Definition
• Divides rows into n approximately equal groups (tiles).
• Each group is assigned a number from 1 to n.

144
[Link].2 Use Case
• Segmenting data into quartiles, percentiles, or custom groups.

SELECT
EmployeeID,
Name,
Salary,
DepartmentID,
NTILE(2) OVER (
PARTITION BY DepartmentID
ORDER BY Salary DESC
) AS Tile
FROM employees;

[Link].3 Example Query

• Output:
– Employees in each department are split into 2 groups (high/low salary).
– Group 1: Top 50% salaries; Group 2: Bottom 50%.

[Link].4 Applications
• Customer segmentation (e.g., high-value vs. low-value clients).
• Risk stratification (e.g., dividing patients by health metrics).

[Link] 4.4 FIRST_VALUE() and NTH_VALUE(n)

[Link].1 4.4.1 FIRST_VALUE()

• Returns the first value in an ordered partition.
• Example: Finding the highest salary per department.

SELECT
EmployeeID,
Name,
Salary,
DepartmentID,
FIRST_VALUE(Salary) OVER (
PARTITION BY DepartmentID
ORDER BY Salary DESC
) AS HighestSalary
FROM employees;

[Link].2 Example Query

• Output:
– For each department, HighestSalary shows the top salary.

145
[Link].3 4.4.2 NTH_VALUE(n)
• Returns the value of the nth row in the ordered partition.
• Example: Finding the second-highest salary.

SELECT
EmployeeID,
Name,
Salary,
DepartmentID,
NTH_VALUE(Salary, 2) OVER (
PARTITION BY DepartmentID
ORDER BY Salary DESC
) AS SecondHighestSalary
FROM employees;

[Link].4 Example Query

[Link].5 Applications
• Benchmarking (e.g., comparing against top performers).
• Anomaly detection (e.g., identifying outliers).

[Link] 4.5 CUME_DIST()

[Link].1 Definition
• Computes the cumulative distribution of a value within a partition.
• Returns a value between 0 and 1, representing the percentage of rows <= current row.

[Link].2 Use Case

• Analyzing relative standing (e.g., salary percentiles per department).

SELECT
EmployeeID,
Name,
Salary,
DepartmentID,
CUME_DIST() OVER (
PARTITION BY DepartmentID
ORDER BY Salary DESC
) AS CumulativeDist
FROM employees;

[Link].3 Example Query

• Output:
– A value of 0.75 means the employee’s salary is higher than 75% of their department.

146
[Link].4 Applications
• Performance percentiles (e.g., “Top 10% earners”).
• Equity analysis (e.g., salary distribution fairness).

[Link] 4.6 PERCENT_RANK()

[Link].1 Definition
• Calculates the relative rank of a row as a percentage (0 to 1).
• Formula:
(rank - 1) / (total rows - 1)

[Link].2 Use Case

• Determining percentage-based rankings (e.g., “This employee is in the top 5%”).

SELECT
EmployeeID,
Name,
Salary,
DepartmentID,
PERCENT_RANK() OVER (
ORDER BY Salary DESC
) AS PercentRank
FROM employees;

[Link].3 Example Query

• Output:
– Highest salary → PercentRank = 0.
– Lowest salary → PercentRank = 1.

[Link].4 Applications
• Compensation analysis (e.g., identifying underpaid employees).
• Market positioning (e.g., product ranking by revenue).

4.5.5 5. Practical Applications of Window Functions

[Link] 5.1 Ranking Employees by Salary
• Functions: ROW_NUMBER(), RANK(), DENSE_RANK().
• Use Case:
– Identify top earners per department.
– Analyze salary distribution across the organization.

[Link] 5.2 Cumulative Sales Totals

• Function: SUM() OVER (ORDER BY date).
• Use Case:

147
 – Track running sales totals per representative.
– Identify sales trends over time.

[Link] 5.3 Moving Averages of Stock Prices

• Function: AVG() OVER (ORDER BY date ROWS BETWEEN 2 PRECEDING AND CURRENT ROW).
• Use Case:
– Smooth price fluctuations for trend analysis.
– Support investment decisions.

[Link] 5.4 Customer Segmentation

• Function: NTILE(4).
• Use Case:
– Divide customers into quartiles by purchase history.
– Tailor marketing strategies to each segment.

4.5.6 6. Summary of Key Concepts

Function Purpose Example Use Case

ROW_NUMBER() Assigns unique sequential numbers. Ranking employees per
department.
RANK() Assigns ranks with gaps for ties. Sales performance leaderboards.
NTILE(n) Divides data into n equal groups. Customer segmentation (e.g.,
quartiles).
FIRST_VALUE() Returns the first value in a partition. Highest salary per department.
CUME_DIST() Computes cumulative distribution (0 to 1). Salary percentiles.
PERCENT_RANK() Calculates relative rank as a percentage. Identifying top 10% performers.

[Link] 6.1 Key Takeaways

1. Window functions preserve individual rows while performing calculations.
2. PARTITION BY enables group-specific analysis without GROUP BY.
3. Common functions include:
• Ranking: ROW_NUMBER(), RANK(), DENSE_RANK().
• Aggregations: SUM(), AVG() with OVER().
• Distribution: CUME_DIST(), PERCENT_RANK().
• Segmentation: NTILE().
4. Applications span finance (moving averages), HR (salary analysis), and marketing (customer segmentation).

4.6 Left Join & Right Join

4.6.1 Introduction to Left and Right Joins
• Objective: Understand the definition, use cases, and practical applications of LEFT JOIN and RIGHT
JOIN in SQL.
• Key Skills:
– Execute LEFT JOIN and RIGHT JOIN operations.
– Combine joins with GROUP BY for advanced queries.
– Differentiate between LEFT JOIN, RIGHT JOIN, and INNER JOIN.

148
4.6.2 1. Left Join (LEFT OUTER JOIN)
[Link] 1.1 Definition
• A LEFT JOIN combines rows from two tables:
– All records from the left table (Table 1).
– Matching records from the right table (Table 2).
• If no match is found in the right table, the result contains NULL for the right table’s columns.

SELECT columns
FROM table1
LEFT JOIN table2 ON [Link] = [Link];

[Link] 1.2 Syntax

• Explanation:
– table1 = Left table (all records included).
– table2 = Right table (only matching records included; NULL if no match).

[Link] 1.3 Example Scenario

• Tables:
– Employees (Left table: EmployeeID, Name, Salary, DepartmentID).
– Departments (Right table: DepartmentID, DepartmentName).
• Query:
SELECT EmployeeID, Name, Salary, DepartmentName
FROM Employees
LEFT JOIN Departments ON [Link] = [Link];

• Result:
– All employees are listed, including those without a department (e.g., Eve with DepartmentName =
NULL).

[Link] 1.4 Use Cases

• Data Completeness: Ensure all records from the left table are included, even without matches.
– Example: List all customers, including those who never placed an order.
• Reporting: Generate reports where left table data is mandatory, and right table data is optional.

[Link] 1.5 Left Join with GROUP BY

• Objective: Calculate total salary per department.
• Query:
SELECT DepartmentName, SUM(Salary) AS TotalSalary
FROM Employees
INNER JOIN Departments ON [Link] = [Link]
GROUP BY DepartmentName;

• Key Points:

149
 – INNER JOIN ensures only matching records are included.
– GROUP BY aggregates data by DepartmentName.
– SUM() calculates the total salary for each department.

[Link] 1.6 Left Join with WHERE Clause (Filtering NULLs)

• Objective: Find employees without a department.
• Query:
SELECT EmployeeID, Name, Salary
FROM Employees
LEFT JOIN Departments ON [Link] = [Link]
WHERE [Link] IS NULL;

• Result: Only employees with no matching department (e.g., Eve).

4.6.3 2. Right Join (RIGHT OUTER JOIN)

[Link] 2.1 Definition
• A RIGHT JOIN combines rows from two tables:
– All records from the right table (Table 2).
– Matching records from the left table (Table 1).
• If no match is found in the left table, the result contains NULL for the left table’s columns.

SELECT columns
FROM table1
RIGHT JOIN table2 ON [Link] = [Link];

[Link] 2.2 Syntax

• Explanation:
– table1 = Left table (only matching records included).
– table2 = Right table (all records included; NULL if no match).

[Link] 2.3 Example Scenario

• Tables:
– Employees (Left table: EmployeeID, Name, Salary, DepartmentID).
– Departments (Right table: DepartmentID, DepartmentName).
• Query:
SELECT EmployeeID, Name, Salary, DepartmentName
FROM Employees
RIGHT JOIN Departments ON [Link] = [Link];

• Result:
– All departments are listed, including those without employees (e.g., Sales department with Employ-
eeID = NULL).

150
[Link] 2.4 Use Cases
• Data Completeness: Ensure all records from the right table are included, even without matches.
– Example: List all departments, including those without employees.
• Reporting: Generate reports where right table data is mandatory, and left table data is optional.

4.6.4 3. Comparing LEFT JOIN, RIGHT JOIN, and INNER JOIN

Join
Type Description Syntax Example Key Difference
LEFT All records from left table + SELECT * FROM table1 LEFT Includes all left table rows,
JOIN matching records from right JOIN table2 ON [Link] = even without matches.
table. [Link];
RIGHT All records from right table + SELECT * FROM table1 RIGHT Includes all right table rows,
JOIN matching records from left JOIN table2 ON [Link] = even without matches.
table. [Link];
INNER Only matching records from SELECT * FROM table1 INNER Excludes non-matching rows
JOIN both tables. JOIN table2 ON [Link] = from both tables.
[Link];

[Link] 3.1 When to Use Each Join

Join Type Use Case

LEFT JOIN When all records from the left table must appear (e.g., all customers).
RIGHT JOIN When all records from the right table must appear (e.g., all departments).
INNER JOIN When only matching records are needed (e.g., employees with departments).

[Link] 3.2 Key Observations

• LEFT JOIN and RIGHT JOIN are mirror images of each other.
– Swapping table order in a LEFT JOIN makes it equivalent to a RIGHT JOIN.
• INNER JOIN is more restrictive—it excludes non-matching rows.
• LEFT/RIGHT JOINs are essential for preserving data integrity in reports.

4.6.5 4. Practical Applications & Advanced Queries

[Link] 4.1 Left Join with Aggregation (GROUP BY)
• Use Case: Calculate total sales per customer, including customers with no orders.
• Query:
SELECT [Link], COALESCE(SUM([Link]), 0) AS TotalSpent
FROM Customers
LEFT JOIN Orders ON [Link] = [Link]
GROUP BY [Link];

• Key Functions:
– COALESCE() replaces NULL with 0 for customers with no orders.

151
[Link] 4.2 Right Join for Unassigned Records
• Use Case: Find departments with no employees.
• Query:
SELECT DepartmentName, COUNT(EmployeeID) AS EmployeeCount
FROM Employees
RIGHT JOIN Departments ON [Link] = [Link]
GROUP BY DepartmentName;

• Result: Departments with EmployeeCount = 0 have no assigned employees.

[Link] 4.3 Combining Joins with WHERE for Filtering

• Use Case: Find employees in a specific department (e.g., “HR”).
• Query:
SELECT EmployeeID, Name
FROM Employees
LEFT JOIN Departments ON [Link] = [Link]
WHERE [Link] = 'HR';

• Note: Using WHERE with LEFT JOIN filters after the join, ensuring only HR employees are listed.

4.6.6 5. Common Pitfalls & Best Practices

[Link] 5.1 Pitfalls
1. Accidental Data Loss with INNER JOIN:
• Using INNER JOIN when LEFT/RIGHT JOIN is needed excludes non-matching records.
2. Ambiguous Column Names:
• Always qualify columns (e.g., [Link] vs. [Link]).
3. Performance Issues:
• Joins on non-indexed columns can slow queries.

[Link] 5.2 Best Practices

1. Use LEFT JOIN for Primary Table Focus:
• If the left table is the primary focus, use LEFT JOIN.
2. Avoid RIGHT JOIN (Use LEFT JOIN Instead):
• Most queries can be rewritten with LEFT JOIN for better readability.
3. Test with Small Datasets:
• Verify join logic before running on large databases.
4. Use Aliases for Clarity:
• Example: FROM Employees e LEFT JOIN Departments d ON [Link] = [Link].

4.6.7 6. Summary & Key Takeaways

• LEFT JOIN:
– Returns all left table records + matching right table records (NULL if no match).
– Use for preserving left table data (e.g., all customers).
• RIGHT JOIN:
– Returns all right table records + matching left table records (NULL if no match).

152
 – Use for preserving right table data (e.g., all departments).
• INNER JOIN:
– Returns only matching records from both tables.
– Use for strictly related data (e.g., employees with departments).
• Advanced Use Cases:
– Combine joins with GROUP BY, WHERE, and aggregate functions (e.g., SUM, COUNT).
– Filter NULL results to find unmatched records.
End of Notes

4.7 One-to-Many Relationships

4.7.1 Introduction to One-to-Many Relationships
[Link] Definition
• A one-to-many relationship exists when a single record in one table (parent table) is associated with
multiple records in another table (child table).
• This is a fundamental concept in relational database design, enabling efficient data organization and re-
trieval.

[Link] Key Characteristics

• Parent Table (One Side):
– Contains the primary key (PK).
– Represents the “one” in the relationship (e.g., a single customer).
• Child Table (Many Side):
– Contains a foreign key (FK) that references the parent table’s primary key.
– Represents the “many” in the relationship (e.g., multiple orders per customer).

[Link] Example: Customers and Orders

• Customers Table (Parent):
– CustomerID (Primary Key)
– FirstName, LastName
• Orders Table (Child):
– OrderID (Primary Key)
– Amount, CustomerID (Foreign Key referencing CustomerID in Customers)
• Relationship:
– One customer (e.g., John, CustomerID = 1) can place multiple orders (e.g., OrderID = 101, 103,
105).
– Each order is linked to one and only one customer via the foreign key.

4.7.2 Importance of One-to-Many Relationships

[Link] 1. Efficient Data Organization and Retrieval
• Structured Storage:
– Data is stored in a normalized manner, reducing redundancy.
– Example: Storing customer details once (in Customers) and linking orders (in Orders) via CustomerID
avoids duplicating customer data.
• Query Efficiency:
– Enables fast retrieval of related records.

153
 – Example: Fetching all orders for a specific customer is optimized by the foreign key relationship.

[Link] 2. Data Integrity and Consistency

• Referential Integrity:
– Foreign keys enforce constraints to ensure that a child record (e.g., an order) must reference a valid
parent record (e.g., an existing customer).
– Example: An order with CustomerID = 99 cannot exist if no customer with CustomerID = 99 exists.
• Prevention of Orphaned Records:
– Ensures no child record is left “dangling” without a parent.

[Link] 3. Modeling Real-World Scenarios

• Naturally represents hierarchical relationships common in real life:
– Library System: One author → Many books.
– School System: One teacher → Many classes.
– E-Commerce: One customer → Many orders.

4.7.3 Real-World Examples of One-to-Many Relationships

[Link] 1. Customers and Orders
• Scenario: A single customer can place multiple orders.
• Tables:
– Customers (CustomerID, Name)
– Orders (OrderID, Amount, CustomerID [FK])
• Relationship: One customer → Many orders.

[Link] 2. Authors and Books

• Scenario: One author can write multiple books.
• Tables:
– Authors (AuthorID, Name)
– Books (BookID, Title, AuthorID [FK])
• Example: J.K. Rowling (AuthorID = 1) is linked to all books in the Harry Potter series.

[Link] 3. Teachers and Classrooms

• Scenario: One teacher can teach multiple classes.
• Tables:
– Teachers (TeacherID, Name)
– Classes (ClassID, Subject, TeacherID [FK])
• Example: A math teacher (TeacherID = 5) teaches Algebra, Geometry, and Calculus.

4.7.4 Implementing One-to-Many Relationships in SQL

[Link] Step 1: Create the Parent Table
• The parent table contains the primary key that the child table will reference.
• Example: Customers Table

154
 CREATE TABLE Customers (
CustomerID INT PRIMARY KEY,
CustomerName VARCHAR(100)
);

– CustomerID: Primary key (uniquely identifies each customer).

– CustomerName: Stores the customer’s name (max 100 characters).

[Link] Step 2: Create the Child Table with a Foreign Key

• The child table includes a foreign key that references the parent table’s primary key.
• Example: Orders Table
CREATE TABLE Orders (
OrderID INT PRIMARY KEY,
OrderDate DATE,
CustomerID INT,
FOREIGN KEY (CustomerID) REFERENCES Customers(CustomerID)
);

– OrderID: Primary key (uniquely identifies each order).

– OrderDate: Stores the date the order was placed.
– CustomerID: Foreign key referencing CustomerID in Customers.
– Foreign Key Constraint:
* Ensures CustomerID in Orders must exist in Customers.
* Syntax: FOREIGN KEY (child_column) REFERENCES ParentTable(parent_column).
[Link] Key Points About Foreign Keys
1. Enforces Relationships:
• Links the child table to the parent table.
• Example: CustomerID in Orders must match a CustomerID in Customers.
2. Maintains Data Integrity:
• Prevents invalid operations, such as:
– Inserting an order with a non-existent CustomerID.
– Deleting a customer who has existing orders (unless cascading deletes are configured).
3. Reflects Real-World Constraints:
• Ensures database relationships mirror real-world logic (e.g., an order must belong to a valid customer).

4.7.5 Querying One-to-Many Relationships with Joins

[Link] Purpose of Joins
• Combines data from related tables based on the foreign key relationship.
• Essential for retrieving parent-child data in a single result set.

[Link] Example: Retrieving All Orders for a Customer

• SQL Query:
SELECT
[Link],

155
 [Link],
[Link]
FROM
Customers
JOIN
Orders ON [Link] = [Link];

• Breakdown:
– SELECT: Specifies columns to retrieve (CustomerName, OrderID, OrderDate).
– FROM Customers: Primary table (parent).
– JOIN Orders: Secondary table (child).
– ON [Link] = [Link]: Join condition (matches foreign key to primary
key).
• Result:
– A list of customers with their corresponding orders (e.g., John → Order 101, Order 103).

[Link] Types of Joins (Relevant to One-to-Many)

1. INNER JOIN:
• Returns only matching rows from both tables.
• Example: Customers with at least one order.
2. LEFT JOIN:
• Returns all rows from the parent table and matching rows from the child table (or NULL if no match).
• Example: All customers, including those with no orders.

4.7.6 Summary of Key Concepts

Concept Description
One-to-Many One parent record → Many child records.
Foreign Key Column in child table referencing the parent’s primary key.
Referential Integrity Ensures foreign key values match existing primary keys.
Join Operation Combines tables based on foreign key relationships for data retrieval.
Normalization Reduces redundancy by separating data into related tables.

[Link] Best Practices

1. Always define foreign keys to enforce relationships.
2. Use meaningful column names (e.g., CustomerID instead of ID).
3. Index foreign keys to improve join performance.
4. Consider cascading actions (e.g., ON DELETE CASCADE) for automatic updates/deletes.

4.8 Practical Applications of Window Functions

4.8.1 1. Introduction to Window Functions
[Link] 1.1 Definition and Purpose
• Window functions perform calculations across a set of table rows related to the current row.

156
 • Unlike aggregate functions (e.g., SUM, AVG), window functions do not collapse rows into a single output row.
Instead, each row retains its individual identity while computations are applied over a defined “window”
of rows.

[Link] 1.2 Common Use Cases Window functions are widely used for: - Ranking (e.g., assigning ranks to
employees based on salary). - Aggregating (e.g., calculating running totals or department-wise sums). - Moving
averages (e.g., computing a 3-month rolling average of sales). - Cumulative sums (e.g., tracking year-to-date
revenue).

4.8.2 2. Combining Multiple Window Functions in a Single Query

[Link] 2.1 Motivation
• Combining multiple window functions allows deeper data insights without requiring multiple queries or
complex joins.
• Example: Calculating total salary, average salary, and ranking employees within a department in one
query.

SELECT
employee_id,
name,
department_id,
salary,
SUM(salary) OVER (PARTITION BY department_id) AS total_department_salary,
AVG(salary) OVER (PARTITION BY department_id) AS avg_department_salary,
ROW_NUMBER() OVER (PARTITION BY department_id ORDER BY salary DESC) AS salary_rank
FROM
employees;

[Link] 2.2 Example Query

[Link].1 Breakdown of the Query:

1. SUM(salary) OVER (PARTITION BY department_id)
• Computes the total salary per department.
2. AVG(salary) OVER (PARTITION BY department_id)
• Computes the average salary per department.
3. ROW_NUMBER() OVER (PARTITION BY department_id ORDER BY salary DESC)
• Assigns a unique rank to each employee within their department, ordered by highest salary first.

[Link].2 Key Observations:

• The PARTITION BY clause groups rows by department.
• The ORDER BY clause sorts employees by salary before assigning ranks.
• All window functions operate independently but are computed in a single pass over the data.

4.8.3 3. Techniques for Optimizing Window Function Performance

Optimizing window functions is critical for large datasets to avoid excessive computation time. Below are key
strategies:

157
[Link] 3.1 Limit Row Processing
• Use WHERE clauses to filter rows before applying window functions.
• Reduces the number of rows processed, improving efficiency.
• Example:
SELECT
employee_id,
ROW_NUMBER() OVER (ORDER BY salary DESC) AS rank
FROM
employees
WHERE
salary > 50000; -- Filters rows early

[Link] 3.2 Efficient Partitioning

• Choose PARTITION BY columns wisely to minimize the number of partitions.
• Fewer partitions = less overhead in managing window frames.
• Example:
– Good: PARTITION BY department_id (if departments are few).
– Bad: PARTITION BY employee_id (each row becomes its own partition, defeating the purpose).

[Link] 3.3 Maximize Sorting Efficiency

• Use ORDER BY judiciously to avoid unnecessary sorting.
• If the data is pre-sorted (e.g., via an index), the database can skip sorting steps.
• Example:
-- If an index exists on (department_id, salary), this query benefits:
SELECT
ROW_NUMBER() OVER (PARTITION BY department_id ORDER BY salary) AS rank
FROM
employees;

[Link] 3.4 Avoid Nested Subqueries

• Replace nested subqueries with Common Table Expressions (CTEs) for:
– Better readability.
– Improved performance (CTEs are materialized once and reused).
• Example:
WITH FilteredEmployees AS (
SELECT * FROM employees WHERE salary > 3000
)
SELECT
employee_id,
SUM(salary) OVER (PARTITION BY department_id) AS total_salary,
ROW_NUMBER() OVER (PARTITION BY department_id ORDER BY salary DESC) AS rank

158
 FROM
FilteredEmployees;

– Benefit: The WHERE clause filters rows before window functions are applied.

4.8.4 4. Practical Example: Optimizing Window Functions with CTEs

[Link] 4.1 Scenario
• Goal: Calculate total salary and assign row numbers only for employees earning > £3,000.

WITH FilteredEmployees AS (
SELECT * FROM employees WHERE salary > 3000
)
SELECT
employee_id,
department_id,
salary,
SUM(salary) OVER (PARTITION BY department_id) AS total_department_salary,
ROW_NUMBER() OVER (PARTITION BY department_id ORDER BY salary DESC) AS salary_rank
FROM
FilteredEmployees;

[Link] 4.2 Optimized Query

[Link].1 Why This is Efficient:

1. Early Filtering: The WHERE salary > 3000 clause reduces the dataset before window functions are
applied.
2. CTE Usage: The filtered result is materialized once and reused, avoiding repeated subquery execution.
3. Performance Gain: Fewer rows = faster window function computations.

4.8.5 5. Using Indexes to Enhance Window Function Performance

[Link] 5.1 Role of Indexes
• Indexes accelerate operations involving:
– PARTITION BY columns (for grouping).
– ORDER BY columns (for sorting).
• Without indexes, the database may perform full table scans or expensive sorts.

-- Assume indexes exist on (department_id) and (salary)

SELECT
employee_id,
SUM(salary) OVER (PARTITION BY department_id) AS total_salary,
ROW_NUMBER() OVER (PARTITION BY department_id ORDER BY salary DESC) AS rank
FROM
employees;

[Link] 5.2 Example: Indexed Query

159
[Link].1 Performance Impact:
• The index on department_id speeds up partitioning.
• The index on salary eliminates the need for a full sort when ordering.

[Link] 5.3 Best Practices for Indexing

1. Create indexes on columns used in:
• PARTITION BY.
• ORDER BY.
2. Composite indexes (e.g., (department_id, salary)) can further optimize queries.
3. Avoid over-indexing, as indexes consume storage and slow down writes.

4.8.6 6. Analyzing Query Plans for Optimization

[Link] 6.1 Importance of Query Plans
• The execution plan reveals how the database processes a query.
• Helps identify bottlenecks (e.g., full table scans, inefficient joins).

[Link] 6.2 Using EXPLAIN

• Syntax: EXPLAIN [query] (varies by DBMS; some use EXPLAIN ANALYZE).
• Key Metrics to Review: | Column | Description | |—————–|—————————————————
————————–| | id | Sequence of operations. | | select_type | Type of query (e.g., SIMPLE, SUBQUERY,
DERIVED for CTEs). | | table | Tables involved. | | type | Join/type of access (e.g., ALL = full scan, ref = index
lookup). | | rows | Estimated rows processed. | | Extra | Additional info (e.g., “Using filesort” = inefficient
sorting). |

EXPLAIN
SELECT
ROW_NUMBER() OVER (PARTITION BY department_id ORDER BY salary DESC) AS rank
FROM
employees;

[Link] 6.3 Example: Interpreting a Query Plan

[Link].1 Possible Output (Simplified):

id select_type table type rows Extra

1 SIMPLE employees ALL 1000 Using filesort

[Link].2 Analysis:
• type: ALL → Full table scan (inefficient).
• Using filesort → Sorting in memory/disk (slow for large datasets).
• Optimization: Add an index on (department_id, salary DESC).

[Link] 6.4 Common Bottlenecks and Fixes

160
Bottleneck Solution
Full table scan (ALL) Add an index on filtered/partitioned columns.
Using filesort Ensure ORDER BY columns are indexed.
High rows estimate Narrow the dataset with WHERE.
Nested subqueries Replace with CTEs or joins.

4.8.7 7. Summary of Best Practices

[Link] 7.1 Combining Window Functions
• Use multiple window functions in a single query for complex analytics.
• Example: Compute totals, averages, and ranks simultaneously.

[Link] 7.2 Optimization Techniques

1. Filter Early: Use WHERE to reduce rows before window functions.
2. Partition Wisely: Choose PARTITION BY columns that minimize partitions.
3. Sort Efficiently: Leverage indexes for ORDER BY to avoid sorts.
4. Avoid Nested Subqueries: Use CTEs for clarity and performance.

[Link] 7.3 Indexing Strategies

• Index PARTITION BY and ORDER BY columns.
• Use composite indexes for multi-column window functions.

[Link] 7.4 Query Plan Analysis

• Use EXPLAIN to identify inefficiencies.
• Look for:
– Full table scans (type: ALL).
– Sort operations (Using filesort).
– High row estimates.

[Link] 7.5 Key Takeaways

• Window functions retain row identity while performing calculations.
• Optimization is critical for large datasets.
• CTEs, indexing, and query plans are essential tools for performance tuning.
End of Notes

4.9 Recording of Building Database Applications Week 3 - Live Session on 26-03-27

4.9.1 1. Introduction to SQL Relationships
[Link] 1.1 Overview of Relationships in SQL
• SQL relationships define how tables interact and share data.
• Key types of relationships:
– One-to-Many (1:M)
– Many-to-Many (M:N)
– One-to-One (1:1)
• Relationships are established using foreign keys and joins.

161
4.9.2 2. One-to-Many (1:M) Relationship
[Link] 2.1 Definition
• A one-to-many relationship exists when:
– A single record in Table A (parent) is associated with multiple records in Table B (child).
– Example: One employee can be assigned to many projects.

[Link] 2.2 Implementation Using Foreign Keys

• A foreign key (FK) is a column (or set of columns) in a child table that references the primary key (PK)
of a parent table.
• Purpose:
– Ensures referential integrity (only valid data from the parent table can be used in the child table).
– Prevents orphaned records (child records without a valid parent).

[Link] 2.3 Example: Chemical Factory Database

[Link].1 Parent Table: Chemical

Chemical_Name (PK) Formula Stock_Quantity

A H2O 500
B NaCl 300
C CO2 200

[Link].2 Child Table: Shipment

Shipment_ID (PK) Chemical_Name (FK) Quantity Ship_To

101 A 100 UK
102 B 50 India

• Foreign Key Constraint:

– Chemical_Name in Shipment must exist in the Chemical table.
– Prevents invalid entries (e.g., shipping “Chemical D” if it doesn’t exist in Chemical).

[Link] 2.4 Referential Integrity

• Ensures that:
– A foreign key value in the child table must match a primary key value in the parent table.
– No orphaned records (child records without a parent) can exist.
• Example:
– If Chemical_Name = "D" is inserted into Shipment but does not exist in Chemical, the DBMS rejects
the insertion.

4.9.3 3. Foreign Key Constraints

[Link] 3.1 Definition
• A foreign key is a field (or combination of fields) in one table that references the primary key of another
table.

162
 • Syntax (MySQL):
CREATE TABLE ChildTable (
child_id INT PRIMARY KEY,
parent_id INT,
FOREIGN KEY (parent_id) REFERENCES ParentTable(parent_id)
);

[Link] 3.2 Key Properties

1. Referential Integrity:
• Ensures that a foreign key value must exist in the referenced primary key column.
2. Naming Conventions:
• If not explicitly named, MySQL auto-generates names (e.g., FK_1, FK_2).
• Best practice: Explicitly name foreign keys (e.g., FK_Professor).
3. Multiple Foreign Keys:
• A table can have multiple foreign keys, each referencing different parent tables.

[Link] 3.3 Example: Professor and Course Database

[Link].1 Parent Table: Professor

Professor_ID (PK) Professor_Name

101 Dr. Ashok
102 Dr. Varma
103 Dr. Patel

[Link].2 Child Table: Course_Professor

Course_ID (PK) Professor_ID (FK)

1 101
2 102
3 103

• Foreign Key Constraint:

CREATE TABLE Course_Professor (
Course_ID INT PRIMARY KEY,
Professor_ID INT,
FOREIGN KEY (Professor_ID) REFERENCES Professor(Professor_ID)
);

• Error Case:
– Inserting Professor_ID = 104 in Course_Professor fails because 104 does not exist in Professor.

[Link] 3.4 Dropping Foreign Keys

• Check Existing Constraints:

163
 SHOW CREATE TABLE Course_Professor;

– Output shows auto-generated or explicit constraint names (e.g., CONSTRAINT FK_Professor).

• Drop a Foreign Key:
ALTER TABLE Course_Professor
DROP FOREIGN KEY FK_Professor;

4.9.4 4. Database Normalization

[Link] 4.1 Definition
• Normalization is the process of:
– Eliminating redundancy (duplicate data).
– Preventing anomalies (insertion, update, deletion issues).
• Achieved through normal forms (1NF, 2NF, 3NF, BCNF, etc.).

[Link] 4.2 Problems Without Normalization

[Link].1 Example: University Database

• Table 1: Student (used by Admissions) | Student_ID (PK) | Student_Name | Address | Contact | Par-
ent_Details | |———————|————–|———|———|—————–| | 1 | Alice | NYC | 12345 | John
Doe |
• Table 2: Placement (used by Placement Cell) | Student_ID (FK) | Student_Name | Contact | Experience
| Previous_Package | |———————|————–|———|————|——————| | 1 | Alice | 12345 | 2
years | $50K |

[Link].2 Issues (Anomalies)

1. Redundancy:
• Student_Name and Contact are duplicated in both tables.
2. Update Anomaly:
• If Contact changes in Student but not in Placement, inconsistency occurs.
3. Insertion Anomaly:
• Cannot add a student to Placement without first adding them to Student.
4. Deletion Anomaly:
• Deleting a student from Student orphans their record in Placement.

[Link] 4.3 Normal Forms

[Link].1 4.3.1 First Normal Form (1NF)

• Rule: Every column must contain atomic (indivisible) values.
• Violation Example:
– Storing multiple phone numbers in a single cell (e.g., 12345, 67890).
• Solution:
– Split into separate rows or use a secondary table.

164
[Link].2 4.3.2 Second Normal Form (2NF)
• Prerequisites: Must satisfy 1NF.
• Rule: No partial dependencies (non-key attributes must depend on the entire primary key, not part of it).
• Example:
– Composite Primary Key: (Student_ID, Course_ID) → Grade.
– Violation: If Student_Name depends only on Student_ID (partial dependency).
– Solution: Move Student_Name to a separate Student table.

[Link].3 4.3.3 Third Normal Form (3NF)

• Prerequisites: Must satisfy 2NF.
• Rule: No transitive dependencies (non-key attributes must not depend on other non-key attributes).
• Example:
– Student_ID → Department_ID → Department_Name.
– Violation: Department_Name depends on Department_ID, which depends on Student_ID.
– Solution: Move Department_Name to a Department table.

[Link].4 4.3.4 Boyce-Codd Normal Form (BCNF)

• Stricter than 3NF.
• Rule: For every functional dependency X → Y, X must be a superkey (candidate key).
• Example:
– If Professor_ID → Department and Department → Professor_ID, both must be keys.

4.9.5 5. SQL Joins

[Link] 5.1 Definition
• Joins combine rows from two or more tables based on a related column (usually a foreign key).
• Used to retrieve data from multiple tables without redundancy.

[Link] 5.2 Types of Joins

[Link].1 5.2.1 Cross Join (Cartesian Product)

• Definition: Returns all possible combinations of rows from both tables.
• Use Case: Generating permutations (e.g., assigning all employees to all projects).
• Syntax:
SELECT P.Professor_Name, C.Course_ID
FROM Professor P
CROSS JOIN Course_Professor C;

• Example Output (3 professors × 3 courses = 9 rows): | Professor_Name | Course_ID | |—————-|——

—–| | Dr. Ashok | 1 | | Dr. Ashok | 2 | | … | … |

[Link].2 5.2.2 Inner Join

• Definition: Returns only matching rows from both tables based on a condition.
• Use Case: Retrieving employees assigned to at least one project.

165
 • Syntax:
SELECT P.Professor_Name, C.Course_ID
FROM Professor P
INNER JOIN Course_Professor C ON P.Professor_ID = C.Professor_ID;

• Example Output: | Professor_Name | Course_ID | |—————-|———–| | Dr. Ashok | 1 | | Dr. Varma | 2 |

[Link].3 5.2.3 Left Join (Left Outer Join)

• Definition: Returns all rows from the left table and matching rows from the right table (or NULL if no
match).
• Use Case: Listing all employees, including those without assigned projects.
• Syntax:
SELECT P.Professor_Name, C.Course_ID
FROM Professor P
LEFT JOIN Course_Professor C ON P.Professor_ID = C.Professor_ID;

• Example Output: | Professor_Name | Course_ID | |—————-|———–| | Dr. Ashok | 1 | | Dr. Varma | 2 |

| Dr. Patel | NULL |

[Link].4 5.2.4 Right Join (Right Outer Join)

• Definition: Returns all rows from the right table and matching rows from the left table (or NULL if no
match).
• Use Case: Listing all projects, including those without assigned employees.
• Syntax:
SELECT P.Professor_Name, C.Course_ID
FROM Professor P
RIGHT JOIN Course_Professor C ON P.Professor_ID = C.Professor_ID;

• Example Output: | Professor_Name | Course_ID | |—————-|———–| | Dr. Ashok | 1 | | Dr. Varma | 2 |

| NULL | 3 |

[Link].5 5.2.5 Full Join (Full Outer Join)

• Definition: Returns all rows from both tables, with NULL for non-matching rows.
• Use Case: Combining left and right joins to include all records.
• Syntax (MySQL does not natively support FULL JOIN; use UNION of LEFT and RIGHT joins):
SELECT * FROM Professor P LEFT JOIN Course_Professor C ON P.Professor_ID = C.Professor_ID
UNION
SELECT * FROM Professor P RIGHT JOIN Course_Professor C ON P.Professor_ID = C.Professor_ID;

[Link] 5.3 Table Aliases

• Definition: Short names (AS) for tables to simplify queries.
• Example:

166
 SELECT P.Professor_Name, C.Course_ID
FROM Professor AS P
JOIN Course_Professor AS C ON P.Professor_ID = C.Professor_ID;

4.9.6 6. Window Functions

[Link] 6.1 Definition
• Window functions perform calculations across a set of table rows related to the current row.
• Unlike aggregate functions (GROUP BY), they do not collapse rows.

[Link] 6.2 Common Window Functions

[Link].1 6.2.1 ROW_NUMBER()

• Assigns a unique sequential number to each row within a partition.
• Syntax:
SELECT ROW_NUMBER() OVER (ORDER BY Student_Name) AS RowNum, Student_Name
FROM Student;

• Example Output: | RowNum | Student_Name | |——–|————–| | 1 | Alice | | 2 | Bob |

[Link].2 6.2.2 RANK()

• Assigns a rank to each row, with gaps for ties.
• Example:
– Scores: 25, 25, 24 → Ranks: 1, 1, 3.
• Syntax:
SELECT RANK() OVER (ORDER BY Marks DESC) AS Rank, Student_Name
FROM Student;

[Link].3 6.2.3 DENSE_RANK()

• Assigns a rank without gaps (ties get the same rank, next rank is incremented by 1).
• Example:
– Scores: 25, 25, 24 → Ranks: 1, 1, 2.
• Syntax:
SELECT DENSE_RANK() OVER (ORDER BY Marks DESC) AS DenseRank, Student_Name
FROM Student;

4.9.7 7. Practical Demonstration (SQL Examples)

-- Parent Table
CREATE TABLE Professor (
Professor_ID INT PRIMARY KEY,

167
 Professor_Name VARCHAR(50)
);

-- Child Table with Foreign Key

CREATE TABLE Course_Professor (
Course_ID INT PRIMARY KEY,
Professor_ID INT,
FOREIGN KEY (Professor_ID) REFERENCES Professor(Professor_ID)
);

[Link] 7.1 Creating Tables with Foreign Keys

-- Insert into Parent Table

INSERT INTO Professor VALUES (101, 'Dr. Ashok');
INSERT INTO Professor VALUES (102, 'Dr. Varma');
INSERT INTO Professor VALUES (103, 'Dr. Patel');

-- Insert into Child Table (Valid Foreign Key)

INSERT INTO Course_Professor VALUES (1, 101);
INSERT INTO Course_Professor VALUES (2, 102);

-- Attempt Invalid Insert (Fails)

INSERT INTO Course_Professor VALUES (3, 104); -- Error: No Professor_ID 104

[Link] 7.2 Inserting Data

-- Cross Join (All Combinations)

SELECT P.Professor_Name, C.Course_ID
FROM Professor P
CROSS JOIN Course_Professor C;

-- Inner Join (Matching Rows)

SELECT P.Professor_Name, C.Course_ID
FROM Professor P
INNER JOIN Course_Professor C ON P.Professor_ID = C.Professor_ID;

-- Left Join (All Professors, Even Without Courses)

SELECT P.Professor_Name, C.Course_ID
FROM Professor P
LEFT JOIN Course_Professor C ON P.Professor_ID = C.Professor_ID;

[Link] 7.3 Executing Joins

4.9.8 8. Summary of Key Concepts

168
Concept Definition Example
One-to-Many One record in Table A relates to many in Table Employee → Projects
Relationship B.
Foreign Key Column in child table referencing a primary Professor_ID in Course_Professor
key in parent table. references Professor(Professor_ID).
Referential Ensures foreign key values exist in the Cannot insert Professor_ID=104 if it
Integrity referenced primary key. doesn’t exist in Professor.
1NF All columns contain atomic values. No comma-separated lists (e.g.,
12345,67890).
2NF No partial dependencies (non-key attributes Move Student_Name from (Student_ID,
depend on the entire primary key). Course_ID) → Grade.
3NF No transitive dependencies. Move Department_Name from Student to
Department.
BCNF Stricter 3NF: Only candidate keys determine If A → B and B → A, both must be keys.
other attributes.
Cross Join Returns all possible row combinations. 3 professors × 3 courses = 9 rows.
Inner Join Returns only matching rows. Professors with assigned courses.
Left Join All rows from left table + matching right table All professors, even those without courses.
(or NULL).
Right Join All rows from right table + matching left table All courses, even those without professors.
(or NULL).
Full Join All rows from both tables (with NULL for Combines left and right joins.
non-matches).
Window Perform calculations across a set of rows. ROW_NUMBER(), RANK(), DENSE_RANK().
Functions

4.10 Using PARTITION BY

4.10.1 Introduction to PARTITION BY
• Definition: The PARTITION BY clause is a powerful SQL tool that divides a result set into partitions (seg-
ments).
• Purpose: It groups rows into distinct subsets, allowing window functions to process each partition indepen-
dently.
• Use Case: Essential for performing operations on subsets of data within a large dataset (e.g., ranking em-
ployees by department).

4.10.2 Basic Syntax of PARTITION BY

• Structure:
SELECT
column_name,
window_function() OVER (PARTITION BY column_name)
FROM table_name;

• Components:
– column_name: The column used to partition the data.
– window_function: Any window function (e.g., ROW_NUMBER(), RANK(), SUM()).
– OVER clause: Specifies the partitioning logic.

169
4.10.3 Window Functions Compatible with PARTITION BY
[Link] 1. Row-Level Functions
• ROW_NUMBER():
– Assigns a unique sequential integer to each row within a partition.
– Use Case: Generating unique identifiers (e.g., row IDs per department).
• RANK():
– Assigns a rank to each row within a partition, with gaps for ties.
– Use Case: Ranking items while preserving rank gaps (e.g., employee performance rankings).
• NTILE(n):
– Divides rows into n groups (tiles) and assigns a group number to each row.
– Use Case: Splitting data into percentiles or quartiles.

[Link] 2. Aggregate Functions

• SUM():
– Calculates the total of a numeric column within each partition.
– Example: Total salary per department.
– Query:
SELECT
employee_name,
salary,
department_id,
SUM(salary) OVER (PARTITION BY department_id) AS department_total_salary
FROM employees;
• AVG():
– Computes the average of a numeric column within each partition.
– Example: Average salary per department.
– Query:
SELECT
employee_id,
employee_name,
salary,
department_id,
AVG(salary) OVER (PARTITION BY department_id) AS department_avg_salary
FROM employees;
• COUNT():
– Counts the number of rows within each partition.
– Example: Number of employees per department.
– Query:
SELECT
employee_id,
employee_name,
COUNT(*) OVER (PARTITION BY department_id) AS department_employee_count
FROM employees;
• MIN()/MAX():
– Finds the minimum/maximum value of a numeric column within each partition.
– Example: Lowest and highest salary per department.

170
 – Query:
SELECT
employee_id,
employee_name,
salary,
department_id,
MIN(salary) OVER (PARTITION BY department_id) AS department_min_salary,
MAX(salary) OVER (PARTITION BY department_id) AS department_max_salary
FROM employees;
– Note: Requires numeric data types.

4.10.4 Advanced Partitioning Techniques

[Link] 1. Using the RANGE Clause
• Definition: Defines a window frame based on a range of values relative to the current row.
• Use Case: Dynamic calculations (e.g., running total of salaries in descending order per department).
• Example Query:
SELECT
employee_id,
employee_name,
salary,
department_id,
SUM(salary) OVER (
PARTITION BY department_id
ORDER BY salary DESC
RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
) AS running_total_salary
FROM employees;

• Key Clauses:
– ORDER BY salary DESC: Sorts salaries in descending order.
– RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW: Includes all preceding rows in the
partition.

[Link] 2. Using the ROWS Clause

• Definition: Defines a window frame based on a fixed number of rows relative to the current row.
• Use Case: Fixed-size calculations (e.g., average salary per department using the current row and the previous
row).
• Example Query:
SELECT
employee_id,
employee_name,
salary,
department_id,
AVG(salary) OVER (

171
 PARTITION BY department_id
ORDER BY salary DESC
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW
) AS moving_avg_salary
FROM employees;

• Key Clauses:
– ORDER BY salary DESC: Sorts salaries in descending order.
– ROWS BETWEEN 1 PRECEDING AND CURRENT ROW: Includes the current row and the previous row.

4.10.5 Key Takeaways

• Scope of Functions: All functions (window or aggregate) operate only within their partition, not the entire
table.
• Flexibility: PARTITION BY can be combined with ORDER BY and window framing clauses (RANGE/ROWS) for
advanced analysis.
• Practical Applications:
– Department-level statistics (sum, average, count).
– Ranking and percentiles within groups.
– Running totals or moving averages.

172
5 Module 5: Database Design
5.1 Data Models
5.1.1 1. Introduction to Data Models
[Link] 1.1 Definition and Purpose
• Data models represent the structure, relationships, and constraints of data in a database.
• They serve as a blueprint for designing and managing databases, ensuring consistency and data integrity.
• Data models abstract and organize data to reflect real-world scenarios and business requirements.

[Link] 1.2 Importance of Data Models

• Provide a structured framework for data management and retrieval.
• Facilitate communication between:
– Stakeholders (business analysts, end-users)
– Developers (software engineers, database designers)
– Database administrators (DBAs)
• Act as a common language to bridge the gap between business requirements and technical implementa-
tion.
• Help avoid misunderstandings and ensure the database meets its intended purpose.

5.1.2 2. Types of Data Models

There are three main types of data models:

[Link] 2.1 Conceptual Data Model

• Provides a high-level view of data from a business perspective.
• Identifies key entities and relationships but does not specify:
– Detailed attributes
– Primary keys
– Data types
• Used for initial planning and requirements gathering.

[Link] 2.2 Logical Data Model

• Offers a detailed view of data structure, including:
– Entities (tables)
– Attributes (columns)
– Relationships (how entities interact)
• Independent of any DBMS (Database Management System).
• Defines:
– Primary keys (unique identifiers)
– Foreign keys (relationships between tables)
– Constraints (rules for data integrity)
• Example: Entity-Relationship Diagram (ERD).

[Link] 2.3 Physical Data Model

• Focuses on the actual implementation of the database in a specific DBMS.
• Includes technical details such as:

173
 – Tables (physical storage structures)
– Columns (with defined data types)
– Constraints (e.g., NOT NULL, UNIQUE)
– Indexes (for performance optimization)
– Storage parameters (e.g., partitioning, clustering)
• Tailored to the DBMS (e.g., MySQL, Oracle, SQL Server).

5.1.3 3. Logical vs. Physical Data Models

[Link] 3.1 Key Differences

Aspect Logical Data Model Physical Data Model

Purpose Abstract representation of data structure. Actual implementation in a DBMS.
DBMS Dependency Independent (universal). Dependent (specific to a DBMS).
Details Included Entities, attributes, relationships. Tables, columns, data types, constraints,
indexes.
Example ERD (Entity-Relationship Diagram). SQL CREATE TABLE statements.
Audience Business analysts, developers. Database administrators, developers.

[Link] 3.2 Example Comparison

• Logical Model (ERD):
– Entity: Customer
– Attributes: CustomerID, Name, Email
– Relationship: Customer places Order.
• Physical Model (SQL):
CREATE TABLE Customer (
CustomerID INT PRIMARY KEY,
Name VARCHAR(255) NOT NULL,
Email VARCHAR(255) UNIQUE
);

5.1.4 4. Practical Steps for Creating Data Models

[Link] 4.1 Step-by-Step Process
1. Identify Entities and Relationships
• Determine the main entities (e.g., Customer, Order, Product).
• Define how they relate (e.g., one-to-many, many-to-many).
2. Define Attributes
• Specify attributes for each entity (e.g., CustomerID, Name, Email).
• Identify primary keys (unique identifiers) and foreign keys (relationship links).
3. Create an Entity-Relationship Diagram (ERD)
• Visually represent:
– Entities (rectangles)
– Attributes (ovals/ellipses)
– Relationships (diamonds with connecting lines).
4. Convert ERD to a Physical Model

174
 • Translate the logical model into DBMS-specific structures:
– Define tables and columns.
– Assign data types (e.g., INT, VARCHAR, DATE).
– Implement constraints (e.g., PRIMARY KEY, FOREIGN KEY).
– Optimize for performance (e.g., indexing, partitioning).

5.1.5 5. Example: E-Commerce Data Model

[Link] 5.1 Entities and Attributes

Entity Attributes Key Type

Customer CustomerID, Name, Email CustomerID (PK)
Order OrderID, OrderDate, CustomerID OrderID (PK), CustomerID
(FK)
Product ProductID, Name, Price ProductID (PK)
Payment PaymentID, OrderID, Amount, PaymentDate PaymentID (PK), OrderID
(FK)

[Link] 5.2 Relationships

1. Customer Places Order
• One-to-Many: A customer can place multiple orders.
• Foreign Key: CustomerID in Order table links to Customer.
2. Order Contains Products
• Many-to-Many: An order can include multiple products, and a product can be in multiple orders.
• Resolution: Use an order detail table (junction table) with OrderID and ProductID as foreign keys.
3. Payment for Order
• One-to-One or One-to-Many: An order can have one or more payments.
• Foreign Key: OrderID in Payment table links to Order.

[Link] 5.3 Logical Model (ERD)

• Visual Representation:
– Entities: Customer, Order, Product, Payment.
– Relationships:
* Customer → Order (1:N)
* Order → Product (M:N via order detail)
* Order → Payment (1:N)

-- Customers Table
CREATE TABLE Customer (
CustomerID INT PRIMARY KEY,
Name VARCHAR(255) NOT NULL,
Email VARCHAR(255) UNIQUE
);

-- Orders Table
CREATE TABLE Order (
OrderID INT PRIMARY KEY,

175
 OrderDate DATE NOT NULL,
CustomerID INT,
FOREIGN KEY (CustomerID) REFERENCES Customer(CustomerID)
);

-- Products Table
CREATE TABLE Product (
ProductID INT PRIMARY KEY,
Name VARCHAR(255) NOT NULL,
Price DECIMAL(10, 2) NOT NULL
);

-- Order Details (Junction Table)

CREATE TABLE OrderDetail (
OrderDetailID INT PRIMARY KEY,
OrderID INT,
ProductID INT,
Quantity INT,
FOREIGN KEY (OrderID) REFERENCES Order(OrderID),
FOREIGN KEY (ProductID) REFERENCES Product(ProductID)
);

-- Payments Table
CREATE TABLE Payment (
PaymentID INT PRIMARY KEY,
OrderID INT,
Amount DECIMAL(10, 2) NOT NULL,
PaymentDate DATE,
FOREIGN KEY (OrderID) REFERENCES Order(OrderID)
);

[Link] 5.4 Physical Model (SQL Implementation)

5.1.6 6. Example: Healthcare Data Model

[Link] 6.1 Entities and Attributes

Entity Attributes Key Type

Patient PatientID, Name, DateOfBirth PatientID (PK)
Doctor DoctorID, Name, Specialization DoctorID (PK)
Appointment AppointmentID, PatientID, DoctorID, Date, Type AppointmentID (PK),
PatientID (FK),
DoctorID (FK)
Treatment TreatmentID, AppointmentID, Description, Cost TreatmentID (PK),
AppointmentID (FK)

[Link] 6.2 Relationships

1. Patient Has Appointments

176
 • One-to-Many: A patient can have multiple appointments.
2. Doctor Conducts Appointments
• One-to-Many: A doctor can have multiple appointments.
3. Appointment Involves Treatments
• One-to-Many: An appointment can have one or more treatments.

[Link] 6.3 Key Insights

• Foreign keys ensure data integrity (e.g., PatientID in Appointment links to Patient).
• Specialization in Doctor helps categorize medical expertise.
• Treatment Cost is critical for billing and financial records.

5.1.7 7. Example: Educational Data Model

[Link] 7.1 Entities and Attributes

Entity Attributes Key Type

Student StudentID, Name, Major StudentID (PK)
Course CourseID, Title, Credits CourseID (PK)
Enrollment EnrollmentID, StudentID, CourseID, Grade EnrollmentID (PK),
StudentID (FK), CourseID
(FK)
Instructor InstructorID, Name, Department InstructorID (PK)

[Link] 7.2 Relationships

1. Instructor Teaches Courses
• One-to-Many: An instructor can teach multiple courses.
2. Student Enrolls in Courses
• Many-to-Many: A student can enroll in multiple courses, and a course can have multiple students.
• Resolution: Enrollment table acts as a junction table.
3. Enrollment Links Student and Course
• Foreign Keys: StudentID and CourseID in Enrollment link to Student and Course.

[Link] 7.3 Key Insights

• Grade in Enrollment tracks student performance.
• Department in Instructor helps organize faculty by subject area.

5.1.8 8. Summary of Key Concepts

• Data models define the structure, relationships, and constraints of data in a database.
• Three types of data models:
1. Conceptual (high-level, business-focused).
2. Logical (detailed, DBMS-independent).
3. Physical (implementation-specific, optimized for performance).
• Logical vs. Physical Models:
– Logical: Abstract (ERDs, entities, attributes).
– Physical: Implementation (SQL tables, data types, indexes).
• Steps to Create a Data Model:

177
 1. Identify entities and relationships.
2. Define attributes (primary/foreign keys).
3. Draw an ERD.
4. Convert to a physical model (SQL).
• Industry Examples:
– E-Commerce: Customers, orders, products, payments.
– Healthcare: Patients, doctors, appointments, treatments.
– Education: Students, courses, enrollments, instructors.

5.2 Designing for Scalability and Performance

5.2.1 1. Introduction to Scalability and Performance in Database Design
[Link] 1.1 Overview
• This lecture focuses on advanced database design techniques for scalability and performance.
• Key objectives:
– Understand principles of designing databases for scalability.
– Learn techniques to ensure databases can handle growth.
– Explore performance optimization methods (e.g., indexing, query optimization, partitioning).
– Apply practical examples of these techniques.

5.2.2 2. Scalability in Database Design

[Link] 2.1 Definition of Scalability
• Scalability: The ability of a database to handle increased load (data volume, user traffic) without compro-
mising performance.
• Ensures the database can grow efficiently in terms of:
– Data volume (storage capacity).
– User traffic (concurrent requests).

[Link] 2.2 Importance of Scalability

• Critical for applications expected to grow over time.
• Prevents performance bottlenecks as data/user load increases.
• Maintains a smooth user experience by ensuring consistent performance.

5.2.3 3. Principles of Database Design for Scalability

[Link] 3.1 Horizontal Scaling (Sharding)
• Definition: Distributing data across multiple servers (shards), where each shard handles a subset of the
data.
• Advantages:
– Allows the database to handle large data volumes by adding more servers.
– Improves parallel processing and load distribution.
• Use Case: Ideal for large-scale applications with high growth expectations.

[Link] 3.2 Vertical Scaling

• Definition: Adding more resources (CPU, RAM, storage) to a single server.
• Advantages:

178
 – Simpler to implement than horizontal scaling.
– Suitable for smaller databases with predictable growth.
• Limitations:
– Hardware constraints (maximum capacity of a single machine).
– Not infinitely scalable (eventual performance plateau).

[Link] 3.3 Comparison: Horizontal vs. Vertical Scaling

Aspect Horizontal Scaling Vertical Scaling

Approach Distribute data across servers Upgrade single server resources
Scalability High (theoretically unlimited) Limited by hardware
Complexity High (requires sharding logic) Low (simple hardware upgrade)
Use Case Large, distributed applications Small to medium databases

[Link] 3.4 Replication

• Definition: Creating copies of the database across multiple servers.
• Purposes:
– Load distribution: Read requests can be handled by replicas.
– High availability: Ensures data accessibility even if one server fails.
– Disaster recovery: Redundant copies prevent data loss.
• Types:
– Master-Slave Replication: One primary (master) server for writes, multiple secondary (slave) servers
for reads.
– Multi-Master Replication: Multiple servers handle both reads and writes (complex but highly avail-
able).

[Link] 3.5 Load Balancing

• Definition: Distributing incoming requests across multiple servers to prevent bottlenecks.
• Benefits:
– Enhances performance by avoiding overloading a single server.
– Improves reliability (no single point of failure).
– Ensures high availability during traffic spikes.
• Implementation:
– Uses load balancers (hardware/software) to route requests.
– Can be combined with replication for read-heavy workloads.

5.2.4 4. Techniques to Ensure Database Scalability

[Link] 4.1 Partitioning
• Definition: Dividing a large database/table into smaller, manageable pieces (partitions).
• Benefits:
– Improves query performance by reducing the data scanned per query.
– Distributes load across storage/processing resources.
– Simplifies maintenance (e.g., archiving old data).
• Types of Partitioning:
– Range Partitioning: Data divided by ranges (e.g., dates, IDs).
– List Partitioning: Data divided by predefined lists (e.g., regions, categories).

179
 – Hash Partitioning: Data distributed using a hash function (even distribution).

[Link] 4.2 Data Archiving

• Definition: Moving rarely accessed data to cheaper, slower storage (e.g., cold storage, tapes).
• Benefits:
– Keeps the main database lean and fast.
– Reduces storage costs for active data.
– Improves query performance on frequently accessed data.
• Implementation:
– Use automated policies to archive old records (e.g., data older than 2 years).
– Ensure archived data remains accessible when needed.

[Link] 4.3 Choosing the Right Storage Engine

• Definition: Selecting a database storage engine based on application requirements.
• Examples:
– InnoDB (MySQL):
* Supports transactions (ACID compliance).
* Ideal for write-heavy workloads.
– MyISAM (MySQL):
* Faster for read-heavy workloads (no transaction support).
* Less overhead for simple queries.
• Considerations:
– Transaction support (ACID vs. non-ACID).
– Concurrency (row-level vs. table-level locking).
– Performance characteristics (indexing, caching).

5.2.5 5. Performance Optimization in Databases

[Link] 5.1 Definition of Performance Optimization
• Goal: Enhance the speed and efficiency of database operations by:
– Reducing query response time.
– Improving overall throughput.
• Importance:
– Ensures responsive applications.
– Enhances user experience.
– Reduces server load and resource consumption.

[Link] 5.2 Techniques for Performance Optimization

[Link].1 5.2.1 Indexing

• Definition: Creating indexes to speed up data retrieval by allowing the database to locate rows quickly.
• Types of Indexes:
– B-tree Index: Efficient for range queries (e.g., WHERE age > 30).
– Hash Index: Optimized for exact-match queries (e.g., WHERE id = 123).
• Best Practices:
– Index frequently queried columns (e.g., foreign keys, search fields).
– Avoid over-indexing (slows down writes).

180
 – Use composite indexes for multi-column queries.

[Link].2 5.2.2 Query Optimization

• Definition: Analyzing and improving SQL queries to ensure efficient execution.
• Techniques:
– Avoid SELECT *: Retrieve only necessary columns to reduce data transfer.
– Use EXPLAIN: Analyze query execution plans to identify bottlenecks.
– Optimize Joins: Ensure proper indexing on joined columns.
– Limit Result Sets: Use LIMIT to reduce data processing.
• Example:
-- Inefficient
SELECT * FROM Orders WHERE OrderID = 123;

-- Optimized
SELECT CustomerID, OrderDate FROM Orders WHERE OrderID = 123;

[Link].3 5.2.3 Partitioning (Revisited for Performance)

• Performance Benefits:
– Reduces I/O overhead by scanning only relevant partitions.
– Enables parallel processing (queries run on subsets of data).
• Example (Range Partitioning by Year):
CREATE TABLE Sales (
SaleID INT,
SaleDate DATE,
Amount DECIMAL(10, 2)
)
PARTITION BY RANGE (YEAR(SaleDate)) (
PARTITION p0 VALUES LESS THAN (2020),
PARTITION p1 VALUES LESS THAN (2021),
PARTITION p2 VALUES LESS THAN (2022)
);

– Partition p0: All sales before 2020.

– Partition p1: Sales from 2020 (inclusive) to 2021 (exclusive).
– Partition p2: Sales from 2021 (inclusive) to 2022 (exclusive).

5.2.6 6. Practical Examples

[Link] 6.1 Indexing Example
• Scenario: A retail application with an Orders table where each order is linked to a CustomerID.
• Problem: Without an index, searching for orders by CustomerID requires a full table scan (slow for large
tables).
• Solution: Create an index on CustomerID.

181
 CREATE INDEX idx_customerid ON Orders(CustomerID);

• Result:
– The database uses the index to quickly locate orders for a given customer.
– Significantly faster queries (logarithmic time complexity with B-tree indexes).

[Link] 6.2 Query Optimization Example

• Scenario: Retrieving order details for a specific OrderID.
• Inefficient Query:
SELECT * FROM Orders WHERE OrderID = 123;

– Problem: Retrieves all columns, increasing data transfer and processing.

• Optimized Query:
SELECT CustomerID, OrderDate FROM Orders WHERE OrderID = 123;

– Benefit: Only retrieves necessary columns, reducing overhead.

[Link] 6.3 Partitioning Example

• Scenario: A Sales table with millions of records spanning multiple years.
• Problem: Queries filtering by SaleDate are slow due to full table scans.
• Solution: Partition the table by year.
CREATE TABLE Sales (
SaleID INT,
SaleDate DATE,
Amount DECIMAL(10, 2)
)
PARTITION BY RANGE (YEAR(SaleDate)) (
PARTITION p0 VALUES LESS THAN (2020),
PARTITION p1 VALUES LESS THAN (2021),
PARTITION p2 VALUES LESS THAN (2022)
);

• Result:
– Queries filtering by SaleDate only scan the relevant partition.
– Faster performance and better resource utilization.

5.2.7 7. Summary and Key Takeaways

• Scalability ensures databases can handle growth in data and user traffic without performance degradation.
• Key Principles:
– Horizontal Scaling (Sharding): Distribute data across servers.
– Vertical Scaling: Upgrade single-server resources.
– Replication: Maintain redundant copies for load distribution and availability.
– Load Balancing: Distribute requests to prevent bottlenecks.
• Techniques for Scalability:

182
 – Partitioning: Divide large tables into smaller chunks.
– Data Archiving: Move old data to cheaper storage.
– Storage Engine Selection: Choose based on workload (e.g., InnoDB for transactions).
• Performance Optimization:
– Indexing: Speed up data retrieval.
– Query Optimization: Write efficient SQL (avoid SELECT *, use EXPLAIN).
– Partitioning: Improve query performance by reducing data scans.
• Practical Applications:
– Use indexes on frequently queried columns.
– Optimize queries to reduce data processing.
– Partition tables by logical ranges (e.g., dates).

5.3 Entity-Relationship Diagrams (ERDs) - Advanced

5.3.1 1. Introduction to Advanced ERDs
• Objective: Document database applications using Entity-Relationship Diagrams (ERDs) with advanced
features.
• Learning Outcomes:
– Represent complete attributes in ERDs.
– Map cardinality constraints in ERDs.
– Express weak entity sets in ERDs.

5.3.2 2. Attributes in ERDs

Attributes describe properties of entities and can be categorized based on their structure and behavior.

[Link] 2.1 Simple Attributes

• Definition: Single-value attributes where each entity has only one value for the attribute.
• Examples:
– Social Security Number (SSN)
– Date of Birth (DoB)
• Key Point: No subdivision; atomic in nature.

[Link] 2.2 Composite Attributes

• Definition: Attributes composed of multiple subparts (sub-attributes).
• Example:
– Address → Street, City, State, Zip Code
• Characteristics:
– Can be multivalued (e.g., multiple phone numbers for one entity).
– Subparts can be represented individually in the ERD for clarity.

[Link] 2.3 Derived Attributes

• Definition: Attributes calculated or derived from other attributes.
– Not stored directly; computed on demand.
• Example:
– Age (derived from Date of Birth).
• Purpose:
– Avoid redundancy by storing only base attributes.

183
 – Ensure data consistency (e.g., age updates automatically when DoB changes).
• Representation in ERDs:
– Typically shown with a dashed oval or distinct notation to indicate they are computed.

[Link] 2.4 Representing Complex Attributes in ERDs

• Composite Attributes:
– Broken down into subparts (e.g., Address → Street, City, State, Zip).
– Subparts are connected to the main entity.
• Derived Attributes:
– Marked differently (e.g., dashed lines) to signify they are not stored but computed.
• Visual Example:
– An Address entity may include:
* Street Number
* Street Name
* Apartment Number (if applicable)
* City, State, Zip Code

5.3.3 3. Cardinality in ERDs

Cardinality defines the numerical relationship between entities, specifying how many instances of one entity relate
to instances of another.

[Link] 3.1 Types of Cardinality Four primary types:

1. One-to-One (1:1)
• Definition: Each entity in Set A is associated with at most one entity in Set B, and vice versa.
• Example:
– A Person has one unique Passport, and each Passport belongs to one Person.
• Notation: Line with single arrowheads pointing to both entities.
2. One-to-Many (1:N)
• Definition: Each entity in Set A can be associated with multiple entities in Set B, but each entity in
Set B is associated with at most one in Set A.
• Example:
– A Teacher teaches multiple Courses, but each Course is taught by one Teacher.
• Notation: Line with a single arrow pointing to the “one” side (Set A) and an undirected line to the
“many” side (Set B).
3. Many-to-One (N:1)
• Definition: Multiple entities in Set A are associated with one entity in Set B, but each entity in Set B
can relate to multiple in Set A.
– Note: This is the reverse of One-to-Many.
• Example:
– Multiple Employees work in one Department, but each Department can have multiple Em-
ployees.
• Notation: Same as One-to-Many but with arrow direction reversed.
4. Many-to-Many (M:N)
• Definition: Entities in Set A can be associated with multiple entities in Set B, and vice versa.
• Example:
– A Student enrolls in multiple Courses, and each Course has multiple Students.
• Notation: Undirected lines on both sides (no arrows).

184
[Link] 3.2 Participation Constraints Indicates whether an entity must participate in a relationship.
1. Total Participation (Mandatory)
• Definition: Every entity in the set must participate in the relationship.
• Notation: Double line (||) connecting the entity to the relationship.
• Example:
– A Course Section must belong to a Course (cannot exist independently).
2. Partial Participation (Optional)
• Definition: Only some entities in the set participate in the relationship.
• Notation: Single line (|) connecting the entity to the relationship.
• Example:
– A Professor may or may not advise a Student (optional relationship).

5.3.4 4. Weak Entity Sets

Entities that cannot be uniquely identified by their own attributes and depend on a strong (owner) entity for
identification.

[Link] 4.1 Key Characteristics

• Dependency: Weak entities rely on a strong entity (owner) for existence.
• Identification: Cannot be identified solely by their attributes; require a foreign key from the owner.
• Examples:
– A Course Section depends on a Course (strong entity).
– A Dependent (e.g., child) depends on an Employee (strong entity).

[Link] 4.2 Representation in ERDs

1. Double Rectangle
• Weak entities are depicted with a double-bordered rectangle (distinct from strong entities, which use
a single rectangle).
2. Discriminator Attribute
• A partial key (unique within the context of the owner) that helps identify weak entities.
• Notation: Dashed underline (e.g., SectionNumber for a Course Section).
3. Double Diamond
• The relationship between a weak entity and its strong entity is represented with a double diamond.
• Emphasizes the dependency of the weak entity on the strong entity.

[Link] 4.3 Example: Course and Course Section

• Strong Entity: Course (identified by CourseID).
• Weak Entity: Course Section (identified by CourseID + SectionNumber).
– SectionNumber is the discriminator (dashed underline).
– The relationship is shown with a double diamond connecting Course (strong) to Course Section
(weak).

5.3.5 5. Summary of Key Concepts

Concept Definition Notation/Representation

Simple Attribute Single-value, atomic attribute (e.g., SSN, DoB). Standard oval.

185
Concept Definition Notation/Representation
Composite Attribute with subparts (e.g., Address → Street, Broken into sub-attributes connected
Attribute City). to the entity.
Derived Computed from other attributes (e.g., Age from Dashed oval or distinct marking.
Attribute DoB).
One-to-One (1:1) One entity in A relates to one in B, and vice versa. Line with arrows on both ends.
One-to-Many One entity in A relates to many in B; one in B relates Arrow to “one” side; undirected line
(1:N) to one in A. to “many” side.
Many-to-One Many in A relate to one in B; one in B relates to Reverse of One-to-Many.
(N:1) many in A.
Many-to-Many Many in A relate to many in B, and vice versa. Undirected lines on both sides.
(M:N)
Total Every entity must participate in the relationship. Double line (
Participation
Partial Only some entities participate. Single line (
Participation
Weak Entity Depends on a strong entity for identification. Double rectangle, discriminator
(dashed underline).
Strong Entity Independent entity that owns weak entities. Single rectangle.
Double Diamond Relationship between weak and strong entities. Double-bordered diamond.

5.3.6 6. Practical Implications

• Accurate ERDs ensure:
– Data integrity (correct relationships).
– Non-redundancy (derived attributes avoid duplication).
– Clarity in complex structures (composite attributes broken down).
• Weak entities are critical for modeling dependent objects (e.g., order items depending on orders).

5.4 Entity-Relationship Diagrams (ERDs)

5.4.1 1. Introduction to Entity-Relationship Diagrams (ERDs)
[Link] 1.1 Definition and Purpose
• Entity-Relationship Diagrams (ERDs) are a type of flowchart that visually represent how entities (e.g.,
people, objects, concepts) relate to each other within a system.
• They serve as a visual representation of a database’s data model, illustrating how tables (entities) are
interconnected through relationships.
• Key Purpose:
– Database Design & Documentation: Helps visualize database structure and relationships, making it
easier to communicate design to stakeholders.
– Early Issue Detection: Useful in the initial stages of database design to identify potential inconsis-
tencies or problems, saving time and resources.
– High-Level System View: Provides a conceptual overview of the system’s structure before implemen-
tation.

[Link] 1.2 Terminology Note

• The terms “Entity-Relationship Diagram,” “ER Diagram,” and “ERD” are used interchangeably
throughout the course.

186
5.4.2 2. Core Components of ERDs
ERDs consist of three fundamental components:

[Link] 2.1 Entities

• Definition: Objects or concepts about which data is stored (e.g., real-world objects like people, places, or
abstract concepts).
• Examples:
– In a university database: Instructor, Student, Course.
– In an e-commerce system: Customer, Order, Product.
• Graphical Representation:
– Depicted as a rectangle in the ERD.
– Entity Set: A collection of similar entities (e.g., all students in a university form the Student entity
set).

[Link] 2.2 Attributes

• Definition: Properties or characteristics that describe an entity.
• Examples:
– Instructor entity: ID, Name, Salary.
– Student entity: Student_ID, Name, Total_Credit.
• Graphical Representation:
– Depicted as ovals connected to their respective entities.
– Primary Key Attributes:
* Uniquely identify each entity (e.g., Student_ID for Student).
* Underlined in the ERD to denote their role.
[Link] 2.3 Relationships
• Definition: Associations or interactions between entities.
• Examples:
– In a university: An Instructor advises a Student.
– In e-commerce: A Customer places an Order.
• Graphical Representation:
– Depicted as a diamond connected to the related entities via lines.
– Relationship Attributes (optional):
* Additional data about the relationship (e.g., Date for the Advisor relationship between Instruc-
tor and Student).

5.4.3 3. Steps to Create an ERD

The process of creating an ERD involves four key steps:

[Link] 3.1 Step 1: Identify Entities

• Action: Determine the main objects/concepts in the system being modeled.
• Guidelines:
– Focus on nouns in the system’s description (e.g., “instructor,” “student,” “course”).
– Define the scope of the data to avoid unnecessary complexity.
• Example:
– University system: Entities = Instructor, Student, Course.

187
[Link] 3.2 Step 2: Define Attributes
• Action: List the properties that describe each entity.
• Guidelines:
– Include essential attributes (e.g., identifiers, names, quantities).
– Designate a primary key (unique identifier) for each entity.
• Example:
– Instructor: ID (PK), Name, Salary.
– Student: Student_ID (PK), Name, Total_Credit.

[Link] 3.3 Step 3: Establish Relationships

• Action: Determine how entities interact with each other.
• Guidelines:
– Use verbs to describe relationships (e.g., “advises,” “places,” “contains”).
– Specify cardinality (e.g., one-to-one, one-to-many, many-to-many).
• Example:
– Instructor advises Student (one-to-many).
– Customer places Order (one-to-many).
– Order contains Product (many-to-many).

[Link] 3.4 Step 4: Draw the Diagram

• Action: Visually represent the entities, attributes, and relationships.
• Graphical Conventions:
– Entities: Rectangles.
– Attributes: Ovals (connected to entities).
– Relationships: Diamonds (connected to entities via lines).
– Primary Keys: Underlined attributes.
• Tools:
– UML Diagrams (Unified Modeling Language) for standardized notation.
– MySQL Workbench (for database-specific ERDs).
– Paper/Pen (for quick drafting).

5.4.4 4. Types of Relationships

Relationships in ERDs are categorized by cardinality (how many instances of one entity relate to instances of
another):

[Link] 4.1 One-to-One (1:1)

• Definition: A single instance of Entity A relates to one instance of Entity B.
• Example:
– A Student has one Student_ID_Card.

[Link] 4.2 One-to-Many (1:N)

• Definition: A single instance of Entity A relates to multiple instances of Entity B.
• Example:
– A Customer can place many Orders.
– An Instructor can advise many Students.

188
[Link] 4.3 Many-to-Many (M:N)
• Definition: Multiple instances of Entity A relate to multiple instances of Entity B.
• Example:
– An Order can contain many Products.
– A Product can appear in many Orders.
• Implementation Note:
– Often resolved using a junction table (associative entity) in relational databases.

5.4.5 5. Practical Example: E-Commerce System ERD

[Link] 5.1 Entities and Attributes

Entity Attributes
Customer Customer_ID (PK), Name, Email
Order Order_ID (PK), Order_Date, Customer_ID (FK)
Product Product_ID (PK), Name, Price

[Link] 5.2 Relationships

1. Customer Places Order (One-to-Many):
• A Customer can place many Orders.
• An Order belongs to one Customer.
2. Order Contains Product (Many-to-Many):
• An Order can include multiple Products.
• A Product can appear in multiple Orders.
• Resolution: Requires a junction table (e.g., Order_Product with Order_ID and Product_ID as for-
eign keys).

5.4.6 6. Creating ERDs in MySQL Workbench

[Link] 6.1 From an Existing Database (Reverse Engineering) Steps: 1. Open MySQL Workbench. 2.
Select Database > Reverse Engineer. 3. Choose the target database to analyze. 4. Follow prompts to generate
the ERD automatically. - Purpose: - Helps document and understand existing database structures. - Identifies
areas for improvement or modification.

[Link] 6.2 From Scratch Steps: 1. Open MySQL Workbench > File > New Model. 2. Right-click on the
canvas > Place a New Table. 3. Define Tables: - Enter table names (e.g., Student, Course). - Add columns
(attributes) with data types (e.g., INT, VARCHAR). - Set primary keys (e.g., Student_ID). 4. Define Relationships:
- Use the Foreign Key tool to link tables. - Specify relationship type (1:1, 1:N, M:N). - Set foreign key constraints
(e.g., Course_ID in Enrollment table). 5. Repeat for all necessary tables.
Output: - The ERD will display tables, attributes, and relationships with clear visual indicators for cardinality
(e.g., crow’s foot notation for one-to-many).

5.4.7 7. Key Takeaways

1. ERDs are essential for visualizing and designing databases.
2. Three Core Components:
• Entities (rectangles).
• Attributes (ovals; primary keys underlined).

189
 • Relationships (diamonds with connecting lines).
3. Steps to Create an ERD:
• Identify entities → Define attributes → Establish relationships → Draw the diagram.
4. Relationship Types:
• One-to-one (1:1), one-to-many (1:N), many-to-many (M:N).
5. Tools:
• MySQL Workbench (for database-specific ERDs).
• UML Diagrams (standardized notation).
• Manual Sketching (for quick drafting).
6. Practical Application:
• Use reverse engineering for existing databases.
• Build from scratch for new designs.

5.4.8 8. Summary
• ERDs provide a blueprint for database design, ensuring clarity, consistency, and efficiency.
• Mastery of entities, attributes, and relationships is foundational for effective database modeling.
• MySQL Workbench is a powerful tool for creating and managing ERDs, but alternative methods (e.g.,
pen/paper) are also viable.
• Cardinality (relationship types) must be accurately defined to maintain data integrity.

5.5 Extended ER Models

5.5.1 1. Introduction to Extended ER Models
[Link] 1.1 Overview
• Extended Entity-Relationship (EER) models build upon the basic ER model by introducing advanced con-
cepts to represent complex real-world scenarios more accurately.
• EER models enhance database design by incorporating:
– Generalization
– Specialization
– Aggregation

[Link] 1.2 Importance of Extended ER Models

• Enhanced Modeling Capabilities: Allow for more precise and organized representation of complex
database structures.
• Reduced Redundancy: Facilitate modular and efficient database design by abstracting common features
and relationships.
• Better Abstraction: Support hierarchical and categorical relationships among entities.

5.5.2 2. Basic ER Model vs. Extended ER Model

[Link] 2.1 Basic ER Model Components
• Entities: Objects or concepts (e.g., Student, Course).
• Attributes: Properties of entities (e.g., StudentID, Name).
• Relationships: Associations between entities (e.g., Enrollment between Student and Course).
• Cardinalities: Define relationship constraints (e.g., one-to-many, many-to-many).
• Dependency Types:
– Weak Relationships: Exist only if a related entity exists (e.g., Dependent relies on Employee).

190
 – Strong Relationships: Independent existence (e.g., Department and Employee).

[Link] 2.2 Extended ER Model Enhancements

• Introduces three key concepts absent in basic ER models:
1. Generalization
2. Specialization
3. Aggregation
• Provides more detailed and accurate representations of real-world scenarios.

5.5.3 3. Components of Extended ER Models

[Link] 3.1 Generalization

[Link].1 3.1.1 Definition

• A bottom-up process where multiple specialized entities are combined into a single generalized entity.
• Purpose: Abstracts common attributes/relationships from specialized entities to avoid redundancy.

[Link].2 3.1.2 Key Characteristics

• Combining Entities: Similar entities (e.g., Student, Instructor) are merged into a higher-level entity (e.g.,
Person).
• Hierarchy Inversion: Generalization is the inverse of specialization (abstracting commonalities rather than
detailing specifics).
• Types of Generalization:
– Total Generalization:
* Every entity in the generalized set must belong to at least one specialized set.
* Example: All Employees must be either Managers or Staff.
– Partial Generalization:
* Some entities in the generalized set may not belong to any specialized set.
* Example: Some Vehicles may not be categorized as Cars or Trucks.
[Link].3 3.1.3 Symbolic Representation
• Represented using a triangle pointing upward (from specialized to generalized entities).
• Double line: Total generalization.
• Single line: Partial generalization.

[Link] 3.2 Specialization

[Link].1 3.2.1 Definition

• A top-down process where a generalized entity is divided into specialized subgroups.
• Purpose: Captures specific attributes/relationships unique to subgroups.

[Link].2 3.2.2 Key Characteristics

• Designating Subgroups: Entities in a high-level set (e.g., Person) are divided into lower-level sets (e.g.,
Student, Instructor).
• Low-Level Attributes: Specialized entities may have additional attributes not applicable to the generalized
entity.

191
 – Example:
* Person (generalized) → Name, Address.
* Student (specialized) → StudentID, Major.
* Instructor (specialized) → InstructorID, Department.
[Link].3 3.2.3 Types of Specialization
1. Overlapping Specialization:
• An entity can belong to multiple specialized groups.
• Example: A Person can be both an Employee and a Student.
2. Disjoint Specialization:
• An entity belongs to only one specialized group.
• Example: An Employee can be either an Instructor or a Secretary, but not both.
3. Total Specialization:
• Every entity in the generalized set must belong to at least one specialized set.
• Represented by a double line from the generalized to specialized entities.
4. Partial Specialization:
• Some entities in the generalized set may not belong to any specialized set.
• Represented by a single line.

[Link].4 3.2.4 Symbolic Representation

• Represented using a triangle pointing downward (from generalized to specialized entities).

[Link] 3.3 Aggregation

[Link].1 3.3.1 Definition

• A process where a relationship set is treated as a high-level entity.
• Purpose: Simplifies complex interactions by abstracting relationships into entities.

[Link].2 3.3.2 Key Characteristics

• Abstracting Relationships: A relationship (e.g., Project between Student and Instructor) can be modeled
as an entity.
• Modular Design: Allows relationships to participate in other relationships, reducing redundancy.
– Example:
* Project (relationship between Student and Instructor) → Treated as an entity.
* Project can then participate in Project Funding (relationship with Funding Agent).
• Elimination of Redundancy: Avoids introducing new entities unnecessarily.

[Link].3 3.3.3 Symbolic Representation

• Represented using a double diamond (distinct from standard relationship diamonds).

5.5.4 4. Practical Steps for Creating Extended ER Models

[Link] 4.1 Step 1: Identify Entities and Relationships
• Basic Building Blocks:
– List main entities (e.g., Student, Instructor, Course).
– Define relationships (e.g., Enrollment between Student and Course).

192
 • Example (University Database):
– Entities: Student, Instructor, Course.
– Relationships: Enrollment, Teaches.

[Link] 4.2 Step 2: Apply Generalization and Specialization

1. Identify Commonalities:
• Look for shared attributes/relationships across entities.
• Example: Student and Instructor both have Name and Address → Generalize into Person.
2. Define Specialized Entities:
• Add specific attributes to specialized entities.
• Example:
– Student: StudentID, Major.
– Instructor: InstructorID, Department.

[Link] 4.3 Step 3: Apply Aggregation

1. Identify Relationships for Abstraction:
• Determine which relationships can be treated as entities.
• Example: Project (involving Student and Instructor) → Abstract into an entity.
2. Model Complex Interactions:
• Allow the abstracted entity to participate in new relationships.
• Example: Project entity can relate to Funding Agent via Project Funding.

[Link] 4.4 Step 4: Draw the Extended ER Diagram

• Symbol Guide:
– Entities: Rectangles.
– Attributes: Ovals.
– Relationships: Diamonds.
– Aggregated Entities: Double diamonds.
– Generalization/Specialization: Triangles (direction indicates process).
• Example Diagram Components:
– Person (generalized) → Student, Instructor (specialized).
– Project (aggregated entity) → Linked to Student, Instructor, and Funding Agent.

5.5.5 5. Summary of Key Concepts

Concept Definition Types Symbol

Generalization Bottom-up abstraction of common features into a Total, Partial Triangle
generalized entity. (upward)
Specialization Top-down division of a generalized entity into Overlapping, Disjoint, Triangle
specialized subgroups. Total, Partial (downward)
Aggregation Treating a relationship as a high-level entity. N/A Double diamond

[Link] 5.1 Key Takeaways

• Extended ER models enhance basic ER models by introducing generalization, specialization, and aggre-
gation.
• Generalization reduces redundancy by combining similar entities.

193
 • Specialization captures unique attributes of subgroups.
• Aggregation simplifies complex relationships by abstracting them into entities.
• Symbolic representation is critical for clear and accurate EER diagrams.
End of Notes

5.6 Introduction to Domain Models

5.6.1 1. Overview of Domain Models
[Link] 1.1 Definition
• Domain models represent the concepts, relationships, and data within a specific domain of interest.
• They abstract real-world entities and their interactions within the context of the system being designed.
• A domain model provides a conceptual representation of the system’s structure and behavior, acting as
a bridge between the real world and system design.

[Link] 1.2 Purpose

• Serves as a high-level abstraction of the system’s key components.
• Facilitates clear communication among stakeholders, developers, and designers.
• Ensures a shared understanding of the domain, aligning the system with business requirements and user
needs.

5.6.2 2. Importance of Domain Models

[Link] 2.1 Clarity and Organization
• Provides a clear and organized representation of system components and their relationships.
• Helps define key entities and their interactions in a structured manner.

[Link] 2.2 Stakeholder Communication

• Acts as a common language between:
– Business stakeholders (e.g., managers, clients)
– Developers (e.g., software engineers, database designers)
– Designers (e.g., UX/UI, system architects)
• Ensures all parties have a consistent understanding of the domain.

[Link] 2.3 Blueprint for System Development

• Serves as a foundation for developing:
– Detailed system models
– Database schemas (logical and physical data models)
• Ensures consistency and alignment with business requirements.

[Link] 2.4 Transition from Conceptual Design to Implementation

• Provides a high-level view of the system, aiding in:
– System structure understanding
– Behavioral analysis
– Correct implementation of business logic

194
5.6.3 3. Steps to Create a Domain Model
[Link] 3.1 Identify Key Entities
• Entities are the core objects in the domain (e.g., Customer, Order, Product).
• Each entity should have:
– Attributes (properties, e.g., CustomerID, Name, Email)
– Behaviors (actions/methods, e.g., placeOrder(), updateProfile())

[Link] 3.2 Define Relationships Between Entities

• Relationships describe how entities interact. Common types include:
1. Association – A general connection (e.g., Customer places Order).
2. Aggregation – A “whole-part” relationship where the part can exist independently (e.g., Department
contains Employees).
3. Composition – A strong “whole-part” relationship where the part cannot exist without the whole (e.g.,
Order contains OrderItems).

[Link] 3.3 Use Visual Modeling Tools

• Unified Modeling Language (UML) is the standard for representing domain models.
• UML Class Diagrams help visualize:
– Entities (Classes)
– Attributes
– Relationships
– Multiplicity (e.g., One-to-Many, Many-to-Many)

[Link] 3.4 Example: E-Commerce Domain Model

Entity Attributes Relationships

Customer CustomerID, Name, Email Places → Order
Order OrderID, OrderDate Contains → Product, Paid by → Payment, Shipped
via → Shipping
Product ProductID, Name, Price Belongs to → Order
Payment PaymentID, Amount, Method Associated with → Order
Shipping ShippingID, Method, Status Linked to → Order

Key Interactions: - A Customer places an Order. - An Order contains multiple Products. - A Payment is made
for an Order. - An Order is shipped using a Shipping method.

5.6.4 4. Domain Models Across Industries

Domain models are industry-agnostic but tailored to specific business needs. Below are examples from different
sectors:

[Link] 4.1 Healthcare Domain Model

Entity Attributes Relationships

Patient PatientID, Name, MedicalHistory Has → Appointment, Receives → Prescription

195
Entity Attributes Relationships
Doctor DoctorID, Name, Specialization Schedules → Appointment, Writes →
Prescription
Appointment AppointmentID, Date, Time Involves → Patient & Doctor
Prescription PrescriptionID, Medication, Dosage Issued by → Doctor, For → Patient

[Link] 4.2 Education Domain Model

Entity Attributes Relationships

Student StudentID, Name, EnrollmentDate Enrolls in → Course, Receives → Grade
Course CourseID, Title, Credits Taught by → Instructor, Enrolled by → Student
Instructor InstructorID, Name, Department Teaches → Course, Assigns → Grade
Grade GradeID, Score, Semester Awarded to → Student, For → Course

[Link] 4.3 Finance Domain Model

Entity Attributes Relationships

Account AccountID, Balance, Type Owned by → Customer, Has → Transaction
Customer CustomerID, Name, ContactInfo Owns → Account, Takes → Loan
Transaction TransactionID, Amount, Date Belongs to → Account
Loan LoanID, Amount, InterestRate Associated with → Customer, Has → Payment
Payment PaymentID, Amount, DueDate Linked to → Loan

5.6.5 5. Summary of Key Concepts

[Link] 5.1 Definition Recap
• A domain model is a conceptual representation of a system’s structure and behavior within a specific
domain.
• It abstracts real-world entities and their interactions.

[Link] 5.2 Importance Recap

• Facilitates communication among stakeholders.
• Ensures alignment with business requirements.
• Serves as a blueprint for database and system design.
• Guides implementation by providing a high-level view.

[Link] 5.3 Creation Process Recap

1. Identify key entities (e.g., Customer, Order).
2. Define attributes and behaviors for each entity.
3. Establish relationships (Association, Aggregation, Composition).
4. Use UML diagrams for visualization.

196
[Link] 5.4 Industry Applications
• E-Commerce (Customer, Order, Product)
• Healthcare (Patient, Doctor, Appointment)
• Education (Student, Course, Instructor)
• Finance (Account, Transaction, Loan)

[Link] 5.5 Final Takeaway

• Domain models are essential for understanding system structure and ensuring correct implementation.
• They bridge the gap between real-world problems and technical solutions.

5.7 Normalisation and Denormalisation

5.7.1 1. Introduction to Database Design Techniques
This lecture focuses on normalisation and denormalisation, two fundamental techniques in database design. By
the end of this lecture, students should be able to: - Understand the features of good relational design. - Learn the
concept of functional dependency. - Apply principles of decomposition using functional dependencies. - Recog-
nise and implement normal forms. - Realise the importance of denormalisation for performance optimisation.

5.7.2 2. Features of Good Relational Design

A well-designed relational database exhibits the following key characteristics:

[Link] 2.1 Minimal Redundancy

• Redundancy refers to the unnecessary duplication of data.
• Problems caused by redundancy:
– Anomalies (insertion, update, deletion).
– Increased storage requirements.
– Potential inconsistencies (e.g., conflicting updates to duplicated data).
• Solution: Normalisation reduces redundancy by structuring data efficiently.

[Link] 2.2 Facilitating Updates and Deletions

• A well-designed database allows data modifications without anomalies.
• Updates and deletions should be straightforward and not compromise data integrity.
• Example: If a customer’s address is stored in only one place, updating it requires a single operation.

[Link] 2.3 Support for Data Integrity

• Data integrity ensures that data remains accurate and consistent.
• Constraints (e.g., primary keys, foreign keys) enforce real-world rules.
• Example: A student_id should uniquely identify a student; no duplicate IDs should exist.

[Link] 2.4 Optimal Data Retrieval

• Efficient querying is crucial, especially in large databases.
• Poor design leads to slow queries and performance bottlenecks.
• Goal: Structure data to minimise query complexity while maintaining integrity.

197
5.7.3 3. Functional Dependency (FD)
Functional dependency is a fundamental concept in database normalisation.

[Link] 3.1 Definition

• Attribute B is functionally dependent on Attribute A if, for every valid instance of A, the value of A
uniquely determines the value of B.
• Notation: A → B (read as “A determines B”).
• Example:
– In a Customer table, customer_id → customer_address (each customer ID uniquely determines
their address).

[Link] 3.2 Role in Database Design

• Helps identify relationships between attributes.
• Used to eliminate redundancy by structuring tables based on dependencies.
• Forms the basis for normalisation.

5.7.4 4. Decomposition of Relations

Decomposition involves breaking down a large table into smaller, more manageable tables to reduce redun-
dancy and anomalies.

[Link] 4.1 Types of Decomposition

Type Description Outcome

Lossless Decomposition Original table can be reconstructed by No data loss; preserves integrity.
joining decomposed tables.
Lossy Decomposition Original table cannot be fully reconstructed. Data loss or duplication may
occur.

[Link] 4.2 Example of Lossless Decomposition

• Original Relation R: (A, B, C)
• Decomposed into:
– R1 (A, B)
– R2 (B, C)
• Reconstruction: R1 JOIN R2 (join on B) should return the original R.

[Link] 4.3 Goals of Decomposition

• Preserve data integrity (no loss of information).
• Avoid redundancy (minimise duplicated data).
• Ensure reconstructability (original table can be retrieved via joins).

5.7.5 5. Normalisation
Normalisation is the process of organising data to: - Minimise redundancy. - Improve data integrity. - Reduce
anomalies (insertion, update, deletion).

198
[Link] 5.1 Importance of Normalisation
• Ensures data consistency (no conflicting updates).
• Makes databases easier to maintain (changes are localised).
• Reduces anomalies (e.g., preventing orphaned records).
• Critical for efficient, reliable data storage.

[Link] 5.2 Normalisation Principles (Normal Forms) Normalisation proceeds through stages called normal
forms (NF), each with stricter rules.

[Link].1 5.2.1 First Normal Form (1NF) Rules: 1. Eliminate duplicate columns (no repeating groups). 2.
Create separate tables for related data. 3. Identify each set of related data with a primary key.
Example: - Before 1NF: | OrderID | Customer | Product1 | Product2 | |———|———-|———-|———-| | 101 |
Alice | Laptop | Mouse |
• After 1NF:
– Orders Table: | OrderID | Customer | |———|———-| | 101 | Alice |
– OrderItems Table: | OrderID | Product | |———|———-| | 101 | Laptop | | 101 | Mouse |

[Link].2 5.2.2 Second Normal Form (2NF) Rules: 1. Must be in 1NF. 2. Remove subsets of data that
apply to multiple rows (eliminate partial dependencies). 3. Place them in separate tables and link via foreign
keys.
Example: - Before 2NF (Partial Dependency): | OrderID | Product | Customer | CustomerAddress | |———|—
——-|———-|—————–| | 101 | Laptop | Alice | 123 Main St | | 101 | Mouse | Alice | 123 Main St |
(Here, CustomerAddress depends only on Customer, not the full primary key OrderID + Product.)
• After 2NF:
– Orders Table: | OrderID | Product | |———|———| | 101 | Laptop | | 101 | Mouse |
– Customers Table: | Customer | CustomerAddress | |———-|—————–| | Alice | 123 Main St |

[Link].3 5.2.3 Third Normal Form (3NF) Rules: 1. Must be in 2NF. 2. Eliminate transitive dependencies
(non-key attributes should not depend on other non-key attributes).
Example: - Before 3NF (Transitive Dependency): | StudentID | CourseID | Instructor | InstructorPhone | |———
–|———-|————|—————–| | S001 | C101 | Dr. Smith | 555-1234 |
(Here, InstructorPhone depends on Instructor, not directly on StudentID or CourseID.)
• After 3NF:
– StudentCourses Table: | StudentID | CourseID | Instructor | |———–|———-|————| | S001 | C101
| Dr. Smith |
– Instructors Table: | Instructor | InstructorPhone | |————|—————–| | Dr. Smith | 555-1234 |
Benefits of 3NF: - Eliminates redundancy (e.g., storing InstructorPhone only once). - Ensures data integrity
(updates to InstructorPhone need only one change). - Suitable for most practical database designs.

[Link].4 5.2.4 Boyce-Codd Normal Form (BCNF) Definition: - A stricter version of 3NF. - A table is in
BCNF if: - It is in 3NF. - For every functional dependency X → Y, X must be a superkey (i.e., uniquely identifies
rows).
Example: - Before BCNF: | StudentID | CourseID | Instructor | |———–|———-|————| | S001 | C101 |
Dr. Smith | | S002 | C101 | Dr. Smith |

199
(Here, CourseID → Instructor, but CourseID is not a superkey—multiple students can take the same course.)
• After BCNF:
– StudentCourses Table: | StudentID | CourseID | |———–|———-| | S001 | C101 | | S002 | C101 |
– CourseInstructors Table: | CourseID | Instructor | |———-|————| | C101 | Dr. Smith |
Comparison: BCNF vs. 3NF | Feature | BCNF | 3NF | |——————-|———————————–|—————
——————–| | Strictness | Stricter (all determinants must be superkeys) | Less strict (allows some redundancy)
| | Redundancy | Fully eliminated | Some redundancy may remain | | Use Case | Critical data integrity required |
Practical balance for most apps |
Primary Goals of Normalisation: 1. Achieve BCNF (eliminate all redundancy). 2. Ensure lossless-join decom-
position (original data can be reconstructed). 3. Preserve functional dependencies (maintain data relationships).
4. Balance normalisation and performance (avoid over-normalisation).

5.7.6 6. Denormalisation
Denormalisation is the intentional reversal of normalisation to improve read performance at the cost of redun-
dancy.

[Link] 6.1 Definition

• Combines normalised tables to reduce the number of joins in queries.
• Introduces controlled redundancy to speed up data retrieval.

[Link] 6.2 When to Use Denormalisation

• High-read, low-write environments (e.g., reporting databases).
• Performance-critical scenarios (e.g., real-time analytics).
• Complex queries where normalised tables lead to slow joins.

[Link] 6.3 Techniques for Denormalisation

1. Combining Tables
• Example: Merge Orders and Customers into a single table to avoid joins.
2. Pre-computing Values
• Store aggregated data (e.g., total_sales) to avoid recalculating.
3. Duplicating Data
• Store customer_name in the Orders table to avoid joining with Customers.

[Link] 6.4 Trade-offs of Denormalisation

Benefit Drawback
Faster read operations Increased storage usage
Simpler queries Risk of update anomalies
Reduced join complexity More complex application logic

[Link] 6.5 When to Consider Denormalisation

• Query performance is critical (e.g., dashboards, OLAP systems).
• Write operations are infrequent (reduces risk of inconsistencies).
• Normalised schema leads to overly complex queries.

200
[Link] 6.6 Risks and Mitigations
• Update Anomalies: Redundant data must be kept in sync (use triggers or application logic).
• Storage Overhead: Additional space required for duplicated data.
• Maintenance Complexity: Requires careful handling of redundant data.

5.7.7 7. Summary of Key Concepts

Concept Definition Purpose

Functional A → B means A uniquely determines B. Identifies relationships for normalisation.
Dependency
1NF No repeating groups; atomic values; primary Eliminates duplicate columns.
key defined.
2NF No partial dependencies (non-key attributes Removes redundant data in composite keys.
depend on full primary key).
3NF No transitive dependencies (non-key attributes Ensures all attributes depend only on the
don’t depend on other non-keys). primary key.
BCNF Every determinant is a superkey. Eliminates all redundancy (stricter than 3NF).
Denormalisation Intentional redundancy to improve read Optimises query speed in read-heavy
performance. systems.
Lossless Decomposed tables can be joined to Ensures no data loss during normalisation.
Decomposition reconstruct original data.

5.7.8 8. Conclusion
• Normalisation is essential for data integrity, minimising redundancy, and reducing anomalies.
• Denormalisation is a performance optimisation technique used when read efficiency is prioritised over
write consistency.
• Balancing normalisation and denormalisation is key to designing efficient, scalable database applica-
tions.

5.8 Primary Terminologies Used in Database Design

5.8.1 Introduction to Database Design Terminologies
This lecture introduces the primary terminologies used in database design, which are essential for: - Understand-
ing database structures. - Creating efficient and effective databases. - Applying physical examples to reinforce
key concepts.
By the end of this lecture, learners should be able to: 1. Define primary database design terminologies. 2. Apply
these terms to real-world examples. 3. Understand how these concepts contribute to data integrity, efficiency,
and organization.

5.8.2 1. Entity
[Link] Definition An entity is an object or concept about which data is stored in a database. - Represents a
category of data (e.g., a real-world object, event, or concept). - Serves as a fundamental building block of a
database.

201
[Link] Examples
• In a school database:
– Student (an entity representing individuals enrolled in the school).
– Course (an entity representing classes offered by the school).
• In a business database:
– Customer (an entity representing clients).
– Product (an entity representing items sold).

[Link] Key Characteristics

• Each entity has attributes (properties) that describe it.
• Entities are independent but can be related to other entities.

5.8.3 2. Attribute
[Link] Definition An attribute is a property or characteristic of an entity that stores specific data about it. -
Helps define and describe the data collected for each entity. - Ensures the database is organized and meaningful.

[Link] Examples
• For the Student entity:
– Student ID (unique identifier).
– Name (student’s full name).
– Age (student’s age).
• For the Course entity:
– Course ID (unique identifier).
– Course Name (title of the course).
– Credits (number of credit hours).

[Link] Key Characteristics

• Attributes store actual data values (e.g., “John Doe” for a student’s name).
• They describe the entity in detail, making the database useful for queries and analysis.

5.8.4 3. Relationship
[Link] Definition A relationship describes how entities interact with each other in a database. - Establishes
meaningful connections between entities. - Essential for linking data in a structured way.

[Link] Types of Relationships

1. One-to-One (1:1)
• A single record in Table A relates to only one record in Table B.
• Example: A student has one unique student ID card.
2. One-to-Many (1:N)
• A single record in Table A relates to multiple records in Table B.
• Example: A course can have many students enrolled.
3. Many-to-Many (M:N)
• Multiple records in Table A relate to multiple records in Table B.
• Example: Students can enroll in multiple courses, and courses can have multiple students.

202
[Link] Example in a School Database
• Student (entity) enrolls in (relationship) Course (entity).
– This relationship connects students to their registered courses.
– Helps in retrieving data (e.g., “Which students are enrolled in Database Design?”).

[Link] Key Characteristics

• Relationships define how data is linked across tables.
• They enable complex queries (e.g., joining tables to extract meaningful insights).

5.8.5 4. Primary Key

[Link] Definition A primary key is a unique identifier for a record (row) in a table. - Ensures each record
can be distinctly identified. - Prevents duplicate or null values in the key column.

[Link] Examples
• In the Student table:
– Student ID (e.g., S1001, S1002) is the primary key.
– No two students can have the same Student ID.
• In the Course table:
– Course ID (e.g., CSE101, MATH202) is the primary key.

[Link] Key Characteristics

• Uniqueness: Guarantees no two rows have the same primary key.
• Data Integrity: Ensures accurate and consistent data retrieval.
• Efficiency: Speeds up search and join operations in queries.

5.8.6 5. Foreign Key

[Link] Definition A foreign key is a field in one table that references the primary key of another table. -
Establishes a link between two tables. - Ensures referential integrity (i.e., relationships between tables remain
consistent).

[Link] Example
• In an Enrollment table:
– Student ID (foreign key) refers to the Student table’s primary key.
– Course ID (foreign key) refers to the Course table’s primary key.
• This linkage ensures:
– Each enrollment record is tied to a valid student and valid course.
– Prevents orphaned records (e.g., an enrollment without a corresponding student).

[Link] Key Characteristics

• Maintains relationships between tables.
• Prevents data inconsistencies (e.g., deleting a student while their enrollments still exist).
• Supports complex queries (e.g., finding all courses a student is enrolled in).

203
5.8.7 6. Schema
[Link] Definition A schema is the overall structure of a database, including: - Tables (entities). - Columns
(attributes). - Relationships between tables. - Constraints (e.g., primary keys, foreign keys).

[Link] Purpose
• Serves as a blueprint for how data is stored and organized.
• Helps database designers plan and visualize the database structure.

[Link] Example: School Database Schema

Table Attributes (Columns) Relationships

Student Student ID (PK), Name, Age One-to-many with Enrollment
Course Course ID (PK), Course Name, Credits One-to-many with Enrollment
Enrollment Enrollment ID (PK), Student ID (FK), Many-to-one with Student and Course
Course ID (FK), Grade

[Link] Key Characteristics

• Defines data organization (how tables relate).
• Ensures data integrity (through constraints).
• Guides database development (from design to implementation).

5.8.8 7. Normalization
[Link] Definition Normalization is the process of organizing data to: - Reduce redundancy (duplicate data).
- Improve data integrity (accuracy and consistency). - Minimize anomalies (errors during data operations).

[Link] Process
1. Decompose large tables into smaller, related tables.
2. Eliminate repeating groups (e.g., storing multiple course enrollments in a single student row).
3. Ensure each piece of data is stored in only one place.

[Link] Example: Before and After Normalization

[Link].1 Before (Denormalized Table: Students and Courses)

Student ID Name Age Course 1 Course 2 Course 3

S1001 Alice 20 CSE101 MATH202 PHY103
S1002 Bob 21 CSE101

Problems: - Redundancy: Course names are repeated. - Update Anomaly: If CSE101 changes, multiple rows
must be updated. - Insertion Anomaly: Adding a student with no courses requires null values.

204
[Link].2 After (Normalized Tables) Students Table: | Student ID | Name | Age | |————|——-|—–| | S1001
| Alice | 20 | | S1002 | Bob | 21 |
Courses Table: | Course ID | Course Name | |———–|————-| | CSE101 | Intro to CS | | MATH202 | Calculus
| | PHY103 | Physics |
Enrollment Table: | Enrollment ID | Student ID | Course ID | |—————|————|———–| | E001 | S1001 |
CSE101 | | E002 | S1001 | MATH202 | | E003 | S1001 | PHY103 | | E004 | S1002 | CSE101 |
Benefits: - No redundancy: Each course is stored once. - Easier updates: Changing CSE101 requires updating
only one row in the Courses table. - Flexible inserts: Students can be added without course data.

[Link] Key Characteristics

• Follows normal forms (1NF, 2NF, 3NF, BCNF).
• Reduces data duplication.
• Improves query performance (smaller, optimized tables).

5.8.9 8. Denormalization
[Link] Definition Denormalization is the process of intentionally introducing redundancy by combining
normalized tables to: - Improve read performance (faster queries). - Reduce the need for complex joins.

[Link] When to Use Denormalization

• Read-heavy applications (e.g., reporting, analytics).
• Scenarios where query speed is critical (e.g., e-commerce product listings).
• When storage efficiency is less important than performance.

[Link] Example: Denormalized Student-Enrollment Table

Student ID Name Age Course ID Course Name Grade

S1001 Alice 20 CSE101 Intro to CS A
S1001 Alice 20 MATH202 Calculus B
S1002 Bob 21 CSE101 Intro to CS A-

Trade-offs: - Pros: - Faster reads (no need to join Students, Courses, and Enrollment tables). - Simpler queries
(all data is in one place). - Cons: - Redundancy: Student and course details are repeated. - Update anomalies:
Changing CSE101 requires updating multiple rows. - Storage overhead: More data is stored than necessary.

[Link] Key Characteristics

• Balances performance and integrity.
• Used selectively (not all databases should be denormalized).
• Requires careful planning to avoid excessive redundancy.

5.8.10 Summary of Key Database Design Terminologies

Term Definition Example

Entity Object/concept about which data is stored. Student, Course

205
Term Definition Example
Attribute Property/characteristic of an entity. Student ID, Name, Age
Relationship Describes how entities interact. Student enrolls in Course
Primary Unique identifier for a record in a table. Student ID (S1001)
Key
Foreign Field linking to a primary key in another table. Student ID in Enrollment table
Key
Schema Blueprint of database structure (tables, School database with Students, Courses,
relationships, constraints). Enrollment tables
NormalizationProcess of organizing data to reduce redundancy Splitting Students and Courses into separate
and improve integrity. tables
Combining tables to improve read performance
Denormalization Merging Student and Enrollment data into one
(at the cost of redundancy). table for faster queries

5.8.11 Conclusion
Understanding these primary terminologies is fundamental to: - Designing efficient databases. - Ensuring data
integrity and consistency. - Optimizing performance (through normalization and selective denormalization). -
Creating scalable and maintainable database systems.
These concepts form the foundation of database design and are essential for building real-world applications.

5.9 Recording of Building Database Applications Week 4 - Live Session on 26-04-03

5.9.1 1. Introduction to Database Design Fundamentals
[Link] 1.1 Schema
• Definition: The schema is the overall structure of a database, defining:
– Organization of tables.
– Columns within each table.
– Relationships between tables.
• Analogy: Acts as a blueprint for how data is stored and organized.
• Purpose: Provides a flow of data, ensuring structured storage and retrieval.

5.9.2 2. Normalization and Denormalization

[Link] 2.1 Normalization
• Definition: A process used to:
– Reduce redundancy (duplicate data).
– Improve data integrity (accuracy and consistency).
• Goal: Eliminate anomalies (insertion, update, deletion) by decomposing tables into smaller, related tables.

[Link] 2.2 Denormalization

• Definition: The reverse process of normalization, where normalized tables are combined to:
– Improve read performance (faster queries).
– Reduce joins in complex queries.
• Trade-off: May introduce redundancy but enhances performance for read-heavy applications.

206
5.9.3 3. Data Models
[Link] 3.1 Definition of a Data Model
• Represents the structure, relationships, and constraints of data in a database.
• Serves as a blueprint for:
– Designing the database.
– Managing data.
– Ensuring consistency and data integrity.

[Link] 3.2 Data Consistency vs. Data Integrity

[Link].1 3.2.1 Data Consistency

• Definition: Ensures that data is the same everywhere it is referenced.
– No contradictions across tables or rows.
– Uniformity in data representation.
• Example 1 (Student-Department Inconsistency):
– Student Table: | StudentID | Name | DepartmentID | |———–|——-|————–| | 1 | Ravi | 10 |
– Department Table: | DepartmentID | DepartmentName | |————–|—————-| | 10 | CSE |
– Inconsistency: If another table lists Ravi’s DepartmentID as 20, it violates consistency.
• Example 2 (Bank Account Balance):
– A joint account shows:
* User A’s view: Balance = ₹10,000
* User B’s view: Balance = ₹8,000
– Problem: Same account, different balances without transactions → inconsistency.

[Link].2 3.2.2 Data Integrity

• Definition: Ensures data is:
– Correct (accurate).
– Valid (adheres to rules).
– Reliable (no corruption).
• Types of Integrity Violations:
1. Primary Key Violation:
– Example: StudentID = NULL (primary keys cannot be null).
2. Referential Integrity Violation:
– Example: A Student record references DepartmentID = 50, but no such department exists in
the Department table.

5.9.4 4. Types of Data Models

[Link] 4.1 Conceptual Data Model
• Purpose: Provides a high-level view of data.
• Focus:
– Business logic (what data is needed).
– Key attributes and relationships between entities.
• Exclusions:
– Does not specify:
* Data types.
* Primary/foreign keys.

207
 * Detailed constraints.

[Link] 4.2 Logical Data Model

• Purpose: A detailed view of the database structure.
• Includes:
– Entities, attributes, and relationships.
– Data structures (tables, columns).
– Primary/foreign keys.
– Constraints (e.g., NOT NULL).
• Characteristics:
– Independent of any DBMS (universal design).
– Example: Like a house blueprint—specifies rooms and layout but not construction materials.

[Link] 4.3 Physical Data Model

• Purpose: Focuses on actual implementation of the database.
• Includes:
– Tables, columns, indexes.
– Storage details (e.g., file organizations).
– DBMS-specific optimizations.
• Process: Translates the logical model into an implemented database schema.

5.9.5 5. Entity-Relationship (ER) Diagrams

[Link] 5.1 Definition
• A visual representation of a data model.
• Components:
1. Entities (tables).
2. Attributes (columns).
3. Relationships (links between tables).

[Link] 5.2 Entities

[Link].1 5.2.1 Strong Entity

• Definition: An entity with a primary key.
• Example: Student (with StudentID as primary key).

[Link].2 5.2.2 Weak Entity

• Definition: An entity without a primary key; depends on another entity (strong entity).
• Example: Installment depends on Loan (an installment cannot exist without a loan).
• Representation: Double rectangle in ER diagrams.

[Link] 5.3 Attributes

[Link].1 5.3.1 Types of Attributes

1. Composite Attribute:
• Definition: An attribute composed of sub-attributes.

208
 • Examples:
– Name → FirstName, MiddleName, LastName.
– PhoneNumber → CountryCode, AreaCode, LocalNumber.
• Representation: Oval with sub-ovals.
2. Multivalued Attribute:
• Definition: An attribute with multiple values.
• Examples:
– PhoneNumber (a person may have multiple numbers).
– Address (permanent and temporary).
• Representation: Double oval.
3. Derived Attribute:
• Definition: An attribute calculated from other attributes.
• Example: Age derived from DateOfBirth.
• Characteristics:
– Not stored directly in the table.
– Computed on-the-fly (e.g., via SQL functions).
• Representation: Dashed oval.
4. Prime Attribute:
• Definition: An attribute that is part of a primary key.
• Representation: Underlined.

[Link] 5.4 Relationships

[Link].1 5.4.1 Types of Relationships

1. One-to-One (1:1):
• Definition: One record in Table A relates to exactly one record in Table B.
• Example: Husband ↔ Wife (assuming monogamy).
• Representation: Single line with 1 on both ends.
2. One-to-Many (1:N):
• Definition: One record in Table A relates to many records in Table B.
• Example: Scientist → Inventions (one scientist can invent many things).
• Representation: Single line with 1 on one side and N (or crow’s foot) on the other.
3. Many-to-Many (M:N):
• Definition: Many records in Table A relate to many records in Table B.
• Example: Employee ↔ Project (employees work on multiple projects; projects have multiple em-
ployees).
• Representation: Lines with M or N on both ends.

[Link].2 5.4.2 Participation Constraints

1. Total Participation:
• Definition: Every entity in the set must participate in the relationship.
• Example: A Student must enroll in a Course to exist.
• Representation: Double line (thick line).
2. Partial Participation:
• Definition: Not all entities in the set participate in the relationship.
• Example: A Course (e.g., Civil Engineering) may exist even if no students enroll.
• Representation: Single line.

209
[Link].3 5.4.3 Weak Relationships
• Definition: A relationship where one entity is weak (dependent).
• Example: Loan ↔ Installment (installments depend on loans).
• Representation: Double diamond.

5.9.6 6. Advanced ER Modeling Techniques

[Link] 6.1 Generalization
• Definition: Bottom-up approach to extract common properties from multiple entities.
• Example:
– Student and Faculty → Both are Persons.
– Inheritance: Child entities (Student, Faculty) inherit attributes from the parent (Person).
• Attributes:
– Person: PersonID, Name, Address.
– Student: StudentID, CourseRegistered (inherits Person attributes).
– Faculty: FacultyID, CourseTaught (inherits Person attributes).

[Link] 6.2 Specialization

• Definition: Top-down approach to classify a general entity into specific subtypes.
• Example:
– Employee → Developer, Tester, Manager.
– Doctor → Cardiologist, Pathologist, Gynecologist.
• Characteristics:
– Subtypes inherit attributes from the supertype.
– May have additional specialized attributes.

[Link] 6.3 Aggregation

• Definition: Treats a relationship as an entity to simplify complex relationships.
• Example 1 (Student-Course-Subject):
– Student attends Course → Course has Subjects.
– Problem: ER diagrams cannot directly link relationships.
– Solution: Combine Student + Course into a single entity, then link to Subjects.
• Example 2 (Employee-Project-Missionary):
– Employee works on Project → Project requires Missionary.
– Aggregation: Treat Employee-Project as one entity, then link to Missionary.

5.9.7 7. Transforming ER Diagrams into Database Tables

[Link] 7.1 Step-by-Step Process
1. Identify Entities:
• Each rectangle in the ER diagram → one table.
• Example: Account, Branch, Customer, Loan.
2. Map Attributes to Columns:
• Each oval → column in the table.
• Underlined attributes → Primary Key.
3. Handle Relationships:
• 1:1 or 1:N:

210
 – Add the primary key of the “one” side as a foreign key in the “many” side.
– Example: Account (many) → Branch (one) → Add BranchName (FK) to Account.
• M:N:
– Create a junction table with foreign keys from both tables.
– Example: Depositor (links Account and Customer) with columns AccountNumber (FK) and
CustomerName (FK).
4. Weak Entities:
• Include the primary key of the strong entity as part of the weak entity’s primary key.
• Example: Installment (weak) includes LoanNumber (FK from Loan).

[Link] 7.2 Example: Bank Database ER Diagram

• Entities:
1. Account (AccountNumber [PK], Balance).
2. Branch (BranchName [PK], BranchCity, Assets).
3. Customer (CustomerName [PK], CustomerStreet, CustomerCity).
4. Loan (LoanNumber [PK], Amount).
• Relationships:
1. Depositor (M:N):
– Junction table: Depositor (AccountNumber [FK], CustomerName [FK]).
2. Borrower (M:N):
– Junction table: Borrower (CustomerName [FK], LoanNumber [FK]).
3. Account-Branch (1:N):
– Add BranchName (FK) to Account.
4. Loan-Branch (1:N):
– Add BranchName (FK) to Loan.

5.9.8 8. Advanced Database Design Techniques

[Link] 8.1 Scalability
• Definition: The ability of a database to handle growth without performance degradation.
• Principles:
1. Horizontal Scaling (Sharding):
– Definition: Distribute data across multiple servers.
– Use Case: Large datasets (e.g., social media platforms).
– Benefit: Each server handles a subset of data, improving load distribution.
2. Vertical Scaling:
– Definition: Upgrade hardware (CPU, RAM, storage).
– Use Case: Small to medium databases.
– Limitation: Hardware constraints (cost, physical limits).
3. Replication:
– Definition: Create copies of the database to:
* Distribute read load.
* Improve availability.
* Enable disaster recovery.
4. Load Balancing:
– Definition: Distribute requests across multiple servers.
– Benefit: Prevents single-server overload, improving responsiveness.

[Link] 8.2 Performance Optimization

211
 • Definition: Techniques to enhance speed, reduce query time, and improve user experience.
• Techniques:
1. Indexing:
– Definition: Create indexes on columns to speed up searches.
– Example:
* Without index: Scanning 500 pages to find “Normalization” (slow).
* With index: Directly jump to page 301 (fast).
– SQL Syntax:
CREATE INDEX idx_name ON table_name (column_name);
2. Query Optimization:
– Definition: Write efficient SQL queries to retrieve only necessary data.
– Example:
* Bad: SELECT * FROM Orders (retrieves all columns/rows).
* Good: SELECT CustomerID, OrderDate FROM Orders WHERE OrderID IN (1, 2, 3)
(targeted retrieval).
3. Partitioning:
– Definition: Split large tables into smaller, manageable parts.
– Types:
* Range Partitioning: By date ranges (e.g., orders before/after 2020).
* List Partitioning: By specific values (e.g., regions).
* Hash Partitioning: Distribute data using a hash function.
– Benefit: Faster queries on partitioned subsets.

5.9.9 9. Summary of Key Concepts

Concept Definition Example

Schema Blueprint of database structure (tables, relationships). Student, Course, Enrollment
tables.
Normalization Reduces redundancy, improves integrity. Splitting StudentCourse into
Student + Course.
Denormalization Combines tables for performance. Merging Order + Customer for
faster reads.
Data Consistency Uniform data across all references. Same DepartmentID for a
student everywhere.
Data Integrity Correct, valid, and accurate data. No NULL in primary keys.
ER Diagram Visual model of entities, attributes, relationships. Rectangles (entities), ovals
(attributes).
Generalization Bottom-up extraction of common properties. Student + Faculty → Person.
Specialization Top-down classification of entities. Employee → Developer,
Tester.
Aggregation Treats a relationship as an entity. Student-Course → linked to
Subjects.
Scalability Handles growth via sharding, replication, load Distributing data across servers.
balancing.
Performance Speeds up queries via indexing, partitioning, efficient Creating an index on
Optimization SQL. CustomerID.

212
5.10 Why Database Design is Important
5.10.1 Introduction to Database Design
• Database design is a foundational aspect of building database applications.
• It directly impacts application performance, scalability, and maintainability.
• Poor database design leads to inefficiencies, while good design ensures optimal data management.

5.10.2 Importance of Good Database Design

Good database design is essential for three key reasons:

[Link] 1. Enhances Application Performance

• Efficient data retrieval and storage reduce query execution time.
• Example (E-commerce Application):
– A poorly designed database may lack proper indexing, leading to slow product searches and a poor
user experience.
– Data redundancy (e.g., duplicate customer records) complicates updates and increases errors.
• Solution:
– Optimized indexing speeds up queries.
– Normalization eliminates redundancy, ensuring data integrity and faster operations.

[Link] 2. Supports Scalability

• A well-designed database can handle increasing data volumes and user loads without performance degra-
dation.
• Example:
– As an e-commerce platform grows, a poorly structured database may struggle with high traffic, leading
to crashes or slowdowns.
– A scalable design allows seamless expansion (e.g., adding new tables, optimizing queries).

[Link] 3. Improves Maintainability

• A structured database is easier to update, modify, and debug.
• Example:
– Without proper relationships between tables, updating customer information in multiple places becomes
error-prone and time-consuming.
– A well-normalized database simplifies maintenance by reducing redundancy and ensuring consistent
data updates.

5.10.3 The Database Design Lifecycle

The database design process follows a structured five-stage lifecycle to ensure alignment with business objectives
and user needs.

[Link] 1. Requirement Analysis

• Objective: Gather and document stakeholder requirements to understand data needs.
• Key Activities:
– Identify key entities (e.g., objects or concepts relevant to the system).
– Define attributes (properties of entities).
– Determine relationships between entities.

213
 – Ensure requirements align with business goals and user expectations.
• Example (Library Management System):
– Entities: Books, Authors, Borrowers.
– Attributes:
* Books: Title, ISBN, Publication Year.
* Authors: Name, Author ID.
* Borrowers: Borrower ID, Contact Details.
– Relationships:
* A Borrower borrows Books.
* An Author writes Books.
[Link] 2. Conceptual Design
• Objective: Create a high-level model of the database using an Entity-Relationship Diagram (ERD).
• Key Activities:
– Define entities and their attributes.
– Identify primary keys (unique identifiers for each entity).
– Establish relationships (e.g., one-to-many, many-to-many).
– Ensure the model accurately represents real-world data requirements.
• Example (Library Management System ERD):
– Entities:
* Books (ISBN, Title, Publication Year).
* Authors (Author ID, Name).
* Borrowers (Borrower ID, Name, Contact).
– Relationships:
* Author → Books (One-to-Many: An author writes multiple books).
* Borrower → Books (Many-to-Many: A borrower can borrow multiple books, and a book can be
borrowed by multiple users over time).

[Link] 3. Logical Design

• Objective: Convert the conceptual model into a logical schema (database-independent structure).
• Key Activities:
– Define tables (based on entities).
– Specify columns (attributes) and data types (e.g., VARCHAR, INT, DATE).
– Establish relationships using foreign keys.
– Apply normalization rules (e.g., 1NF, 2NF, 3NF) to minimize redundancy.
• Example (Library Management System Logical Schema):
– Tables:
* Books (ISBN [PK], Title, Publication Year, Author ID [FK]).
* Authors (Author ID [PK], Name).
* Borrowers (Borrower ID [PK], Name, Contact).
* Loans (Loan ID [PK], Borrower ID [FK], ISBN [FK], Loan Date, Return Date).
– Foreign Keys:
* Author ID in Books references Authors.
* Borrower ID and ISBN in Loans reference Borrowers and Books, respectively.
[Link] 4. Physical Design
• Objective: Optimize the storage structure for performance.
• Key Activities:

214
 – Design indexes to speed up queries on frequently accessed columns.
– Implement partitioning for large tables to distribute data efficiently.
– Configure storage parameters (e.g., tablespaces, file organizations).
– Ensure the physical design supports the logical model and meets performance requirements.
• Example (Library Management System Physical Design):
– Indexes:
* Create an index on ISBN in the Books table for faster searches.
* Index Borrower ID in the Borrowers table for quick lookups.
– Partitioning:
* Partition the Loans table by Loan Date to improve query performance for historical data.
[Link] 5. Implementation
• Objective: Build and deploy the database using SQL and DBMS tools.
• Key Activities:
– Create tables based on the logical schema.
– Populate the database with initial data.
– Test queries and relationships to ensure correctness.
– Validate performance against requirements.
• Example (Library Management System Implementation):
– SQL Commands:
CREATE TABLE Authors (
AuthorID INT PRIMARY KEY,
Name VARCHAR(100)
);

CREATE TABLE Books (

ISBN VARCHAR(20) PRIMARY KEY,
Title VARCHAR(200),
PublicationYear INT,
AuthorID INT,
FOREIGN KEY (AuthorID) REFERENCES Authors(AuthorID)
);

CREATE TABLE Borrowers (

BorrowerID INT PRIMARY KEY,
Name VARCHAR(100),
Contact VARCHAR(50)
);

CREATE TABLE Loans (

LoanID INT PRIMARY KEY,
BorrowerID INT,
ISBN VARCHAR(20),
LoanDate DATE,
ReturnDate DATE,
FOREIGN KEY (BorrowerID) REFERENCES Borrowers(BorrowerID),
FOREIGN KEY (ISBN) REFERENCES Books(ISBN)
);

215
 – Testing:
* Run queries to verify relationships (e.g., “List all books borrowed by a specific user”).
* Check performance metrics (e.g., query execution time).
5.10.4 Summary of Key Points

Aspect Key Takeaways

Importance of Design Improves performance, scalability, and maintainability.
Poor Design Risks Slow queries, data redundancy, update errors, scalability issues.
Good Design Benefits Fast retrieval, minimal redundancy, easy updates, efficient scaling.
Design Lifecycle 1. Requirement Analysis, 2. Conceptual Design, 3. Logical Design, 4. Physical
Design, 5. Implementation.
Requirement Analysis Identify entities, attributes, relationships; align with business goals.
Conceptual Design Create ERD with entities, attributes, and relationships.
Logical Design Define tables, columns, data types, and foreign keys; apply normalization.
Physical Design Optimize indexes, partitioning, and storage for performance.
Implementation Build tables, populate data, test queries, and validate performance.

5.10.5 Conclusion
• Good database design is critical for high-performance, scalable, and maintainable applications.
• The five-stage lifecycle ensures the database meets user needs and business objectives.
• Real-world examples (e.g., e-commerce, library systems) demonstrate the impact of design choices on ef-
ficiency and reliability.

216
6 Module 6: Monitoring and Maintaining Database Applications
6.1 Backup Strategies
6.1.1 Introduction to Backup Strategies
• Backup strategies are critical for monitoring and maintaining database applications.
• The goal is to protect data integrity, ensure recoverability, and minimize downtime in case of failures.
• This lecture covers four primary backup types:
1. Full Backup
2. Incremental Backup
3. Differential Backup
4. Synthetic Backup
• Each type has distinct methods, advantages, disadvantages, and use cases.

6.1.2 Types of Database Backups

[Link] 1. Full Backup

[Link].1 Definition
• A full backup creates a complete copy of all data and files in the database at a specific point in time.
• Every piece of data is duplicated, including unchanged files from previous backups.

[Link].2 Characteristics
• Standalone backup: Can be used independently for data restoration without relying on other backups.
• Most comprehensive: Ensures all data is preserved in a single backup set.

[Link].3 Advantages
• Simplicity in restoration: Only one backup file is needed for full recovery.
• High reliability: Guarantees complete data protection at the time of backup.

[Link].4 Disadvantages
• Time-consuming: Requires significant time to complete, especially for large databases.
• High storage requirements: Consumes large amounts of storage space due to full data duplication.
• Resource-intensive: Can impact system performance during backup operations.

[Link].5 Best Use Case

• Ideal for small businesses or small datasets where storage and time constraints are minimal.

[Link] 2. Incremental Backup

[Link].1 Definition
• An incremental backup captures only the data that has changed since the last backup (whether full or
incremental).
• Does not duplicate unchanged data.

217
[Link].2 Characteristics
• Efficient in storage and time: Only new or modified files are saved.
• Reduces backup size: Minimizes storage requirements compared to full backups.
• Faster execution: Completes quickly due to smaller data volume.

[Link].3 Advantages
• Low storage consumption: Ideal for environments with frequent but small data changes.
• Minimal performance impact: Less strain on system resources during backup.
• Cost-effective: Reduces storage costs over time.

[Link].4 Disadvantages
• Complex restoration process:
– Requires last full backup + all subsequent incremental backups for complete recovery.
– Time-consuming recovery if multiple incremental backups must be applied sequentially.
• Risk of backup chain dependency: If any incremental backup is corrupted, the entire recovery process
may fail.

[Link].5 Best Use Case

• Suitable for large databases with frequent, small updates (e.g., transactional systems).

[Link] 3. Differential Backup

[Link].1 Definition
• A differential backup records all changes made since the last full backup.
• Unlike incremental backups, it does not reset after each backup; it accumulates changes until the next full
backup.

[Link].2 Characteristics
• Middle ground between full and incremental:
– Faster than full backups but slower than incremental.
– Simpler restoration than incremental but more storage-intensive.
• Grows in size over time: As more data changes, the backup expands until the next full backup.

[Link].3 Advantages
• Simpler recovery process:
– Only requires the last full backup + the most recent differential backup.
– Faster restoration compared to incremental backups (which require all intermediate backups).
• Balanced storage usage: More efficient than full backups but less so than incremental.

[Link].4 Disadvantages
• Increasing storage requirements: Over time, differential backups consume more space than incremental
backups.
• Slower than incremental: Takes longer to complete than incremental backups.

218
[Link].5 Best Use Case
• Ideal for medium-sized databases where moderate data changes occur between full backups.

[Link] 4. Synthetic Full Backup

[Link].1 Definition
• A synthetic full backup combines a previous full backup with subsequent incremental backups to create
an updated full backup without recopying all data.
• Uses existing backups to reconstruct a full backup logically rather than physically recopying files.

[Link].2 Characteristics
• Hybrid approach: Merges benefits of full and incremental backups.
• Reduces network and storage load: Only changed data is transferred during synthesis.
• Does not require a full data recopy: Avoids the time and resource costs of traditional full backups.

[Link].3 Advantages
• Faster backup completion: Since it reuses existing backups, it reduces backup time.
• Lower storage requirements: Avoids duplicating unchanged data.
• Reduced network workload: Only modified data is transferred during synthesis.
• Cost-effective: Lowers storage and bandwidth costs.

[Link].4 Disadvantages
• Overwrites existing backups: The synthesis process modifies the original full backup, which may risk
data integrity if not managed properly.
• Complexity in management: Requires specialized backup software to handle synthetic operations.

[Link].5 Best Use Case

• Best for large-scale databases where full backups are impractical due to time or storage constraints.

6.1.3 Comparison of Backup Strategies

Backup Storage Backup Recovery

Type Data Copied Usage Speed Speed ComplexityBest For
Full All data (complete High Slow Fast Low Small databases,
Backup snapshot) simplicity
Incremental Only changes since Low Fast Slow High Frequent small
Backup last backup changes
Differential All changes since last Medium Medium Medium Medium Moderate data
Backup full backup changes
Synthetic Combines full + Medium Fast Fast High Large databases, cost
Full incremental logically efficiency

219
6.1.4 Choosing the Best Backup Strategy
[Link] Key Factors Influencing Backup Strategy Selection
1. Amount of Data
• Large datasets → Prefer incremental or synthetic backups to save storage.
• Small datasets → Full backups may be sufficient.
2. Time Constraints
• Limited backup windows → Incremental or differential backups (faster execution).
• No time restrictions → Full backups (simpler recovery).
3. Software and Operating System Compatibility
• Some backup tools only support specific methods (e.g., RMAN for Oracle).
• Cloud-based solutions (AWS, Google Cloud) may dictate backup approaches.
4. Recovery Speed Requirements
• Fast recovery needed → Full or synthetic backups (simpler restoration).
• Can tolerate slower recovery → Incremental backups (but require chain restoration).

[Link] Common Hybrid Backup Strategy Most businesses adopt a multi-tiered approach: 1. Initial Full
Backup (baseline). 2. Differential or Incremental Backups (daily/weekly). 3. Occasional Full or Synthetic
Full Backup (monthly/quarterly). - Ensures balance between storage efficiency and recovery speed.

6.1.5 Backup Strategies for Different Database Types

[Link] 1. Relational Databases

Database Backup Tool/Method

PostgreSQL Built-in pg_dump / pg_basebackup
Microsoft SQL Server Native BACKUP command, SQL Server Agent
Oracle RMAN (Recovery Manager)
MySQL mysqldump, mysqlbackup (Enterprise)

[Link] 2. NoSQL and Semi-Structured Databases

Database Backup Tool/Method

MongoDB mongodump / mongorestore
Cassandra Snapshot-based backups (nodetool snapshot)
Amazon DynamoDB AWS Backup service

[Link] 3. Large-Scale and File System Backups

Use Case Backup Solution

Long-term archival Amazon S3 Glacier
Cost-effective storage Google Cloud Storage Nearline
Enterprise-scale backup IBM Spectrum Protect

220
[Link] Key Considerations for Database-Specific Backups
• Consistency: Ensure backups capture transactionally consistent data (e.g., using hot backups in Oracle
RMAN).
• Automation: Schedule backups during low-traffic periods to minimize performance impact.
• Validation: Regularly test backups to ensure recoverability.
• Encryption: Protect backup files with encryption for security compliance.

6.1.6 Summary and Best Practices

[Link] Key Takeaways
1. Full Backups are comprehensive but resource-heavy; best for small, critical datasets.
2. Incremental Backups are storage-efficient but complex to restore; ideal for frequent, small changes.
3. Differential Backups offer a balance between speed and simplicity; suitable for moderate change rates.
4. Synthetic Full Backups combine efficiency and completeness; optimal for large-scale environments.

[Link] Best Practices for Implementing Backup Strategies

• Follow the 3-2-1 Rule:
– 3 copies of data (primary + 2 backups).
– 2 different media types (e.g., disk + tape/cloud).
– 1 offsite backup (protected against physical disasters).
• Automate Backup Schedules: Use cron jobs, SQL Agent, or cloud scheduling to ensure consistency.
• Monitor Backup Jobs: Set up alerts for failures and log reviews.
• Test Recovery Procedures: Conduct regular drills to verify backup integrity.
• Document Backup Policies: Maintain clear documentation on backup types, schedules, and recovery steps.

[Link] Final Recommendation

• Hybrid strategies (e.g., full + differential + synthetic) provide flexibility and resilience.
• Tailor the approach based on database type, size, and business continuity requirements.
Conclusion: A well-designed backup strategy is essential for data protection, compliance, and business conti-
nuity. By understanding the strengths and trade-offs of each backup type, organizations can optimize storage,
recovery time, and cost efficiency.

6.2 Bulk Uploads

6.2.1 Introduction to Bulk Uploads
[Link] Purpose of the Lecture
• The lecture focuses on bulk uploads, a critical process in database management for handling large datasets
efficiently.
• By the end of the lecture, students should:
– Understand the principles of efficient data handling in large-scale databases.
– Learn techniques to streamline bulk uploads.
– Recognize the importance of data integrity and performance optimization.

6.2.2 Importance of Efficient Data Handling

[Link] Challenges with Large Datasets

221
 • Large datasets can slow down database systems, leading to:
– Delayed query processing.
– Inefficient use of computational resources (CPU, memory, disk I/O).
• Inefficient data management results in:
– Increased hardware and infrastructure costs (scaling up servers, storage).
– Performance bottlenecks affecting application responsiveness.

[Link] Benefits of Efficient Data Handling

• Cost reduction: Optimized processes minimize hardware demands.
• Performance optimization: Faster queries, updates, and deletions.
• Data integrity: Ensures accuracy and reliability throughout the data lifecycle.
• Consistency: Prevents discrepancies during bulk operations.

6.2.3 Key Techniques for Managing Large Datasets

[Link] 1. Data Partitioning
• Definition: Dividing data into chunks (batches) instead of processing one row at a time.
• Advantages:
– Reduces the number of individual operations, decreasing system load.
– Improves throughput by processing data in parallelizable segments.
• Example:
– Uploading 10,000 records in batches of 1,000 instead of row-by-row.

[Link] 2. Index Optimization

• Definition: Ensuring database indexes are structured for optimal query and update performance.
• Key Points:
– Poorly optimized indexes slow down INSERT/UPDATE/DELETE operations.
– Proper indexing speeds up data retrieval and bulk modifications.
• Best Practices:
– Use composite indexes for frequently queried columns.
– Avoid over-indexing, which can degrade write performance.

[Link] 3. Bulk Operations

• Definition: Database features designed to handle large datasets in single operations.
• Examples in SQL Databases:
– BULK INSERT (SQL Server).
– COPY (PostgreSQL).
– LOAD DATA INFILE (MySQL).
• Advantages:
– Reduces transaction overhead (fewer commits/rollbacks).
– Minimizes network latency (single command vs. multiple statements).

[Link] 4. Parallel Processing

• Definition: Distributing workload across multiple processes/servers to process data simultaneously.
• Implementation:
– Multi-threading: Single machine divides tasks among CPU cores.
– Distributed systems: Multiple servers handle different data partitions.

222
 • Benefits:
– Reduces total processing time (e.g., uploading 1M records in 10 minutes instead of 1 hour).
– Improves scalability for growing datasets.

6.2.4 Bulk Uploads: Definition and Process

[Link] Definition
• A method to quickly import large volumes of data from an external source (e.g., CSV, JSON) into a
database.
• Uses specialized commands/tools optimized for high-volume data transfer.

[Link] How Bulk Uploads Work

1. Source File: Data is stored in a structured format (e.g., CSV, TSV).
2. Database Command: A bulk operation (e.g., BULK INSERT) reads the file and loads data into a table.
3. Terminators:
• Field Terminator: Character separating columns (e.g., comma , in CSV).
• Row Terminator: Character separating rows (e.g., newline \n).
4. Execution: The database processes the file in a single optimized operation.

BULK INSERT Customers

FROM 'C:\data\[Link]'
WITH (
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
);

[Link] Example: SQL Server BULK INSERT

• FIELDTERMINATOR: Specifies the delimiter between columns (, for CSV).
• ROWTERMINATOR: Specifies the end-of-row marker (newline).
• Use Case: Importing thousands of customer records in one operation.

6.2.5 Best Practices for Bulk Uploads

[Link] 1. File Format Specifications
• Format: CSV (Comma-Separated Values) is the standard.
• Delimiter: Use a comma (,) to separate values.
• File Size Limit: <= 52 MB to avoid performance issues (larger files may require chunking).
• Structure:
– Tabular format (rows and columns).
– First row: Must define column headers matching the database table.

[Link] 2. Tools for Creating Import Files

Tool Use Case Output Format

Microsoft Excel Easy editing, sorting, and validation of data. .csv
Google Sheets Collaborative editing; export as CSV. .csv
Notepad/TextEdit Manual editing for small files; verify formatting. .txt/.csv

223
Tool Use Case Output Format

[Link] 3. Validation Steps

1. Create Data: Use Excel/Sheets to organize data into columns.
2. Save as CSV:
• In Excel: **File → Save As → CSV (Comma delimited) (*.csv)**.
3. Verify Format:
• Open the CSV in a text editor (e.g., Notepad).
• Confirm values are comma-separated and rows are newline-separated.
• Example:
id,name,email
1,John Doe,john@[Link]
2,Jane Smith,jane@[Link]
4. Check for Errors:
• Missing delimiters.
• Mismatched columns (e.g., extra/comma in a field).
• Special characters (e.g., quotes " in CSV may need escaping).

[Link] 4. Example Workflow

1. Scenario: Import a group dataset from Excel into a database.
2. Steps:
• Organize data in Excel with headers (e.g., group_id, group_name).
• Save as [Link].
• Open in Notepad to confirm formatting:
group_id,group_name
1,Admin
2,Users
3,Guests
• Execute BULK INSERT in SQL Server.

6.2.6 Summary of Key Concepts

1. Efficient Data Handling:
• Critical for performance, cost savings, and data integrity.
2. Techniques for Large Datasets:
• Partitioning, index optimization, bulk operations, parallel processing.
3. Bulk Uploads:
• Use specialized commands (e.g., BULK INSERT) for high-volume imports.
• Follow file format best practices (CSV, delimiters, size limits).
4. Tools:
• Excel/Sheets for editing; text editors for validation.
5. Validation:
• Always verify CSV structure before upload to prevent errors.

6.2.7 Conclusion
• Bulk uploads are essential for managing large-scale database applications.
• By applying the techniques and best practices discussed, developers can:

224
 – Optimize performance.
– Maintain data integrity.
– Reduce operational costs.
• Key Takeaway: Proper planning and validation ensure smooth, efficient bulk data transfers.

6.3 Ensuring Data Integrity

6.3.1 1. Introduction to Data Integrity
[Link] 1.1 Definition of Data Integrity
• Data integrity refers to the accuracy, consistency, and reliability of data throughout its entire lifecycle
(storage, modification, transfer, deletion).
• Ensures that data remains unaltered, complete, and trustworthy within an organization.
• Two primary categories:
1. Physical Data Integrity – Focuses on how data is stored and accessed, ensuring secure and reliable
physical storage.
2. Logical Data Integrity – Prevents human errors and maintains consistency through rules, con-
straints, and relationships within the data.

[Link] 1.2 Importance of Data Integrity

• Critical for business operations – Compromised data can lead to disastrous consequences, such as:
– Financial losses (e.g., spoiled shipments in temperature-sensitive logistics).
– Legal and compliance risks (e.g., regulatory violations).
– Operational failures (e.g., incorrect medical treatments due to inaccurate patient records).
– Reputational damage (e.g., loss of customer trust).
• Key benefits of maintaining data integrity:
– Prevents costly errors (e.g., incorrect financial transactions).
– Ensures regulatory compliance (e.g., GDPR, HIPAA, KYC).
– Supports informed decision-making (e.g., accurate business analytics).
– Business necessity, not just a technical requirement.

6.3.2 2. Common Causes of Data Integrity Loss

Data integrity can be compromised due to multiple factors, leading to inaccuracy, inconsistency, and unreliability
in databases.

[Link] 2.1 Human Errors

• Mistakes during data entry, updates, or deletions (e.g., typos, incorrect formatting).
• Example: Entering an incorrect patient’s medical history due to a keystroke error.

[Link] 2.2 Transfer Errors

• Issues during data transfer, such as:
– Incomplete transfers (partial data loss).
– Corruption during transit (e.g., network interruptions).
• Example: A bank transfer record getting corrupted mid-transmission.

225
[Link] 2.3 Cyber Threats
• Malicious attacks that alter, delete, or corrupt data:
– Hacking (unauthorized access).
– Malware/Ransomware (data encryption or deletion).
• Example: A ransomware attack encrypting hospital patient records.

[Link] 2.4 Security Issues

• Weak access controls allowing unauthorized users to modify or delete data.
• Insufficient security measures (e.g., lack of encryption, weak passwords).
• Example: An employee without proper clearance altering financial records.

[Link] 2.5 Hardware or Infrastructure Issues

• Storage device failures (e.g., hard drive crashes).
• Power outages leading to unsaved data loss.
• Network issues causing data corruption.
• Example: A server crash corrupting a company’s customer database.

6.3.3 3. Strategies to Maintain Data Integrity

Organizations must implement multi-layered strategies to protect data integrity across all stages of data manage-
ment.

[Link] 3.1 Data Entry & Training

• Train staff on proper data entry procedures to minimize human errors.
• Standardize data formats (e.g., date formats, naming conventions).

[Link] 3.2 Input Validation & Data Cleaning

• Implement validation rules to ensure only correct and expected data is entered.
– Example: Restricting a “Date of Birth” field to accept only valid dates.
• Remove duplicate data through regular database cleaning to prevent inconsistencies.

[Link] 3.3 Backup & Recovery

• Regular backups to restore data in case of corruption or loss.
• Automated backup schedules (daily, weekly, incremental).
• Example: A nightly backup of a retail inventory database.

[Link] 3.4 Access Control & Audit Trails

• Role-based access control (RBAC) to restrict data modification to authorized personnel.
• Maintain audit logs to track who made changes, when, and what was modified.
– Example: Logging all changes to a patient’s medical records in a hospital system.

[Link] 3.5 Security Measures

• Penetration testing & security audits to identify vulnerabilities.
• SSL/TLS encryption for secure data transfer between client and server.
• Example: Encrypting credit card transactions in an e-commerce database.

226
[Link] 3.6 Stress Testing & Redundancy
• Test databases under high-load conditions to ensure stability during peak usage.
• Implement RAID (Redundant Array of Independent Disks) for data redundancy in case of hardware
failure.

[Link] 3.7 Process Standardization

• Create process maps for consistent data handling (e.g., SOPs for data entry, updates, deletions).
• Example: A standardized workflow for updating student records in a university database.

[Link] 3.8 Cybersecurity Awareness

• Train employees on cybersecurity best practices (e.g., phishing awareness, password hygiene).
• Example: Conducting quarterly cybersecurity training for bank employees.

[Link] 3.9 Cultural Emphasis on Data Integrity

• Foster a workplace culture where data integrity is a priority.
• Encourage reporting of anomalies (e.g., suspicious data changes).

6.3.4 4. SQL Constraints for Enforcing Data Integrity

Constraints in SQL databases enforce rules to maintain data integrity at the schema level.

[Link] 4.1 Types of Constraints

Constraint Purpose Example

CHECK Ensures column values meet specific CHECK (age > 0) – Ensures age cannot be
Constraint conditions. negative.
UNIQUE Prevents duplicate entries in a column or UNIQUE (email) – Ensures no two users have
Constraint group of columns. the same email.
PRIMARY Uniquely identifies each record in a table PRIMARY KEY (student_id) – Each student
KEY (cannot be NULL). has a unique ID.
FOREIGN Maintains referential integrity by ensuring a FOREIGN KEY (customer_id) REFERENCES
KEY value exists in another table. Customers(id) – Orders must reference a
valid customer.

[Link] 4.2 Practical Applications of Constraints

• CHECK Constraint Example:
CREATE TABLE Employees (
employee_id INT PRIMARY KEY,
salary DECIMAL(10,2) CHECK (salary >= 0)
);

– Ensures salaries are non-negative.

• UNIQUE Constraint Example:

227
 CREATE TABLE Users (
user_id INT PRIMARY KEY,
email VARCHAR(100) UNIQUE
);

– Prevents duplicate email addresses.

• FOREIGN KEY Example:
CREATE TABLE Orders (
order_id INT PRIMARY KEY,
customer_id INT,
FOREIGN KEY (customer_id) REFERENCES Customers(customer_id)
);

– Ensures orders reference valid customers.

6.3.5 5. Triggers for Advanced Data Integrity

When basic constraints are insufficient, triggers provide custom logic to enforce complex rules.

[Link] 5.1 Types of Triggers

Trigger
Type Description Use Case
AFTER Executes after an INSERT, UPDATE, or DELETE Logging changes to an audit table after a
Trigger operation. record is updated.
INSTEAD Replaces the triggering action with custom Validating data before allowing an insert (e.g.,
OF Trigger code. checking inventory levels).

CREATE TRIGGER log_employee_changes

AFTER UPDATE ON Employees
FOR EACH ROW
BEGIN
INSERT INTO AuditLog (employee_id, old_salary, new_salary, change_date)
VALUES (OLD.employee_id, [Link], [Link], NOW());
END;

[Link] 5.2 Example: AFTER Trigger for Audit Logging

• Purpose: Tracks salary changes for compliance.

CREATE TRIGGER validate_order_quantity

INSTEAD OF INSERT ON Orders
FOR EACH ROW
BEGIN
IF [Link] > (SELECT stock FROM Inventory WHERE product_id = NEW.product_id)
THEN
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT = 'Insufficient stock';

228
 ELSE
INSERT INTO Orders VALUES (NEW.*);
END IF;
END;

[Link] 5.3 Example: INSTEAD OF Trigger for Validation

• Purpose: Prevents orders exceeding available inventory.

6.3.6 6. Categories of Data Integrity

Data integrity is categorized based on different aspects of data management.

[Link] 6.1 Physical Integrity

• Ensures data remains intact despite hardware/environmental issues.
• Methods:
– RAID (Redundant Array of Independent Disks) for fault tolerance.
– Backup power supplies (UPS) to prevent data loss during outages.

[Link] 6.2 Logical Integrity

[Link].1 6.2.1 Entity Integrity

• Ensures each record is unique (e.g., via primary keys).
• Example: Assigning a unique Social Security Number (SSN) to each employee in an HR database.

[Link].2 6.2.2 Referential Integrity

• Maintains consistency between related tables via foreign keys.
• Example: An Orders table cannot reference a non-existent Customer.

[Link].3 6.2.3 Domain Integrity

• Ensures data adheres to defined formats, ranges, or values.
• Example: A date_of_birth column only accepts valid dates (e.g., not 31-Feb-2023).

[Link].4 6.2.4 User-Defined Integrity

• Enforces business-specific rules not covered by standard constraints.
• Example: Ensuring employee salaries fall within approved pay scales for their job title.

6.3.7 7. Real-World Examples of Data Integrity by Sector

[Link] 7.1 Healthcare
• Critical for: Accurate patient records, diagnoses, and treatment plans.
• Example: Electronic Health Records (EHR) must reflect correct allergies to prevent life-threatening med-
ication errors.
• Consequences of failure: Misdiagnosis, incorrect prescriptions, legal liabilities.

229
[Link] 7.2 Financial Institutions
• Critical for: Transaction accuracy, fraud detection, and regulatory compliance.
• Example: Banks use KYC (Know Your Customer) protocols to verify identities and prevent money laun-
dering.
• Consequences of failure: Financial losses, regulatory fines, reputational damage.

[Link] **7.3 Education

• Critical for: Student records, enrollment management, and academic tracking.
• Example: Accurate GPA calculations for scholarship eligibility.
• Consequences of failure: Incorrect graduations, misallocated resources.

[Link] 7.4 Logistics & Supply Chain

• Critical for: Inventory tracking, shipment status, and temperature-sensitive goods.
• Example: A dry ice temperature data logger must maintain accurate readings to prevent spoiled ship-
ments.
• Consequences of failure: Product losses, customer dissatisfaction.

6.3.8 8. Summary & Key Takeaways

• Data integrity ensures accuracy, consistency, and reliability of data across its lifecycle.
• Two main types:
– Physical integrity (hardware/storage protection).
– Logical integrity (rules, constraints, relationships).
• Common threats: Human error, transfer issues, cyberattacks, hardware failures.
• Mitigation strategies:
– Constraints (CHECK, UNIQUE, PRIMARY KEY, FOREIGN KEY).
– Triggers (AFTER, INSTEAD OF) for complex rules.
– Backups, access control, encryption, audits.
• Sector-specific importance:
– Healthcare (patient safety), Finance (fraud prevention), Education (academic records).
• Final Thought: Data integrity is not optional—it is a fundamental requirement for trustworthy, com-
pliant, and efficient database systems.

6.4 Handling Large Data Sets

6.4.1 1. Introduction to Handling Large Data Sets
• Objective: Identify challenges of handling large datasets and understand techniques for evaluation and se-
lection.
• Significance: Effective management of large-scale data is critical for storage, processing, and maintaining
data quality.

6.4.2 2. Challenges of Handling Large Data Sets

[Link] 2.1 Storage Challenges
• Scalability Requirement: Need for scalable storage solutions to accommodate vast amounts of data.
• Infrastructure Strain: Traditional storage systems may become inadequate as data volume grows.

230
[Link] 2.2 Processing Challenges
• Resource Intensity: Large datasets demand substantial processing power, leading to:
– Slow query response times.
– Bottlenecks in system performance.
• Infrastructure Limitations: Existing hardware may struggle to handle increased computational loads.

[Link] 2.3 Security Challenges

• Unauthorized Access Risks: Large datasets are prime targets for cyber threats (e.g., breaches, ransomware).
• Compliance Requirements: Must adhere to regulatory standards (e.g., GDPR, HIPAA) to protect sensitive
data.

[Link] 2.4 Data Quality Challenges

• Common Issues:
– Inaccuracies: Errors in data collection or entry.
– Inconsistencies: Discrepancies across datasets (e.g., conflicting formats).
– Duplication: Redundant data entries wasting storage and processing resources.
• Impact: Poor data quality undermines analytical reliability and decision-making.

6.4.3 3. Approaches to Fix Data Quality Issues

[Link] 3.1 Correcting Information in the Original Database
• Direct Repair: Fix inaccuracies at the source to prevent propagation of errors.
• High-Accuracy Methods: Use validated techniques (e.g., data validation rules, automated cleansing tools)
to ensure data integrity.

[Link] 3.2 Scaling Big Data Systems

• Key Techniques:
1. Database Sharding:
– Horizontal partitioning of data across multiple servers.
– Improves performance by distributing load.
2. Memory Caching:
– Stores frequently accessed data in RAM for faster retrieval (e.g., Redis, Memcached).
3. Cloud Merging:
– Integrates on-premise and cloud data for hybrid scalability.
4. Read-Write Separation:
– Dedicated databases for read-only and write-active operations to optimize performance.

6.4.4 4. Techniques for Evaluating and Managing Large Datasets

[Link] 4.1 Hadoop Ecosystem
• Purpose: Distributed storage and processing of large datasets across clusters.
• Components:
– HDFS (Hadoop Distributed File System): Scalable, fault-tolerant storage.
– MapReduce: Parallel processing framework.
– YARN: Resource management layer.

231
[Link] 4.2 Apache Spark
• Definition: Open-source unified analytics engine for large-scale data processing.
• Advantages:
– In-memory computation for faster performance.
– Supports batch processing, streaming, machine learning, and graph processing.

[Link] 4.3 NoSQL Databases

• Design: Non-relational databases optimized for large-scale data storage and retrieval.
• Types:
– Document Stores (e.g., MongoDB).
– Key-Value Stores (e.g., Cassandra).
– Column-Family Stores (e.g., HBase).
– Graph Databases (e.g., Neo4j).
• Use Cases: High-speed read/write operations, flexible schema design.

[Link] 4.4 R Software

• Purpose: Programming language and environment for statistical computing and data analysis.
• Features:
– Extensive libraries for data visualization (e.g., ggplot2).
– Advanced statistical modeling capabilities.

[Link] 4.5 Predictive Analytics

• Definition: Uses historical data to forecast future outcomes.
• Applications:
– Demand forecasting.
– Risk assessment.
– Customer behavior prediction.

[Link] 4.6 Prescriptive Analytics

• Definition: Recommends actions based on data analysis to achieve desired outcomes.
• Examples:
– Optimization of supply chain logistics.
– Personalized marketing strategies.

6.4.5 5. Industry-Specific Challenges: Healthcare

[Link] 5.1 Data Sources
• Electronic Health Records (EHRs).
• Genomic Sequencing.
• Medical Imaging (e.g., X-rays, MRIs).
• Clinical Research Data.

[Link] 5.2 Key Challenges

1. Implementation Costs:
• High expenses for large-scale data systems (hardware, software, maintenance).
2. Data Compilation and Cleaning:

232
 • Heterogeneous data formats from multiple sources (e.g., hospitals, labs, wearables).
• Requires standardization and deduplication.
3. Security and Compliance:
• Protection of sensitive patient data under HIPAA (Health Insurance Portability and Accountability Act).
• Risk of breaches and unauthorized access.
4. Communication Gaps:
• Lack of interoperability between healthcare providers/systems.
• Potential for fragmented patient care due to siloed data.

6.4.6 6. Cloud-Based Data Management Challenges

[Link] 6.1 Regulatory Compliance
• Key Regulations:
– GDPR (General Data Protection Regulation): EU data privacy law.
– HIPAA: U.S. healthcare data protection standard.
• Requirements:
– Data encryption (at rest and in transit).
– Access controls and audit logs.

[Link] 6.2 Governance and Control

• Policies Needed:
– Role-based access control (RBAC).
– Data usage monitoring and reporting.
• Remote Security Model:
– Secure remote applications and data access.
– Zero-trust architecture principles.

[Link] 6.3 Cost Management

• Challenges:
– Unpredictable costs with pay-as-you-go cloud models.
– Storage and compute expenses scale with data volume.
• Solutions:
– Resource optimization (e.g., auto-scaling, reserved instances).
– Cost-monitoring tools (e.g., AWS Cost Explorer).

[Link] 6.4 Performance Evaluation

• Metrics to Monitor:
– Query latency.
– Throughput (data processed per unit time).
– System uptime and reliability.
• Improvement Strategies:
– Load balancing.
– Caching frequently accessed data.

6.4.7 7. Strategies for Efficient and Secure Storage

[Link] 7.1 Distributed File Systems

233
 • Examples:
– Hadoop Distributed File System (HDFS): Scalable, fault-tolerant storage.
– Cloud Storage (e.g., AWS S3, Google Cloud Storage): Elastic and durable.
• Benefits:
– Horizontal scalability.
– Redundancy for data availability.

[Link] 7.2 Columnar Storage Formats

• Example: Apache ORC (Optimized Row Columnar).
• Advantages:
– Reduces storage overhead via compression.
– Improves query performance for analytical workloads.

[Link] 7.3 Data Partitioning

• Definition: Dividing data into smaller, manageable subsets (e.g., by date, region).
• Benefits:
– Parallel processing (e.g., Spark partitions).
– Faster query execution (reduced I/O).

[Link] 7.4 Data Compression

• Algorithms:
– Snappy: Fast compression/decompression (used in Hadoop).
– Gzip: Higher compression ratio (slower but space-efficient).
• Trade-offs:
– Compression ratio vs. CPU overhead.

6.4.8 8. Techniques for Managing Large Datasets

[Link] 8.1 Compression
• Methods:
– Lossless compression (e.g., ZIP, Gzip).
– Domain-specific compression (e.g., delta encoding for time-series data).
• Goal: Reduce storage footprint without data loss.

[Link] 8.2 Partitioning

• Approaches:
– Horizontal Partitioning: Splitting rows (e.g., by customer ID).
– Vertical Partitioning: Splitting columns (e.g., separating frequently accessed fields).
• Use Case: Distributed databases (e.g., Cassandra).

[Link] 8.3 Data Transformation

• Objective: Convert data into processing-friendly formats.
• Examples:
– Flattening nested JSON into relational tables.
– Normalizing text data (e.g., lowercase conversion, stemming).

234
[Link] 8.4 Cache Optimization
• Techniques:
– In-Memory Caching: Store hot data in RAM (e.g., Redis).
– Query Result Caching: Reuse results for repeated queries.
• Benefits:
– Reduced latency for frequent access patterns.
– Lower database load.

6.4.9 9. Case Studies: Real-World Applications

[Link] 9.1 Hospitality
• Airbnb:
– Uses data science to analyze customer feedback and optimize listings.
• Qantas:
– Applies predictive analytics to minimize operational losses.

[Link] 9.2 Healthcare

• Novo Nordisk & AstraZeneca:
– Leverage big data for drug discovery and personalized medicine.
• Johnson & Johnson:
– Used data science to combat COVID-19 (e.g., vaccine research, supply chain optimization).

[Link] 9.3 E-Commerce

• Amazon:
– Personalization engines driven by customer behavior data.
– Recommendation systems increase sales and satisfaction.

[Link] 9.4 Supply Chain & Logistics

• UPS:
– Optimizes routes using data analytics, reducing fuel costs.
• IMD (India Meteorological Department):
– Predicted Cyclone Fani (2019), enabling 1.2 million evacuations.

[Link] 9.5 Entertainment

• Netflix:
– Personalized content recommendations via collaborative filtering.
• Spotify:
– Uses big data for music recommendations and playlist generation.

[Link] 9.6 Banking & Finance

• HDFC Bank:
– Fraud detection and customer segmentation via big data analytics.

235
[Link] 9.7 Urban Planning
• Smart Cities (Pune, Bhubaneshwar):
– Traffic management systems analyze real-time data to reduce congestion.

[Link] 9.8 Agriculture

• Farmers Edge (Canada):
– Precision farming using satellite imagery and IoT sensors.

[Link] 9.9 Transportation

• Uber:
– Optimizes ride-sharing and delivery routes with real-time analytics.

[Link] 9.10 Environmental Science

• NASA:
– Predicts natural disasters (e.g., hurricanes, wildfires) using satellite data.
• World Wildlife Fund:
– Tracks deforestation and biodiversity loss via remote sensing.

6.4.10 10. Cross-Industry Applications of Large Datasets

[Link] 10.1 Advertising & Marketing
• Use Cases:
– Personalized ads based on browsing history.
– Customer segmentation for targeted campaigns.

[Link] 10.2 Education

• Applications:
– Dropout prediction using student performance data.
– Adaptive learning platforms (e.g., Khan Academy).

[Link] 10.3 Healthcare

• Innovations:
– AI-driven diagnostics (e.g., IBM Watson).
– Wearable device integration for remote monitoring.

[Link] 10.4 Transport & Logistics

• Optimizations:
– Route planning for fuel efficiency.
– Predictive maintenance for fleets.

[Link] 10.5 Banking & Finance

• Key Functions:
– Fraud detection via anomaly detection algorithms.
– Credit scoring using alternative data (e.g., social media activity).

236
[Link] 10.6 Agriculture
• Technologies:
– Drone imagery for crop health monitoring.
– Soil sensors for irrigation optimization.

6.4.11 11. Summary and Key Takeaways

• Challenges:
– Storage scalability, processing bottlenecks, security risks, and data quality issues.
• Solutions:
– Technologies: Hadoop, Spark, NoSQL, R.
– Strategies: Partitioning, compression, caching, cloud governance.
• Industry Impact:
– Large datasets drive innovation across sectors (healthcare, finance, agriculture, etc.).
• Future Directions:
– Continued adoption of AI/ML for advanced analytics.
– Emphasis on ethical data use and regulatory compliance.

6.5 Importance of Backups

6.5.1 1. Definition of Database Backup
• Database Backup: An exact copy of a database stored in a separate location to preserve all data.
– Purpose:
* Restore lost data in emergencies (e.g., data loss due to outages, system failures).
* Ensure long-term data preservation for compliance and archival.
– Process:
* Involves copying and storing data, often in multiple locations, to guarantee availability when
needed.
– Key Distinction from Data Replication:
* Backup: Focuses on long-term data prevention and recovery.
* Replication: Aims to minimize recovery time and ensure business continuity during disasters.
6.5.2 2. Four Critical Factors in Database Backup Strategy
A robust backup strategy must consider four key factors:

[Link] 2.1 Frequency

• Databases updated frequently require more frequent backups to minimize data loss between backup inter-
vals.
• Example: A high-transaction e-commerce database may need hourly backups, whereas a static reference
database may only need daily or weekly backups.

[Link] 2.2 Amount of Data

• Impact on Backup Process:
– Storage Requirements: Larger datasets demand more storage capacity.
– Time to Complete Backup: Large volumes increase backup duration.
• Solutions for Large Datasets:
– Incremental Backups: Only back up changes since the last backup.

237
 – Differential Backups: Back up changes since the last full backup.

[Link] 2.3 Urgency

• Critical Data Access Needs:
– If immediate restoration is required (e.g., financial transactions, real-time inventory), more frequent
backups are essential.
– Example: A stock trading platform cannot afford data loss; hence, near real-time backups are neces-
sary.

[Link] 2.4 Type of Data

• Different data types require tailored backup methods:
– Transactional Data (e.g., orders, payments):
* Requires frequent backups due to constant updates.
– Static Data (e.g., historical records):
* Less frequent backups suffice.
– Sensitive Data (e.g., customer PII, financial records):
* Encryption during backup is mandatory for security.
– Multimedia Files (e.g., videos, high-res images):
* Need specialized storage solutions due to size and format.
6.5.3 3. Potential Threats Mitigated by Backups
Database backups protect against multiple risks:

[Link] 3.1 Data Corruption and Loss

• Causes:
– Natural Disasters (fires, floods).
– Hardware Failures (disk crashes, server malfunctions).
– Power Outages (unexpected shutdowns).
– Human Errors (accidental deletions, misconfigurations).
– Cyber Attacks (ransomware, malware, hacking).

[Link] 3.2 Business Continuity (BC) and Disaster Recovery (DR)

• Business Continuity (BC):
– Ensures entire business operations remain functional during and after a disaster.
• Disaster Recovery (DR):
– Focuses on restoring technology infrastructure (e.g., servers, databases).
• Role of Backups:
– Enable quick recovery of data, minimizing downtime.
– Maintain operational stability by restoring critical systems.

[Link] 3.3 Case Study: E-Commerce Platform (XYZ)

• Scenario:
– Handles thousands of daily transactions (orders, payments, inventory).
– Threats:
* Hardware failure → Data loss.
* Ransomware attack → Data encryption by hackers.
238
 • Solution:
– Disaster Recovery Plan with regular backups (cloud-based storage).
– Outcome:
* After hardware failure, IT team restores the latest backup, recovering lost data.
* During ransomware attack, team ignores ransom demand and restores from an unaffected
backup, ensuring continuity without security compromise.

6.5.4 4. Database-Specific Backup Considerations

Different databases require customized backup approaches:

[Link] 4.1 Traditional Relational Databases

• Examples: Oracle, Microsoft SQL Server, MySQL.
• Structure: Data stored in structured tables with defined relationships.
• Preferred Backup Methods:
– Incremental Backups: Save only changes since the last backup (any type).
– Differential Backups: Save all changes since the last full backup.

[Link] 4.2 Distributed Databases (NoSQL)

• Examples: MongoDB, Cassandra, Hadoop.
• Structure: Data spread across multiple nodes (distributed architecture).
• Backup Challenges:
– Requires coordinated backups across all nodes to ensure consistency.
– May need specialized tools for distributed data synchronization.

[Link] 4.3 SaaS and Cloud-Based Databases

• Examples: Microsoft 365, AWS RDS.
• Backup Considerations:
– Provider-Managed Backups: Cloud providers often handle backups, but additional redundancy is
critical.
– Recommended Strategy:
* Regular exports from cloud to on-premise storage or a secondary cloud provider.
* Protects against provider outages, data breaches, or accidental deletions.
[Link] 4.4 Example Database Scenarios

Database Type Use Case Backup Strategy

Oracle (Accounting) Financial transactions Frequent backups + encryption
SAP (Inventory) Supply chain management Real-time replication to avoid stock
discrepancies
MongoDB (NoSQL) Unstructured big data Distributed node coordination
AWS RDS (Cloud) Scalable web applications Cross-cloud exports + on-premise copies

6.5.5 5. Benefits of Database Backups

[Link] 5.1 Fast Data Recovery and Replication
• Enables quick restoration after disasters, failures, or corruption.

239
 • Reduces downtime and accelerates business resumption.

[Link] 5.2 Storage Data Security

• Protects against:
– Cyber attacks (ransomware, hacking).
– Accidental deletion (human error).
– Corruption (software bugs, hardware issues).

[Link] 5.3 Easier Data Management

• Streamlines handling of large datasets through:
– Scheduled backups (automated, aligned with business needs).
– Organized storage (versioning, categorization).

[Link] 5.4 Improved System Performance

• Regular backups help:
– Optimize database performance (e.g., archiving old data).
– Reduce load on primary systems by offloading to backup storage.

[Link] 5.5 Controlled Costs

• Optimized storage usage through:
– Full backups (complete copies, less frequent).
– Incremental/Differential backups (space-efficient, frequent).
• Reduces financial impact of data loss (e.g., downtime costs, recovery expenses).

[Link] 5.6 Better Compliance

• Regulatory Requirements:
– Many industries (finance, healthcare) mandate data redundancy and protection.
– Example: GDPR, HIPAA, SOX.
• Audit Readiness:
– Backups provide verifiable proof of data protection during audits or legal proceedings.

6.5.6 6. Types of Database Backups

[Link] 6.1 Full Backup
• Definition: A complete copy of the entire database.
• Pros:
– Simplest restoration (single file to recover).
– No dependency on other backups.
• Cons:
– Time-consuming (especially for large databases).
– High storage requirements.

[Link] 6.2 Incremental Backup

• Definition: Copies only data changed since the last backup (full, differential, or incremental).
• Pros:
– Faster execution (smaller data volume).

240
 – Lower storage usage.
• Cons:
– Complex restoration (requires all incremental backups since the last full backup).
– Chain dependency (if one incremental fails, restoration may be incomplete).

[Link] 6.3 Differential Backup

• Definition: Copies all changes since the last full backup (not since the last differential).
• Pros:
– Faster restoration than incremental (only needs last full + latest differential).
– Balanced storage usage (more efficient than full, less complex than incremental).
• Cons:
– Grows larger over time (accumulates changes since last full backup).
– Slower than incremental for frequent backups.

6.5.7 7. Five Steps to Create a Database Backup Plan

[Link] 7.1 Step 1: Identify Critical Data
• Action: Determine which data is most important to protect.
• Considerations:
– Business impact of data loss (e.g., customer records vs. logs).
– Legal/regulatory requirements (e.g., financial data, PII).

[Link] 7.2 Step 2: Define Recovery Objectives

• Recovery Time Objective (RTO):
– Maximum acceptable downtime to restore data after a disaster.
– Example: “Restore within 2 hours.”
• Recovery Point Objective (RPO):
– Maximum acceptable data loss (e.g., “No more than 15 minutes of data loss”).
– Example: If RPO = 1 hour, backups must run at least hourly.

[Link] 7.3 Step 3: Choose Backup Location (Online vs. Offline)

Option Pros Cons

Online (Cloud) - Accessible from anywhere. - Dependent on internet connectivity.
- Automated backups. - Potential security risks (if misconfigured).
Offline - Full control over security. - Manual intervention required.
(On-Premise)
- Air-gapped (protected from cyber - Physical vulnerabilities (theft, disasters).
attacks).

[Link] 7.4 Step 4: Select Backup Strategy

• Options:
– Full-only: Simple but storage-intensive.
– Full + Incremental: Balances speed and storage.
– Full + Differential: Easier restoration than incremental.
– Hybrid (Combination): Tailored to specific needs (e.g., daily full + hourly incremental).

241
 • Factors to Consider:
– Backup frequency (how often).
– Data volume (size of database).
– Restoration speed (how quickly data must be recovered).

[Link] 7.5 Step 5: Implement Automation and Monitoring

• Tools:
– Automated backup software (e.g., Oracle RMAN, SQL Server Backup, MongoDB Ops Manager).
– Replication tools (e.g., AWS Database Migration Service, MySQL Replica).
• Key Features:
– Scheduling (automated backups at set intervals).
– Alerts (notifications for failures or anomalies).
– Multi-location replication (geographic redundancy).
– Verification (automated integrity checks).

6.5.8 8. Summary and Key Takeaways

• Why Backups Matter:
– Protect against data loss, corruption, and cyber threats.
– Ensure business continuity and disaster recovery.
• Backup Types:
– Full (complete copy), Incremental (changes since last backup), Differential (changes since last full).
• Strategy Development:
– Assess frequency, data volume, urgency, and data type.
– Define RTO and RPO based on business needs.
– Choose online/offline storage and automate processes.
• Best Practices:
– Regularly test backups to ensure recoverability.
– Update the backup plan as data growth and business needs evolve.
– Comply with regulations (e.g., GDPR, HIPAA) through documented backup procedures.

6.5.9 9. Final Recommendations

• Review and Update: Periodically reassess backup strategies to adapt to:
– New threats (e.g., emerging cyber attacks).
– Data growth (scaling storage needs).
– Regulatory changes (updated compliance requirements).
• Employee Training: Ensure IT staff and users understand:
– Backup procedures.
– Recovery protocols in case of emergencies.

6.6 Importing and Exporting Data

6.6.1 Introduction to Data Loading Techniques
[Link] Definition and Importance
• Data loading refers to the process of transferring data between:
– Different databases.
– A database and an external file.
• Critical applications include:

242
 – Data migration: Moving data between systems.
– Backup and recovery: Creating copies for disaster recovery.
– Database integration: Connecting databases with other systems.
– Data feeding: Transferring operational data to analytical platforms.

[Link] Key Terms and Definitions

1. Exporting
• The process of copying database data to an external file (e.g., .csv, .sql).
• Typically formatted for importing into another database (same or different DBMS).
2. Importing
• The reverse of exporting: copying data from external files into a database.
• Files are usually generated from a prior export operation.
3. Unloading
• Similar to exporting but copies data to a text file (e.g., .txt, .csv).
• Designed for use in non-database applications (e.g., spreadsheets, data analysis tools).
4. Loading
• The process of copying data into a database from an external text file.
• Supports formats like:
– CSV (Comma-Separated Values)
– **Oracle SQL*Loader-compatible formats**

6.6.2 Real-World Applications of Data Loading

• Database Migration: Moving data between different systems (e.g., from MySQL to PostgreSQL).
• Backup and Disaster Recovery: Creating redundant copies to restore data in case of failure.
• Data Integration: Combining data from multiple sources into a unified system.
• Analytical Platforms: Feeding operational data into data warehouses or business intelligence (BI) tools.

6.6.3 Techniques and Tools for Importing and Exporting Data

[Link] 1. WinSQL Techniques WinSQL provides multiple methods for data transfer:

[Link].1 A. Data Movement Between Relational Databases

• Functionality:
– Maps data types between source and target systems.
– Ensures compatibility during transfer.
• Use Cases:
– Migrating databases with different schemas.
– Synchronizing data across environments (e.g., development → production).

[Link].2 B. Backup to Local Files

• Process:
– Exports database data to a local file (e.g., .sql, .bak).
– Can be stored for backup purposes or transferred to another environment.
• Advantages:
– Portability: Files can be moved across systems.
– Recovery: Enables restoration in case of data loss.

243
[Link].3 C. Export to/Import from Text Files
• Exporting:
– Converts database tables into text-based formats (e.g., .csv, .txt).
– Useful for non-database applications (e.g., Excel, statistical software).
• Importing:
– Reads text files and loads data into database tables.
– Requires format compatibility (e.g., delimiter matching).

[Link].4 D. Generating SQL INSERT Statements

• Process:
– Creates SQL INSERT statements for existing data.
– Statements can be executed on another database to replicate data.
• Example:
INSERT INTO employees (id, name, department) VALUES (1, 'John Doe', 'IT');

• Use Cases:
– Data replication between identical schemas.
– Testing with production-like data in a staging environment.

[Link] 2. Drag-and-Drop Data Export

[Link].1 How It Works

• User Interface (UI) Method:
– Drag a table from one database in a management tool (e.g., WinSQL, SQL Server Management Studio).
– Drop it into another database to transfer data.
• Abstraction:
– Hides complex SQL or scripting.
– Simplifies the process for non-technical users.

[Link].2 Benefits
1. Streamlined Process:
• Reduces manual effort in writing export/import scripts.
2. User-Friendly:
• Accessible to users without SQL expertise.
3. Flexible File Support:
• Works with multiple formats (e.g., .csv, .xlsx, .txt).
4. Speed:
• Faster than manual scripting for small-to-medium datasets.

[Link] 3. Exporting SQL Query Results

[Link].1 Process
1. Write a SQL Query:

244
 • Example:
SELECT employee_id, first_name, last_name, salary
FROM employees
WHERE department = 'Marketing';
2. Execute in Database Tool (e.g., WinSQL, MySQL Workbench).
3. Export Results:
• Save output to formats like:
– Text (.txt, .csv)
– Microsoft Excel (.xlsx)
– Microsoft Access (.mdb, .accdb)

[Link].2 Benefits
1. Customizable Data Extraction:
• Users can filter and select only necessary data.
• Reduces unnecessary data handling.
2. Multi-Format Support:
• Enables sharing with non-technical stakeholders (e.g., Excel for analysts).
3. Analytical Flexibility:
• Exported data can be analyzed in external tools (e.g., Python, R, Tableau).

[Link] 4. SQL Server Import/Export Tools

• Built-in Tools:
– SQL Server Import and Export Wizard:
* Guided UI for transferring data between sources (e.g., SQL Server → Excel).
– Bulk Copy Program (BCP):
* Command-line tool for high-speed data transfer.
– SQL Server Integration Services (SSIS):
* Advanced ETL (Extract, Transform, Load) capabilities.
• Features:
– Data type mapping between source and destination.
– Error handling during transfer.
– Scheduling for automated imports/exports.

6.6.4 Summary of Key Concepts

1. Data Loading Fundamentals:
• Exporting: Database → External file.
• Importing: External file → Database.
• Unloading/Loading: Text-file-based transfers.
2. Techniques:
• WinSQL: Data movement, backups, SQL INSERT generation.
• Drag-and-Drop: UI-based, user-friendly transfers.
• SQL Query Exports: Customizable, multi-format results.
3. Tools:
• SQL Server: Import/Export Wizard, BCP, SSIS.
• Third-Party Tools: WinSQL, MySQL Workbench, Oracle SQL*Loader.
4. Real-World Use Cases:
• Migration, backups, analytics, integration.

245
 5. Best Practices:
• Preserve data integrity during transfers.
• Choose the right format (e.g., .csv for spreadsheets, .sql for replication).
• Automate repetitive tasks (e.g., scheduled backups).

6.7 Monitoring Database Performance

6.7.1 1. Introduction to Database Monitoring
[Link] 1.1 Definition of Database Monitoring
• Database monitoring (also called database performance monitoring) refers to the real-time tracking of
various metrics to assess the health and behavior of a database system.
• It involves continuous observation of performance indicators to detect issues early and take corrective ac-
tions before they impact system performance.

[Link] 1.2 Importance of Database Monitoring

• Databases are a critical component of IT infrastructure; any performance degradation can:
– Slow down application and service response times.
– Lead to downtime, affecting business operations.
– Cause data loss or corruption if issues are not addressed promptly.
• Key benefits of effective monitoring:
– Rapid identification and resolution of performance bottlenecks.
– Maintaining database availability (ensuring the database remains operational).
– Ensuring reliable service delivery for applications and end-users.
– Consistent performance, preventing sudden slowdowns or failures.

6.7.2 2. Key Factors Influencing Database Performance

Database performance depends on multiple interrelated factors that must be monitored and optimized regularly.

[Link] 2.1 Workload

• Definition: The amount of work or number of transactions the database handles in a given time.
• Impact:
– High workloads can lead to resource exhaustion (CPU, memory, disk I/O).
– Poorly managed workloads result in slow query execution and increased latency.

[Link] 2.2 Throughput

• Definition: The amount of work performed by the database over a given period, typically measured in:
– Transactions per second (TPS)
– Queries per second (QPS)
• Impact:
– Reflects the database’s ability to handle high volumes of data and transactions.
– Low throughput indicates bottlenecks in processing capacity.

[Link] 2.3 Resources

• Definition: The hardware and software resources available to the database, including:
– CPU (processing power)
– Memory (RAM) (caching and query execution)

246
 – Storage (disk I/O) (read/write operations)
– Network bandwidth (data transfer speed)
• Impact:
– Inadequate resources lead to slow performance and system crashes.
– Poor resource allocation (e.g., memory leaks, CPU starvation) degrades efficiency.

[Link] 2.4 Optimization

• Definition: The process of fine-tuning database components to enhance performance, including:
– Database configuration (settings like buffer pool size, cache parameters).
– SQL query optimization (rewriting inefficient queries, using proper joins).
– Indexing strategies (creating, modifying, or removing indexes to speed up searches).
– Partitioning and sharding (distributing data to reduce load).
• Impact:
– Improves response times.
– Reduces resource consumption (CPU, memory, disk usage).

[Link] 2.5 Contention

• Definition: Occurs when multiple processes or transactions compete for the same resources or data.
• Common issues:
– Locking conflicts (e.g., two transactions trying to modify the same row).
– Deadlocks (where transactions wait indefinitely for each other to release locks).
– Resource starvation (e.g., CPU or memory bottlenecks).
• Impact:
– Leads to delays, timeouts, and performance degradation.
– May cause transaction failures if not managed properly.

[Link] 2.6 Summary of Performance Factors

Factor Key Metrics to Monitor Potential Issues if Neglected

Workload Transactions/sec, active sessions Overload, slow response times
Throughput Queries/sec, batch processing speed Bottlenecks, inability to handle peak loads
Resources CPU usage, memory consumption, disk I/O Crashes, high latency
Optimization Query execution plans, index usage Inefficient queries, high resource usage
Contention Lock waits, deadlocks, blockings Transaction failures, timeouts

6.7.3 3. Key Performance Indicators (KPIs) for Database Monitoring

Monitoring KPIs helps assess database health and performance. The most critical KPIs include:

[Link] 3.1 Query Response Time

• Definition: The average time it takes for the database to respond to a query.
• Why it matters:
– Directly impacts user experience (slow queries frustrate users).
– Helps identify bottlenecks (e.g., poorly optimized queries, missing indexes).
• Monitoring approach:
– Track average, minimum, and maximum response times.
– Set thresholds for acceptable performance (e.g., queries should not exceed 200ms).

247
[Link] 3.2 Database Throughput
• Definition: The volume of work the database processes over time (e.g., queries/sec, transactions/sec).
• Why it matters:
– Indicates whether the database can handle current and future loads.
– Helps in capacity planning (scaling resources as needed).
• Monitoring approach:
– Measure peak vs. average throughput.
– Compare against historical data to detect trends.

[Link] 3.3 Errors

• Definition: The number and type of errors generated by the database (e.g., query failures, connection
errors).
• Why it matters:
– Errors can indicate:
* Data integrity issues (e.g., constraint violations).
* System failures (e.g., disk full, memory leaks).
* Security breaches (e.g., unauthorized access attempts).
• Monitoring approach:
– Categorize errors (e.g., syntax errors, timeouts, deadlocks).
– Set up alerts for critical errors (e.g., failed backups, corruption).

[Link] 3.4 Open Connections

• Definition: The number of active connections to the database at any given time.
• Why it matters:
– Too many open connections can overload the database.
– Connection leaks (unclosed connections) waste resources.
• Monitoring approach:
– Track connection count trends.
– Set limits on maximum connections to prevent overload.

[Link] 3.5 Most Frequent Queries

• Definition: The queries executed most often in the database.
• Why it matters:
– Optimizing frequent queries provides the biggest performance gains.
– Helps identify repetitive, inefficient operations (e.g., full table scans).
• Monitoring approach:
– Use query logging to identify high-frequency queries.
– Analyze execution plans to optimize them (e.g., add indexes, rewrite queries).

6.7.4 4. Challenges in Database Monitoring

[Link] 4.1 Determining What to Monitor
• Problem: With hundreds of potential metrics, it can be overwhelming to decide which to track.
• Solution:
– Start with essential metrics (response time, throughput, errors, connections).
– Gradually expand monitoring as the database grows.
– Prioritize metrics that directly impact user experience and system stability.

248
[Link] 4.2 Avoiding Information Overload
• Problem: Monitoring too many metrics can lead to alert fatigue and inefficient troubleshooting.
• Solution:
– Focus on high-impact metrics (those affecting performance the most).
– Use threshold-based alerts to filter noise.
– Automate analysis where possible (e.g., anomaly detection).

[Link] 4.3 Scaling Monitoring with Database Growth

• Problem: As databases grow in size and complexity, monitoring systems must scale accordingly.
• Solution:
– Ensure monitoring tools support large-scale databases.
– Distribute monitoring (e.g., separate resource and query monitoring).
– Use sampling for high-volume metrics (e.g., log only a percentage of queries).

6.7.5 5. Critical Metrics to Monitor

The following metrics are essential for effective database performance monitoring:

Metric Description Impact of Poor Performance

Response Time Average time for query execution. Slow user experience, application
timeouts.
Throughput Number of queries/transactions processed per second. Bottlenecks, inability to handle
peak loads.
Open Number of active database connections. Resource exhaustion, connection
Connections refusals.
Errors Count and type of database errors (e.g., timeouts, Data corruption, downtime.
deadlocks).
Most Frequent Queries executed most often. High load from inefficient queries.
Queries
CPU Usage Percentage of CPU consumed by database processes. Slow query execution, system lag.
Memory Usage RAM consumption (e.g., buffer pool usage). Swapping to disk, performance
degradation.
Disk I/O Read/write operations per second. Slow data retrieval, storage
bottlenecks.
Network Delay in data transmission between database and Slow application response times.
Latency applications.
Lock Waits Time transactions spend waiting for locks. Deadlocks, transaction failures.

6.7.6 6. Types of Database Monitors

Different types of monitors focus on specific aspects of database performance.

[Link] 6.1 Resource Monitor

• Purpose: Tracks hardware and system resources used by the database.
• Key Metrics:
– CPU usage (percentage utilized by database processes).
– Memory consumption (buffer pool, cache hit ratio).

249
 – Storage usage (disk space, I/O operations).
• Tools: top, vmstat, Performance Monitor (Windows), Nagios.

[Link] 6.2 Network Monitor

• Purpose: Observes network performance between the database and connected applications.
• Key Metrics:
– Latency (delay in data transmission).
– Bandwidth usage (data transfer rates).
– Packet loss (indicates network issues).
• Tools: Wireshark, ping, traceroute, Nagios Network Monitoring.

[Link] 6.3 Application Performance Monitor (APM)

• Purpose: Monitors how applications interact with the database.
• Key Metrics:
– Application response time (end-to-end latency).
– Query performance (slow queries from the application).
– Connection pooling efficiency.
• Tools: New Relic, Dynatrace, AppDynamics.

[Link] 6.4 Third-Party Component Monitor

• Purpose: Tracks external plugins, extensions, or integrations used by the database.
• Key Metrics:
– Plugin response time.
– Error rates in third-party components.
– Resource usage by external tools.
• Tools: Custom scripts, Nagios plugins, Prometheus exporters.

6.7.7 7. Choosing Monitoring Tools: Open Source vs. Commercial

Criteria Open Source Tools Commercial Tools

Cost Free (but may require development effort). Licensed (can be expensive).
Customization Highly customizable (modify source code). Limited to vendor-provided features.
Features Basic to advanced (depends on tool). Often include AI-driven analytics,
predictive alerts.
Support Community-based (forums, Dedicated vendor support (SLAs).
documentation).
Scalability May require manual scaling. Often enterprise-ready (handles large
databases).
Examples Prometheus, Grafana, Zabbix, Nagios. Datadog, SolarWinds, New Relic.

[Link] 7.1 Best Practices for Tool Selection

1. Assess organizational needs (budget, database size, required features).
2. Evaluate ease of use (UI, setup complexity).
3. Check integration capabilities (compatibility with existing systems).
4. Consider future scalability (will the tool grow with the database?).
5. Test before full deployment (use trials or pilot programs).

250
6.7.8 8. Best Practices for Database Monitoring
[Link] 8.1 Monitor Availability and Resource Consumption
• Availability:
– Ensure the database is always accessible (uptime monitoring).
– Use heartbeat checks to detect outages.
• Resource Consumption:
– Set thresholds for CPU, memory, and disk usage.
– Alert when resources near capacity (e.g., 90% CPU usage).

[Link] 8.2 Monitor Slow Queries

• Identify slow queries using:
– Query logs (e.g., MySQL slow query log, SQL Server Profiler).
– Execution plan analysis (look for full table scans, missing indexes).
• Optimize slow queries by:
– Adding indexes on frequently queried columns.
– Rewriting queries (avoid SELECT *, use proper joins).
– Caching results for repeated queries.

[Link] 8.3 Measure Throughput

• Track throughput trends to:
– Detect performance degradation over time.
– Plan for scaling (e.g., add more servers, optimize queries).
• Use benchmarking to compare against industry standards.

[Link] 8.4 Monitor Logs

• Regularly review database logs for:
– Errors (e.g., failed logins, deadlocks).
– Unusual activity (e.g., sudden spikes in queries).
– Security events (e.g., unauthorized access attempts).
• Automate log analysis where possible (e.g., log parsing scripts).

[Link] 8.5 Set Up Alerts and Notifications

• Configure alerts for:
– Critical errors (e.g., database crashes).
– Resource thresholds (e.g., disk space < 10%).
– Performance anomalies (e.g., response time > 1s).
• Use multiple notification channels (email, SMS, Slack).

[Link] 8.6 Regularly Review and Update Monitoring Strategies

• Conduct periodic reviews of monitoring setup.
• Update thresholds as database usage changes.
• Adopt new tools/features as technology evolves.

251
6.7.9 9. Case Study: Monitoring MS SQL Server with Nagios XI
[Link] 9.1 Overview of Nagios XI
• Purpose: A commercial monitoring tool that supports MS SQL Server monitoring.
• Key Features:
– Microsoft SQL Server Wizard for easy setup.
– Predictive analysis to forecast resource needs.
– Customizable dashboards for real-time insights.

[Link] 9.2 Key Metrics Monitored by Nagios XI for MS SQL

Metric Description Impact

Connection Time Time taken to establish a database connection. Slow connections indicate
network issues.
Buffer Hit Ratio Percentage of pages found in the buffer pool (vs. disk Low ratio → excessive disk I/O.
reads).
Page Looks Number of times SQL Server searches for a page in the High looks → memory pressure.
buffer.
Free Pages Number of unused pages in the buffer pool. Low free pages → memory
shortage.
Target Pages Pages SQL Server aims to keep in the buffer. Helps tune memory allocation.
Stolen Pages Pages used by SQL Server for internal operations. High stolen pages → memory
contention.
Lazy Writes Pages written to disk due to memory pressure. Indicates buffer pool stress.
Read-Ahead Pages preloaded into the buffer for expected use. Low read-ahead → poor query
Pages planning.
Page Disk I/O operations (reads/writes per second). High I/O → storage bottleneck.
Reads/Writes
Lock Re- Number of lock requests and timeouts. High locks → contention issues.
quests/Timeouts
Deadlocks Instances where transactions are stuck waiting for each Causes transaction failures.
other.
Page Splits Occurs when an index page fills up and splits. Leads to fragmentation.
Log Wait Time Time transactions wait for log writes. High wait → log disk bottleneck.
Average Wait Average time queries spend waiting for resources. High wait → performance
Time degradation.

[Link] 9.3 Benefits of Using Nagios XI for MS SQL Monitoring

1. Increased Application Availability
• Proactively detects database outages before they affect users.
2. Improved Database Performance
• Identifies bottlenecks (e.g., slow queries, lock contention).
3. Fast Detection of Issues
• Alerts for table corruption, failed backups, deadlocks.
4. Predictive Analysis
• Forecasts future resource needs (e.g., CPU, memory, storage).
5. Integration & Customization
• Works with other monitoring tools (e.g., Grafana, Prometheus).

252
 • Allows custom scripts and plugins for specialized monitoring.

6.7.10 10. Conclusion

[Link] 10.1 Summary of Key Takeaways
• Database monitoring is essential for maintaining performance, availability, and reliability.
• Key performance factors include workload, throughput, resources, optimization, and contention.
• Critical KPIs to monitor: response time, throughput, errors, open connections, frequent queries.
• Types of monitors: Resource, network, APM, third-party component monitors.
• Best practices:
– Monitor availability and resource usage.
– Optimize slow queries.
– Measure throughput trends.
– Review logs regularly.
– Use alerts for proactive issue resolution.
• Tools like Nagios XI provide comprehensive monitoring for databases like MS SQL Server.

[Link] 10.2 Final Recommendations

1. Start with essential metrics (response time, errors, connections).
2. Gradually expand monitoring as the database grows.
3. Use a mix of open-source and commercial tools based on needs.
4. Automate where possible to reduce manual effort.
5. Regularly review and update monitoring strategies to adapt to changes.

6.8 Optimising Database Performance

6.8.1 Introduction to Database Performance Optimisation
• Objective: Understand mechanisms for optimising APIs and database performance, apply query optimisa-
tion techniques, and utilise monitoring tools to identify and resolve performance bottlenecks.
• Key Focus Areas:
– Caching mechanisms to reduce load times and improve query performance.
– Database query optimisation techniques.
– Monitoring tools and practices for real-time performance analysis.
– Real-world case studies (Netflix, Airbnb).

6.8.2 Commonly Used Mechanisms for Optimising Database Performance

[Link] 1. Caching Mechanisms Caching is essential for reducing database load, improving query performance,
and ensuring a smooth user experience.

[Link].1 A. In-Memory Caching

• Definition: Stores frequently accessed data in memory (RAM) to minimise disk I/O operations.
• Purpose:
– Reduces the time required to fetch data from the database.
– Improves response times for read-heavy applications.
• Use Cases:
– Session storage.
– Frequently accessed query results.

253
 – Temporary data processing.

[Link].2 B. Content Delivery Network (CDN)

• Definition: Distributes static and dynamic content across geographically dispersed servers.
• Purpose:
– Reduces latency by serving content from the nearest edge server.
– Improves access speed for global users.
• Use Cases:
– Media streaming (videos, images).
– Web applications with global user bases.
– Static asset delivery (CSS, JavaScript, images).

[Link].3 C. HTTP Caching

• Definition: Stores server responses locally (browser or proxy cache) to avoid redundant requests.
• Purpose:
– Reduces server load by serving cached responses for repeated requests.
– Decreases bandwidth usage.
• Mechanisms:
– Cache-Control Headers: Define caching policies (e.g., max-age, no-cache).
– ETags (Entity Tags): Validate cached content to avoid re-downloading unchanged resources.
• Use Cases:
– Static web pages.
– API responses with infrequent updates.

[Link].4 D. Full-Page Caching

• Definition: Stores entire rendered pages in cache to serve them directly without dynamic generation.
• Purpose:
– Reduces server processing load.
– Improves response times for high-traffic pages.
• Use Cases:
– E-commerce product pages.
– Blog posts with infrequent updates.
– Landing pages with static content.

6.8.3 Techniques for Optimising Database Queries

[Link] 1. Query Plan Analysis and Optimisation
• Definition: A query execution plan is a step-by-step breakdown of how a database engine retrieves data.
• Purpose:
– Identify inefficient operations (e.g., full table scans, unnecessary joins).
– Optimise query structure for faster execution.
• Tools:
– EXPLAIN (PostgreSQL, MySQL): Displays the query execution plan.
– SQL Server Execution Plan: Graphical representation of query steps.
• Optimisation Strategies:
– Avoid SELECT *; fetch only required columns.
– Use appropriate join types (e.g., INNER JOIN vs. LEFT JOIN).
– Limit result sets with WHERE, LIMIT, or OFFSET.

254
[Link] 2. Query Rewriting
• Definition: Restructuring queries to improve efficiency without changing functionality.
• Techniques:
– Simplification: Break complex queries into smaller, manageable subqueries.
– Avoiding Nested Queries: Replace correlated subqueries with joins where possible.
– Using Common Table Expressions (CTEs): Improve readability and performance for complex logic.
• Example:
– Original: SELECT * FROM orders WHERE customer_id IN (SELECT id FROM customers WHERE
status = 'active')
– Optimised: SELECT o.* FROM orders o JOIN customers c ON o.customer_id = [Link] WHERE
[Link] = 'active'

[Link] 3. Indexing
• Definition: A data structure that improves the speed of data retrieval operations.
• How It Works:
– Acts like a book index, allowing the database to locate data without scanning the entire table.
– Uses B-trees, hash indexes, or bitmap indexes depending on the database system.
• Best Practices:
– Columns to Index:
* Primary keys (automatically indexed).
* Foreign keys.
* Frequently queried columns (e.g., WHERE, JOIN, ORDER BY clauses).
– Avoid Over-Indexing:
* Too many indexes slow down INSERT, UPDATE, and DELETE operations.
* Regularly review and remove unused indexes.
• Types of Indexes:
– Single-Column Index: Index on one column.
– Composite Index: Index on multiple columns (order matters).
– Unique Index: Ensures no duplicate values in the indexed column(s).
– Full-Text Index: Optimised for text search operations.

[Link].1 Benefits of Indexing

• Faster Query Execution: Reduces the time to locate data, especially in large tables.
• Reduced Server Load: Efficient queries lower CPU and memory usage.
• Improved Scalability: Helps maintain performance as data volume grows.

[Link] 4. Database-Specific Optimisations

• Partitioning (Oracle, SQL Server, PostgreSQL):
– Definition: Divides large tables into smaller, manageable partitions based on criteria (e.g., range, list,
hash).
– Benefits:
* Improves query performance by scanning only relevant partitions.
* Simplifies maintenance (e.g., archiving old data).
– Example: Partitioning sales data by month (sales_2023_01, sales_2023_02).
• Materialised Views (Oracle, PostgreSQL):
– Definition: Pre-computed query results stored as tables.
– Use Case: Complex aggregations that are frequently accessed but rarely updated.

255
[Link] 5. Denormalisation
• Definition: Introduces redundancy by combining tables to reduce the need for joins.
• Purpose:
– Improves read performance for frequently accessed data.
– Reduces join complexity in queries.
• Trade-offs:
– Pros: Faster reads, simpler queries.
– Cons: Increased storage usage, potential data inconsistency.
• Use Cases:
– Reporting databases where read performance is critical.
– Applications with heavy read loads and infrequent writes.

6.8.4 Data Sharding

[Link] 1. Definition
• A horizontal partitioning technique that distributes data across multiple database instances (shards).
• Each shard contains a subset of the data and operates independently.

[Link] 2. How It Works

• Sharding Key: A column (e.g., user_id, region) used to distribute data evenly.
• Shard Distribution:
– Range-Based: Data split by value ranges (e.g., user_id 1-1000 on Shard 1, 1001-2000 on Shard 2).
– Hash-Based: Data distributed using a hash function for even distribution.
– Directory-Based: A lookup table maps keys to shards.

[Link] 3. Benefits
• Scalability: Adds more shards to handle increased load.
• Performance: Parallel processing across shards reduces query latency.
• Fault Isolation: A failure in one shard does not affect others.

[Link] 4. Challenges
• Complexity: Requires careful design for sharding keys and query routing.
• Cross-Shard Queries: Joins or aggregations across shards can be inefficient.
• Rebalancing: Adding/removing shards may require data redistribution.

[Link] 5. Use Cases

• High-Traffic Applications: Social media platforms, e-commerce sites.
• Global Applications: Distributing data by geographic region.

6.8.5 Monitoring Database Performance

[Link] 1. Importance of Monitoring
• Identifies performance bottlenecks before they impact users.
• Ensures database resources (CPU, memory, disk I/O) are optimally utilised.
• Provides data for capacity planning and optimisation.

256
[Link] 2. Monitoring Tools and Practices

[Link].1 A. Application Performance Monitoring (APM) Tools

• Definition: Software that tracks application and database performance metrics.
• Features:
– Real-time monitoring of query execution times.
– Alerts for slow queries or failures.
– Transaction tracing to identify bottlenecks.
• Examples:
– New Relic.
– Datadog.
– AppDynamics.

[Link].2 B. Logging and Tracing

• Definition: Records database activities, queries, and transactions for analysis.
• Purpose:
– Diagnose slow queries or errors.
– Audit database usage and security events.
• Best Practices:
– Log critical queries and their execution times.
– Use structured logging (JSON, XML) for easier analysis.
– Implement distributed tracing for microservices architectures.

[Link].3 C. System Resource Monitoring

• Metrics to Track:
– CPU Usage: High CPU may indicate inefficient queries or lack of indexing.
– Memory Usage: Insufficient memory leads to disk swapping and slow performance.
– Disk I/O: High read/write latency can bottleneck performance.
– Network Latency: Affects distributed databases and replication.
• Tools:
– OS-Level: top, htop, vmstat (Linux); Task Manager (Windows).
– Database-Specific:
* MySQL: SHOW PROCESSLIST, Performance Schema.
* PostgreSQL: pg_stat_activity.
* SQL Server: Dynamic Management Views (DMVs).
[Link].4 D. Real User Monitoring (RUM)
• Definition: Measures performance from the end-user perspective.
• Metrics:
– Page load times.
– API response times.
– User interaction delays.
• Purpose:
– Correlate database performance with user experience.
– Identify geographic or device-specific issues.
• Tools:
– Google Analytics.

257
 – New Relic Browser Monitoring.
– Sentry.

6.8.6 Case Studies: Real-World Applications

[Link] 1. Netflix
• Challenge: Deliver streaming content to millions of users globally with minimal latency.
• Solutions:
– Multi-Level Caching:
* CDN Caching: Distributes videos and static assets globally.
* In-Memory Caching (Redis, Memcached): Stores user session data and frequently accessed
metadata.
– Database Optimisations:
* Read replicas for scaling read operations.
* Sharding for user-specific data (e.g., watch history).
• Results:
– Reduced database load by serving cached content.
– Improved response times and streaming quality.

[Link] 2. Airbnb
• Challenge: Handle vast amounts of data (listings, bookings, user profiles) while scaling infrastructure.
• Solutions:
– Data Sharding:
* Horizontally partitioned databases by geographic regions.
* Each shard handles a subset of listings and users.
– Denormalisation:
* Combined frequently joined tables to reduce query complexity.
– Query Optimisation:
* Indexed search-relevant columns (e.g., location, price).
* Used materialised views for reporting.
• Results:
– Improved scalability to handle millions of users.
– Faster search and booking operations.

6.8.7 Summary and Key Takeaways

• Optimisation Mechanisms:
– Caching (in-memory, CDN, HTTP, full-page) reduces load times and improves performance.
– Query optimisation (indexing, rewriting, partitioning) ensures efficient database operations.
• Scaling Techniques:
– Sharding distributes data across servers for horizontal scalability.
– Denormalisation and materialised views optimise read-heavy workloads.
• Monitoring:
– APM tools, logging, and RUM provide insights into performance bottlenecks.
– System resource monitoring ensures optimal database operation.
• Real-World Applications:
– Netflix and Airbnb demonstrate the impact of caching, sharding, and query optimisation on scalability
and user experience.

258
Final Note: Optimising database performance is an ongoing process requiring continuous monitoring, testing, and
refinement to adapt to evolving application demands.

6.9 Performing Backup

6.9.1 Introduction to Database Backup
• Objective: Understand tools, software, and processes (manual/automated) for performing database backups.
• Key Outcomes:
– Identify and utilize built-in and third-party backup tools.
– Execute manual and automated backup processes step-by-step.
– Implement best practices for reliable database backups.

6.9.2 Backup Tools and Software

[Link] 1. Built-in Database Tools
• Definition: Native tools provided by database management systems (DBMS) for fundamental backup oper-
ations.
• Examples:
– Microsoft Access:
* Basic backup functionality via the File tab.
* Supports manual backups through Save As options.
– SQL Server Management Studio (SSMS) (for Microsoft SQL Server):
* Streamlined backup operations within the interface.
* Integrates directly with the DBMS for seamless backups.
• Advantages of Built-in Tools:
– Convenience: No additional software required.
– Integration: Directly accessible from the database management interface.
– Simplicity: Ideal for small-scale or straightforward backup needs.

[Link] 2. Third-Party Backup Solutions

• Definition: Enterprise-grade tools offering advanced backup capabilities beyond native DBMS features.
• Examples:
– Veritas Backup Exec
– Veeam Backup & Replication
– Commvault
• Key Features:
– Deduplication: Reduces storage usage by eliminating redundant data.
– Encryption: Secures backups to prevent unauthorized access.
– Cloud Storage Integration: Supports backups to cloud platforms (e.g., AWS, Azure).
– Comprehensive Management: Centralized control for large-scale or complex database environments.
• Use Cases:
– Large-scale databases.
– Organizations requiring high availability, security, and compliance.
– Environments with complex backup/recovery needs.

6.9.3 Manual Backup Process

• Context: Step-by-step guide using SQL Server Management Studio (SSMS) as an example.

259
[Link] Steps:
1. Navigate to the File Tab:
• Open the database in Microsoft Access or SSMS.
2. Select “Save As”:
• Choose the option to save the database under a new name.
3. Choose Database File Type:
• Select the appropriate format (e.g., .accdb for Access, .bak for SQL Server).
4. Advanced Settings:
• Click on Backup Database (or equivalent option).
5. Save As Backup:
• Specify a backup file name and location.
• Confirm to create the backup.
• Advantages:
– Simplicity: Straightforward process with minimal steps.
– Immediate Availability: Ensures a recent backup is available for recovery in case of failures.
• Limitations:
– Human Error: Risk of forgetting to perform backups manually.
– Inconsistency: May not be executed regularly without discipline.

6.9.4 Automated Backup Process

• Context: Using VBScript and Windows Task Scheduler for automation in Microsoft Access/SSMS.

[Link] Steps:
1. Automate with VBScript:
• Write a script to copy the current database file to a backup location.
• Example:
' Sample VBScript to copy a database file to a backup location
Set fso = CreateObject("[Link]")
[Link] "C:\Databases\[Link]", "C:\Backups\[Link]", True
2. Add a Delay (Optional):
• Introduce a 10-second delay to ensure the database is not in use during the copy.
• Example:
[Link] 10000 ' Delay in milliseconds
3. Execute the Script:
• Run the script via Windows Command Prompt or Task Scheduler.
• Advantages:
– Consistency: Eliminates human error by automating the process.
– Efficiency: Reduces manual intervention, ensuring backups are performed on schedule.
– Reliability: Minimizes the risk of missed backups.

6.9.5 Using Third-Party Backup Tools

[Link] 1. Installation and Setup
• Steps:
1. Install the Tool:

260
 – Download and install the preferred third-party solution (e.g., Veeam, Commvault).
2. Follow Setup Wizard:
– Configure initial settings (e.g., license activation, network permissions).
3. Configure Backup Settings:
– Define backup frequency (daily, weekly, monthly).
– Specify storage location (local, cloud, hybrid).
4. Customize Preferences:
– Select backup type (full, incremental, differential).
– Enable encryption for security.
– Set retention policies (how long backups are kept).
• Benefits:
– Tailored Solutions: Meets specific organizational requirements (e.g., compliance, RTO/RPO).
– Advanced Features: Deduplication, encryption, and cloud integration.
– Scalability: Suitable for growing or complex database environments.

[Link] 2. Running the Backup

• Steps:
1. Initiate Backup:
– Start the backup process from the tool’s interface.
2. Monitor Progress:
– Track the backup operation for alerts or errors.
– Verify data integrity post-backup.
3. Ensure Recovery Readiness:
– Test backup restoration periodically to confirm usability.
• Key Considerations:
– Data Integrity: Ensure backups are corruption-free.
– Alerts: Configure notifications for failures or anomalies.
– Quick Recovery: Optimize backup settings for fast restoration when needed.

6.9.6 Best Practices for Database Backup

[Link] 1. Regular Backup Schedule
• Definition: A structured plan to ensure backups are performed consistently.
• Components:
– Daily Backups:
* Capture the most recent changes.
* Minimize data loss in case of failure.
– Weekly Backups:
* Serve as checkpoints for mid-term recovery.
* Provide multiple restore points within a week.
– Monthly Backups:
* Long-term archival for historical data.
* Enable retrieval of specific versions from past months.
• Implementation:
– Use automation (e.g., scripts, Task Scheduler) to enforce consistency.
– Document the schedule and assign ownership for accountability.

[Link] 2. Automating the Backup Process

261
 • Methods:
– VBScript Automation:
* Script the copy process for database files.
* Example:
' Automated backup script
Set fso = CreateObject("[Link]")
source = "C:\Databases\[Link]"
destination = "C:\Backups\ProductionDB_" & Year(Now) & Month(Now) & Day(Now) & ".accdb"
[Link] source, destination, True
– Windows Task Scheduler:
* Schedule the script to run at predetermined intervals (e.g., nightly at 2 AM).
• Advantages:
– Reduced Workload: Minimizes manual effort for IT teams.
– Reliability: Ensures backups are not missed due to human oversight.

[Link] 3. Using Windows Task Scheduler

• Setup Steps:
1. Create a New Task:
– Open Task Scheduler ([Link]).
– Click Create Task.
2. Configure Triggers:
– Set the frequency (daily, weekly) and start time.
3. Define Actions:
– Specify the script or program to run (e.g., VBScript file).
4. Customize Settings:
– Adjust conditions (e.g., run only if the system is idle).
– Set error handling (e.g., retry on failure).
5. Test the Task:
– Run the task manually to verify functionality.
• Best Practices:
– Logging: Enable task history to track execution and failures.
– Notifications: Configure email alerts for task failures.
– Redundancy: Store backups in multiple locations (local + cloud).

6.9.7 Summary of Key Takeaways

1. Tools:
• Use built-in tools (SSMS, Access) for simplicity or third-party solutions (Veeam, Commvault) for
advanced features.
2. Processes:
• Manual backups are straightforward but prone to human error.
• Automated backups (via scripts/Task Scheduler) ensure consistency.
3. Best Practices:
• Implement a multi-tiered backup schedule (daily/weekly/monthly).
• Automate backups to reduce risk and workload.
• Monitor and test backups regularly to ensure recoverability.
• Final Note: A robust backup strategy is critical for data integrity, disaster recovery, and business conti-
nuity. By leveraging the right tools and adhering to best practices, organizations can mitigate data loss risks

262
 effectively.

6.10 Rebuilding Indexes

6.10.1 Introduction to Rebuilding Indexes
[Link] Definition of Rebuilding an Index
• Rebuilding an index involves deleting the existing index and replacing it with a new one.
• This process is essential for maintaining data retrieval efficiency and overall database performance.
• Key actions during rebuilding:
– Eliminates fragmentation.
– Compacts pages based on existing fill factor settings.
– Reclaims storage space.
• The Database Management System (DBMS):
– Drops the existing index.
– Recreates it from scratch.

[Link] Difference Between Rebuilding and Reorganizing

Aspect Rebuilding Reorganizing

Scope Full index restructuring Focuses on leaf-level adjustments
Process Drops and recreates index Reorders physical pages without
full recreation
Fragmentation Level Recommended when Suitable for mild fragmentation
fragmentation >= 40%
Impact Significantly reduces Moderate performance
fragmentation improvement
Use Case When index is highly When minor optimizations are
disorganized needed

6.10.2 Importance of Indexing in Database Management

[Link] Fundamental Role of Indexing
• Indexing is a core technique for optimizing query performance.
• Without proper indexing, databases become slow and inefficient as they grow in size and complexity.
• Analogy: Indexes function like a table of contents in a book, enabling faster data retrieval.

[Link] Benefits of Proper Indexing

1. Reduces Disk I/O Operations
• Disk I/O is one of the slowest operations in computing.
• Indexes minimize the amount of data read from disk by directing queries to relevant rows.
2. Accelerates Query Performance
• Frequently accessed columns (e.g., those used in WHERE clauses, JOINs, or filters) benefit most
from indexing.
3. Optimizes Resource Usage
• Reduces CPU and memory load by avoiding full table scans.

263
[Link] Tips for Selecting the Right Columns to Index
1. Analyze Query Patterns
• Identify columns frequently used in searches, joins, and filters.
• Example: If CustomerID is often used in WHERE clauses, it should be indexed.
2. Prioritize High-Selectivity Columns
• Selectivity = Ratio of unique values to total rows in a column.
• High selectivity (e.g., Email, SSN) is better for indexing than low selectivity (e.g., Gender, Status).
3. Index Columns Used in Joins Between Large Tables
• Joins on unindexed columns in large tables are resource-intensive and slow.
4. Avoid Over-Indexing
• Drawbacks of over-indexing:
– Increased storage requirements (each index consumes space).
– Slower write operations (INSERT, UPDATE, DELETE must update all indexes).
• Best Practice: Only index columns that provide measurable performance benefits.

6.10.3 Index Fragmentation: Causes, Impacts, and Solutions

[Link] Definition of Index Fragmentation
• Physical disorganization of an index’s data pages on disk or in memory.
• Initially, indexes are well-structured, but data modifications (INSERT, UPDATE, DELETE) cause scat-
tering of pages.
• Result: The DBMS must perform extra work to locate and retrieve data.

[Link] Primary Causes of Fragmentation

1. Data Modifications
• Insertions: New data may not fit in existing pages, forcing page splits.
• Updates: Changing key values may require moving rows to maintain index order.
• Deletions: Remove rows, leaving gaps in pages.
2. Fill Factor Settings
• Fill factor = Percentage of space left empty in index pages to accommodate future growth.
• Low fill factor → More frequent page splits → Higher fragmentation.

[Link] Impacts of High Fragmentation

Impact Explanation
Increased Disk I/O DBMS must read multiple scattered pages, slowing queries.
Higher Memory Usage More memory required to track fragmented pages.
Larger Database Size Unused gaps from deletions and splits bloat storage.
Longer Backups & Recovery Fragmented indexes increase backup size and slow
restoration.

[Link] Methods to Reduce Fragmentation

1. Index Reorganization (Reorg)
• Repairs physical order of leaf-level pages without full recreation.
• Less resource-intensive than rebuilding.
• Best for: Mild fragmentation (< 40%).

264
 2. Index Rebuilding
• Drops and recreates the index from scratch.
• Eliminates all fragmentation and reclaims unused space.
• Best for: Severe fragmentation (>= 40%).
3. Regular Maintenance
• Schedule periodic reorg/rebuild tasks to prevent fragmentation buildup.
• Monitor fragmentation levels using DBMS tools (e.g., SQL Server’s sys.dm_db_index_physical_stats).

[Link] Best Practices for Minimizing Fragmentation

1. Monitor Fragmentation Levels
• Use SQL Server Management Studio (SSMS) or T-SQL queries to check fragmentation.
• Example query:
SELECT * FROM sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, 'DETAILED');
2. Set Appropriate Fill Factors
• Balance between storage efficiency and performance:
– High fill factor (e.g., 90-100%) → Less fragmentation but more page splits.
– Low fill factor (e.g., 70-80%) → More free space but higher storage usage.
3. Optimize Query Plans
• Review execution plans to identify inefficient index usage.
• Avoid unnecessary scans by ensuring queries use indexed columns.
4. Automate Maintenance
• Schedule regular reorg/rebuild jobs (e.g., weekly or monthly).
• Use tools like Ola Hallengren’s scripts for automated maintenance.

6.10.4 Step-by-Step: Reorganizing an Index in SQL Server Management Studio (SSMS)

1. Navigate to the Index
• Expand the database containing the target table.
• Expand the Tables folder → Select the specific table.
• Expand the Indexes folder.
2. Initiate Reorganization
• Right-click the index → Select Reorganize.
3. Configure Reorganization
• In the Reorganize Indexes dialog:
– Verify the correct index is selected.
– (Optional) Check Compact large object column data to optimize LOB (Large Object) data.
• Click OK to execute.

6.10.5 Automating Index Maintenance

[Link] Importance of Automation
• Ensures consistency in database performance.
• Reduces manual workload for DBAs.
• Prevents performance degradation from neglected maintenance.

[Link] Tools for Automation

1. Ola Hallengren’s Maintenance Scripts
• Pre-built scripts for SQL Server to automate:

265
 – Index reorganization.
– Index rebuilding.
– Fragmentation threshold-based maintenance.
• Download: [Link]
2. SQL Server Maintenance Plans
• GUI-based tool in SSMS to schedule:
– Index rebuilds.
– Database integrity checks.
– Backups.
3. SQL Server Agent Jobs
• Create custom jobs to run T-SQL scripts at scheduled intervals.
• Example:
-- Rebuild indexes with fragmentation > 30%
EXEC sp_MSforeachtable 'IF EXISTS (SELECT * FROM sys.dm_db_index_physical_stats(DB_ID(), OBJ
BEGIN
ALTER INDEX ALL ON ? REBUILD;
END';

[Link] Advanced Automation Techniques

1. Parallel Processing
• Rebuild multiple indexes simultaneously to reduce downtime.
• Use MAXDOP (Maximum Degree of Parallelism) to control CPU usage.
2. Stop/Restart Protocols
• Pause and resume long-running rebuilds to minimize disruptions.
• Use checkpoints to track progress.
3. Handling Large Indexes
• Partition large indexes to rebuild in smaller chunks.
• Schedule off-peak hours to avoid impacting production workloads.

[Link] Scheduling Index Maintenance

1. SQL Server Agent Jobs
• Create a new job in SQL Server Agent.
• Define steps (e.g., run Ola Hallengren’s script).
• Set a schedule (e.g., weekly during low-traffic periods).
2. Optimal Timing
• Avoid peak hours to prevent query slowdowns.
• Consider database usage patterns (e.g., nightly for OLTP systems).
3. Advanced Scheduling Options
• Customize frequency (daily, weekly, monthly).
• Use multiple schedules for different maintenance tasks.
• Monitor and adjust based on performance metrics.

6.10.6 Summary of Key Takeaways

1. Rebuilding vs. Reorganizing
• Rebuilding = Full recreation (best for high fragmentation).
• Reorganizing = Leaf-level adjustments (best for mild fragmentation).
2. Indexing Best Practices

266
 • Index high-selectivity columns used in WHERE, JOIN, ORDER BY.
• Avoid over-indexing to prevent storage bloat and slow writes.
3. Fragmentation Management
• Monitor regularly using DMVs (Dynamic Management Views).
• Set appropriate fill factors to balance storage and performance.
• Automate maintenance using scripts or SQL Agent jobs.
4. Automation Strategies
• Use Ola Hallengren’s scripts for efficient, threshold-based maintenance.
• Schedule jobs during off-peak hours to minimize impact.
• Leverage parallel processing for large databases.

[Link] Final Note

• Proactive index maintenance is critical for sustained database performance.
• Regular monitoring and automation ensure optimal query speeds and resource efficiency.

6.11 Recording of Building Database Applications Week 5 - Live Session on 26-04-10

6.11.1 1. Introduction to Database Monitoring and Maintenance
• Database systems must not only create and store data but also:
– Protect data from loss or corruption.
– Maintain data integrity and availability.
– Optimize performance for efficient retrieval.
• Key focus areas for this session:
1. Database Backup and Recovery – Ensuring data survival in case of failures.
2. Import/Export Operations – Moving data between systems.
3. Indexing – Improving query performance.

6.11.2 2. Importance of Database Backup

[Link] 2.1 Definition of Backup
• A backup is a copy of database data stored separately to enable recovery in case of:
– Data loss.
– Corruption.
– Accidental deletion.
– System failures.

[Link] 2.2 Why Backups Are Essential

• Failures are inevitable and can occur due to:
– System crashes (OS/hardware failure).
– Hard disk failures (physical damage).
– Human error (accidental deletion/modification).
– Malware/ransomware attacks (data corruption).
– Power failures (interrupted transactions).
• Backup is not optional—it is a safety precaution.

[Link] 2.3 Real-World Examples of Data Loss Impact

267
Scenario Impact of Data Loss
College Student Marks Unable to publish results; loss of recent updates.
DB
E-commerce Order DB Cannot track purchases, payments, or shipments; business disruption.
Banking Transaction DB Loss of financial records; legal and operational consequences.
Hospital Patient Records Critical medical history lost; risk to patient safety.

• Key Insight:
– Losing data is worse than losing software because:
* Software can be reinstalled/updated.
* Data, if lost without backup, is irrecoverable.
6.11.3 3. Factors Affecting Backup Strategy
[Link] 3.1 Frequency of Backup
• Definition: How often backups should be performed.
• Depends on:
– Rate of data change (how frequently data is updated).
– Business criticality (how quickly data must be recoverable).

Database Type Data Change Frequency Recommended Backup Frequency

Online Banking Continuous transactions Real-time or hourly
Stock Trading Seconds/minutes Near real-time
School Library Few times a day Daily or weekly
Archive System Rarely updated Monthly or as needed

• Rule of Thumb:
– Higher data volatility → More frequent backups.

[Link] 3.2 Volume of Data (Amount)

• Definition: The size of the database being backed up.
• Challenges with Large Databases:
– Longer backup time (more data = more time).
– Higher storage requirements.
– More system resources (CPU, memory, I/O).
• Backup Strategies:
– Full Backup: Copies entire database (time-consuming but comprehensive).
– Incremental Backup: Copies only changes since last backup (faster, but requires full backup for
restoration).

[Link] 3.3 Urgency of Recovery

• Definition: How quickly the system needs to be restored after a failure.
• Examples:
– Stock Market DB: Millisecond delays can cause financial losses → Instant recovery needed.
– Archive System: Delayed recovery may be acceptable.
• Business Continuity Consideration:
– If downtime is unacceptable, backup/restore must be fast and automated.

268
[Link] 3.4 Type of Data
• Not all data is equal—backup strategies vary by:
– Sensitivity (e.g., passwords, financial records → encrypted backups).
– Size (e.g., multimedia files → compression or separate storage).
– Regulatory requirements (e.g., GDPR, HIPAA compliance).
• Best Practice:
– Backup the right data in the right way at the right time (not just a blind copy).

6.11.4 4. Database Backup Tools

Different databases use different tools for backups:

Database
System Backup Tool Description
PostgreSQL pg_dump Command-line utility for database dumping.
SQL Server SQL Server GUI-based backup and restore.
Management Studio
(SSMS)
Oracle Recovery Manager Oracle’s built-in backup and recovery tool.
(RMAN)
MySQL mysqldump Command-line tool for logical backups (SQL dump files).

6.11.5 5. Hands-on Demo: MySQL Backup and Restore

[Link] 5.1 Setting Up the Database
1. Create a Database:
CREATE DATABASE college_demo;
USE college_demo;

2. Create a Table (student):

CREATE TABLE student (
student_id INT PRIMARY KEY,
student_name VARCHAR(50) NOT NULL,
age INT,
course VARCHAR(50)
);

3. Insert Sample Data:

INSERT INTO student VALUES
(1, 'Asha', 20, 'MBA'),
(2, 'Ravi', 22, 'BBA'),
(3, 'Sneha', 21, 'MBA');

4. Verify Data:
SELECT * FROM student;

Output:

269
 +------------+--------------+-----+-------+
| student_id | student_name | age | course|
+------------+--------------+-----+-------+
| 1 | Asha | 20 | MBA |
| 2 | Ravi | 22 | BBA |
| 3 | Sneha | 21 | MBA |
+------------+--------------+-----+-------+

[Link] 5.2 Backup Using Command Line (mysqldump)

1. Open Command Prompt (CMD).
2. Run Backup Command:
mysqldump -u root -p college_demo > C:\Users\itsup\college_demo_backup.sql

• -u root: MySQL username (root).

• -p: Prompt for password.
• college_demo: Database to back up.
• >: Redirects output to a file.
• C:\Users\itsup\college_demo_backup.sql: Backup file path.

3. Verify Backup File:

• Open the .sql file in a text editor (e.g., Notepad).
• Contents should include:
– CREATE TABLE statements.
– INSERT INTO statements with all records.

[Link] 5.3 Restore from Backup

1. Create a New Database for Restoration:
CREATE DATABASE college_demo_restore;
USE college_demo_restore;

2. Import Backup File:

• Method 1: Command Line
mysql -u root -p college_demo_restore < C:\Users\itsup\college_demo_backup.sql

• Method 2: MySQL Workbench GUI

– Go to Server → Data Import.
– Select Import from Self-Contained File.
– Choose the .sql backup file.
– Set Target Schema to college_demo_restore.
– Click Start Import.
3. Verify Restored Data:
USE college_demo_restore;
SELECT * FROM student;

Expected Output: Same as original table.

270
[Link] 5.4 Backup Using MySQL Workbench GUI
1. Export Data:
• Go to Server → Data Export.
• Select college_demo database.
• Choose Export to Self-Contained File.
• Set destination path (e.g., C:\Users\itsup\[Link]).
• Click Start Export.
2. Import Data:
• Go to Server → Data Import.
• Select Import from Self-Contained File.
• Choose the .sql file.
• Select target schema (college_demo_restore).
• Click Start Import.

6.11.6 6. Database Indexing

[Link] 6.1 What is an Index?
• Analogy: Like an index in a book—allows quick lookup without scanning every page.
• Purpose:
– Speeds up data retrieval (especially for large tables).
– Reduces search time from O(n) (full scan) to O(log n) (indexed search).

[Link] 6.2 When to Use Indexing

• Ideal for:
– Columns frequently used in WHERE clauses (e.g., WHERE department = 'IT').
– Columns used in JOIN operations.
– Primary keys (automatically indexed in most DBMS).
• Example Scenarios:
– Searching for a student by ID in a table with 10,000 records.
– Filtering orders by date in an e-commerce database.

[Link] 6.3 Hands-on Demo: Creating an Index

1. Create an employee Table:
CREATE TABLE employee (
emp_id INT PRIMARY KEY,
emp_name VARCHAR(30),
department VARCHAR(20),
salary INT
);

2. Insert Sample Data:

INSERT INTO employee VALUES
(1, 'John', 'HR', 30000),
(2, 'Mike', 'Finance', 50000),
(3, 'Shalini', 'IT', 30000),
(4, 'Sharing', 'IT', 50000),

271
 (5, 'Isha', 'HR', 25000),
(6, 'Anna', 'Marketing', 40000);

3. Create an Index on department:

CREATE INDEX idx_department ON employee(department);

• Syntax:
– CREATE INDEX index_name ON table_name(column_name);
• Common Errors:
– Typo in column name (e.g., deparment instead of department).
– Column does not exist in the table.
4. Verify Index Creation:
SHOW INDEX FROM employee;

Output:
+----------+------------+----------------+--------------+-------------+-----------+--
-----------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_pa
+----------+------------+----------------+--------------+-------------+-----------+--
-----------+----------+--------+------+------------+---------+---------------+
| employee | 0 | PRIMARY | 1 | emp_id |A | 6| NULL | NULL |
| employee | 1 | idx_department | 1 | department | A | 3| NULL | NULL |
+----------+------------+----------------+--------------+-------------+-----------+--
-----------+----------+--------+------+------------+---------+---------------+

• Observations:
– PRIMARY index (automatically created for emp_id).
– idx_department (manually created for department).
5. Test Query Performance:
SELECT * FROM employee WHERE department = 'IT';

• Without Index: Full table scan (slower for large datasets).

• With Index: Uses B-tree structure for faster lookup.

[Link] 6.4 Benefits and Drawbacks of Indexing

Benefits Drawbacks
Faster search queries (WHERE, JOIN). Slower writes (INSERT, UPDATE, DELETE) because
indexes must be updated.
Efficient sorting and grouping. Additional storage overhead.
Improves overall query performance. Over-indexing can degrade performance.

• Best Practices:
– Index columns used in frequent searches.
– Avoid indexing rarely used columns.
– Monitor index usage and remove unused indexes.

272
6.11.7 7. Key Takeaways
1. Backup is non-negotiable—data loss can cripple operations.
2. Backup strategy depends on:
• Frequency (how often data changes).
• Volume (size of the database).
• Urgency (how fast recovery is needed).
• Data type (sensitivity and size).
3. MySQL Backup Methods:
• Command-line (mysqldump).
• GUI (MySQL Workbench).
4. Indexing improves read performance but has trade-offs for write operations.
5. Use indexes wisely—only on columns that benefit from faster searches.

6.11.8 8. Common Pitfalls and Troubleshooting

Issue Solution
Backup file not created Check file path permissions; ensure mysqldump command is correct.
Restore fails Verify the target database exists; check for SQL syntax errors in the dump.
Index creation fails Double-check column names and table existence.
Slow queries despite indexing Analyze query execution plan (EXPLAIN in MySQL).
Storage full during backup Use incremental backups or compress the dump file.

6.12 Restoring Databases

6.12.1 1. Introduction to Database Restoration
[Link] 1.1 Definition of Restoration
• Restoration is the process of copying data from a backup and applying any large transactions to bring the
database to a specific state.
• It involves converting a backup file into a fully operational database.
• Restoration is critical for disaster recovery—without it, backups are ineffective.

[Link] 1.2 Significance of Restoration

• Ensures data recovery in case of:
– Hardware failures
– Human errors
– Cyberattacks (e.g., ransomware)
– Software corruption
• Maintains business continuity by minimizing downtime.
• Validates the effectiveness of backups—if a backup cannot be restored, it is useless.

6.12.2 2. Restoring a Database in Microsoft SQL Server (SSMS)

[Link] 2.1 Prerequisites To restore a database in Microsoft SQL Server Management Studio (SSMS), the
following must be in place: 1. SSMS installed on the target machine. 2. A backup file stored in the default
backup location (or accessible path).

[Link] 2.2 Step-by-Step Restoration Process

273
[Link].1 Step 1: Log into the Target Computer
• Access the machine where the database will be restored.

[Link].2 Step 2: Open SQL Server Management Studio (SSMS)

• Launch SSMS and connect to the SQL Server instance.

[Link].3 Step 3: Initiate the Restore Process

1. In the left navigation bar, right-click on Databases.
2. Select Restore Database from the context menu.
• This opens the Restore Database dialog.

[Link].4 Step 4: Select the Backup Source

1. Under Source, choose Device.
2. Click the three-dot button (…) to browse for the backup file.
3. In the pop-up window:
• Click Add.
• Navigate to and select the backup file.
• Click OK to confirm.

[Link].5 Step 5: Configure Restoration Options

1. In the left navigation menu, click Options.
2. Under Restore options, select:
• Override the existing database (WITH REPLACE) – Replaces the current database if it exists.
• Close existing connections to the destination database – Ensures no active connections interfere with
restoration.
3. Click OK to begin the restoration.

[Link].6 Step 6: Verification

• After completion, verify the database is fully restored and operational.
• Check for:
– Data integrity (no corruption).
– Functionality (queries, stored procedures work as expected).

6.12.3 3. Best Practices for Successful Backup and Restoration

[Link] 3.1 Reliable Data Backup
• The quality of restoration depends on the quality of backups.
• Use a robust backup solution (e.g., SQL Server Backup, third-party tools like Veeam, Commvault).

[Link] 3.2 The 3-2-1 Backup Rule A proven strategy to ensure data redundancy: 1. 3 copies of data: -
Original + two backups. 2. 2 different storage media: - E.g., local disk + external HDD. 3. 1 offsite backup: -
Cloud storage (Azure, AWS) or a remote physical location.

274
[Link] 3.3 Regular Testing of Restorations
• Test restores periodically to confirm backups are viable.
• Simulate disaster recovery scenarios to ensure quick recovery.

[Link] 3.4 Daily Monitoring and Reporting

• Generate daily backup status reports.
• Monitor for failures (e.g., incomplete backups, corruption).
• Address issues promptly to prevent data loss.

6.12.4 4. Understanding Backup Failures

[Link] 4.1 Definition of Backup Failure
• Occurs when:
– A backup operation does not complete successfully.
– The backup file is unusable during restoration.
• No backup system is 100% fail-proof—studies show significant failure rates.

[Link] 4.2 Risks of Backup Failure Backup failures pose serious threats to organizations:

Risk Type Description

Lost Productivity Without reliable backups, recovery time
increases, leading to downtime.
Reputation Costs Data loss damages trust with
customers/clients, harming brand reputation.
Penalties & Legal Issues Failure to comply with data protection laws
(e.g., GDPR, HIPAA) can result in fines or
lawsuits.

6.12.5 5. Common Causes of Backup Failures

[Link] 5.1 Infrastructure Issues
• Network problems (especially in WAN or cloud-based backups) can disrupt backup operations.
• Example: Slow internet causing timeouts during cloud backups.

[Link] 5.2 Media Issues

• Physical media degradation (e.g., failing hard drives, corrupted tapes).
• Storage issues (e.g., insufficient disk space, permission errors).

[Link] 5.3 Software Issues

• Backup software bugs (e.g., SQL Server Agent failures).
• Misconfigurations (e.g., incorrect backup schedules, wrong file paths).

[Link] 5.4 Human Errors

• Mistakes in configuration (e.g., excluding critical tables).
• Execution errors (e.g., manually stopping a backup job).
• Poor maintenance (e.g., not updating backup software).

275
6.12.6 6. Handling Backup Failures
[Link] 6.1 Ensure Complete Coverage
• Verify backup jobs include all necessary files/data.
• Example: Ensure transaction logs are backed up alongside full backups.

[Link] 6.2 Regular Test Restores

• Routinely restore critical backups to a test environment.
• Goal: Confirm backups are not corrupted and can be fully recovered.

[Link] 6.3 Automated Notifications

• Configure backup software to send status alerts (success/failure) after each job.
• Example: Email notifications via SQL Server Agent or third-party tools.

[Link] 6.4 Daily Review of Backup Reports

• Check backup logs daily for:
– Failed jobs.
– Warnings (e.g., slow backups, partial completions).
• Take immediate action if issues are detected.

6.12.7 7. Summary and Key Takeaways

[Link] 7.1 Importance of Database Restoration
• Restoration is essential for disaster recovery and business continuity.
• Backups are useless if they cannot be restored.

[Link] 7.2 Best Practices Recap

Practice Action Item

Reliable Backups Use enterprise-grade backup solutions.
3-2-1 Rule Maintain multiple copies across different media.
Test Restores Regularly verify backup integrity.
Monitoring Review backup reports daily.

[Link] 7.3 Mitigating Backup Failures

• Proactive measures (automated alerts, testing) reduce risks.
• Contingency plans (e.g., secondary backups) ensure recovery is possible even if primary backups fail.

[Link] 7.4 Final Thought

“A backup is only as good as its last successful restore.” - Prioritize restoration testing to ensure
data resilience.

276
7 Module 7: Introduction and Project Setup
7.1 Creating Initial Database Tables and Testing Connections
7.1.1 Introduction
By the end of this lecture, you will be able to: - Create database tables using SQL scripts or JPA annotations. -
Auto-generate tables by running a Spring Boot application. - Test database connections and verify table structures.

7.1.2 1. Creating Tables Using SQL Scripts

SQL scripts provide a direct and explicit method for defining a database schema.

[Link] 1.1. Key Characteristics of SQL Scripts

• Straightforward approach: Schema is defined using standard SQL CREATE TABLE statements.
• Explicit control: Allows precise definition of tables, columns, data types, constraints, and relationships.
• Execution flexibility: Can be run via:
– MySQL Workbench (GUI tool)
– MySQL Command Line Interface (CLI)

[Link] 1.2. Example Tables Three tables are created in the example: 1. authors 2. categories 3. books
(includes foreign keys linking to authors and categories)

CREATE TABLE authors (

id INT AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(100) NOT NULL,
birth_date DATE
);

[Link].1 1.2.1. SQL Script for authors Table

• Columns:
– id (Primary Key, auto-incremented)
– name (String, max 100 chars, non-nullable)
– birth_date (Date type)

CREATE TABLE categories (

id INT AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(50) NOT NULL
);

[Link].2 1.2.2. SQL Script for categories Table

• Columns:
– id (Primary Key, auto-incremented)
– name (String, max 50 chars, non-nullable)

CREATE TABLE books (

id INT AUTO_INCREMENT PRIMARY KEY,

277
 title VARCHAR(200) NOT NULL,
publication_year INT,
author_id INT,
category_id INT,
FOREIGN KEY (author_id) REFERENCES authors(id),
FOREIGN KEY (category_id) REFERENCES categories(id)
);

[Link].3 1.2.3. SQL Script for books Table (with Foreign Keys)
• Columns:
– id (Primary Key, auto-incremented)
– title (String, max 200 chars, non-nullable)
– publication_year (Integer)
– author_id (Foreign Key → [Link])
– category_id (Foreign Key → [Link])

[Link] 1.3. Executing SQL Scripts

• Method 1: MySQL Workbench
– Open the script file in Workbench.
– Execute the script to create tables in the database.
• Method 2: MySQL Command Line
– Run the script directly in the MySQL CLI.
– Example command:
mysql -u [username] -p [database_name] < [Link]
• Outcome: Tables are created and ready for data insertion.

7.1.3 2. Creating Tables with JPA Annotations

Java Persistence API (JPA) allows defining database schemas programmatically using Java classes and anno-
tations.

[Link] 2.1. Key Concepts

• Entity Classes: Each class represents a database table.
• Annotations:
– @Entity → Marks a class as a JPA entity.
– @Table → Maps the class to a database table.
– @Id → Defines the primary key.
– @GeneratedValue → Configures auto-increment.
– Relationship Annotations (@OneToMany, @ManyToOne) → Define table relationships.

[Link] 2.2. Example: JPA Entity Definitions Three entities are defined: 1. Author (Maps to authors table)
2. Category (Maps to categories table) 3. Book (Maps to books table, with foreign keys)

@Entity
@Table(name = "authors")
public class Author {

278
 @Id
@GeneratedValue(strategy = [Link])
private Long id;

@Column(nullable = false, length = 100)

private String name;

@Column(name = "birth_date")
private LocalDate birthDate;

// Getters and Setters

}

[Link].1 2.2.1. Author Entity

• Annotations:
– @Entity → Declares this as a JPA entity.
– @Table(name = "authors") → Maps to the authors table.
– @Id + @GeneratedValue → Auto-incremented primary key.
– @Column → Configures column properties (e.g., nullable = false).

@Entity
@Table(name = "categories")
public class Category {
@Id
@GeneratedValue(strategy = [Link])
private Long id;

@Column(nullable = false, length = 50)

private String name;

// Getters and Setters

}

[Link].2 2.2.2. Category Entity

• Similar structure to Author, but maps to categories.

@Entity
@Table(name = "books")
public class Book {
@Id
@GeneratedValue(strategy = [Link])
private Long id;

@Column(nullable = false, length = 200)

private String title;

@Column(name = "publication_year")

279
 private Integer publicationYear;

@ManyToOne
@JoinColumn(name = "author_id")
private Author author;

@ManyToOne
@JoinColumn(name = "category_id")
private Category category;

// Getters and Setters

}

[Link].3 2.2.3. Book Entity (with Relationships)

• Relationships:
– @ManyToOne → A book belongs to one author and one category.
– @JoinColumn → Specifies the foreign key column (author_id, category_id).

[Link] 2.3. Advantages of JPA Approach

• Abstraction: Hides direct SQL, reducing boilerplate code.
• Auto Schema Generation: JPA can create/update tables based on entity classes.
• Maintainability: Schema changes are managed via Java code, not SQL scripts.

7.1.4 3. Auto-Generating Tables with JPA

JPA (via Hibernate) can automatically generate database tables when the application starts.

[Link] 3.1. Configuration via [Link] The [Link]-auto property

controls schema generation behavior.

Property Value Behavior

none No action – Schema is not modified.
update Updates schema – Adds new
tables/columns but does not drop existing
ones.
create Drops existing schema and recreates it
on startup.
create-drop Drops schema when the application stops
(useful for testing).

# [Link]
[Link]-auto=update

[Link].1 Example Configuration

• With update, JPA will:
– Create missing tables/columns.
– Not delete existing tables or data.

280
[Link] 3.2. How Auto-Generation Works
1. Application Startup:
• Spring Boot initializes.
• Hibernate reads entity classes (Author, Category, Book).
• Generates corresponding SQL to create/update tables.
2. Database Execution:
• SQL is executed against the database.
• Tables are created/modified without manual SQL scripts.

[Link] 3.3. Running the Application

• Command:
mvn spring-boot:run

(or via IDE execution)

• Result:
– Tables (authors, categories, books) are auto-created in the database.

7.1.5 4. Verifying Tables with SQL Commands

After table creation (via SQL scripts or JPA), verification is essential.

[Link] 4.1. Methods for Verification

1. Graphical Tools (e.g., MySQL Workbench)
2. MySQL Command Line Interface (CLI)

[Link] 4.2. Essential SQL Commands

Command Purpose Example

SHOW TABLES; Lists all tables in the database. SHOW TABLES;
DESCRIBE [table_name]; Shows the structure (columns, DESCRIBE authors;
types, constraints) of a table.
SELECT * FROM [table_name]; Retrieves all data from a table. SELECT * FROM books;

[Link].1 4.2.1. Example Workflow

1. Connect to MySQL:
mysql -u [username] -p

2. Select Database:
USE [database_name];

3. List Tables:
SHOW TABLES;

Expected Output:

281
 +---------------------+
| Tables_in_[db_name] |
+---------------------+
| authors |
| books |
| categories |
+---------------------+

4. Inspect Table Structure:

DESCRIBE books;

Expected Output:
+------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------+--------------+------+-----+---------+----------------+
| id | int | NO | PRI | NULL | auto_increment |
| title | varchar(200) | NO | | NULL | |
| publication_year | int | YES | | NULL | |
| author_id | int | YES | MUL | NULL | |
| category_id | int | YES | MUL | NULL | |
+------------------+--------------+------+-----+---------+----------------+

5. Check Data:
SELECT * FROM authors;

(Initially empty if no data inserted.)

[Link] 4.3. Use Cases for Verification

• Development: Ensure schema matches expectations.
• Debugging: Check if tables/columns were created correctly.
• Testing: Verify data integrity after auto-generation.

7.1.6 5. Summary of Key Takeaways

1. SQL Scripts:
• Explicit, manual control over schema.
• Requires execution via MySQL Workbench or CLI.
2. JPA Annotations:
• Schema defined in Java classes.
• Auto-generation via [Link]-auto.
• Supports relationships (@OneToMany, @ManyToOne).
3. Auto-Generation:
• Configured in [Link].
• Options: none, update, create, create-drop.
4. Verification:
• Use SHOW TABLES, DESCRIBE, and SELECT in MySQL CLI.
• Essential for development and debugging.
End of Lecture Notes

282
7.2 Designing the Database Schema for the Library Application
7.2.1 1. Introduction to Database Schema Design
[Link] 1.1 Definition of Database Schema
• A database schema defines the structure, relationships, and constraints of data stored in a database.
• Acts as a blueprint for organizing and accessing data.
• Ensures data integrity, optimizes performance, and simplifies maintenance.

[Link] 1.2 Importance of a Well-Designed Schema

• Prevents inefficiency in database operations.
• Avoids issues such as:
– Data redundancy (duplicate data).
– Inconsistency (conflicting or outdated information).
– Poor query performance (slow retrieval of data).
• Goal: Create efficient, reliable, and maintainable databases.

7.2.2 2. Core Principles of Database Schema Design

[Link] 2.1 Normalization
• Definition: A technique to reduce redundancy and dependency by organizing data into distinct tables
based on relationships.
• Example:
– Separate books and authors into individual tables.
– Avoid storing author details repeatedly with each book record.
• Benefits:
– Prevents duplicate data.
– Enhances data consistency.
– Simplifies updates and maintenance.

[Link] 2.2 Denormalization

• Definition: The reverse of normalization—intentionally introducing redundancy by combining tables
to improve query performance.
• Trade-offs:
– Pros:
* Reduces the number of tables, speeding up searches.
* Improves read performance (faster data retrieval).
– Cons:
* Increases storage requirements.
* Adds complexity in data maintenance (e.g., updates must be applied in multiple places).
• Decision Criteria:
– Choose between normalized (separate tables) or denormalized (combined tables) based on:
* Application performance needs.
* Query frequency vs. update frequency.
[Link] 2.3 Data Integrity
• Definition: Ensures data remains accurate and consistent over time.
• Methods to Enforce Integrity:

283
 – Primary Keys: Uniquely identify records (e.g., BookID in a books table).
– Foreign Keys: Link tables and enforce referential integrity (e.g., AuthorID in books references Au-
thorID in authors).
– Unique Constraints: Ensure no duplicate values in a column (e.g., ISBN must be unique).
– Check Constraints: Validate data (e.g., PublishedDate must be a valid date).

[Link] 2.4 Scalability

• Definition: Ensures the database performs efficiently as data grows.
• Techniques:
– Indexing: Speeds up data retrieval (e.g., index on Title for faster book searches).
– Partitioning: Divides large tables into smaller, manageable parts (e.g., partition books by Publica-
tionYear).
– Sharding: Distributes data across multiple servers to handle large volumes.
• Example:
– Partitioning the books table by PublicationYear improves query performance for time-based
searches.

7.2.3 3. Best Practices in Database Schema Design

[Link] 3.1 Planning and Requirements Analysis
• Steps:
1. Understand requirements: Identify what data needs to be stored and how it will be used.
2. Identify entities: Determine key objects (e.g., books, authors, categories).
3. Create diagrams: Use Entity-Relationship Diagrams (ERDs) to visualize relationships.

[Link] 3.2 Simplicity

• Avoid unnecessary complexity:
– Keep the schema clean and intuitive.
– Eliminate redundant tables or columns.

[Link] 3.3 Documentation

• Comprehensive documentation should include:
– Table structures (columns, data types, constraints).
– Relationships between tables.
– Business rules (e.g., “An author can write multiple books”).
• Purpose: Ensures maintainability and collaboration among developers.

[Link] 3.4 Regular Review and Refactoring

• Continuously improve the schema:
– Adapt to changing requirements.
– Optimize for performance bottlenecks.
– Refactor to eliminate inefficiencies.

[Link] 3.5 Benefits of Following Best Practices

• Creates a robust and scalable database.
• Meets current and future needs.

284
 • Ensures long-term efficiency and reliability.

7.2.4 4. Creating Tables

[Link] 4.1 Identifying Entities
• Key entities for a library application:
– Books
– Authors
– Categories

[Link] 4.2 Defining Columns and Data Types

• Example: books Table | Column | Data Type | Description | Constraint | |—————-|—————-|——
—————————|———————| | BookID | INT | Unique identifier for a book | Primary Key | |
Title | VARCHAR(255) | Title of the book | | | AuthorID | INT | References the author | Foreign Key | |
CategoryID | INT | References the category | Foreign Key | | PublishedDate| DATE | Publication date | | |
ISBN | VARCHAR(20) | International Standard Book Number | Unique |

• Example: authors Table | Column | Data Type | Description | Constraint | |————-|—————-|——

———————-|—————–| | AuthorID | INT | Unique identifier for author | Primary Key | | Name |
VARCHAR(100) | Author’s name | | | BirthDate | DATE | Author’s birth date | |
• Example: categories Table | Column | Data Type | Description | Constraint | |—————-|—————-
|—————————|—————–| | CategoryID | INT | Unique identifier for category | Primary Key | |
CategoryName | VARCHAR(50) | Name of the category | |

[Link] 4.3 Establishing Constraints

• Primary Keys:
– Ensure each record is uniquely identifiable (e.g., BookID, AuthorID).
• Foreign Keys:
– Enforce referential integrity (e.g., AuthorID in books must exist in authors).
• Unique Constraints:
– Prevent duplicate values (e.g., ISBN must be unique).
• Purpose:
– Maintains data integrity.
– Enables efficient querying.

7.2.5 5. Establishing Relationships Between Tables

[Link] 5.1 Importance of Relationships
• Maintains data integrity.
• Enables complex queries (e.g., “Find all books by Author X in Category Y”).
• Reflects real-world connections between entities.

[Link] 5.2 Types of Relationships

[Link].1 5.2.1 One-to-One (1:1)

• Definition: A record in Table A corresponds to exactly one record in Table B.
• Example:

285
 – Users and UserDetails tables.
– Each user has one set of details (e.g., address, phone number).
• Use Case:
– Organizes data for performance optimization (e.g., frequently accessed details stored separately).

[Link].2 5.2.2 One-to-Many (1:N)

• Definition: A record in Table A can relate to multiple records in Table B.
• Example:
– authors (1) to books (N).
– One author can write many books.
– Each book has one author (foreign key AuthorID in books).
• Implementation:
– Foreign key in the many-side table (e.g., AuthorID in books).
• Benefits:
– Efficiently retrieves all books by an author.
– Maintains data integrity through foreign keys.

[Link].3 5.2.3 Many-to-Many (M:N)

• Definition: Records in Table A relate to multiple records in Table B, and vice versa.
• Example:
– books and categories.
– A book can belong to multiple categories.
– A category can contain multiple books.
• Implementation:
– Uses a junction table (e.g., book_categories).
– Junction table contains foreign keys from both tables.
• Example Junction Table (book_categories): | Column | Data Type | Description | |—————-|———
–|———————————| | BookID | INT | References books table | | CategoryID | INT | References
categories table |

[Link] 5.3 Practical Example: Library Application Relationships

• Tables:
– books (foreign keys: AuthorID, CategoryID).
– authors (primary key: AuthorID).
– categories (primary key: CategoryID).
• Relationships:
– One-to-Many:
* authors → books (one author, many books).
* categories → books (one category, many books).
– Many-to-Many (if books can have multiple categories):
* Requires a junction table (book_categories).
• Benefits:
– Ensures data integrity (e.g., no orphaned books).
– Enables efficient queries (e.g., “Find all books in the ‘Science’ category”).

7.2.6 6. Summary of Key Learnings

[Link] 6.1 Principles Covered

286
 • Normalization vs. Denormalization:
– Normalization reduces redundancy.
– Denormalization improves query performance at the cost of storage.
• Data Integrity:
– Enforced via primary keys, foreign keys, unique constraints, and check constraints.
• Scalability:
– Achieved through indexing, partitioning, and sharding.

[Link] 6.2 Table Creation

• Identify entities (books, authors, categories).
• Define columns and data types.
• Apply constraints (primary keys, foreign keys, uniqueness).

[Link] 6.3 Relationships

• One-to-One: Rare; used for specialized data separation.
• One-to-Many: Common (e.g., authors to books).
• Many-to-Many: Requires a junction table (e.g., books to categories).

[Link] 6.4 Best Practices

• Plan before building.
• Keep the schema simple.
• Document thoroughly.
• Review and refactor regularly.

[Link] 6.5 Application to Library Database

• Tables: books, authors, categories, and optionally book_categories (for many-to-many).
• Relationships:
– authors → books (1:N).
– categories → books (1:N or M:N via junction table).
• Outcome: A well-structured, efficient, and maintainable database schema.

7.3 Initialising a Spring Boot Project

7.3.1 1. Introduction to Spring Boot Project Setup
[Link] 1.1 Learning Objectives By the end of this lecture, students will be able to: - Demonstrate how to set
up a new Spring Boot project using Spring Initializr. - Configure project metadata (e.g., build tool, language,
packaging). - Add necessary dependencies (e.g., Spring Web, Spring Data JPA). - Explain the structure of a
Spring Boot project. - Perform key operations such as: - Building the project. - Running the application. -
Verifying the setup.

[Link] 1.2 Purpose of Spring Initializr

• Simplifies Spring application setup by providing a pre-configured project template.
• Enables rapid deployment and development by automating initial configurations.
• Reduces manual setup errors by generating a standardized project structure.

287
7.3.2 2. Creating a New Spring Boot Project
[Link] 2.1 Accessing Spring Initializr
• Website: [Link]
• Function: A web-based tool for generating Spring Boot projects with customizable settings.

[Link] 2.2 Configuring Project Metadata The following configurations must be specified in Spring Initial-
izr:
1. Build Tool:
• Maven (default) or Gradle (alternative).
• Determines dependency management and project lifecycle.
2. Language:
• Java (primary choice for Spring Boot).
• Other options (e.g., Kotlin, Groovy) may be available but are less common.
3. Spring Boot Version:
• Select the latest stable version (recommended) or a specific version based on project requirements.
4. Packaging:
• JAR (Java Archive): Default for standalone applications (embedded server).
• WAR (Web Application Archive): For deployment in external servlet containers (e.g., Tomcat).
5. Project Metadata:
• Group ID: Typically follows reverse domain notation (e.g., [Link]).
• Artifact ID: Name of the project (e.g., myapp).
• Name: Human-readable project name (optional).
• Description: Brief project summary (optional).
• Package Name: Defaults to [Link].[artifact-id] but can be customized.

[Link] 2.3 Adding Dependencies Dependencies are libraries/modules required for the project. Common de-
pendencies include: - Spring Web: For building web applications (REST APIs, MVC). - Spring Data JPA: For
database interactions using Java Persistence API (JPA). - Thymeleaf: Templating engine for server-side HTML
rendering. - H2 Database: In-memory database for development/testing. - Lombok: Reduces boilerplate code
(e.g., getters, setters).
Process: 1. Search for dependencies in the Spring Initializr interface. 2. Select required dependencies (e.g.,
Spring Web, Spring Data JPA). 3. Click “Generate” to download a ZIP file containing the project.

[Link] 2.4 Downloading and Extracting the Project

1. Download the generated ZIP file from Spring Initializr.
2. Extract the ZIP file into a project folder (e.g., myapp).
3. The extracted folder contains the pre-configured Spring Boot project.

7.3.3 3. Spring Boot Project Structure

The generated project follows a standardized Maven/Gradle structure. Key components:

[Link] 3.1 Source Code Directory (src/)

[Link].1 3.1.1 Main Application Code (src/main/java/)

• Contains the primary application code.

288
 • Key File: [ProjectName][Link] (e.g., [Link]).
– Purpose: Entry point of the Spring Boot application.
– Annotations:
* @SpringBootApplication: Combines:
ꞏ @Configuration: Marks the class as a source of bean definitions.
ꞏ @EnableAutoConfiguration: Automatically configures Spring based on dependencies.
ꞏ @ComponentScan: Scans for Spring components (e.g., @Service, @Repository).

[Link].2 3.1.2 Application Properties (src/main/resources/[Link])

• Contains configuration settings for the application.
• Examples:
– Server port: [Link]=8080
– Database URL: [Link]=jdbc:h2:mem:testdb
– Logging levels: [Link]=INFO

[Link].3 3.1.3 Static Resources (src/main/resources/static/)

• Stores static files (e.g., CSS, JavaScript, images).

[Link].4 3.1.4 Templates (src/main/resources/templates/)

• Stores Thymeleaf/HTML templates (if using server-side rendering).

[Link] 3.2 Test Code Directory (src/test/java/)

• Contains unit and integration tests.
• Key File: [ProjectName][Link] (e.g., [Link]).
– Purpose: Basic test class to verify the application context loads.

[Link] 3.3 Project Configuration Files

[Link].1 3.3.1 Maven ([Link])

• Purpose: Manages dependencies, plugins, and build configurations.
• Key sections:
– <dependencies>: Lists all project dependencies (e.g., Spring Web, JPA).
– <build>: Configures plugins (e.g., Spring Boot Maven Plugin).
– <properties>: Defines project metadata (e.g., Java version).

[Link].2 3.3.2 Gradle ([Link])

• Alternative to [Link] if Gradle is selected.
• Uses Groovy/Kotlin DSL for configuration.

[Link] 3.4 Documentation ([Link])

• Provides basic project information (e.g., setup instructions, dependencies).
• Often auto-generated by Spring Initializr.

7.3.4 4. Building and Running the Application

[Link] 4.1 Using Maven

289
[Link].1 4.1.1 Via Command Line
1. Open a terminal (e.g., VS Code integrated terminal).
2. Navigate to the project directory:
cd /path/to/myapp

3. Run the application using the Spring Boot Maven plugin:

./mvnw spring-boot:run

• Note: mvnw is the Maven Wrapper, ensuring consistent Maven versions across environments.

[Link].2 4.1.2 Expected Output

• Startup logs will appear in the terminal, indicating:
– Dependency resolution.
– Application context initialization.
– Embedded server (e.g., Tomcat) starting on [Link]

[Link] 4.2 Using Java (Alternative Method)

1. Build the project to generate a JAR file:
./mvnw clean package

• Creates a standalone JAR in target/[artifact-id]-[version].jar.

2. Run the JAR file:
java -jar target/[Link]

7.3.5 5. Verifying the Application Setup

[Link] 5.1 Checking Application Output
1. Startup Logs:
• Verify that the application starts without errors.
• Look for:
– Started [ProjectName]Application in X seconds.
– Tomcat started on port(s): 8080 (http).
2. Accessing the Application:
• Open a web browser and navigate to:
[Link]
• Expected Result:
– Default Spring Boot whitelabel error page (if no controllers are defined).
– Custom response if endpoints are configured (e.g., @RestController).

[Link] 5.2 Verifying Dependencies and Configurations

1. Dependencies:
• Check [Link] to ensure all required dependencies are listed.
• Run ./mvnw dependency:tree to view the dependency hierarchy.
2. Configuration:
• Verify [Link] for correct settings (e.g., database URL, server port).

290
7.3.6 6. Common Issues and Solutions
[Link] 6.1 Port Already in Use
• Symptom: Application fails to start with:
Port 8080 already in use

• Solution:
– Change the port in [Link]:
[Link]=8088

– Alternatively, terminate the process using the port (e.g., via lsof -i :8080 on Unix).

[Link] 6.2 Dependency Issues

• Symptom: Build fails due to missing or conflicting dependencies.
• Solution:
– Review [Link] for correct dependency versions.
– Use ./mvnw clean install to refresh dependencies.
– Check for version conflicts (e.g., incompatible Spring Boot and JPA versions).

[Link] 6.3 Configuration Errors

• Symptom: Application starts but behaves unexpectedly (e.g., cannot connect to database).
• Solution:
– Verify [Link] for typos or missing properties.
– Enable debug logging for troubleshooting:
[Link]=DEBUG

7.3.7 7. Summary of Key Concepts

[Link] 7.1 Spring Initializr
• Web tool for generating pre-configured Spring Boot projects.
• Configures metadata, dependencies, and build tools.

[Link] 7.2 Project Structure

Directory/File Purpose
src/main/java/ Main application code (e.g., @SpringBootApplication).
[Link] Configuration settings (e.g., server port, database).
[Link] Maven build configuration (dependencies, plugins).
src/test/java/ Test classes.

[Link] 7.3 Building and Running

• Maven: ./mvnw spring-boot:run
• Java: java -jar target/[artifact].jar

291
[Link] 7.4 Verification
• Check logs for successful startup.
• Access [Link]
• Validate dependencies and configurations.

[Link] 7.5 Troubleshooting

Issue Solution
Port in use Change [Link] in [Link].
Dependency conflicts Review [Link] and run mvn clean install.
Configuration errors Check [Link] for accuracy.

7.4 Integrating MySQL with Spring Boot

7.4.1 Introduction
This lecture provides a step-by-step guide to configuring Spring Boot with MySQL using VS Code on Ubuntu.
The session covers: - Setting up a Spring Boot project. - Configuring MySQL database connectivity. - Building
a CRUD (Create, Read, Update, Delete) application. - Testing endpoints using Postman.
By the end of this lecture, students will be able to: 1. Configure a Spring Boot application to connect with MySQL.
2. Analyze key Spring Boot annotations and their purposes. 3. Develop a complete CRUD application with
entities, repositories, and controllers. 4. Test API endpoints using Postman.

7.4.2 Prerequisites
Before starting, ensure the following tools are installed: - JDK (Java Development Kit) – Required for Java
development. - Maven – A build automation tool for Java projects that: - Manages dependencies. - Compiles
code. - Packages applications. - Handles deployment. - MySQL Server – Database management system. - VS
Code (Visual Studio Code) – A versatile, open-source code editor supporting multiple languages, including Java.
- Postman – A tool for testing APIs by sending HTTP requests.

7.4.3 Setting Up the Development Environment in VS Code

[Link] Installing Required Extensions To enhance VS Code for Java and Spring Boot development, install
the following extensions: 1. Java Extension Pack – Provides Java language support. 2. Spring Boot Extension
Pack – Adds Spring Boot-specific features.
Steps to Install Extensions: 1. Open VS Code. 2. Press Ctrl+Shift+X to open the Extensions view. 3. Search
for: - “Java Extension Pack” → Install. - “Spring Boot Extension Pack” → Install.

7.4.4 Creating a Spring Boot Project

There are two methods to set up a Spring Boot project: 1. Using Spring Initializr (Recommended for beginners).
2. Manual Maven Setup (For advanced users).

[Link] Method 1: Using Spring Initializr

1. Visit [Link].
2. Select project options:
• Project: Maven.

292
 • Language: Java.
• Spring Boot Version: Latest stable release.
3. Add dependencies (e.g., Spring Web, Spring Data JPA, MySQL Driver).
4. Generate and download the project.
5. Extract the ZIP file and open the project in VS Code.

[Link] Method 2: Manual Maven Setup

1. Run the following Maven command to generate a project:
mvn archetype:generate -DgroupId=[Link] -DartifactId=demo -DarchetypeArtifactId=maven-arche

2. Navigate to the project directory:

cd demo

3. Open the project in VS Code.

7.4.5 Project Structure Overview

After creating the project, the following key directories and files are present:

File/Directory Purpose
src/main/java/ Contains Java source code.
[Link] Main entry point of the Spring
Boot application.
[Link] Entity class representing the Book
table in the database.
[Link] Repository interface for CRUD
operations (Create, Read, Update,
Delete).
[Link] REST Controller to handle
HTTP requests (GET, POST,
etc.).
[Link] Configuration file for database
and application settings.
src/main/resources/static/ Contains static assets (HTML,
CSS, JavaScript, images).
src/main/resources/templates/ Contains server-side templates
(e.g., Thymeleaf).
[Link] Maven configuration file (defines
dependencies, build settings).
src/test/ Contains test classes (e.g.,
[Link]).

7.4.6 Configuring [Link] (Maven Project Configuration)

The [Link] file is the core of a Maven project, defining: - Project metadata (groupId, artifactId, version). -
Dependencies (libraries required for the project). - Build settings (plugins, compilation options).

293
[Link] Key Dependencies for MySQL & Spring Boot To enable JDBC (Java Database Connectivity) and
JPA (Java Persistence API), add the following dependencies in [Link]:
<dependencies>
<!-- Spring Boot Starter Web (for REST APIs) -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>

<!-- Spring Boot Starter Data JPA (for database operations) -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>

<!-- MySQL Connector (for MySQL database connectivity) -->

<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>runtime</scope>
</dependency>

<!-- Spring Boot Starter Test (for testing) -->

<dependency>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

7.4.7 Configuring [Link] for MySQL

The [Link] file stores database connection settings and Hibernate configurations.

# Database Configuration
[Link]=jdbc:mysql://localhost:3306/library_db
[Link]=root
[Link]=your_password

# Hibernate Properties
[Link]-auto=update
[Link]-sql=true
[Link]=[Link].MySQL8Dialect

[Link] Essential Database Properties

[Link] Explanation of Properties

294
Property Description
[Link] JDBC URL for MySQL database (e.g.,
jdbc:mysql://localhost:3306/library_db).
[Link] Database username (default: root).
[Link] Database password.
[Link]-auto Database schema generation strategy (update, create,
validate, none).
[Link]-sql Logs SQL queries in the console (true/false).
[Link] SQL dialect for Hibernate (e.g., MySQL8Dialect).

[Link] Steps to Set Up MySQL Database

1. Install MySQL Server (if not already installed).
2. Create a database (e.g., library_db) using MySQL Workbench or the command line:
CREATE DATABASE library_db;

3. Verify connection by running the Spring Boot application.

7.4.8 Key Spring Boot Annotations

Spring Boot uses annotations to reduce boilerplate code and simplify configuration.

[Link] 1. @SpringBootApplication
• Combines three annotations:
– @Configuration – Marks the class as a Spring configuration source.
– @EnableAutoConfiguration – Enables auto-configuration of Spring Boot.
– @ComponentScan – Scans for Spring components (e.g., @Service, @Repository).
• Usage: Applied to the main application class.

[Link] 2. @RestController
• Combines:
– @Controller – Marks the class as a Spring MVC controller.
– @ResponseBody – Ensures return values are written to the HTTP response body.
• Usage: Used to create RESTful web services.

[Link] 3. @RequestMapping
• Maps web requests to handler methods or classes.
• Example:
@RequestMapping("/books")
public class BookController { ... }

[Link] 4. @PostMapping & @GetMapping

• Shortcuts for @RequestMapping with specific HTTP methods.
– @PostMapping → Handles POST requests (used for submitting data).

295
 – @GetMapping → Handles GET requests (used for retrieving data).
• Example:
@PostMapping("/add")
public Book addBook(@RequestBody Book book) { ... }

@GetMapping("/all")
public List<Book> getAllBooks() { ... }

[Link] 5. @Entity, @Id, @GeneratedValue

• @Entity – Marks a class as a JPA entity (mapped to a database table).
• @Id – Specifies the primary key of the entity.
• @GeneratedValue – Defines the primary key generation strategy (e.g., AUTO, IDENTITY).
• Example:
@Entity
public class Book {
@Id
@GeneratedValue(strategy = [Link])
private Long id;
private String title;
private String author;
// Getters & Setters
}

[Link] 6. @Autowired, @Component, @Service, @Repository

Annotation Purpose
@Autowired Injects dependencies automatically (e.g.,
repositories into services).
@Component Generic Spring-managed component.
@Service Indicates a service layer component (business
logic).
@Repository Indicates a repository component (data access
layer).

7.4.9 Creating the Book Entity & Repository

[Link] 1. Book Entity ([Link]) Represents a database table (Book) with fields mapped to columns.
@Entity
public class Book {
@Id
@GeneratedValue(strategy = [Link])
private Long id;
private String title;
private String author;

296
 private int year;

// Getters & Setters

public Long getId() { return id; }
public void setId(Long id) { [Link] = id; }
public String getTitle() { return title; }
public void setTitle(String title) { [Link] = title; }
public String getAuthor() { return author; }
public void setAuthor(String author) { [Link] = author; }
public int getYear() { return year; }
public void setYear(int year) { [Link] = year; }
}

[Link] 2. Book Repository ([Link]) Extends JpaRepository to provide CRUD operations

without manual implementation.
import [Link];

public interface BookRepository extends JpaRepository<Book, Long> {

// Spring Data JPA automatically provides:
// save(), findAll(), findById(), deleteById(), etc.
}

7.4.10 Creating the Book Controller ([Link])

Handles HTTP requests (GET, POST) for Book operations.
@RestController
@RequestMapping("/books")
public class BookController {

@Autowired
private BookRepository bookRepository;

// POST: Add a new book

@PostMapping("/add")
public Book addBook(@RequestBody Book book) {
return [Link](book);
}

// GET: Retrieve all books

@GetMapping("/all")
public List<Book> getAllBooks() {
return [Link]();
}

// GET: Retrieve a book by ID

@GetMapping("/{id}")
public ResponseEntity<Book> getBookById(@PathVariable Long id) {
Optional<Book> book = [Link](id);

297
 return [Link](ResponseEntity::ok)
.orElseGet(() -> [Link]().build());
}

// PUT: Update a book

@PutMapping("/update/{id}")
public ResponseEntity<Book> updateBook(@PathVariable Long id, @RequestBody Book bookDetails) {
return [Link](id)
.map(book -> {
[Link]([Link]());
[Link]([Link]());
[Link]([Link]());
Book updatedBook = [Link](book);
return [Link](updatedBook);
})
.orElseGet(() -> [Link]().build());
}

// DELETE: Remove a book

@DeleteMapping("/delete/{id}")
public ResponseEntity<Void> deleteBook(@PathVariable Long id) {
[Link](id);
return [Link]().build();
}
}

7.4.11 Testing the API with Postman

Postman is used to test HTTP endpoints by sending GET, POST, PUT, DELETE requests.

[Link] Steps to Test the API

1. Start the Spring Boot application:
mvn spring-boot:run

2. Open Postman and create a new request.

[Link] Example Requests

[Link].1 1. POST Request (Add a Book)

• Endpoint: POST [Link]
• Body (JSON):
{
"title": "Spring Boot in Action",
"author": "Craig Walls",
"year": 2019
}

298
 • Expected Response:
{
"id": 1,
"title": "Spring Boot in Action",
"author": "Craig Walls",
"year": 2019
}

[Link].2 2. GET Request (Get All Books)

• Endpoint: GET [Link]
• Expected Response:
[
{
"id": 1,
"title": "Spring Boot in Action",
"author": "Craig Walls",
"year": 2019
}
]

[Link].3 3. GET Request (Get Book by ID)

• Endpoint: GET [Link]
• Expected Response:
{
"id": 1,
"title": "Spring Boot in Action",
"author": "Craig Walls",
"year": 2019
}

[Link].4 4. PUT Request (Update a Book)

• Endpoint: PUT [Link]
• Body (JSON):
{
"title": "Spring Boot in Action (Updated)",
"author": "Craig Walls",
"year": 2020
}

• Expected Response:
{
"id": 1,
"title": "Spring Boot in Action (Updated)",
"author": "Craig Walls",

299
 "year": 2020
}

[Link].5 5. DELETE Request (Remove a Book)

• Endpoint: DELETE [Link]
• Expected Response: 204 No Content

7.4.12 Summary of Key Concepts

1. Spring Boot + MySQL Integration:
• Configured via [Link].
• Uses JDBC & JPA for database operations.
2. Maven ([Link]):
• Manages dependencies (Spring Web, JPA, MySQL Connector).
3. Annotations:
• @SpringBootApplication, @RestController, @Entity, @Repository.
• @GetMapping, @PostMapping, @Autowired.
4. CRUD Operations:
• Create (POST), Read (GET), Update (PUT), Delete (DELETE).
5. Postman Testing:
• Verifies API endpoints by sending HTTP requests.

7.4.13 Conclusion
This lecture covered: [OK] Setting up a Spring Boot project (via Spring Initializr or Maven). [OK] Configuring
MySQL database connectivity. [OK] Understanding key Spring Boot annotations. [OK] Building a CRUD
application with entities, repositories, and controllers. [OK] Testing APIs using Postman.
By following these steps, you can now integrate MySQL with Spring Boot and develop fully functional
database-driven applications.

7.5 Introduction to the Library Management Application

7.5.1 1. Overview of the Library Management Application
[Link] 1.1 Purpose and Scope
• The Library Management Application is a database-driven system designed to modernize and streamline
library operations.
• Focuses on managing book inventory efficiently, ensuring accessibility and organization of library re-
sources.
• Aims to automate manual processes, reducing human error and improving operational efficiency.

[Link] 1.2 Core Functionalities The application will implement the following key features: 1. Book Stock
Management - Adding new books to the library catalog. - Updating existing records (e.g., new editions, cor-
rected metadata). - Deleting entries for obsolete or lost books. 2. Book Search and Retrieval - Enables patrons
and librarians to search the catalog for specific books. - Supports filtering by title, author, genre, or availability.
3. Availability Tracking - Monitors loan status (checked out, available, reserved). - Tracks overdue books and
sends notifications. - Identifies books that need reordering or replacement. 4. User Management (Implied) -
Manages librarian and patron accounts (though not explicitly detailed in this lecture).

300
[Link] 1.3 Benefits of the System
• Seamless operation of library services.
• Accurate and up-to-date records of book inventory.
• Improved accessibility for users (patrons and staff).
• Reduction in manual workload for librarians.

7.5.2 2. Requirement Analysis for the Library Management Application

[Link] 2.1 Importance of Requirement Analysis
• Critical for successful software development—ensures the final product aligns with stakeholder needs.
• Prevents scope creep, miscommunication, and inefficient development.

[Link] 2.2 Step-by-Step Requirement Gathering Process

[Link].1 2.2.1 Identify and Engage Stakeholders

• Stakeholders include:
– Librarians (primary users).
– Library administrators.
– Patrons (indirect users).
– IT support staff.
• Engagement methods:
– Regular meetings.
– Feedback sessions.

[Link].2 2.2.2 Gather Requirements

• Techniques used:
– Interviews (structured or unstructured).
– Surveys (for quantitative data).
– Observation (watching current workflows).
– Workshops (collaborative brainstorming).

[Link].3 2.2.3 Document Requirements

• Categorization:
– Functional Requirements (what the system must do):
* Example: “The system shall allow librarians to add new books.”
– Non-Functional Requirements (how the system performs):
* Example: “The system shall respond to search queries within 2 seconds.”
• Clear, concise, and unambiguous documentation is essential.

[Link].4 2.2.4 Prioritize Requirements

• MoSCoW Method (Must-have, Should-have, Could-have, Won’t-have):
– Must-have: Critical features (e.g., book addition/deletion).
– Should-have: Important but not urgent (e.g., advanced search filters).
– Could-have: Nice-to-have (e.g., mobile app integration).
– Won’t-have: Excluded from current scope.
• Delivery timeline:

301
 – Immediate needs (MVP—Minimum Viable Product).
– Optional features (future updates).

[Link].5 2.2.5 Validate and Confirm Requirements

• Review with stakeholders to ensure accuracy.
• Prototyping:
– Create mockups or wireframes for UI/UX feedback.
– Use simple prototypes to demonstrate core functionality.

[Link].6 2.2.6 Manage Changes

• Change tracking:
– Use spreadsheets or tools (e.g., Jira, Trello).
– Document impact of changes on development timeline.
• Communication:
– Keep stakeholders informed about scope adjustments.

7.5.3 3. Key Objectives of the Library Management Application

The application aims to: 1. Provide a User-Friendly Interface - Simplify tasks for librarians (e.g., intuitive
dashboards). 2. Automate Book Stock Management - Reduce manual data entry errors. - Streamline addition,
updating, and deletion of records. 3. Maintain Accurate and Up-to-Date Records - Ensure real-time synchro-
nization between the database and physical inventory. 4. Modernize Library Operations - Replace paper-based
or legacy systems with a digital solution. - Improve efficiency and accessibility for all users.

7.5.4 4. Technologies Used in Development

The application leverages a stack of modern technologies to ensure scalability, performance, and maintainabil-
ity.

[Link] 4.1 Spring Boot

• Role: Backend framework for rapid application development.
• Key Features:
– Convention over configuration: Reduces boilerplate code.
– Embedded servers (e.g., Tomcat) for easy deployment.
– Seamless integration with other frameworks (e.g., JPA, Spring MVC).
• Why Chosen?
– Simplifies setup and deployment.
– Supports microservices architecture (scalable for future expansions).

[Link] 4.2 MySQL

• Role: Relational Database Management System (RDBMS).
• Key Features:
– Reliability and performance for large datasets.
– ACID compliance (ensures data integrity).
– Structured query language (SQL) for efficient data retrieval.
• Use Case:
– Stores book records, user data, and transaction logs.
– Handles concurrent access (multiple librarians using the system simultaneously).

302
[Link] 4.3 Java Persistence API (JPA)
• Role: Object-Relational Mapping (ORM) tool.
• Key Features:
– Maps Java objects to database tables (eliminates manual SQL writing).
– Reduces boilerplate code (e.g., CRUD operations).
– Supports transactions and caching.
• Why Chosen?
– Enhances code readability and maintainability.
– Ensures data consistency across the application.

[Link] 4.4 JavaServer Pages (JSP)

• Role: Dynamic web page generation.
• Key Features:
– Embeds Java code in HTML for dynamic content.
– Supports tag libraries (e.g., JSTL for looping/conditionals).
– Integrates with Spring MVC for seamless data flow.
• Use Case:
– Renders librarian dashboards, book lists, and forms.
– Enables interactive features (e.g., search, filters, updates).

[Link] 4.5 Integration of Technologies

Technology Purpose Benefit

Spring Boot Backend framework Rapid development, scalability
MySQL Database management Reliable, high-performance storage
JPA ORM for database operations Reduces code complexity, ensures integrity
JSP Dynamic frontend rendering Interactive UI, seamless data display

• Collective Advantage:
– Modular architecture (easy to maintain/extend).
– High performance (optimized queries, efficient rendering).
– User-friendly experience (responsive interfaces).

7.5.5 5. MVC (Model-View-Controller) Architecture

The application follows the MVC design pattern to ensure separation of concerns, scalability, and maintain-
ability.

[Link] 5.1 Model Component

• Responsibility: Manages data and business logic.
• Implementation:
– JPA Entities: Represent database tables (e.g., Book, User).
– Repositories: Handle CRUD operations (e.g., BookRepository).
– Service Layer: Contains business rules (e.g., validation, transactions).
• Example:
– A Book entity maps to the books table in MySQL.
– The BookService class enforces rules like “A book cannot be deleted if it is checked out.”

303
[Link] 5.2 View Component
• Responsibility: Presents data to the user.
• Implementation:
– JSP Pages: Dynamically generate HTML based on model data.
– Spring MVC: Renders views and handles user interactions.
• Example:
– A JSP page displays a list of books fetched from the database.
– Includes forms for adding/editing books.

[Link] 5.3 Controller Component

• Responsibility: Acts as an intermediary between Model and View.
• Implementation:
– Spring MVC Controllers: Handle HTTP requests (GET, POST, etc.).
– Processes input (e.g., form submissions).
– Interacts with the Service Layer to update the model.
• Example:
– A BookController receives a POST request to add a new book.
– Validates input, calls BookService, and redirects to a success page.

[Link] 5.4 Benefits of MVC in This Application

1. Separation of Concerns:
• Model: Data logic.
• View: Presentation logic.
• Controller: Request handling.
2. Easier Maintenance:
• Changes to the UI (View) do not affect the database (Model).
3. Scalability:
• New features can be added without rewriting existing components.
4. Testability:
• Each component can be unit-tested independently.

7.5.6 6. Project Development Approach

[Link] 6.1 Example-Driven Learning
• The course uses a Library Management System as a case study.
• Students must:
– Choose their own project (e.g., inventory system, student portal).
– Define custom requirements (similar to the library example).
– Develop progressively (weekly assignments).

[Link] 6.2 Weekly Progress Tracking

• Assignments:
– Submit incremental progress (e.g., database schema, UI mockups).
– Demonstrate functional components (e.g., book search feature).
• Evaluation Criteria:
– Functionality: Does the feature work as intended?
– Code Quality: Is it clean, modular, and well-documented?

304
 – Adherence to MVC: Proper separation of concerns.

[Link] 6.3 Expected Outcomes By the end of the module, students should: 1. Design a database schema
(e.g., tables for books, users, loans). 2. Implement CRUD operations using Spring Boot + JPA. 3. Develop a
dynamic frontend with JSP. 4. Apply MVC principles effectively. 5. Document requirements and changes
systematically.

7.5.7 7. Summary of Key Takeaways

• Library Management Application automates book inventory management.
• Requirement analysis is critical—use MoSCoW prioritization and stakeholder feedback.
• Tech Stack:
– Spring Boot (backend).
– MySQL (database).
– JPA (ORM).
– JSP (frontend).
• MVC Architecture ensures scalable, maintainable code.
• Hands-on development with weekly progress checks.

7.5.8 8. Glossary of Key Terms

Term Definition
CRUD Create, Read, Update, Delete—basic database operations.
ORM Object-Relational Mapping—links database tables to Java objects.
MoSCoW Method Prioritization technique (Must, Should, Could, Won’t have).
JPA Java Persistence API—standard for ORM in Java.
Spring MVC Model-View-Controller framework within Spring Boot.
ACID Atomicity, Consistency, Isolation, Durability—database transaction properties.
Stakeholder Any individual/group affected by the system (e.g., librarians, patrons).

7.6 ORM Setup and Configuration

7.6.1 1. Introduction to Object-Relational Mapping (ORM)
[Link] 1.1 Definition of ORM
• ORM (Object-Relational Mapping) is a technique that bridges the gap between object-oriented pro-
gramming (OOP) and relational databases.
• It allows developers to interact with databases using object-oriented paradigms rather than writing raw
SQL queries.

[Link] 1.2 Motivation Behind ORM ORM is motivated by the following goals: 1. Simplifies data access
– Reduces complexity in database operations. 2. Enhances code quality – Promotes cleaner, more maintainable
code. 3. Promotes an object-oriented approach – Aligns database interactions with Java’s OOP principles.

7.6.2 2. Benefits of Using ORM in Java Applications

ORM provides several advantages over traditional SQL-based database interactions:

305
[Link] 2.1 Reduction of Boilerplate Code
• ORM frameworks automate repetitive SQL operations, eliminating the need for manually writing CRUD
(Create, Read, Update, Delete) statements.
• Example: Instead of writing:
INSERT INTO users (name, email) VALUES ('John', 'john@[Link]');

ORM allows:
User user = new User("John", "john@[Link]");
[Link](user);

[Link] 2.2 Portability Across Databases

• ORM abstracts database-specific details, allowing applications to switch databases (e.g., from MySQL
to PostgreSQL) without modifying application code.
• Achieved through database dialects (e.g., Hibernate dialects for different SQL variants).

[Link] 2.3 Improved Maintainability

• Simplifies data access logic, making the codebase easier to understand and modify.
• Reduces coupling between business logic and database operations.

[Link] 2.4 Object-Oriented Approach

• Developers work with Java objects (e.g., User, Product) instead of tables and rows.
• Aligns with Java’s OOP principles, improving code consistency and developer productivity.

7.6.3 3. Configuring ORM Using Java Persistence API (JPA)

JPA is the standard Java specification for ORM, providing a set of annotations and APIs to map Java objects
to database tables.

[Link] 3.1 Steps to Configure ORM with JPA The configuration process involves the following key steps:

[Link].1 3.1.1 Add JPA Dependencies

• Include a JPA implementation (e.g., Hibernate or EclipseLink) in the project.
• Example (Maven [Link]):
<dependency>
<groupId>[Link]</groupId>
<artifactId>hibernate-core</artifactId>
<version>[Link]</version>
</dependency>

[Link].2 3.1.2 Define Entity Classes

• Create Java classes that represent database tables.
• Annotate them with @Entity to mark them as JPA-managed entities.

306
 • Example:
@Entity
@Table(name = "users")
public class User {
@Id
@GeneratedValue(strategy = [Link])
private Long id;

private String name;

private String email;

// Getters and setters

}

– @Entity: Marks the class as a JPA entity.

– @Table: Specifies the database table name (optional if the class name matches the table).
– @Id: Denotes the primary key.
– @GeneratedValue: Configures auto-increment for the primary key.

[Link].3 3.1.3 Configure the Persistence Unit

• Define a persistence unit in [Link] (located in META-INF/).
• Specifies:
– Database connection details (URL, username, password).
– JPA provider (e.g., Hibernate).
– Database dialect (e.g., [Link].MySQL8Dialect).
• Example ([Link]):
<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="[Link]
version="2.2">
<persistence-unit name="my-persistence-unit">
<provider>[Link]</provider>
<properties>
<property name="[Link]" value="jdbc:mysql://localhost:3306/mydb"
<property name="[Link]" value="root"/>
<property name="[Link]" value="password"/>
<property name="[Link]" value="[Link].MySQL8Dialect"/>
<property name="[Link]" value="update"/>
</properties>
</persistence-unit>
</persistence>

– [Link]: Controls schema generation (update, create, validate, etc.).

[Link].4 3.1.4 Obtain an EntityManager

• The EntityManager is the primary interface for interacting with the database.
• Responsible for persisting, retrieving, updating, and deleting entities.

307
 • Example:
EntityManagerFactory emf = [Link]("my-persistence-unit");
EntityManager entityManager = [Link]();

– EntityManagerFactory: A thread-safe factory for creating EntityManager instances.

– EntityManager: Manages the lifecycle of entities (e.g., persist(), find(), remove()).

[Link].5 3.1.5 Transaction Management

• Database operations must be wrapped in transactions to ensure data integrity.
• Use @Transactional (from Spring or Java EE) to automate transaction handling.
• Example:
@Transactional
public void saveUser(User user) {
[Link](user); // Automatically committed or rolled back
}

– If an exception occurs, the transaction rolls back.

– If successful, the transaction commits.

7.6.4 4. Summary of Key Concepts

Concept Description
ORM Technique to map Java objects to database tables.
JPA Java specification for ORM (implemented by Hibernate, EclipseLink, etc.).
Entity Class A Java class annotated with @Entity to represent a database table.
Persistence Unit Defined in [Link]; configures database connection and JPA settings.
EntityManager Manages CRUD operations and entity lifecycle.
@Transactional Annotation to automate transaction management (commit/rollback).

[Link] 4.1 Key Takeaways

• ORM eliminates repetitive SQL, improving productivity and code quality.
• JPA standardizes ORM in Java, ensuring portability across databases.
• Entity classes, persistence units, and EntityManager are core components of JPA configuration.
• Transactions ensure data consistency and are managed via @Transactional.

7.7 Recording of Building Database Applications Week 6 - Live Session on 26-04-17

7.7.1 1. Introduction to the Library Management Application
[Link] 1.1 Overview of the Application
• Purpose: Develop a Library Management System to manage books in a library inventory.
• Core Functionalities:
– Add new books to the library inventory.
– Update existing book records.
– Delete books from the inventory.
– Search for books (e.g., by title, author, or other attributes).

308
 • Example Use Cases:
– Adding a new book to the library database.
– Updating details (e.g., title, author, publication year) of an existing book.
– Removing a book that is no longer available.
– Searching for books by title or other criteria.

7.7.2 2. MVC (Model-View-Controller) Architecture

[Link] 2.1 Definition and Components
• MVC is a software design pattern used to separate concerns in an application:
– Model: Represents data and business logic.
* Responsible for:
ꞏ Retrieving data from databases or other sources.
ꞏ Data validation and processing.
ꞏ Example: Fetching book records from a database.
– View: Represents the user interface (UI).
* Displays data from the model to the user.
* Receives user input (e.g., search queries, form submissions).
* Example: HTML pages, JSP (JavaServer Pages).
– Controller: Acts as an intermediary between the Model and View.
* Handles user requests (e.g., search, insert, delete).
* Coordinates between the Model (data) and View (UI).
* Example: Processing a user’s request to search for a book.
[Link] 2.2 MVC Workflow
1. User Interaction:
• User interacts with the View (e.g., submits a search form).
2. Controller Handling:
• The Controller receives the request (e.g., “Search for a book”).
• Controller delegates the request to the Model.
3. Model Processing:
• The Model fetches data (e.g., queries the database for books matching the search criteria).
• Model returns the data to the Controller.
4. View Update:
• The Controller passes the data to the View.
• The View renders the data (e.g., displays search results to the user).

[Link] 2.3 Visual Representation of MVC

User → View → Controller → Model → Database → Model → Controller → View → User

• Example:
– User searches for a book titled “Database Systems.”
– View sends the request to the Controller.
– Controller asks the Model to fetch matching books.
– Model queries the database and returns results.
– Controller passes results to the View.
– View displays the books to the user.

309
7.7.3 3. Tools and Technologies
[Link] 3.1 Required Software and Frameworks

Tool/Technology Purpose
Java (JDK 21) Programming language for backend logic.
Maven Build automation and dependency management.
VS Code Integrated Development Environment (IDE) for coding.
Spring Boot Framework for building Java-based web applications (simplifies MVC setup).
MySQL Database management system for storing book records.
JPA (Java Persistence ORM (Object-Relational Mapping) for database interactions.
API)
HTML/CSS/jQuery Frontend technologies for the View layer.
Tomcat Embedded web server for running the Spring Boot application.

[Link] 3.2 Development Environment Setup

1. Install Prerequisites:
• Java JDK 21: Required to run Spring Boot applications.
– Verify installation: Run java -version in the terminal.
• Maven: Used to build the project and manage dependencies.
– Verify installation: Run mvn -version.
• VS Code: Lightweight IDE for development.
– Download from [Link].
2. VS Code Extensions:
• Java Extension Pack: Supports Java development.
• Spring Boot Extension Pack: Tools for Spring Boot projects.
• Maven for Java: Maven integration for Java projects.

7.7.4 4. Project Setup with Spring Boot

[Link] 4.1 Creating a New Spring Boot Project
1. Open VS Code Terminal:
• Use the shortcut Ctrl+Shift+P to open the command palette.
• Type “Spring Initializr: Create a Maven Project” and select it.
2. Configure Project Settings:
• Spring Boot Version: Select 4.0.5 (stable version to avoid issues).
• Language: Java.
• Group ID: Default (e.g., [Link]).
– Represents the base package structure (e.g., [Link]).
• Artifact ID: library-app (or any custom project name).
• Packaging: JAR (simpler for embedded server deployment).
• Java Version: 21 (must match installed JDK).
3. Add Dependencies:
• Search and select “Spring Web” (for web application development).
4. Generate Project:
• Choose a workspace folder (e.g., C:\Users\SpringProjects).
• Click “Generate” to create the project structure.
• Open the generated folder in VS Code.

310
[Link] 4.2 Project Structure Overview
library-app/
├── src/
│ ├── main/
│ │ ├── java/
│ │ │ └── com/example/libraryapp/
│ │ │ └── [Link] (Main class)
│ │ └── resources/
│ │ └── [Link] (Configuration file)
│ └── test/ (for unit tests)
├── [Link] (Maven configuration and dependencies)
└── target/ (compiled output)

• Key Files:
– [Link]: Entry point for the Spring Boot application.
– [Link]: Contains project dependencies (e.g., Spring Web, JPA).
– resources/: Configuration files (e.g., [Link]).

7.7.5 5. Running the Spring Boot Application

[Link] 5.1 Starting the Application
1. Open Terminal in VS Code:
• Navigate to Terminal → New Terminal.
2. Run the Application:
• Execute the command:
mvn spring-boot:run
• Expected Output:
– Maven downloads dependencies (if first run).
– Tomcat server starts on port 8080.
– Log message: "Tomcat started on port 8080".

[Link] 5.2 Verifying the Application

1. Access the Application:
• Open a web browser and navigate to:
[Link]
2. Expected Response:
• White Label Error Page (HTTP 404):
– Indicates the application is running but no homepage is mapped.
– Reason: No controller is defined to handle the root (/) endpoint.

7.7.6 6. Creating the First Controller

[Link] 6.1 Purpose of the Controller
• Acts as the C in MVC.
• Handles HTTP requests (e.g., GET, POST) and returns responses.
• Example: Mapping the root URL (/) to a welcome message.

311
[Link] 6.2 Steps to Create a Controller
1. Create a New File:
• Right-click on the [Link] package → New File.
• Name the file: [Link].
2. Add Controller Code:
package [Link];

import [Link];
import [Link];
import [Link];

@Controller
public class HomeController {

@GetMapping("/")
@ResponseBody
public String home() {
return "Library Application is running successfully!";
}
}

3. Explanation of Annotations:
• @Controller: Marks the class as a Spring MVC controller.
• @GetMapping("/"): Maps HTTP GET requests for / to the home() method.
• @ResponseBody: Returns the string directly as the HTTP response (no view template).

[Link] 6.3 Testing the Controller

1. Restart the Application:
• Stop the running application (Ctrl+C in the terminal).
• Re-run:
mvn spring-boot:run
2. Access the Endpoint:
• Refresh [Link]
• Expected Output:
Library Application is running successfully!
• Without @Controller:
– If @Controller is removed, Spring ignores the class, and the endpoint returns a 404 error.

7.7.7 7. Key Takeaways from the Session

[Link] 7.1 Summary of Topics Covered
1. Introduction to MVC:
• Separation of concerns: Model (data), View (UI), Controller (logic).
• Workflow: User → View → Controller → Model → View.
2. Project Setup:
• Installed Java, Maven, and VS Code.

312
 • Created a Spring Boot project using Spring Initializr.
• Configured dependencies (Spring Web).
3. Running the Application:
• Used mvn spring-boot:run to start the embedded Tomcat server.
• Verified the application via localhost:8080 (initially returned 404).
4. Controller Creation:
• Added a HomeController to handle the root endpoint.
• Used @GetMapping and @ResponseBody to return a custom message.

[Link] 7.2 Common Issues and Fixes

Issue Cause Solution

Port 8080 already in use Previous instance not stopped. Stop the running app (Ctrl+C) or use
a different port (e.g.,
[Link]=8081 in
[Link]).
404 Error on localhost:8080 No controller mapped to /. Add a @Controller with
@GetMapping("/").
Changes not reflected File not saved or app not restarted. Save the file and restart the
application.

[Link] 7.3 Next Steps (Preview of Upcoming Sessions)

• Model Layer:
– Connecting to a MySQL database using JPA.
– Defining entities (e.g., Book class) and repositories.
• View Layer:
– Creating JSP/Thymeleaf templates for the UI.
– Handling user input (e.g., forms for adding/updating books).
• CRUD Operations:
– Implementing Create, Read, Update, Delete for books.
– Example: Adding a new book via a form, displaying search results.

7.7.8 8. Q&A Highlights

[Link] 8.1 Student Questions and Answers
1. Q: Do we need to follow the exact same steps (e.g., imports, controller setup) for our own applications?
• A: Yes, the annotations (@Controller, @GetMapping) and project structure are standard for Spring
Boot MVC applications.
2. Q: Is the HomeController the same as the MVC Controller?
• A: Yes, it is the Controller in the MVC pattern, implemented in Java code.
3. Q: How does the browser display the message from the controller?
• A:
– The browser sends a GET request to /.
– Spring routes the request to the home() method in HomeController.
– The @ResponseBody annotation sends the returned string directly as the HTTP response.

313
7.8 Setting Up MySQL Database
7.8.1 Introduction to MySQL
• Definition: MySQL is a popular open-source Relational Database Management System (RDBMS).
• Purpose: Allows efficient storage and management of data using tables and SQL queries.
• Key Features:
– Reliability: Ensures data integrity and consistency.
– Scalability: Can handle large datasets and high traffic.
• Importance: Essential for database management in applications requiring structured data storage.

7.8.2 Installation of MySQL on Ubuntu

[Link] Step 1: Update Package Index and Upgrade Packages
• Run the following commands to ensure the system is up-to-date:
sudo apt update
sudo apt upgrade

• Purpose: Ensures all existing packages are updated before installing new software.

[Link] Step 2: Install MySQL Server

• Install MySQL using:
sudo apt install mysql-server

• Details:
– Installs MySQL server along with necessary dependencies.
– Sets up the required system services for MySQL operation.

7.8.3 Securing MySQL Installation

[Link] Step 1: Run Security Script
• Execute the following command to initiate the security configuration:
sudo mysql_secure_installation

• Purpose: Protects the MySQL server from unauthorized access and vulnerabilities.

[Link] Step 2: Security Configuration Prompts

• The script will guide through the following security measures:
1. Set Root Password: Establishes a secure password for the root user.
2. Remove Anonymous Users: Prevents unauthorized access via anonymous logins.
3. Disallow Remote Root Login: Restricts root access to local connections only.
4. Remove Test Database: Deletes the default test database to eliminate potential security risks.

7.8.4 Managing MySQL Service

[Link] Step 1: Check MySQL Status
• Verify if MySQL is running:

314
 sudo systemctl status mysql

• Expected Output: Should display active (running) if MySQL is operational.

[Link] Step 2: Start MySQL (If Not Running)

• If MySQL is inactive, start it manually:
sudo systemctl start mysql

[Link] Step 3: Enable MySQL on System Boot

• Ensure MySQL starts automatically during system startup:
sudo systemctl enable mysql

• Purpose: Guarantees MySQL is available without manual intervention after a reboot.

[Link] Understanding systemctl

• Definition: A Linux utility for managing the systemd system and service manager.
• Functions:
– Controls system services (start, stop, restart).
– Manages system states (boot, shutdown).
– Oversees system initialization processes.

[Link] Understanding systemd

• Definition: The initialization system and service manager used in modern Linux distributions.
• Role: Manages system processes, services, and dependencies.

7.8.5 Creating a Database in MySQL

[Link] Step 1: Log in to MySQL Client
• Access MySQL as the root user:
sudo mysql -u root -p

• Note: Replace -p with the root password when prompted.

[Link] Step 2: Create a New Database

• Execute the following SQL command to create a database for a library application:
CREATE DATABASE library_db;

• Purpose: Establishes a dedicated database (library_db) for storing library-related data.

7.8.6 Designing Database Tables

[Link] Step 1: Switch to the Database
• Select the newly created database:

315
 USE library_db;

[Link] Step 2: Create the authors Table

• Define a table to store author details:
CREATE TABLE authors (
AuthorID INT AUTO_INCREMENT PRIMARY KEY,
Name VARCHAR(100) NOT NULL,
BirthDate DATE
);

• Fields:
– AuthorID: Unique identifier for each author (auto-incremented primary key).
– Name: Stores the full name of the author (max 100 characters).
– BirthDate: Records the author’s birth date.

[Link] Step 3: Create Additional Tables

• Similarly, create tables for categories and books to manage:
– Book categories (e.g., Fiction, Non-Fiction).
– Book details (e.g., title, author, category, publication date).
• Example:
CREATE TABLE categories (
CategoryID INT AUTO_INCREMENT PRIMARY KEY,
Name VARCHAR(50) NOT NULL
);

CREATE TABLE books (

BookID INT AUTO_INCREMENT PRIMARY KEY,
Title VARCHAR(100) NOT NULL,
AuthorID INT,
CategoryID INT,
PublicationDate DATE,
FOREIGN KEY (AuthorID) REFERENCES authors(AuthorID),
FOREIGN KEY (CategoryID) REFERENCES categories(CategoryID)
);

• Purpose: Establishes relationships between tables (e.g., books linked to authors and categories).

7.8.7 Populating the Database with Sample Data

[Link] Step 1: Insert Data into authors Table
• Example SQL command:
INSERT INTO authors (Name, BirthDate)
VALUES ('J.K. Rowling', '1965-07-31');

316
[Link] Step 2: Insert Data into categories Table
• Example SQL command:
INSERT INTO categories (Name)
VALUES ('Fantasy');

[Link] Step 3: Insert Data into books Table

• Example SQL command:
INSERT INTO books (Title, AuthorID, CategoryID, PublicationDate)
VALUES ('Harry Potter and the Philosopher''s Stone', 1, 1, '1997-06-26');

• Purpose: Demonstrates data relationships (e.g., a book linked to an author and category).

[Link] Step 4: Verify Data Insertion

• Retrieve all records from a table to confirm data population:
SELECT * FROM authors;

• Expected Output: Displays all inserted author records.

7.8.8 Connecting to MySQL via Command Line

[Link] Step 1: Log in to MySQL with Database Specification
• Connect directly to the library_db database:
mysql -u root -p library_db

• Note: Replace -p with the root password when prompted.

[Link] Step 2: Execute SQL Queries

• Perform operations such as:
SELECT * FROM authors;

• Purpose: Retrieves all records from the authors table for verification.

[Link] Step 3: Exit MySQL Client

• Terminate the MySQL session:
exit;

7.8.9 Summary of Key Concepts

1. MySQL Installation:
• Update system packages (sudo apt update, sudo apt upgrade).
• Install MySQL server (sudo apt install mysql-server).
2. Securing MySQL:
• Run sudo mysql_secure_installation to configure security settings.
3. Service Management:
• Use systemctl to check status, start, and enable MySQL.

317
 4. Database Creation:
• Log in to MySQL (mysql -u root -p).
• Create a database (CREATE DATABASE library_db).
5. Table Design:
• Define tables (authors, categories, books) with appropriate fields and relationships.
6. Data Population:
• Insert sample data using INSERT INTO statements.
7. Query Execution:
• Connect to the database and run SQL queries (e.g., SELECT * FROM authors).
8. Command-Line Interaction:
• Log in, execute queries, and exit the MySQL client.

7.8.10 Learning Outcomes

By the end of this lecture, students should be able to: - Install and secure MySQL on an Ubuntu system. -
Create and manage a database (library_db) for a library application. - Design tables to store structured data
(authors, books, categories). - Populate tables with sample data and verify insertion. - Connect to MySQL via
the command line and execute SQL queries.

7.9 Setting Up the Development Environment

7.9.1 Introduction
This lecture covers the configuration of Visual Studio Code (VS Code) on a Linux environment for Spring Boot
development, including the installation and setup of: - Java JDK - Maven - VS Code extensions - Git for version
control
While the lecture uses Linux, the steps are adaptable to Windows or other operating systems that support the
required tools.

7.9.2 Prerequisites
Before proceeding, ensure the following: - A Windows or Linux-based operating system. - For Linux users,
basic knowledge of terminal commands. - A stable internet connection.

7.9.3 Installing Java Development Kit (JDK)

[Link] Steps to Install OpenJDK
1. Update the package list:
sudo apt update

2. Install OpenJDK (default version or specify a version, e.g., openjdk-17-jdk):

sudo apt install openjdk-17-jdk

3. Verify installation by checking the Java version:

java -version

• This command should display the installed JDK version, confirming successful installation.

318
7.9.4 Installing Visual Studio Code (VS Code)
[Link] Adding Microsoft GPG Key and Repository
1. Add the Microsoft GPG key:
wget -qO- [Link] | gpg --dearmor > [Link]
sudo install -o root -g root -m 644 [Link] /usr/share/keyrings/
sudo sh -c 'echo "deb [arch=amd64,arm64,armhf signed-by=/usr/share/keyrings/[Link].g

2. Update the package list:

sudo apt update

3. Install VS Code:
sudo apt install code

4. Verify installation by launching VS Code:

code

7.9.5 Installing VS Code Extensions for Spring Boot Development

[Link] Required Extensions
1. Java Extension Pack
• Provides Java language support, debugging, and project management.
• Install via VS Code Extensions Marketplace (search for “Java Extension Pack”).
2. Spring Boot Extension Pack
• Adds Spring Boot-specific tools, including project templates and debugging support.
• Install via VS Code Extensions Marketplace (search for “Spring Boot Extension Pack”).

7.9.6 Installing Maven for Project Management

[Link] Steps to Install Maven
1. Update the package list:
sudo apt update

2. Install Maven:
sudo apt install maven

3. Verify installation by checking the Maven version:

mvn -v

• This command should display the installed Maven version, confirming successful installation.

7.9.7 Setting Up a Spring Boot Project

[Link] Downloading and Extracting a Spring Boot Project
1. Generate a Spring Boot project using Spring Initializr.
• Select dependencies (e.g., Spring Web).
• Download the project as a ZIP file (e.g., [Link]).

319
 2. Extract the ZIP file in the desired project directory:
unzip [Link]

3. Open the project in VS Code:

code myapp

[Link] Importing the Maven Project into VS Code

• Option 1 (Automatic Prompt):
– When opening the project folder, VS Code may prompt to “Import Maven Projects”.
– Click “Import” to load the project dependencies.
• Option 2 (Manual Import):
– Open the Command Palette (Ctrl + Shift + P).
– Type and select: “Java: Import Java Projects”.
– Choose the project directory to import.

7.9.8 Running the Spring Boot Application

[Link] Steps to Run the Application
1. Open the main application file (e.g., [Link]).
2. Run the application using one of the following methods:
• Method 1 (VS Code UI):
– Click the “Run” icon in the top-right corner.
• Method 2 (Terminal Command):
– Navigate to the project directory and run:
mvn spring-boot:run
3. Verify the application in a web browser:
• Open: [Link]
• If successful, the Spring Boot application will display in the browser.

7.9.9 Setting Up Git for Version Control

[Link] Introduction to Git
• Git is a distributed version control system used to track changes in source code.
• Key Features:
– Branching and Merging – Allows parallel development.
– Distributed Repository – Every developer has a full copy of the project history.
– History Tracking & Version Control – Enables reverting to previous versions.

[Link] Installing Git on Linux

1. Update the package list:
sudo apt update

2. Install Git:
sudo apt install git

3. Verify installation by checking the Git version:

320
 git --version

[Link] Configuring Git

1. Set global username (replace with your name):
git config --global [Link] "Your Name"

2. Set global email (replace with your email):

git config --global [Link] "[Link]@[Link]"

3. Verify configuration:
git config --list

[Link] Initializing a Git Repository

1. Navigate to the project directory (e.g., myapp):
cd myapp

2. Initialize a Git repository:

git init

3. Stage and commit files:

git add .
git commit -m "Initial commit"

[Link] Connecting to a Remote Repository (GitHub/GitLab/Bitbucket)

1. Create a remote repository (e.g., on GitHub).
2. Link the local repository to the remote:
git remote add origin <remote-repository-url>

3. Push changes to the remote repository:

git push -u origin main

7.9.10 Summary of Key Steps

1. Installed Java JDK (openjdk-17-jdk) and verified with java -version.
2. Installed VS Code by adding the Microsoft repository and verified with code.
3. Installed VS Code Extensions (Java Extension Pack, Spring Boot Extension Pack).
4. Installed Maven and verified with mvn -v.
5. Downloaded and extracted a Spring Boot project from Spring Initializr.
6. Imported the Maven project into VS Code.
7. Ran the Spring Boot application and verified at [Link]
8. Installed and configured Git, initialized a repository, and pushed to a remote (GitHub/GitLab/Bitbucket).

321
7.10 Tools and Technologies Needed
7.10.1 Introduction
This lecture covers the essential tools and technologies required to develop a web-based database application,
such as library management software. The key technologies discussed include: - HTML - CSS - jQuery - Java
Servlets - Spring Boot - JSP (JavaServer Pages) - MySQL - JPA (Java Persistence API)
By the end of this lecture, you will be able to: 1. Identify and describe the role of each technology in a web project.
2. Install and set up each technology for development.

7.10.2 1. HTML (HyperText Markup Language)

[Link] 1.1 Role of HTML
• Standard markup language for creating web pages.
• Provides the structure of a web page.
• Allows inclusion of elements such as:
– Headings (<h1> to <h6>)
– Paragraphs (<p>)
– Images (<img>)
– Links (<a>)
– Multimedia (audio, video)
– Tables (<table>, <tr>, <th>, <td>)
– Lists (<ul>, <ol>, <li>)
– Forms (<form>, <input>, <textarea>, <button>)

[Link] 1.2 How to Install and Set Up HTML

• No installation required—HTML files can be created using any text editor (e.g., Notepad, VI, VS Code).
• Save files with a .html extension.

<!DOCTYPE html>
<html>
<head>
<title>Sample Page</title>
</head>
<body>
<h1>Welcome to HTML</h1>
<p>This is a paragraph.</p>
</body>
</html>

[Link] 1.3 Example: Basic HTML Structure

[Link] 1.4 Key HTML Tags for Web Development

Tag Purpose
<h1> to <h6> Headings (h1 = largest, h6 = smallest)
<a> Hyperlinks (e.g., <a href="[Link]">Link</a>)
<table> Tables (<tr> = row, <th> = header cell, <td> = data cell)

322
Tag Purpose
<img> Embed images (e.g., <img src="[Link]" alt="Description">)
<ul>, <ol>, <li> Unordered, ordered, and list items
<form> Collect user input (e.g., login forms)
<input> Input fields (text, password, checkbox, radio buttons)
<textarea> Multi-line text input
<button> Buttons (Submit, Reset, etc.)

[Link] 1.5 Focus Areas for Project Development

• Forms (user data collection, e.g., username/password).
• Tables (displaying structured data).
• Links (navigation between pages).

7.10.3 2. CSS (Cascading Style Sheets)

[Link] 2.1 Role of CSS
• Used to style and layout web pages.
• Controls appearance of HTML elements:
– Colors (text, background)
– Fonts (family, size, weight)
– Spacing (margins, padding)
– Positioning (alignment, floating elements)
– Responsiveness (adapting to different screen sizes)

[Link] 2.2 How to Install and Set Up CSS

• No installation required—CSS files can be created in any text editor.
• Save files with a .css extension.
• Three ways to include CSS in HTML:
1. Inline CSS (inside HTML elements via style attribute).
2. Internal CSS (inside <style> tags in <head>).
3. External CSS (linking a .css file via <link> tag).

body {
background-color: lightblue;
color: navy;
margin-left: 20px;
}

[Link] 2.3 Example: Basic CSS Styling HTML Integration:

<head>
<link rel="stylesheet" href="[Link]">
</head>

7.10.4 3. jQuery
[Link] 3.1 Role of jQuery

323
 • Fast, small, feature-rich JavaScript library.
• Simplifies:
– HTML document traversal & manipulation (e.g., selecting elements).
– Event handling (e.g., clicks, form submissions).
– Animations & AJAX (asynchronous requests).
• Cross-browser compatibility (works consistently across browsers).

[Link] 3.2 How to Install and Set Up jQuery

1. Download from jQuery official website.
2. Include via CDN (Content Delivery Network) (recommended for faster loading):
<script src="[Link]

3. Add to HTML file inside <script> tags.

<button id="hideButton">Click to Hide Paragraph</button>

<p id="paragraph">This is a paragraph.</p>

<script>
$(document).ready(function() {
$("#hideButton").click(function() {
$("#paragraph").hide();
});
});
</script>

[Link] 3.3 Example: Hiding a Paragraph on Button Click Explanation: - $(document).ready() ensures
the DOM is fully loaded before executing. - $("#hideButton") selects the button by ID. - .click() attaches a
click event handler. - $("#paragraph").hide() hides the paragraph when the button is clicked.

[Link] 3.4 Use Cases in Web Applications

• Dynamic content updates (e.g., hiding/showing elements).
• Form validation (client-side checks before submission).
• AJAX calls (fetching data without page reload).

7.10.5 4. Java Servlets

[Link] 4.1 Role of Java Servlets
• Server-side programs that handle client requests and generate dynamic content.
• Core components of Java web applications.
• Key responsibilities:
– Process form submissions.
– Interact with databases.
– Generate HTML dynamically.

[Link] 4.2 How to Install and Set Up Java Servlets

1. Prerequisites:

324
 • Java Development Kit (JDK) installed.
• Servlet container (e.g., Apache Tomcat).
2. Steps:
• Download and install Tomcat from Apache Tomcat website.
• Configure Tomcat in your IDE (e.g., Eclipse, IntelliJ).
• Create a Dynamic Web Project in your IDE.

import [Link].*;
import [Link].*;
import [Link].*;

public class HelloWorldServlet extends HttpServlet {

protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
[Link]("text/html");
PrintWriter out = [Link]();
[Link]("<html><body>");
[Link]("<h1>Hello, World!</h1>");
[Link]("</body></html>");
}
}

[Link] **4.3 Example: Simple Servlet (HelloWorldServlet) Explanation: - Extends HttpServlet to

handle HTTP requests. - doGet() processes GET requests. - [Link]() sends HTML back to the
client.

[Link] 4.4 Deployment

• Compile the Servlet and deploy it in Tomcat’s webapps directory.
• Access via [Link]

7.10.6 5. Spring Boot

[Link] 5.1 Role of Spring Boot
• Framework for building Java-based applications.
• Simplifies setup & development by providing:
– Default configurations (auto-configuration).
– Embedded server (Tomcat, Jetty—no need for external setup).
– Modular dependencies (easy integration with databases, security, etc.).
• Ideal for microservices and REST APIs.

[Link] 5.2 How to Install and Set Up Spring Boot

1. Prerequisites:
• JDK 8+ installed.
• Build tool (Maven or Gradle).
2. Steps:
• Use Spring Initializr ([Link] to generate a project skeleton.
• Select dependencies (e.g., Spring Web, Spring Data JPA).

325
 • Import into an IDE (Eclipse, IntelliJ).

import [Link];
import [Link];

@SpringBootApplication
public class MyApplication {
public static void main(String[] args) {
[Link]([Link], args);
}
}

[Link] **5.3 Example: Basic Spring Boot Application Explanation: - @SpringBootApplication enables
auto-configuration and component scanning. - [Link]() starts the embedded server.

[Link] 5.4 Key Features

• Auto-configuration (reduces boilerplate code).
• Standalone executable JARs (easy deployment).
• Integration with Spring ecosystem (Spring MVC, Spring Data, Spring Security).

7.10.7 6. JSP (JavaServer Pages)

[Link] 6.1 Role of JSP
• Technology for creating dynamic web content by embedding Java code in HTML.
• Server-side rendering (generates HTML on the server before sending to the client).
• Used in conjunction with Servlets (Servlets handle logic, JSP handles presentation).

[Link] 6.2 How to Install and Set Up JSP

1. Prerequisites:
• Servlet container (e.g., Tomcat).
• Java Development Kit (JDK).
2. Steps:
• Create a file with .jsp extension.
• Place it in the webapp directory of a Spring Boot project.

<%@ page language="java" contentType="text/html; charset=UTF-8" %>

<html>
<head>
<title>Hello JSP</title>
</head>
<body>
<h1>Hello JSP!</h1>
</body>
</html>

[Link] **6.3 Example: Simple JSP Page Explanation: - <%@ page %> directive specifies JSP settings. -
Can embed Java code using <% ... %> (scriptlets) or ${...} (EL expressions).

326
[Link] 6.4 Use Cases
• Dynamic web pages (e.g., displaying database records).
• Form handling (processing user input).
• Template inclusion (reusing headers/footers).

7.10.8 7. MySQL
[Link] 7.1 Role of MySQL
• Relational Database Management System (RDBMS).
• Stores and manages application data (e.g., user records, library books).
• Key features:
– ACID compliance (reliable transactions).
– Scalability (handles large datasets).
– SQL support (structured query language).

[Link] 7.2 How to Install and Set Up MySQL

1. Download from MySQL official website.
2. Install and configure:
• Set root password.
• Create a new database for the project.
3. Basic Commands:
-- Create a database
CREATE DATABASE mydatabase;

-- Select the database

USE mydatabase;

-- Create a table
CREATE TABLE users (
id INT PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(50) NOT NULL
);

-- Insert data
INSERT INTO users (name) VALUES ('John Doe');

-- Retrieve data
SELECT * FROM users;

[Link] **7.3 CRUD Operations (Review from Earlier Modules)

Operation SQL Command Example

Create INSERT INTO table VALUES (...) INSERT INTO users (name) VALUES
('Alice')
Read SELECT * FROM table SELECT * FROM users

327
Operation SQL Command Example
Update UPDATE table SET column=value WHERE UPDATE users SET name='Bob' WHERE id=1
...
Delete DELETE FROM table WHERE ... DELETE FROM users WHERE id=1

[Link] 7.4 Integration with Java Applications

• Use JDBC (Java Database Connectivity) or JPA/Hibernate for database interactions.

7.10.9 8. JPA (Java Persistence API)

[Link] 8.1 Role of JPA
• Java specification for managing relational data.
• Provides Object-Relational Mapping (ORM):
– Maps Java objects to database tables.
– Eliminates the need for manual SQL in most cases.
• Implementations:
– Hibernate (most popular).
– EclipseLink, OpenJPA.

[Link] **8.2 How to Install and Set Up JPA

1. Add dependencies (Maven or Gradle):
• Maven ([Link]):
<dependency>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>

• Gradle ([Link]):
implementation '[Link]:spring-boot-starter-data-jpa'
implementation 'mysql:mysql-connector-java'

2. Configure [Link]:
[Link]=jdbc:mysql://localhost:3306/mydatabase
[Link]=root
[Link]=yourpassword
[Link]-auto=update

import [Link].*;

@Entity

328
@Table(name = "users")
public class User {
@Id
@GeneratedValue(strategy = [Link])
private Long id;

@Column(name = "name")
private String name;

// Getters and Setters

}

[Link] **8.3 Example: JPA Entity Class Explanation: - @Entity marks the class as a JPA entity. - @Table
maps to the database table. - @Id and @GeneratedValue define the primary key. - @Column maps fields to table
columns.

import [Link];

public interface UserRepository extends JpaRepository<User, Long> {

// Custom queries can be added here
}

[Link] **8.4 JPA Repository Interface Explanation: - Extends JpaRepository for CRUD operations. -
Spring Data JPA provides auto-implemented methods (e.g., save(), findAll()).

[Link] 8.5 Benefits of JPA

• Reduces boilerplate code (no manual SQL for basic operations).
• Database-agnostic (switch databases with minimal changes).
• Supports transactions & caching.

7.10.10 9. Summary of Tools & Technologies

Technology Role Setup Requirements

HTML Structure of web pages Text editor (no installation)
CSS Styling and layout of web pages Text editor (no installation)
jQuery Dynamic client-side interactions CDN or downloaded library
Java Server-side request handling JDK + Tomcat
Servlets
Spring Boot Java application framework (simplifies setup) JDK + Maven/Gradle
JSP Dynamic web content with embedded Java Tomcat + .jsp files
MySQL Relational database for data storage MySQL server installation
JPA ORM for database interactions Hibernate + Spring Data JPA
dependencies

7.10.11 10. Project Setup Workflow

1. Frontend:

329
 • Create HTML pages for structure.
• Style with CSS.
• Add interactivity with jQuery.
2. Backend:
• Set up Spring Boot for business logic.
• Use Java Servlets (if not using Spring MVC).
• Render dynamic content with JSP.
3. Database:
• Install MySQL and create tables.
• Integrate using JPA/Hibernate.
4. Deployment:
• Package as a WAR (for Servlets/JSP) or JAR (for Spring Boot).
• Deploy to Tomcat or another server.

7.10.12 11. Key Takeaways

• HTML & CSS handle structure and styling.
• jQuery adds client-side dynamism.
• Java Servlets & Spring Boot manage server-side logic.
• JSP bridges Java and HTML for dynamic content.
• MySQL stores application data.
• JPA simplifies database interactions via ORM.
This stack enables the development of scalable, efficient database applications like library management sys-
tems.

7.11 Understanding the MVC Architecture

7.11.1 1. Introduction to MVC
• Definition: MVC (Model-View-Controller) is a design pattern for software development that divides an
application into three interconnected components:
1. Model
2. View
3. Controller
• Purpose: This separation aids in organizing code, making it more maintainable, scalable, and testable.
• Key Principle: Separation of Concerns (SoC) – Each component handles a distinct aspect of the application,
enhancing modularity and reducing interdependencies.

7.11.2 2. The Three Components of MVC

[Link] 2.1 The Model Component
• Role: Represents the core logic of the application, managing:
– Data
– Business logic
– Rules
• Functionality:
– Retrieves data from a database or other storage.
– Processes data according to business rules.
– Returns processed data to the controller.

330
 • Independence: The model is completely independent of the user interface, allowing reuse across different
interfaces (e.g., web, mobile, desktop).
• Example (Library Management System):
– Book Object:
Book {
title: String,
author: String,
ISBN: String,
availability: Boolean
}
– Methods:
* checkAvailability() → Checks if a book is available.
* updateBookInfo() → Updates book details (e.g., title, author).
* setAvailability() → Changes availability status (e.g., checked out/returned).
– Database Interaction: The model updates the database based on logic (e.g., marking a book as un-
available after checkout).

[Link] 2.2 The View Component

• Role: Responsible for displaying data to the user and providing an interactive interface.
• Functionality:
– Renders the user interface (UI).
– Presents data in a user-friendly format (e.g., tables, forms).
– Queries the model to display up-to-date data.
– Listens to the model and updates dynamically when data changes (e.g., real-time availability updates).
• Example (Library Management System):
– HTML Table Display:
<table>
<thead>
<tr>
<th>Title</th>
<th>Author</th>
<th>ISBN</th>
<th>Availability</th>
</tr>
</thead>
<tbody>
<!-- Dynamically populated from model -->
<tr>
<td>{{ [Link] }}</td>
<td>{{ [Link] }}</td>
<td>{{ [Link] }}</td>
<td>{{ [Link] ? "Available" : "Checked Out" }}</td>
</tr>
</tbody>
</table>
– Dynamic Updates: If a book’s availability changes, the view automatically refreshes to reflect the
new state.

331
[Link] 2.3 The Controller Component
• Role: Acts as an intermediary between the model and the view.
• Functionality:
– Handles user input (e.g., form submissions, button clicks).
– Converts input into commands for the model or view.
– Processes requests by:
1. Calling appropriate model methods (e.g., fetch/update data).
2. Passing results to the view for display.
– Decides which view to render based on the model’s state.
• Example (Library Management System):
– Adding a New Book:
// Controller handles HTTP POST request
function addBook(req, res) {
const { title, author, ISBN } = [Link];
const newBook = [Link](title, author, ISBN);
[Link]('bookList', { books: [Link]() }); // Update view
}
– Workflow:
1. User submits a form (view).
2. Controller receives data and calls [Link]().
3. Model updates the database.
4. Controller refreshes the view to show the updated book list.

7.11.3 3. Benefits of Using MVC

[Link] 3.1 Separation of Concerns
• Definition: Clear division of responsibilities among model, view, and controller.
• Advantages:
– Independent Development: Teams can work on model, view, or controller separately.
– Easier Debugging: Issues can be isolated to a specific component.
– Modularity: Changes in one component do not affect others.
• Example (Library Management System):
– Model: Classes like Book, Patron, Loan (each with its own data and methods).
– View: HTML pages for book lists, patron profiles, loan statuses.
– Controller: Handles HTTP requests (e.g., POST /books, GET /loans).

[Link] 3.2 Reusability and Flexibility

• Decoupled Components: Model, view, and controller can be reused in different contexts.
• Advantages:
– Same Model, Different Views: One model can serve web, mobile, or desktop interfaces.
– Adaptability: Easy to update or replace components without rewriting the entire system.
• Example:
– The Book model can be used in:
* A librarian’s web dashboard (detailed view).
* A patron’s mobile app (simplified view).
– UI Changes: The view can be redesigned without altering the model or controller.

[Link] 3.3 Maintainability and Scalability

332
 • Structured Code: MVC provides a clear organization, simplifying updates.
• Advantages:
– Isolated Changes: Modifying business logic (model) does not impact the UI (view).
– Scalability: New features can be added to specific components without disrupting the entire system.
• Example:
– Adding a notification system for overdue books:
1. Extend the Loan model with sendOverdueNotice().
2. Update the controller to trigger notifications.
3. No changes needed in existing views.

[Link] 3.4 Parallel Development

• Collaboration: Multiple teams can work simultaneously on different components.
• Example (Library Management System):
– Team 1: Develops the Book model (business logic).
– Team 2: Designs the search interface (view).
– Team 3: Implements checkout logic (controller).
• Result: Faster development cycles and improved teamwork.

[Link] 3.5 Testability

• Isolated Testing: Each component can be tested independently.
• Advantages:
– Unit Tests: Verify model methods (e.g., checkAvailability()).
– View Tests: Ensure correct HTML rendering.
– Controller Tests: Validate request handling (e.g., POST /checkout).
• Example:
– Book Model Test:
test('checkAvailability returns true for available books', () => {
const book = new Book({ availability: true });
expect([Link]()).toBe(true);
});
– View Test: Check if the table renders all book fields.
– Controller Test: Simulate a checkout request and verify database updates.

[Link] 3.6 Improved User Experience

• Dynamic Interactions: MVC enables real-time updates without full page reloads.
• Example (Library Management System):
– When a patron checks out a book:
1. Controller updates the Loan model.
2. View instantly reflects the new loan status (e.g., “Checked Out”).
– Result: Faster, smoother interactions for users.

7.11.4 4. MVC Component Interaction Workflow

[Link] 4.1 Example Scenario: Patron Checks Out a Book
1. User Action:
• Patron searches for a book via the view (e.g., search bar).
2. Controller:

333
 • Receives the search request.
• Queries the model for matching books.
3. Model:
• Retrieves book data from the database.
• Returns results to the controller.
4. View:
• Displays search results (e.g., table of books).
5. Checkout Process:
• Patron selects a book → controller receives the request.
• Controller validates the request and updates the model (e.g., setAvailability(false)).
• Model updates the database (e.g., marks book as checked out).
• Controller updates the view to show:
– Confirmation message.
– Updated loan status.

[Link] 4.2 Real-World Analogies

• E-Commerce: MVC manages product listings, cart updates, and checkout.
• Airline Booking: Handles seat availability, reservations, and confirmations.
• General Rule: Any application with CRUD (Create, Read, Update, Delete) operations can benefit from
MVC.

7.11.5 5. Practical Application: Project Guidelines

• Project Task: Develop a database-driven application using MVC.
• Requirements:
– Choose a domain (e.g., library, e-commerce, airline booking).
– Implement full CRUD operations.
– Apply MVC separation:
* Model: Business logic and database interactions.
* View: User interface (HTML, templates).
* Controller: Request handling and workflow coordination.
• Outcome: A well-organized, maintainable, and scalable system.

7.11.6 6. Summary of Key Concepts

Component Role Example (Library System)

Model Manages data and business logic Book class with checkAvailability()
View Displays data to the user HTML table showing book listings
Controller Handles user input and workflow Processes checkout requests

[Link] Benefits Recap:

1. Separation of Concerns → Clean, modular code.
2. Reusability → Same model for multiple views.
3. Maintainability → Easy updates and scaling.
4. Parallel Development → Team collaboration.
5. Testability → Isolated unit tests.
6. User Experience → Real-time updates.

334
**End of Notes**

335
8 Module 8: Implementing the Model Layer
8.1 Creating a Repository Interface for Book
8.1.1 Introduction to Repository Interfaces in Spring Data JPA
• A repository interface in Spring Data JPA is an abstraction layer that simplifies database interactions for
a specific entity (e.g., Book).
• It eliminates the need to write manual SQL queries by providing pre-built methods for common database
operations.
• The repository acts as a bridge between the application and the database, managing CRUD (Create, Read,
Update, Delete) operations efficiently.

8.1.2 Defining the BookRepository Interface

• To interact with the Book entity (which maps to a database table), we define a repository interface.
• This interface extends JpaRepository, which is part of Spring Data JPA and provides built-in methods
for database operations.

public interface BookRepository extends JpaRepository<Book, Long> {

// Custom query methods can be added here
}

[Link] Basic Structure of BookRepository

• JpaRepository<Book, Long>:
– Book: The entity class being managed.
– Long: The data type of the entity’s primary key (ID).
• By extending JpaRepository, the interface inherits standard CRUD methods without requiring manual
implementation.

8.1.3 CRUD Operations Provided by JpaRepository

JpaRepository includes several pre-defined methods for basic database operations:

[Link] 1. Create/Update: save()

• Purpose: Inserts a new entity or updates an existing one.
• Example:
Book newBook = new Book("Effective Java", "Education", "Joshua Bloch");
[Link](newBook); // Saves the new book to the database

[Link] 2. Read: findById() and findAll()

• findById(Long id):
– Retrieves a single entity by its ID.
– Returns an Optional<Book> (to handle cases where the entity may not exist).
– Example:
Optional<Book> book = [Link](1L);
if ([Link]()) {

336
 [Link]([Link]().getTitle());
}
• findAll():
– Retrieves all entities of the given type.
– Returns a List<Book>.
– Example:
List<Book> allBooks = [Link]();

[Link] 3. Delete: deleteById() and delete()

• deleteById(Long id):
– Removes an entity by its ID.
– Example:
[Link](1L); // Deletes the book with ID 1
• delete(Book entity):
– Removes a specific entity instance.
– Example:
Book bookToDelete = [Link](1L).orElseThrow();
[Link](bookToDelete);

[Link] 4. Existence Check: existsById()

• Purpose: Verifies whether an entity with a given ID exists.
• Returns: boolean (true if the entity exists, false otherwise).
• Example:
boolean exists = [Link](1L);

[Link] 5. Pagination: findAll(Pageable pageable)

• Purpose: Retrieves entities in pages (useful for large datasets).
• Example:
Page<Book> booksPage = [Link]([Link](0, 10)); // First 10 books

– [Link](0, 10): Requests the first page (index 0) with 10 items per page.

[Link] 6. Sorting: findAll(Sort sort)

• Purpose: Retrieves entities in a sorted order.
• Example:
List<Book> sortedBooks = [Link]([Link]("title").ascending());

– [Link]("title").ascending(): Sorts books by title in ascending order.

337
8.1.4 Custom Query Methods
While JpaRepository provides standard CRUD operations, custom queries are often needed for specific use
cases. Spring Data JPA allows defining query methods using naming conventions or annotations.

[Link] 1. Query Methods by Naming Convention Spring Data JPA automatically generates queries based
on method names following a specific pattern: - Pattern: findBy[Property][Condition] - Property: Field
name in the entity (e.g., title, category). - Condition: Optional (e.g., Containing, IgnoreCase).

[Link].1 Examples:

Method Signature Generated Query Description

List<Book> findByCategory(String SELECT * FROM Book WHERE Finds books by exact
category) category = ? category.
Long countByAuthor(String SELECT COUNT(*) FROM Book Counts books by a specific
author) WHERE author = ? author.
List<Book> SELECT * FROM Book WHERE title Finds books where the title
findByTitleContaining(String LIKE '%?%' contains a substring.
title)

public interface BookRepository extends JpaRepository<Book, Long> {

List<Book> findByCategory(String category);
Long countByAuthor(String author);
List<Book> findByTitleContaining(String title);
}

[Link].2 Implementation in BookRepository:

[Link] 2. Custom Queries Using @Query Annotation For complex queries that cannot be expressed via nam-
ing conventions, the @Query annotation allows: - JPQL (Java Persistence Query Language): Object-oriented
queries. - Native SQL: Direct database queries.

@Query("SELECT b FROM Book b WHERE [Link] = :title")

List<Book> findBooksByTitle(@Param("title") String title);

[Link].1 a. JPQL Example

• SELECT b FROM Book b: JPQL syntax (operates on entities, not tables).
• :title: A named parameter bound to the method argument via @Param.

@Query(value = "SELECT * FROM books WHERE title = :title", nativeQuery = true)

List<Book> findBooksByTitleNative(@Param("title") String title);

[Link].2 b. Native SQL Example

• nativeQuery = true: Indicates a native SQL query (directly executed on the database).
• books: The actual table name in the database.

338
8.1.5 Key Annotations in Repository Interfaces
Spring Data JPA provides several annotations to enhance repository functionality:

Annotation Purpose
@Repository Marks the interface as a Spring component
and enables exception translation (converts
database exceptions to Spring’s
DataAccessException).
@Query Defines custom JPQL or native SQL queries.
@Param Binds a method parameter to a query
parameter.
@Modifying Indicates that a query method modifies data
(e.g., UPDATE, DELETE). Required for non-select
operations.

@Modifying
@Query("UPDATE Book b SET [Link] = :newTitle WHERE [Link] = :id")
void updateBookTitle(@Param("id") Long id, @Param("newTitle") String newTitle);

[Link] Example with @Modifying

• @Modifying: Signals that this method performs an update operation.

8.1.6 Summary of Key Concepts

1. Repository Interface:
• Acts as a data access layer for an entity (e.g., Book).
• Extends JpaRepository to inherit CRUD methods.
2. Built-in CRUD Methods:
• save(): Create/update.
• findById()/findAll(): Read.
• deleteById()/delete(): Delete.
• existsById(): Check existence.
• Pagination (Pageable) and sorting (Sort).
3. Custom Query Methods:
• Naming conventions: Auto-generates queries (e.g., findByCategory).
• @Query: Defines JPQL or native SQL for complex queries.
• @Param: Binds method parameters to query parameters.
4. Important Annotations:
• @Repository, @Query, @Param, @Modifying.

8.1.7 Conclusion
By defining a BookRepository interface that extends JpaRepository, developers can: - Leverage pre-built
CRUD operations without manual implementation. - Create custom queries using naming conventions or an-
notations. - Handle complex database interactions efficiently with minimal boilerplate code.
This approach reduces development time and improves maintainability by abstracting database operations into
a clean, type-safe interface.

339
8.2 Creating the Book Entity Class
8.2.1 Introduction to Entity Classes in JPA
• Purpose of Entity Classes:
– Entity classes in Java Persistence API (JPA) serve as a bridge between Java objects and database
tables.
– Each instance of an entity class corresponds to a record (row) in a database table.
– Enables object-relational mapping (ORM), allowing database interactions to be managed using Java
objects rather than direct SQL queries.
• Key Benefits of Entity Classes:
– Mapping to Database Tables:
* Fields in the entity class map to columns in the corresponding database table.
* Facilitates seamless interaction between Java applications and relational databases.
– Encapsulation of Data:
* Entity classes encapsulate domain-specific data, ensuring data integrity through controlled ac-
cess (private fields with public getters/setters).
– Primary Key Management:
* Every entity class must have a primary key to uniquely identify records.
* Annotations like @Id and @GeneratedValue are used to define and auto-generate primary keys.
– Persistence Context Management:
* Entities are managed within the persistence context of an EntityManager.
* Changes to entities are automatically synchronized with the database.
– Validation and Business Rules:
* Entity classes can include validation annotations (e.g., @NotNull) to enforce constraints.
– Computed Properties:
* May include derived fields (e.g., calculated values based on other attributes).
– Relationships Between Entities:
* Annotations like @OneToOne, @ManyToOne, @OneToMany, and @ManyToMany define associations
between entities.
* Supports cascading operations (e.g., saving/updating related entities automatically).

8.2.2 Key JPA Annotations for Entity Classes

The following annotations are fundamental to defining an entity class:
1. @Entity
• Marks a class as a JPA entity, indicating it should be mapped to a database table.
• Example:
@Entity
public class Book { ... }
2. @Id
• Designates a field as the primary key of the entity.
• Example:
@Id
private Long id;
3. @GeneratedValue
• Specifies the strategy for generating primary key values (e.g., auto-increment).
• Example:

340
 @GeneratedValue(strategy = [Link])
private Long id;
4. @Column (Optional)
• Customizes the mapping of a field to a database column (e.g., name, constraints).
• Example:
@Column(name = "book_title", nullable = false)
private String title;
5. @NotNull (Validation)
• Ensures a field cannot be null when persisted to the database.
• Example:
@NotNull
private String author;

8.2.3 Creating the Book Entity Class: Step-by-Step

[Link] 1. Class Declaration and Annotations
• The Book class is annotated with @Entity to indicate it is a JPA entity.
• The class name (Book) defaults to the table name in the database (unless overridden with @Table).
@Entity
public class Book {
// Fields, constructors, and methods
}

[Link] 2. Defining the Primary Key

• The id field is marked as the primary key using @Id.
• @GeneratedValue ensures the ID is auto-generated (e.g., via database auto-increment).
@Id
@GeneratedValue(strategy = [Link])
private Long id;

[Link] 3. Adding Attribute Fields

• Fields represent columns in the database table.
• Validation annotations (e.g., @NotNull) enforce constraints.
@NotNull
private String title;

@NotNull
private String author;

@NotNull
private String category;

[Link] 4. Constructors
• Default Constructor:

341
 – Required by JPA for instantiating entities (e.g., during database retrieval).
– Initializes an empty Book object.
public Book() {
// Default constructor
}

• Parameterized Constructor:
– Allows creating a Book with initial values (e.g., from user input).
public Book(String title, String author, String category) {
[Link] = title;
[Link] = author;
[Link] = category;
}

[Link] 5. Getter and Setter Methods

• Getters: Retrieve field values (e.g., for displaying in the view layer).
• Setters: Update field values (e.g., from user input before saving to the database).
// Getters
public Long getId() { return id; }
public String getTitle() { return title; }
public String getAuthor() { return author; }
public String getCategory() { return category; }

// Setters
public void setId(Long id) { [Link] = id; }
public void setTitle(String title) { [Link] = title; }
public void setAuthor(String author) { [Link] = author; }
public void setCategory(String category) { [Link] = category; }

[Link] 6. toString() Method

• Provides a string representation of the Book object for debugging or display purposes.
• Formats fields (e.g., id, title, author, category) into a readable string.
@Override
public String toString() {
return "ID: " + id +
", Title: " + title +
", Author: " + author +
", Category: " + category;
}

8.2.4 Example: Complete Book Entity Class

Below is the full implementation of the Book entity class with all discussed components:
import [Link].*;
import [Link];

342
@Entity
public class Book {

@Id
@GeneratedValue(strategy = [Link])
private Long id;

@NotNull
private String title;

@NotNull
private String author;

@NotNull
private String category;

// Default constructor
public Book() {}

// Parameterized constructor
public Book(String title, String author, String category) {
[Link] = title;
[Link] = author;
[Link] = category;
}

// Getters
public Long getId() { return id; }
public String getTitle() { return title; }
public String getAuthor() { return author; }
public String getCategory() { return category; }

// Setters
public void setId(Long id) { [Link] = id; }
public void setTitle(String title) { [Link] = title; }
public void setAuthor(String author) { [Link] = author; }
public void setCategory(String category) { [Link] = category; }

// toString() method
@Override
public String toString() {
return "ID: " + id +
", Title: " + title +
", Author: " + author +
", Category: " + category;
}
}

343
8.2.5 Key Takeaways
1. Entity Classes map Java objects to database tables, enabling ORM in JPA.
2. Annotations (@Entity, @Id, @GeneratedValue, @NotNull) define the mapping and constraints.
3. Constructors (default and parameterized) support object instantiation and initialization.
4. Getters/Setters ensure controlled access to fields, maintaining encapsulation.
5. toString() provides a readable format for displaying entity data.
6. Validation (e.g., @NotNull) enforces business rules at the database level.

8.2.6 Practical Application

• For your project, define entity classes for other domain objects (e.g., Author, Publisher) using the same
JPA annotations.
• Ensure all entities:
– Have a primary key (@Id).
– Include validation (e.g., @NotNull for required fields).
– Follow encapsulation principles (private fields with getters/setters).
– Override toString() for debugging/display purposes.

8.3 Defining Relationships Between Entities

8.3.1 Introduction to Entity Relationships in JPA
• Java Persistence API (JPA) provides a framework for managing relational data in Java applications.
• Primary focus of JPA: Mapping Java objects (entities) to database tables and managing relationships be-
tween these objects.
• Key objective of this lecture: Understanding and implementing entity relationships using JPA annotations.

8.3.2 Types of Entity Relationships in JPA

Three primary types of entity relationships are covered:

[Link] 1. One-to-One Relationship

• Definition: Each instance of one entity is associated with exactly one instance of another entity.
• Example: An author has one biography, and a biography belongs to one author.
• JPA Annotation: @OneToOne
• Key Attributes:
– mappedBy: Indicates the non-owning side of the relationship (i.e., the entity that does not hold the
foreign key).
– @JoinColumn: Defines the foreign key column in the owning entity.

[Link].1 Implementation Example: Author and Biography

• Author Entity:
@Entity
public class Author {
@Id
private Long id;
private String name;

@OneToOne(mappedBy = "author") // Non-owning side

344
 private Biography biography;
}

• Biography Entity:
@Entity
public class Biography {
@Id
private Long id;
private String text;

@OneToOne
@JoinColumn(name = "author_id") // Owning side (foreign key)
private Author author;
}

• Relationship Dynamics:
– Each Author has one Biography.
– Each Biography is associated with one Author.
– The Biography entity owns the relationship (holds the foreign key author_id).

[Link] 2. One-to-Many and Many-to-One Relationships

• Definition:
– One-to-Many: One instance of an entity is associated with multiple instances of another entity.
– Many-to-One: Multiple instances of an entity are associated with one instance of another entity.
• Example: A category contains multiple books, and each book belongs to one category.
• JPA Annotations:
– @OneToMany (on the “one” side)
– @ManyToOne (on the “many” side)
• Key Attributes:
– mappedBy: Used in @OneToMany to reference the owning side (the “many” side).
– @JoinColumn: Defines the foreign key in the “many” side.

[Link].1 Implementation Example: Category and Book

• Category Entity (“One” side):
@Entity
public class Category {
@Id
private Long id;
private String name;

@OneToMany(mappedBy = "category") // Non-owning side

private List<Book> books;
}

• Book Entity (“Many” side):

@Entity
public class Book {

345
 @Id
private Long id;
private String title;

@ManyToOne
@JoinColumn(name = "category_id") // Owning side (foreign key)
private Category category;
}

• Relationship Dynamics:
– One Category can have many Book instances.
– Each Book belongs to one Category.
– The Book entity owns the relationship (holds the foreign key category_id).

[Link] 3. Many-to-Many Relationship

• Definition: Multiple instances of one entity are associated with multiple instances of another entity.
• Example: A book can have multiple authors, and an author can write multiple books.
• JPA Annotation: @ManyToMany
• Key Attributes:
– @JoinTable: Defines an intermediate table to store the relationship (holds foreign keys for both
entities).
– mappedBy: Indicates the non-owning side of the relationship.

[Link].1 Implementation Example: Book and Author

• Book Entity (Owning Side):
@Entity
public class Book {
@Id
private Long id;
private String title;

@ManyToMany
@JoinTable(
name = "book_author",
joinColumns = @JoinColumn(name = "book_id"),
inverseJoinColumns = @JoinColumn(name = "author_id")
)
private List<Author> authors;
}

• Author Entity (Non-Owning Side):

@Entity
public class Author {
@Id
private Long id;
private String name;

346
 @ManyToMany(mappedBy = "authors") // Non-owning side
private List<Book> books;
}

• Relationship Dynamics:
– A Book can be associated with multiple Author instances.
– An Author can be associated with multiple Book instances.
– The relationship is managed via a join table (book_author) with columns book_id and author_id.
– The Book entity owns the relationship.

8.3.3 Combined Example: Book, Author, and Category

A real-world scenario combining all three relationship types:

[Link] Entity Relationships:

1. Book → Category: Many-to-One (each book belongs to one category).
2. Book → Author: Many-to-Many (each book can have multiple authors, and each author can write multiple
books).
3. Category → Book: One-to-Many (each category contains multiple books).

[Link] Implementation:
• Book Entity:
@Entity
public class Book {
@Id
private Long id;
private String title;

@ManyToOne
@JoinColumn(name = "category_id") // Many-to-One with Category
private Category category;

@ManyToMany
@JoinTable( // Many-to-Many with Author
name = "book_author",
joinColumns = @JoinColumn(name = "book_id"),
inverseJoinColumns = @JoinColumn(name = "author_id")
)
private List<Author> authors;
}

• Category Entity:
@Entity
public class Category {
@Id
private Long id;
private String name;

347
 @OneToMany(mappedBy = "category") // One-to-Many with Book
private List<Book> books;
}

• Author Entity:
@Entity
public class Author {
@Id
private Long id;
private String name;

@ManyToMany(mappedBy = "authors") // Many-to-Many with Book (non-owning)

private List<Book> books;
}

[Link] Relationship Dynamics:

• Book-Category:
– Each Book is linked to one Category (@ManyToOne).
– Each Category can have many Book instances (@OneToMany).
• Book-Author:
– Each Book can have multiple Author instances (@ManyToMany).
– Each Author can be associated with multiple Book instances (@ManyToMany).
– The join table book_author manages the many-to-many relationship.

8.3.4 Key JPA Annotations for Relationships

Annotation Purpose Example Usage

@OneToOne Maps a one-to-one relationship. @OneToOne(mappedBy =
"author")
@OneToMany Maps a one-to-many relationship (on the “one” side). @OneToMany(mappedBy =
"category")
@ManyToOne Maps a many-to-one relationship (on the “many” side). @ManyToOne
@JoinColumn(name =
"category_id")
@ManyToMany Maps a many-to-many relationship. @ManyToMany
@JoinTable(...)
@JoinColumn Specifies the foreign key column for the relationship. @JoinColumn(name =
"author_id")
@JoinTable Defines the join table for many-to-many relationships. @JoinTable(name =
"book_author", ...)
mappedBy Indicates the non-owning side of a bidirectional @OneToMany(mappedBy =
relationship. "category")

8.3.5 Summary of Key Concepts

1. One-to-One:
• Use @OneToOne with mappedBy (non-owning) or @JoinColumn (owning).

348
 • Example: Author ↔ Biography.
2. One-to-Many / Many-to-One:
• Use @OneToMany (non-owning) and @ManyToOne (owning) with @JoinColumn.
• Example: Category ↔ Book.
3. Many-to-Many:
• Use @ManyToMany with @JoinTable (owning) and mappedBy (non-owning).
• Example: Book ↔ Author.
4. Bidirectional vs. Unidirectional:
• Bidirectional: Both entities reference each other (requires mappedBy).
• Unidirectional: Only one entity references the other (no mappedBy).
5. Ownership:
• The owning side holds the foreign key (@JoinColumn or @JoinTable).
• The non-owning side uses mappedBy to delegate ownership.

8.3.6 Practical Applications

• Database Schema Design: JPA annotations automatically generate the appropriate tables and foreign keys.
• Querying: Relationships enable efficient queries (e.g., fetch all books by a category or all authors of a book).
• Data Integrity: Ensures referential integrity (e.g., a book cannot exist without a category if the relationship
is mandatory).

8.3.7 Conclusion
• Mastering JPA relationship annotations is essential for designing efficient, scalable database applications.
• The three relationship types (one-to-one, one-to-many/many-to-one, many-to-many) cover most
real-world use cases.
• Proper use of mappedBy, @JoinColumn, and @JoinTable ensures correct mapping and avoids redundancy.

8.4 Implementing Custom Query Methods

8.4.1 1. Introduction to JPQL and Native SQL Queries
[Link] 1.1 Overview
• Two key components of data access in Java applications:
– JPQL (Java Persistence Query Language)
– Native SQL Queries
• Objective: Understand both approaches, create custom query methods in repositories, and explore common
query examples.

8.4.2 2. Java Persistence Query Language (JPQL)

[Link] 2.1 Definition and Purpose
• JPQL is a query language designed for JPA (Java Persistence API).
• Unlike SQL, which operates on database tables, JPQL operates on entity objects.
• Aligns with object-oriented domain models, leading to more intuitive and maintainable code.

[Link] 2.2 JPQL Syntax Overview

• Similar to SQL but entity-focused rather than table-focused.
• Key clauses:
– SELECT: Specifies entities or fields to retrieve.

349
 – FROM: Defines the entity(ies) to query.
– WHERE: Filters results based on conditions.
– ORDER BY: Orders results as needed.

[Link] 2.3 Basic JPQL Query Examples

1. Select All Instances of an Entity
SELECT b FROM Book b

• Retrieves all Book entities.

2. Query with Conditions
SELECT b FROM Book b WHERE [Link] = 'Effective Java'

• Finds all books matching a specific title.

[Link] 2.4 Advanced JPQL Features

[Link].1 2.4.1 Joining Entities

• Allows querying related entities.
• Example: Find books by a specific publisher.
SELECT b FROM Book b JOIN [Link] p WHERE [Link] = 'OReilly'

[Link].2 2.4.2 Subqueries

• Supports nested queries.
• Example: Find books with a price higher than the average.
SELECT b FROM Book b WHERE [Link] > (SELECT AVG([Link]) FROM Book b2)

[Link].3 2.4.3 Aggregate Functions

• Supports calculations on data:
– COUNT, SUM, AVG, MIN, MAX.
• Example: Count all books in a category.
SELECT COUNT(b) FROM Book b WHERE [Link] = 'Programming'

[Link].4 2.4.4 Parameter Binding

• Positional Parameters (e.g., ?1, ?2).
• Named Parameters (e.g., :title).
SELECT b FROM Book b WHERE [Link] = :title

– Advantages of Named Parameters:

* Improves readability.
* Prevents SQL injection.

350
[Link].5 2.4.5 Join Types in JPQL
1. INNER JOIN
• Fetches records with matching values in both entities.
SELECT b FROM Book b INNER JOIN [Link] p WHERE [Link] = 'OReilly'
2. LEFT JOIN
• Fetches all records from the left entity and matched records from the right.
SELECT b FROM Book b LEFT JOIN [Link] p
3. FETCH JOIN
• Eagerly fetches related entities in a single query (avoids lazy loading issues).
SELECT b FROM Book b JOIN FETCH [Link]

[Link] 2.5 Advantages of JPQL

1. Database Independence
• Queries are written in terms of entities, making them portable across different databases.
2. Object-Oriented Approach
• Operates on the object model, aligning with the domain.
3. Simplified Data Access
• Abstracts the complexity of SQL.
4. Improved Readability and Maintenance
• Uses entity names instead of database-specific syntax.

8.4.3 3. Native SQL Queries

[Link] 3.1 Definition and Use Cases
• Executes direct SQL statements against the database.
• Useful for:
– Leveraging database-specific features.
– Performance optimizations.
• Drawback: Less portable and tightly coupled to the database schema.

SELECT * FROM books WHERE title = 'Effective Java'

[Link] 3.2 Example of a Native SQL Query

• Key Considerations:
– Must manage parameters and SQL syntax directly.
– Provides control but requires careful handling to avoid errors and security risks.

8.4.4 4. Creating Custom Query Methods in Repositories

[Link] 4.1 Using @Query Annotation
• Spring Data JPA allows defining custom query methods using the @Query annotation.
• Supports both JPQL and Native SQL.

351
@Query("SELECT b FROM Book b WHERE [Link] = :category")
List<Book> findBooksByCategory(@Param("category") String category);

[Link].1 4.1.1 Example: JPQL Custom Method

@Query(value = "SELECT * FROM books WHERE author_name = ?1", nativeQuery = true)

List<Book> findBooksByAuthorName(String authorName);

[Link].2 4.1.2 Example: Native SQL Custom Method

[Link] 4.2 Benefits of Custom Query Methods

• Tailored queries to meet application-specific needs.
• Flexibility in choosing between JPQL and Native SQL.

8.4.5 5. Common Query Methods in Repositories

[Link] 5.1 Basic CRUD Operations
1. findAll()
• Lists all records in the entity.
2. findById()
• Retrieves a specific record by ID.

[Link] 5.2 Conditional Queries

1. Find by Title (Substring Match)
List<Book> findByTitleContaining(String substring);

2. Find Books Published After a Date

List<Book> findByPublicationDateAfter(LocalDate date);

3. Find Books by Category Name

List<Book> findByCategoryName(String categoryName);

[Link] 5.3 Derived Query Methods

• Spring Data JPA can auto-generate queries based on method names.
• Example:
List<Book> findByAuthorAndPriceLessThan(String author, double price);

– Equivalent to:
SELECT b FROM Book b WHERE [Link] = ?1 AND [Link] < ?2

8.4.6 6. Comparison: JPQL vs. Native SQL

352
Feature JPQL Native SQL
Approach Object-oriented Database-specific
Portability High (works across databases) Low (tied to schema)
Complexity Simplified (entity-based) Direct control (SQL-based)
Use Case General queries Database optimizations, complex
queries
Security Supports parameter binding Requires manual parameter handling

[Link] 6.1 When to Use Each

• Use JPQL when:
– You need portability.
– Queries align with the object model.
– You want simplified maintenance.
• Use Native SQL when:
– You need database-specific optimizations.
– You require complex queries not easily expressed in JPQL.

8.4.7 7. Best Practices for Query Implementation

[Link] 7.1 Security
• Always use parameterized queries to prevent SQL injection.
– Prefer named parameters (:param) over positional (?1).

[Link] 7.2 Reusability

• Leverage named queries (defined in entities) to avoid duplication.
@NamedQuery(name = "[Link]", query = "SELECT b FROM Book b WHERE [Link] = :title")

[Link] 7.3 Performance Optimization

• Consider indexing for frequently queried fields.
• Design efficient queries (avoid SELECT *, use JOIN FETCH for eager loading).

[Link] 7.4 Testing and Documentation

• Thoroughly test queries for correctness and performance.
• Document queries to improve readability and maintainability.

8.4.8 8. Summary of Key Takeaways

1. JPQL is entity-based, portable, and aligns with object-oriented models.
2. Native SQL provides direct database control but is less portable.
3. Custom query methods in Spring Data JPA can use @Query with JPQL or Native SQL.
4. Common repository methods include findAll, findById, and derived queries.
5. Best practices include:
• Using parameterized queries.
• Optimizing performance with indexing.
• Documenting and testing queries.

353
8.5 Introduction to DAOs and Repositories
8.5.1 1. Overview of the Lecture
This lecture introduces two key concepts in database application development: - Data Access Objects (DAOs) -
Repositories
By the end of this lecture, students will: - Understand the DAO pattern and its benefits. - Learn the role of
repositories in data access. - Be introduced to Spring Data JPA repositories.

8.5.2 2. The DAO (Data Access Object) Pattern

[Link] 2.1 Definition and Purpose The Data Access Object (DAO) pattern is a design pattern that manages
interactions between an application and its data source.
• Primary Goal: Provide a clear separation between business logic and data access logic.
• Key Components:
– DAO Interface: Defines methods for interacting with the data source (e.g., retrieving, modifying data).
– DAO Implementation Class: Contains the actual logic for executing operations (e.g., SQL queries).
– Data Transfer Object (DTO): A simple data structure used to carry data between application layers.
– Data Source: The underlying database or storage system where data is stored.

[Link] 2.2 Benefits of the DAO Pattern

[Link].1 2.2.1 Separation of Concerns

• Business logic is decoupled from data access logic.
• Changes to how data is accessed (e.g., switching databases) do not affect business rules.
• Results in a more modular and maintainable system.

[Link].2 2.2.2 Flexibility

• Allows switching between different data sources (e.g., from SQL to NoSQL) with minimal code changes.
• Data access logic is centralized, making modifications easier.

[Link].3 2.2.3 Improved Testability

• Business logic can be tested independently of data access mechanisms.
• Mock DAOs can be used in unit tests to simulate database interactions without requiring a real database.

[Link].4 2.2.4 Centralized Data Access

• All data-related operations are managed in one place.
• Ensures consistency in data access and enforces data access rules.

[Link].5 2.2.5 Reduced Code Duplication

• Common operations (e.g., CRUD) are abstracted into the DAO, reducing repetitive code across the ap-
plication.

354
8.5.3 3. Repositories in Data Access
[Link] 3.1 Definition and Role A repository is an abstraction layer that simplifies interactions with a data
source.
• Acts as a high-level interface for performing data operations without exposing underlying details (e.g.,
SQL queries, connection handling).
• Encapsulates the complexities of data access, allowing business logic to focus on core functionalities.

[Link] 3.2 Key Features of Repositories

[Link].1 3.2.1 Abstraction Layer

• Provides a clean, high-level API for data operations.
• Hides implementation details (e.g., whether data is fetched from a SQL database, NoSQL, or an in-memory
store).

[Link].2 3.2.2 Encapsulation

• Shields business logic from data access complexities (e.g., connection pooling, transaction management).
• Developers interact with simple method calls rather than raw queries.

[Link].3 3.2.3 Query Methods

• Repositories often include predefined methods for common operations, such as:
– findById()
– findAll()
– save()
– delete()
– existsById()

[Link].4 3.2.4 Integration with ORM (Object-Relational Mapping)

• In frameworks like Spring Data JPA, repositories work seamlessly with ORM tools (e.g., Hibernate).
• Automatically maps Java objects to database tables and vice versa.

[Link] 3.3 Advantages of Using Repositories

• Simplifies data access by providing a consistent interface.
• Reduces boilerplate code (e.g., no need to write repetitive SQL for CRUD operations).
• Enhances maintainability by centralizing data access logic.

8.5.4 4. Spring Data JPA Repositories

[Link] 4.1 Overview Spring Data JPA is a powerful framework that extends the repository concept with
additional features to simplify database interactions.

[Link] 4.2 Key Features

355
[Link].1 4.2.1 Automatic Query Generation
• Method names follow naming conventions, and Spring Data JPA automatically generates the corre-
sponding SQL queries.
– Example:
* Method: findByTitle(String title)
* Generated Query: SELECT * FROM book WHERE title = ?
[Link].2 4.2.2 Built-in CRUD Operations
• Standard CRUD methods are provided out-of-the-box when extending JpaRepository:
– save(T entity) – Inserts or updates an entity.
– findById(ID id) – Retrieves an entity by its ID.
– findAll() – Retrieves all entities.
– delete(T entity) – Deletes an entity.
– count() – Returns the number of entities.

[Link].3 4.2.3 Custom Queries

• Complex queries can be defined using:
– @Query annotation (for JPQL or native SQL).
* Example:
@Query("SELECT b FROM Book b WHERE [Link] = :category")
List<Book> findByCategory(@Param("category") String category);
– Derived query methods (based on method naming conventions).

[Link].4 4.2.4 Pagination and Sorting

• Built-in support for handling large datasets efficiently:
– Pagination: Retrieve data in chunks (e.g., Page<Book> findAll(Pageable pageable)).
– Sorting: Order results (e.g., List<Book> findAll(Sort sort)).

[Link] 4.3 Benefits of Spring Data JPA Repositories

• Reduces boilerplate code (no need to write repetitive SQL).
• Improves productivity by providing predefined methods.
• Enhances maintainability with consistent data access patterns.
• Supports complex queries when needed.

8.5.5 5. Practical Example: Using Spring Data JPA Repositories

[Link] 5.1 Entity Example (Book)
• A Java class representing a database table (e.g., Book).
• Annotated with @Entity to indicate it is a JPA entity.
• Example:
@Entity
public class Book {
@Id
@GeneratedValue(strategy = [Link])

356
 private Long id;
private String title;
private String author;
private String category;

// Getters and Setters

}

[Link] 5.2 Repository Interface (BookRepository)

• Extends JpaRepository<Book, Long> to inherit standard CRUD methods.
• Can define custom query methods.
• Example:
public interface BookRepository extends JpaRepository<Book, Long> {
// Custom query method (automatically implemented by Spring Data JPA)
List<Book> findByCategory(String category);

// Custom query using @Query annotation

@Query("SELECT b FROM Book b WHERE [Link] = :author")
List<Book> findByAuthor(@Param("author") String author);
}

[Link] 5.3 Usage in Application

• Inject the repository into a service or controller.
• Use predefined or custom methods to interact with the database.
• Example:
@Service
public class BookService {
@Autowired
private BookRepository bookRepository;

public List<Book> getBooksByCategory(String category) {

return [Link](category);
}
}

[Link] 5.4 Key Takeaways from the Example

• By extending JpaRepository, we automatically get CRUD operations.
• Custom methods can be added without manual implementation (Spring Data JPA generates the queries).
• Focus shifts to business logic rather than low-level data access.

8.5.6 6. Summary of Key Concepts

[Link] 6.1 DAO Pattern
• Separates business logic from data access.

357
 • Benefits: Modularity, flexibility, testability, reduced duplication.
• Components: DAO Interface, DAO Implementation, DTO, Data Source.

[Link] 6.2 Repositories

• Abstraction layer for data access.
• Encapsulates complexities (e.g., SQL, connections).
• Provides predefined methods (e.g., findById, save).

[Link] 6.3 Spring Data JPA Repositories

• Automatic query generation (via method naming).
• Built-in CRUD operations.
• Supports custom queries (@Query annotation).
• Pagination and sorting for efficient data handling.

[Link] 6.4 Practical Implementation

• Define an entity (@Entity).
• Create a repository interface (extend JpaRepository).
• Use custom methods (derived or annotated queries).
• Inject and use the repository in services/controllers.

8.5.7 7. Conclusion
This lecture covered: [OK] The DAO pattern and its benefits in separating concerns. [OK] The role of reposi-
tories in simplifying data access. [OK] Spring Data JPA repositories and their key features (automatic queries,
CRUD, custom queries, pagination). [OK] A practical example demonstrating how to implement a repository
for a Book entity.
Understanding these concepts is essential for building robust, maintainable database applications in modern
Java frameworks.

8.6 Introduction to JPA and Hibernate

8.6.1 1. Overview of JPA and Hibernate
[Link] 1.1 Introduction
• JPA (Java Persistence API) and Hibernate are critical technologies in Java for managing relational data in
database-driven applications.
• These tools simplify database interactions by allowing developers to work with Java objects instead of writing
raw SQL.

[Link] 1.2 Importance

• Essential for building database applications in Java, particularly in Spring-based projects.
• Enable efficient data management by abstracting low-level database operations.
• Facilitate clean separation of concerns in application architecture.

358
8.6.2 2. What is JPA?
[Link] 2.1 Definition
• JPA (Java Persistence API) is a specification (standard interface) for managing relational data in Java
applications.
• Provides a framework for mapping Java objects to database tables (Object-Relational Mapping, ORM).
• Allows database operations (CRUD: Create, Read, Update, Delete) to be performed via Java code rather
than direct SQL.

[Link] 2.2 Key Features of JPA

1. Object-Relational Mapping (ORM)
• Maps Java classes (entities) to database tables.
• Eliminates the need for manual SQL queries in most cases.
2. JPQL (Java Persistence Query Language)
• A query language similar to SQL but operates on entity objects instead of tables.
• Example:
@Query("SELECT b FROM Book b WHERE [Link] = :title")
Book findBookByTitle(@Param("title") String title);
3. Entity Management
• Manages the lifecycle of entities (e.g., persisting, updating, removing).
• Uses an EntityManager to interact with the persistence context.

8.6.3 3. What is Hibernate?

[Link] 3.1 Definition
• Hibernate is a framework that implements the JPA specification.
• Acts as an ORM (Object-Relational Mapping) tool that simplifies database interactions.

[Link] 3.2 Key Features of Hibernate

1. Automatic Table Generation
• Can generate database tables automatically based on entity classes.
2. Transparent Persistence
• Handles the persistence (saving, updating, deleting) of objects without explicit SQL.
3. Caching
• Improves performance by caching frequently accessed data.
4. Lazy Loading
• Loads associated data only when needed, reducing memory usage.
5. SQL Abstraction
• Generates and executes complex SQL queries behind the scenes, allowing developers to focus on
business logic.

[Link] 3.3 Relationship Between JPA and Hibernate

• JPA is the standard (interface).
• Hibernate is one of the implementations of JPA (other implementations include EclipseLink, OpenJPA).
• Hibernate extends JPA with additional features (e.g., better caching, more query options).

359
8.6.4 4. Key Components in a Spring-Based Application
[Link] 4.1 Overview of Application Layers A typical Spring-based database application (e.g., a Library
Management System) consists of the following layers:

Component Role
Entity Classes Represent database tables (e.g., Book, Author, Category).
Repository Interfaces Handle CRUD operations (e.g., save(), findById(), delete()).
Service Classes Contain business logic (e.g., issuing a book, updating book details).
Controller Classes Manage HTTP requests/responses (REST API or MVC endpoints).

[Link] 4.2 Detailed Breakdown of Components

[Link].1 4.2.1 Entity Classes

• Purpose: Model database tables as Java objects.
• Example:
@Entity
public class Book {
@Id
@GeneratedValue(strategy = [Link])
private Long id;
private String title;
private String author;
// Getters & Setters
}

• Annotations Used:
– @Entity – Marks the class as a JPA entity.
– @Id – Specifies the primary key.
– @GeneratedValue – Auto-generates the primary key.

[Link].2 4.2.2 Repository Interfaces

• Purpose: Provide CRUD operations without writing boilerplate code.
• Example:
public interface BookRepository extends JpaRepository<Book, Long> {
Book findByTitle(String title); // Custom query method
}

• Key Methods:
– save() – Inserts/updates an entity.
– findById() – Retrieves an entity by ID.
– delete() – Removes an entity.

360
[Link].3 4.2.3 Service Classes
• Purpose: Contain business logic (e.g., validation, transactions).
• Example:
@Service
public class BookService {
@Autowired
private BookRepository bookRepository;

@Transactional
public Book addBook(Book book) {
return [Link](book);
}
}

• Key Responsibilities:
– Issuing a book (updating availability status).
– Deleting a book (with validation checks).
– Updating book details (e.g., author, edition).

[Link].4 4.2.4 Controller Classes

• Purpose: Handle HTTP requests and delegate to the service layer.
• Example:
@RestController
@RequestMapping("/books")
public class BookController {
@Autowired
private BookService bookService;

@PostMapping
public Book createBook(@RequestBody Book book) {
return [Link](book);
}
}

• Key Responsibilities:
– Receiving requests (GET, POST, PUT, DELETE).
– Returning responses (JSON, HTML, etc.).
– Delegating to the service layer for business logic.

8.6.5 5. Interaction Between Components

[Link] 5.1 Workflow of a Typical Request
1. Controller receives an HTTP request (e.g., POST /books).
2. Controller calls the Service layer (e.g., [Link](book)).
3. Service layer applies business logic (e.g., validation, transactions).
4. Service layer uses the Repository to interact with the database (e.g., [Link](book)).

361
 5. Repository performs CRUD operations on the Entity.
6. Response is sent back to the client.

[Link] 5.2 Separation of Concerns

• Controller → Handles HTTP communication.
• Service → Handles business logic.
• Repository → Handles database operations.
• Entity → Represents data structure.
This separation ensures modularity, maintainability, and testability.

8.6.6 6. Core ORM Concepts

[Link] 6.1 Key Terminologies

Concept Description
Entity A Java class mapped to a database table (annotated with @Entity).
Primary Key A unique identifier for an entity (annotated with @Id).
Generated Value Auto-incremented primary key (e.g., @GeneratedValue).
Relationships Define associations between entities (e.g., @OneToMany, @ManyToOne).
JPQL Query language for entities (similar to SQL but object-oriented).
Transactions Ensure data integrity by grouping operations into a single unit.
EntityManager Manages the lifecycle of entities (persist, merge, remove, find).

[Link] 6.2 JPQL (Java Persistence Query Language)

• Purpose: Query entities (not tables) using a SQL-like syntax.
• Example:
@Query("SELECT b FROM Book b WHERE [Link] = :title")
Book findBookByTitle(@Param("title") String title);

– @Query – Defines the JPQL query.

– @Param – Binds a method parameter to a query parameter.
– Returns a Book entity (not a database row).

[Link] 6.3 Transactions

• Purpose: Ensure atomicity (all operations succeed or fail together).
• Example:
@Transactional
public Book saveBook(Book book) {
return [Link](book); // Executed within a transaction
}

– @Transactional – Marks the method as transactional.

– If an error occurs, changes are rolled back.

362
8.6.7 7. Practical Example: Saving a Book

@Entity
public class Book {
@Id
@GeneratedValue(strategy = [Link])
private Long id;
private String title;
private String author;
// Getters & Setters
}

[Link] 7.1 Entity Class

public interface BookRepository extends JpaRepository<Book, Long> {

Book findByTitle(String title);
}

[Link] 7.2 Repository Interface

@Service
public class BookService {
@Autowired
private BookRepository bookRepository;

@Transactional
public Book addBook(Book book) {
return [Link](book); // Saved within a transaction
}
}

[Link] 7.3 Service Class (with Transaction)

@RestController
@RequestMapping("/books")
public class BookController {
@Autowired
private BookService bookService;

@PostMapping
public Book createBook(@RequestBody Book book) {
return [Link](book);
}
}

[Link] 7.4 Controller

363
[Link] 7.5 Flow of Execution
1. Client sends a POST /books request with a Book object.
2. Controller receives the request and calls [Link](book).
3. Service (within a transaction) saves the book via [Link](book).
4. Repository persists the book to the database.
5. Response (saved Book object) is returned to the client.

8.6.8 8. Summary of Key Takeaways

1. JPA is a specification for ORM in Java; Hibernate is its most popular implementation.
2. Entity classes model database tables, while repositories handle CRUD operations.
3. Service classes contain business logic, and controllers manage HTTP requests.
4. JPQL allows querying entities (not tables) using a SQL-like syntax.
5. Transactions (@Transactional) ensure data integrity.
6. Separation of concerns (Controller → Service → Repository → Entity) improves maintainability.

8.7 Recording of Building Database Applications Week 7 - Live Session on 26-04-24

8.7.1 1. Overview of the Application Flow
[Link] 1.1 User Interaction and Request Handling
• The application follows a structured flow when a user interacts with it:
– User Action: User enters book details (e.g., title: “Java Basics,” author: “James,” category: “Program-
ming”) and submits the form.
– Controller: Acts as a “receptionist” that receives the request and delegates it to the appropriate service.
– Service Layer: Contains business logic and decides what action to take (e.g., saving a book).
– Repository: Interacts with the database, executing SQL queries without manual SQL writing.
– Database: Stores the data as a record in a table.

[Link] 1.2 Example Flow for Adding a Book

1. User Input: User submits book details via a form.
2. Controller: Receives the request via @RequestMapping (e.g., /addBook).
3. Service: Processes the request (e.g., [Link]()).
4. Repository: Executes the save operation (e.g., [Link]()).
5. Database: Stores the record in the books table.

[Link] 1.3 Example Flow for Viewing Books

1. User Request: User opens the homepage to view books.
2. Controller: Calls the service to fetch data.
3. Service: Calls the repository to retrieve records.
4. Repository: Fetches data from the database.
5. Controller: Sends the data to the view (JSP) for display.

8.7.2 2. Model Layer: Entity Class ([Link])

[Link] 2.1 Purpose of the Entity Class
• Defines the structure of a database table.
• Uses JPA (Java Persistence API) and Hibernate to automatically generate SQL tables and columns.

364
 • Each instance of the entity class becomes a row in the database table.

[Link] 2.2 Key Annotations in [Link]

Annotation Purpose
@Entity Marks the class as a JPA entity (will be mapped
to a database table).
@Table(name = "books") Specifies the table name in the database.
@Id Designates the primary key.
@GeneratedValue(strategy = Auto-generates the primary key.
[Link])
@Column(name = "book_title", nullable = false) Maps a field to a column with constraints (e.g.,
NOT NULL).

@Entity
@Table(name = "books")
public class Book {
@Id
@GeneratedValue(strategy = [Link])
private Long id;

@Column(name = "book_title", nullable = false)

private String title;

@Column(nullable = false)
private String author;

@Column(nullable = false)
private String category;

// Getters and Setters

}

[Link] 2.3 Example: [Link] Code Structure

• Field Mappings:
– title (Java) → book_title (SQL, NOT NULL).
– author (Java) → author (SQL, NOT NULL).
– category (Java) → category (SQL, NOT NULL).

[Link] 2.4 Resulting Database Table Structure

Column Type Constraints

id BIGINT PRIMARY KEY, AUTO_INCREMENT
book_title VARCHAR(255) NOT NULL
author VARCHAR(255) NOT NULL
category VARCHAR(255) NOT NULL

365
8.7.3 3. Repository Layer
[Link] 3.1 Role of the Repository
• Acts as an intermediary between the service layer and the database.
• Extends JpaRepository to inherit CRUD methods (e.g., save(), findAll(), deleteById()).
• Avoids manual SQL queries by using Spring Data JPA methods.

public interface BookRepository extends JpaRepository<Book, Long> {

// Inherits methods like save(), findAll(), findById(), deleteById()
}

[Link] 3.2 Example: [Link]

• Generic Parameters:
– Book: The entity class.
– Long: The type of the primary key (id).

[Link] 3.3 Key Repository Methods

Method Description
save(Book book) Inserts or updates a book record.
findAll() Retrieves all book records.
findById(Long id) Retrieves a book by its ID.
deleteById(Long id) Deletes a book by its ID.

8.7.4 4. Service Layer

[Link] 4.1 Role of the Service Layer
• Contains business logic (e.g., validation, data processing).
• Decouples the controller from the repository (controller should not directly access the database).
• Calls repository methods to perform database operations.

@Service
public class BookService {
@Autowired
private BookRepository bookRepository;

public Book saveBook(Book book) {

return [Link](book);
}

public List<Book> getAllBooks() {

return [Link]();
}

public void deleteBook(Long id) {

[Link](id);

366
 }
}

[Link] 4.2 Example: [Link]

• Dependencies:
– @Autowired: Injects the BookRepository instance.
– @Service: Marks the class as a Spring service component.

[Link] 4.3 Key Service Methods

Method Description
saveBook(Book book) Delegates to [Link]().
getAllBooks() Delegates to [Link]().
deleteBook(Long id) Delegates to [Link]().

8.7.5 5. Controller Layer

[Link] 5.1 Role of the Controller
• Handles HTTP requests (e.g., GET, POST).
• Delegates to the service layer for business logic.
• Returns responses (e.g., renders a JSP view or returns JSON).

@Controller
public class BookController {
@Autowired
private BookService bookService;

@GetMapping("/")
public String home(Model model) {
[Link]("books", [Link]());
return "home";
}

@PostMapping("/addBook")
public String addBook(@ModelAttribute Book book) {
[Link](book);
return "redirect:/";
}
}

[Link] 5.2 Example: [Link]

• Annotations:
– @Controller: Marks the class as a Spring MVC controller.
– @GetMapping("/"): Handles GET requests to the root URL.
– @PostMapping("/addBook"): Handles POST requests to /addBook.
– @ModelAttribute: Binds form data to a Book object.

367
[Link] 5.3 Controller Workflow
1. GET /:
• Fetches all books via [Link]().
• Adds books to the model and renders [Link].
2. POST /addBook:
• Receives a Book object from the form.
• Saves the book via [Link]().
• Redirects to the homepage.

8.7.6 6. View Layer (JSP)

[Link] 6.1 Role of the JSP Page
• Displays data to the user.
• Collects input via HTML forms.
• Dynamically renders data using JSTL (JSP Standard Tag Library).

<form action="/addBook" method="post">

<input type="text" name="title" placeholder="Title" required>
<input type="text" name="author" placeholder="Author" required>
<input type="text" name="category" placeholder="Category" required>
<button type="submit">Add Book</button>
</form>

<table>
<tr>
<th>ID</th>
<th>Title</th>
<th>Author</th>
<th>Category</th>
</tr>
<c:forEach var="book" items="${books}">
<tr>
<td>${[Link]}</td>
<td>${[Link]}</td>
<td>${[Link]}</td>
<td>${[Link]}</td>
</tr>
</c:forEach>
</table>

[Link] **6.2 Example: [Link]

• Form Submission:
– Sends data to /addBook via POST.
– Fields (title, author, category) map to Book object properties.
• Data Display:
– Uses <c:forEach> to loop through ${books} (passed from the controller).
– Renders each book as a table row.

368
8.7.7 7. Application Execution and Testing
[Link] 7.1 Running the Application
1. Start the Spring Boot App:
• Run mvn spring-boot:run (Maven) or via IDE.
• Tomcat starts on port 8080.
2. Access the Homepage:
• Open [Link] in a browser.
3. Add a Book:
• Enter details in the form and submit.
• Verify the book appears in the table.

[Link] 7.2 Database Verification

1. Check MySQL Workbench:
• Run SELECT * FROM books; to confirm records are inserted.
• Example output:
+----+------------+--------+-----------+
| id | book_title | author | category |
+----+------------+--------+-----------+
| 1 | Java | James | Programming|
| 2 | C++ | Bala | Programming|
+----+------------+--------+-----------+
2. Delete a Record:
• Run DELETE FROM books WHERE id = 1;.
• Refresh the browser to confirm the record is removed.

8.7.8 8. Summary of Architecture Layers

Layer Responsibility Key Components

User Interacts with the application. Browser, Forms
Controller Handles HTTP requests/delegates to @Controller, @GetMapping,
service. @PostMapping
Service Contains business logic. @Service, BookService
Repository Interacts with the database. JpaRepository, BookRepository
Database Stores data. MySQL, Tables (books)
View Displays data/collects input. JSP, JSTL

8.7.9 9. Key Takeaways

1. Separation of Concerns:
• Each layer has a distinct role (controller, service, repository, view).
2. JPA/Hibernate:
• Automates table/column creation from entity classes.
• Eliminates manual SQL for CRUD operations.
3. Spring Data JPA:
• Provides built-in repository methods (e.g., save(), findAll()).
4. Form Handling:
• @ModelAttribute binds form data to Java objects.

369
 5. Dynamic Views:
• JSP + JSTL render data dynamically.

8.8 Service Layer Introduction and Implementation

8.8.1 1. Introduction to the Service Layer
[Link] 1.1 Role of the Service Layer
• The service layer is a critical component in a Spring Boot application, acting as an intermediary between
the controller layer and the data access layer (repository layer).
• It manages the flow of data and encapsulates business logic, ensuring a clean separation of concerns.
• A well-structured service layer enhances modularity and maintainability of the application.

[Link] 1.2 Key Functions of the Service Layer The service layer performs several essential functions:
1. Encapsulates Business Logic
• Ensures that business rules are applied consistently across the application.
• Example: Enforcing validation rules before data is saved or modified.
2. Manages Transactions
• Provides data integrity and consistency by controlling transaction boundaries.
• Ensures that operations are atomic (either fully completed or rolled back in case of failure).
3. Acts as an Integration Point
• Coordinates interactions between different components (e.g., repositories, external services).
• Example: Combining data from multiple repositories before returning a response.
4. Enhances Separation of Concerns
• Keeps the controller layer focused on handling HTTP requests/responses.
• Moves business logic out of controllers and repositories, improving code organization.

8.8.2 2. Service Layer Design Principles

[Link] 2.1 Service Interfaces vs. Implementations
• Service Interfaces
– Define the contract (methods) that the service will provide.
– Specify what operations can be performed without exposing implementation details.
– Example:
public interface BookService {
Book saveBook(Book book);
List<Book> getAllBooks();
Book getBookById(Long id);
Book updateBook(Long id, Book book);
void deleteBook(Long id);
boolean borrowBook(Long bookId, Long memberId);
}
• Service Implementations
– Contain the actual business logic.
– Interact with the data access layer (repositories) to perform operations.
– Example:
@Service
public class BookServiceImpl implements BookService {

370
 // Business logic implementation
}

[Link] 2.2 Benefits of This Design

• Loose Coupling: Controllers depend on interfaces, not concrete implementations.
• Easier Testing: Mock implementations can be used for unit testing.
• Flexibility: Different implementations can be swapped without changing the controller.

8.8.3 3. Key Annotations in the Service Layer

Spring Boot uses annotations to configure and manage service layer components.

Annotation Purpose Example Usage

@Service Marks a class as a service provider, indicating it contains @Service public class
business logic. BookServiceImpl
@TransactionalManages transactions at the method or class level, ensuring @Transactional public
atomicity and consistency. void borrowBook()
@Autowired Enables dependency injection, allowing Spring to inject @Autowired private
collaborating beans (e.g., repositories). BookRepository repo;

[Link] 3.1 @Service Annotation

• Indicates that the class is a Spring-managed service.
• Used for auto-detection during component scanning.
• Example:
@Service
public class BookServiceImpl implements BookService {
// Business logic here
}

[Link] 3.2 @Transactional Annotation

• Ensures that a group of operations either all succeed or all fail (atomicity).
• Can be applied at the class level (all methods are transactional) or method level (specific methods).
• Example:
@Transactional
public boolean borrowBook(Long bookId, Long memberId) {
// Business logic with transaction management
}

[Link] 3.3 @Autowired Annotation

• Used for dependency injection (injecting repositories or other services).
• Eliminates the need for manual instantiation.

371
 • Example:
@Service
public class BookServiceImpl {
@Autowired
private BookRepository bookRepository;
}

8.8.4 4. Real-World Scenario: Library System

[Link] 4.1 Business Rules for Book Borrowing Consider a library management system where members can
borrow books. The following business rules must be enforced:
1. A member can borrow up to 5 books at a time.
2. The book must be available (not already borrowed).
3. The member must have a valid membership.

[Link] 4.2 Implementing Business Logic in the Service Layer

• The controller should not handle these rules—it should delegate to the service layer.
• The service layer validates the rules before allowing a book to be borrowed.

@Transactional
public boolean borrowBook(Long bookId, Long memberId) {
// 1. Check if member exists and has a valid membership
Member member = [Link](memberId)
.orElseThrow(() -> new RuntimeException("Member not found"));

if (![Link]()) {
throw new RuntimeException("Invalid membership");
}

// 2. Check if member has already borrowed max books (5)

long borrowedCount = [Link](memberId);
if (borrowedCount >= 5) {
throw new RuntimeException("Maximum borrow limit reached");
}

// 3. Check if book is available

Book book = [Link](bookId)
.orElseThrow(() -> new RuntimeException("Book not found"));

if ([Link]()) {
throw new RuntimeException("Book already borrowed");
}

// 4. Update book status and assign to member

[Link](true);
[Link](member);
[Link](book);

372
 return true;
}

[Link].1 Example: borrowBook Method

[Link] 4.3 Benefits of This Approach

• Clean Controller: The controller only handles HTTP requests and delegates logic to the service.
• Centralized Business Logic: All rules are enforced in one place (service layer).
• Reusability: The same service methods can be used by multiple controllers or other services.

8.8.5 5. Steps to Implement the Service Layer

[Link] 5.1 Step 1: Define the Service Interface
• Declares what operations the service provides.
• Example:
public interface BookService {
Book saveBook(Book book);
List<Book> getAllBooks();
Book getBookById(Long id);
Book updateBook(Long id, Book book);
void deleteBook(Long id);
boolean borrowBook(Long bookId, Long memberId);
}

[Link] 5.2 Step 2: Implement the Service Interface

• Contains the actual business logic.
• Uses @Service to mark it as a Spring component.
• Example:
@Service
public class BookServiceImpl implements BookService {

@Autowired
private BookRepository bookRepository;

@Override
public Book saveBook(Book book) {
// Business logic (e.g., set default values)
if ([Link]() == null) {
[Link]([Link]().getValue());
}
return [Link](book);
}

@Override
@Transactional

373
 public boolean borrowBook(Long bookId, Long memberId) {
// Business logic for borrowing (as shown earlier)
}
}

[Link] 5.3 Step 3: Integrate with the Repository

• The service layer delegates data access to the repository.
• Uses @Autowired to inject the repository.
• Example:
@Autowired
private BookRepository bookRepository;

[Link] 5.4 Step 4: Handle Transactions and Exceptions

• Use @Transactional to ensure data consistency.
• Handle exceptions (e.g., RuntimeException for invalid operations).
• Example:
@Transactional
public void deleteBook(Long id) {
if () {
throw new RuntimeException("Book not found");
}
[Link](id);
}

8.8.6 6. Example Use Case: Adding a Book to the Library

[Link] 6.1 Flow of Data
1. Controller receives book details from the client (e.g., via POST /books).
2. Controller passes the data to the service layer.
3. Service layer:
• Validates input (e.g., checks for null fields).
• Applies business logic (e.g., sets default values).
• Delegates saving to the repository.
4. Repository persists the book to the database.
5. Service layer returns the saved book to the controller.
6. Controller returns an HTTP response (e.g., 201 Created).

[Link] 6.2 Example Code Snippets

@RestController
@RequestMapping("/books")
public class BookController {

374
 @Autowired
private BookService bookService;

@PostMapping
public ResponseEntity<Book> addBook(@RequestBody Book book) {
Book savedBook = [Link](book);
return new ResponseEntity<>(savedBook, [Link]);
}
}

[Link].1 Controller Layer

@Service
public class BookServiceImpl implements BookService {

@Autowired
private BookRepository bookRepository;

@Override
public Book saveBook(Book book) {
// Business logic: Set default publication year if not provided
if ([Link]() == null) {
[Link]([Link]().getValue());
}
return [Link](book);
}
}

[Link].2 Service Layer

public interface BookRepository extends JpaRepository<Book, Long> {

// Spring Data JPA provides default CRUD methods
}

[Link].3 Repository Layer

[Link] 6.3 Key Takeaways from the Example

• Separation of Concerns:
– Controller: Handles HTTP requests/responses.
– Service: Handles business logic.
– Repository: Handles database operations.
• Reusability: The saveBook method can be used by other controllers or services.
• Maintainability: Business logic is centralized in the service layer.

8.8.7 7. Summary of Key Concepts

[Link] 7.1 What is the Service Layer?

375
 • A middle layer between controllers and repositories.
• Encapsulates business logic and manages transactions.

[Link] 7.2 Why Use a Service Layer?

Benefit Description
Separation of Concerns Keeps controllers thin and focused on HTTP handling.
Centralized Logic Business rules are enforced in one place.
Transaction Management Ensures data integrity with @Transactional.
Testability Easier to mock and test business logic in isolation.
Reusability Service methods can be reused across different parts of the application.

[Link] 7.3 Key Annotations

Annotation Purpose
@Service Marks a class as a Spring service component.
@Transactional Ensures method execution is atomic (all-or-nothing).
@Autowired Injects dependencies (e.g., repositories) automatically.

[Link] 7.4 Best Practices

1. Keep Controllers Thin: Delegates business logic to the service layer.
2. Use Interfaces: Define service contracts via interfaces for flexibility.
3. Handle Exceptions: Validate inputs and throw meaningful exceptions.
4. Leverage @Transactional: Ensure data consistency in critical operations.
5. Avoid Business Logic in Repositories: Repositories should only handle data access.

8.9 Testing Entity Classes with a Simple Main Method

8.9.1 Introduction to Testing Entity Classes in Spring Boot
• Testing entity classes is a critical aspect of database application development.
• Ensures data integrity, business logic validation, consistency across operations, error detection, and
refactoring safety.

[Link] Key Reasons for Testing Entity Classes

1. Data Integrity
• Ensures data handled by the application is correctly stored and retrieved from the database.
2. Business Logic Validation
• Verifies that embedded business rules within entities function as expected.
3. Consistency Across Operations
• Maintains data consistency across Create, Read, Update, Delete (CRUD) operations.
4. Error Detection
• Identifies potential bugs or issues in entity mapping and configuration.
5. Refactoring Safety
• Provides confidence that code changes will not break existing functionality.

376
8.9.2 Methods for Testing Entity Classes in Spring Boot
Four primary approaches are available, each with distinct advantages and use cases:
1. Simple Main Method
2. @DataJpaTest Annotation
3. CommandLineRunner Interface
4. @SpringBootTest Annotation

8.9.3 1. Simple Main Method for Testing

[Link] Overview
• A direct, standalone approach that does not require Spring Boot.
• Involves setting up an EntityManager and performing CRUD operations in a basic Java application.

[Link] Advantages
• Simple to implement – No additional dependencies required.
• Quick verification – Useful for basic testing without Spring Boot overhead.

[Link] Disadvantages
• Limited scope – Does not integrate with Spring Boot features.
• Manual verification required – Results must be checked directly in the database.

import [Link];
import [Link];
import [Link];

public class MainTest {

public static void main(String[] args) {
// Create EntityManagerFactory
EntityManagerFactory emf = [Link]("pu-name");
EntityManager em = [Link]();

// Begin transaction
[Link]().begin();

// Perform CRUD operations (e.g., insert a new record)

Book book = new Book();
[Link]("Effective Java");
[Link]("Joshua Bloch");
[Link](book);

// Commit transaction
[Link]().commit();

// Verify by querying the database

Book foundBook = [Link]([Link], [Link]());
[Link]("Retrieved Book: " + [Link]());

377
 // Close resources
[Link]();
[Link]();
}
}

[Link] Example Code Snippet

[Link] Key Observations

• Manual database checks are required to confirm operations (e.g., inserting, updating, deleting records).
• No Spring Boot integration – Pure JPA/Hibernate usage.
• Suitable for quick, isolated tests but not for automated or repeated testing.

8.9.4 2. @DataJpaTest Annotation

[Link] Overview
• Focuses on testing JPA components in isolation.
• Automatically configures an in-memory database (e.g., H2) and rolls back transactions after each test.

[Link] Advantages
• Isolates JPA components – Tests only the repository layer.
• Automatic transaction rollback – Ensures a clean state after each test.
• Fast execution – Uses an in-memory database by default.

[Link] Disadvantages
• Limited to JPA components – Does not test service layers or controllers.
• In-memory database may not reflect production behavior – Differences in SQL dialects or constraints.

import [Link];
import [Link];
import [Link];
import static [Link].*;

@DataJpaTest
public class BookRepositoryTest {

@Autowired
private BookRepository bookRepository;

@Test
public void testFindByTitle() {
// Arrange
Book book = new Book();
[Link]("Effect Java");
[Link]("Joshua Bloch");
[Link](book);

378
 // Act
Book foundBook = [Link]("Effect Java");

// Assert
assertNotNull(foundBook);
assertEquals("Joshua Bloch", [Link]());

// Update and verify

[Link]("Effective Java 3rd Edition");
[Link](foundBook);

Book updatedBook = [Link]("Effective Java 3rd Edition");

assertNotNull(updatedBook);
}
}

[Link] Example Code Snippet

[Link] Key Observations

• Uses @DataJpaTest to load only JPA-related components.
• Automatically rolls back after each test method.
• Ideal for unit testing repository methods without full Spring context.

8.9.5 3. CommandLineRunner Interface

[Link] Overview
• Executes code at application startup (e.g., for data initialization or simple tests).
• Operates within the full Spring Boot context.

[Link] Advantages
• Runs in full Spring Boot context – Useful for testing with real dependencies.
• Good for initialization logic – Can seed a database or verify setup.

[Link] Disadvantages
• Runs only once at startup – Not suitable for repeated unit testing.
• Not ideal for automated tests – Requires manual application restart for retesting.

import [Link];
import [Link];
import [Link];

@Component
public class BookRunner implements CommandLineRunner {

@Autowired
private BookRepository bookRepository;

379
 @Override
public void run(String... args) throws Exception {
// Initialize and test data
Book book = new Book();
[Link]("Clean Code");
[Link]("Robert C. Martin");
[Link](book);

// Verify insertion
Book foundBook = [Link]("Clean Code");
[Link]("Book saved: " + [Link]());
}
}

[Link] Example Code Snippet

[Link] Key Observations

• Runs when the Spring Boot application starts.
• Useful for one-time data setup or verification.
• Not a replacement for unit tests but helpful for integration checks.

8.9.6 4. @SpringBootTest Annotation

[Link] Overview
• Provides comprehensive testing within the full Spring Boot application context.
• Loads the entire application stack (controllers, services, repositories).

[Link] Advantages
• Tests the full application stack – Validates interactions between layers.
• Closest to production environment – Uses real database configurations.

[Link] Disadvantages
• Slower execution – Full context loading takes time.
• Complex setup – Requires proper test configuration (e.g., test profiles).

import [Link];
import [Link];
import [Link];
import static [Link].*;

@SpringBootTest
public class BookIntegrationTest {

@Autowired
private BookRepository bookRepository;

380
 @Test
public void testBookCRUDOperations() {
// Create
Book book = new Book();
[Link]("Design Patterns");
[Link]("Erich Gamma");
[Link](book);

// Read
Book foundBook = [Link]([Link]()).orElse(null);
assertNotNull(foundBook);
assertEquals("Design Patterns", [Link]());

// Update
[Link]("Design Patterns: Elements of Reusable OO Software");
[Link](foundBook);

Book updatedBook = [Link]([Link]()).orElse(null);

assertEquals("Design Patterns: Elements of Reusable OO Software", [Link]());

// Delete
[Link](updatedBook);
assertFalse([Link]([Link]()).isPresent());
}
}

[Link] Example Code Snippet

[Link] Key Observations

• Uses @SpringBootTest to load the full application context.
• Suitable for end-to-end testing (e.g., API + service + repository interactions).
• Best for integration tests but may be overkill for simple repository tests.

8.9.7 Comparison of Testing Methods

Method Scope Pros Cons Best For

Simple Main Standalone JPA Simple, no dependencies Manual verification, Quick, isolated
Method limited scope CRUD tests
@DataJpaTest JPA Repository Fast, isolated, In-memory DB Unit testing
Layer auto-rollback differences repositories
CommandLineRunner
Full Spring Boot Full context, good for Runs once, not for Data setup, one-time
(Startup) initialization repeated tests verification
@SpringBootTest
Full Application Comprehensive, real DB Slow, complex setup End-to-end
Stack testing integration tests

8.9.8 Conclusion
• Simple Main Method is best for quick, manual CRUD testing without Spring Boot.

381
 • @DataJpaTest is ideal for isolated repository unit tests with automatic rollback.
• CommandLineRunner is useful for startup initialization and one-time checks.
• @SpringBootTest provides full-stack testing but is slower and more complex.

[Link] Choosing the Right Method

• For unit testing repositories → Use @DataJpaTest.
• For quick manual checks → Use a Simple Main Method.
• For startup data initialization → Use CommandLineRunner.
• For end-to-end integration tests → Use @SpringBootTest.
By selecting the appropriate method based on testing needs, developers can ensure data integrity, business logic
validation, and system consistency in database applications.

8.10 Using Annotations for Entity Configuration

8.10.1 Introduction to JPA Annotations
• Purpose of JPA Annotations:
– Define how Java objects map to database tables.
– Provide metadata to the JPA (Java Persistence API) provider.
– Simplify code by eliminating XML configurations.
– Enhance readability and maintainability.
• Key Benefits:
– Reduces boilerplate configuration.
– Centralizes mapping logic within the entity class.
– Supports flexible and declarative database schema definitions.

8.10.2 Core JPA Annotations

[Link] 1. @Entity Annotation
• Definition:
– Marks a class as a JPA entity, indicating it should be mapped to a database table.
• Usage:
– Applied at the class level.
• Example:
@Entity
public class User {
// Fields, constructors, methods
}

• Effect:
– JPA recognizes User as an entity and manages its persistence.

[Link] 2. @Table Annotation

• Definition:
– Specifies the database table details for an entity.

382
 – Used when the table name differs from the class name or when additional table properties are required.
• Key Attributes:
– name: Specifies the table name.
– schema: Defines the database schema (optional).
– catalog: Defines the database catalog (optional).
– uniqueConstraints: Specifies unique constraints (optional).

• Example:
@Entity
@Table(name = "employee_records")
public class Employee {
// Fields, constructors, methods
}

• Use Case:
– Mapping an entity to a table with a non-default name (e.g., employee_records instead of Employee).

[Link] 3. @Id and @GeneratedValue Annotations

• @Id:
– Definition: Marks a field as the primary key of the entity.
– Usage: Applied to a single field per entity.
– Example:
@Id
private Long productId;
• @GeneratedValue:
– Definition: Specifies how the primary key value is generated.
– Generation Strategies:
1. AUTO (Default):
* JPA provider selects the generation strategy (e.g., sequence, identity, or table).
* Example:
@GeneratedValue(strategy = [Link])
2. IDENTITY:
* Uses the database’s auto-increment (identity) column.
* Example:
@GeneratedValue(strategy = [Link])
3. SEQUENCE:
* Uses a database sequence to generate values.
* Requires @SequenceGenerator for customization.
* Example:
@GeneratedValue(strategy = [Link], generator = "product_seq")
@SequenceGenerator(name = "product_seq", sequenceName = "seq_product", allocationSize
4. TABLE:
* Uses a database table to simulate sequences.
* Requires @TableGenerator for customization.
* Example:

383
 @GeneratedValue(strategy = [Link], generator = "product_gen")
@TableGenerator(name = "product_gen", table = "id_gen", pkColumnName = "gen_name", va
– Example with @Id:
@Id
@GeneratedValue(strategy = [Link])
private Long productId;

[Link] 4. @Column Annotation

• Definition:
– Customizes the mapping of an entity field to a database column.
– Allows specification of column name, constraints, and other properties.
• Key Attributes:
– name: Column name in the database.
– nullable: Whether the column allows NULL values (default: true).
– unique: Whether the column must have unique values (default: false).
– length: Maximum length for string-based columns.
– precision and scale: For decimal/numeric columns.

• Example:
@Column(name = "full_name", nullable = false, unique = true, length = 100)
private String name;

@Column(name = "email_address", nullable = false)

private String email;

• Use Case:
– Overriding default column names (e.g., full_name instead of name).
– Enforcing constraints (e.g., NOT NULL, UNIQUE).

[Link] 5. @Temporal Annotation

• Definition:
– Specifies the temporal precision for [Link] and [Link] fields.
– Ensures correct mapping to SQL date/time types.
• Temporal Types:
1. DATE: Stores only the date (year, month, day).
2. TIME: Stores only the time (hour, minute, second).
3. TIMESTAMP: Stores both date and time.
• Why Use @Temporal?:
– Without @Temporal, JPA defaults to TIMESTAMP, which may store unnecessary data (e.g., time when
only date is needed).
– Improves storage efficiency by avoiding redundant data.
• Example:

384
 @Temporal([Link])
private Date eventDate;

@Temporal([Link])
private Date eventTime;

• Modern Alternative:
– Java 8’s [Link] API (e.g., LocalDate, LocalDateTime) does not require @Temporal and offers
better precision.

[Link] 6. @Lob Annotation

• Definition:
– Maps a field to a Large Object (LOB) column in the database.
– Used for storing large text (CLOB) or binary data (BLOB).
• Use Cases:
– Storing long text (e.g., articles, descriptions).
– Storing binary data (e.g., images, files).
• Example:
@Lob
private String content; // Maps to CLOB (Character Large Object)

@Lob
private byte[] image; // Maps to BLOB (Binary Large Object)

[Link] 7. @Transient Annotation

• Definition:
– Marks a field as non-persistent (i.e., not stored in the database).
– Used for fields relevant only to application logic.
• Use Case:
– Derived or calculated fields (e.g., age computed from birthDate).
• Example:
@Transient
private int age; // Not persisted to the database

• Effect:
– JPA ignores the field during persistence operations.

8.10.3 Modern Date/Time Handling (Java 8+)

• Java 8 [Link] API:
– Replaces [Link] and [Link].
– Provides more intuitive and precise date/time classes:

385
 * LocalDate: Date without time (e.g., 2023-10-05).
* LocalTime: Time without date (e.g., 14:30:00).
* LocalDateTime: Date and time (e.g., 2023-10-05T14:30:00).
* ZonedDateTime: Date/time with timezone.
– Advantage: No need for @Temporal; JPA automatically maps to appropriate SQL types.
• Example:
private LocalDate eventDate; // Maps to SQL DATE
private LocalDateTime eventDateTime; // Maps to SQL TIMESTAMP

8.10.4 Summary of Key Annotations

Annotation Purpose Example

@Entity Marks a class as a JPA entity. @Entity public class User { ... }
@Table Specifies the database table for the entity. @Table(name = "employee_records")
@Id Marks the primary key field. @Id private Long id;
@GeneratedValue
Configures primary key generation strategy. @GeneratedValue(strategy =
[Link])
@Column Customizes column mapping (name, @Column(name = "full_name", nullable =
constraints). false)
@Temporal Specifies precision for legacy Date/Calendar @Temporal([Link]) private
fields. Date eventDate;
@Lob Maps a field to a large object column @Lob private String content;
(CLOB/BLOB).
@Transient Excludes a field from persistence. @Transient private int age;

8.10.5 Best Practices

1. Use @Entity and @Table:
• Always annotate entity classes with @Entity.
• Use @Table when the table name differs from the class name.
2. Primary Key Generation:
• Prefer [Link] for auto-increment columns.
• Use [Link] for databases supporting sequences (e.g., Oracle, PostgreSQL).
3. Column Customization:
• Explicitly define column names and constraints with @Column for clarity.
4. Avoid Legacy Date/Time:
• Use [Link] classes (LocalDate, LocalDateTime) instead of [Link].
5. LOBs for Large Data:
• Use @Lob for fields storing large text or binary data.
6. Transient Fields:
• Mark non-persistent fields with @Transient to keep the database schema clean.

8.11 Writing Unit Tests for DAOs and Service Methods

8.11.1 1. Introduction to Unit Testing
[Link] 1.1 Definition and Importance
• Unit testing is a critical component of the software development process.

386
 • It involves testing individual components (units) in isolation to ensure they function correctly.
• Purpose:
– Validate code correctness.
– Catch issues early in development.
– Improve maintainability and reliability.

[Link] 1.2 Tools for Unit Testing

• JUnit: A widely used Java testing framework for writing and running tests.
• Mockito: A mocking framework for simulating dependencies in isolated tests.

8.11.2 2. JUnit Fundamentals

[Link] 2.1 Key Annotations JUnit provides annotations to structure and manage test cases:

Annotation Purpose
@Test Marks a method as a test case to be executed.
@BeforeEach Runs before each test (setup logic, e.g., initializing objects).
@AfterEach Runs after each test (cleanup logic, e.g., resetting state).

import [Link];
import static [Link];

public class CalculatorTest {

@Test
public void testAddition() {
int result = 2 + 3;
assertEquals(5, result); // Expected vs. actual comparison
}
}

[Link] 2.2 Basic JUnit Test Example

• Key Points:
– @Test indicates the method is a test case.
– assertEquals(expected, actual) verifies correctness.

8.11.3 3. Mockito Fundamentals

[Link] 3.1 Purpose of Mockito
• Enables mocking dependencies to test components in isolation.
• Avoids reliance on real implementations (e.g., databases, external services).

[Link] 3.2 Key Features

Feature Description
Mock Creation [Link](Class) creates a mock object.
Behavior Definition when([Link]()).thenReturn(value) defines mock behavior.

387
Feature Description
Interaction Verification verify(mock).method() checks if a method was called.

import [Link];
import [Link];
import [Link];
import [Link];
import [Link];
import static [Link].*;

@ExtendWith([Link])
public class BookServiceTest {
@Mock
private BookRepository bookRepository; // Mock dependency

@InjectMocks
private BookService bookService; // Injects mock into service

@Test
public void testFindBookByTitle() {
when([Link]("Clean Code")).thenReturn(new Book());
Book result = [Link]("Clean Code");
verify(bookRepository).findByTitle("Clean Code"); // Verify interaction
}
}

[Link] 3.3 Mockito + JUnit Integration Example

• Key Annotations:
– @ExtendWith([Link]): Integrates Mockito with JUnit.
– @Mock: Creates a mock object (e.g., BookRepository).
– @InjectMocks: Injects mocks into the tested class (e.g., BookService).

8.11.4 4. Testing the Book Entity

[Link] 4.1 Scope of Entity Testing
• Typically part of integration tests but can include unit-level validation.
• Focus Areas:
– Correct database mappings (e.g., @Entity, @Id, @Column).
– Constraints (e.g., @NotNull, @Size).

@Entity
public class Book {
@Id
@GeneratedValue(strategy = [Link])
private Long id;

388
 @NotNull
@Size(min = 1, max = 100)
private String title;

// Getters and setters

}

[Link] **4.2 Example: Book Entity Annotations

• Validation Checks:
– Ensure @Entity is present.
– Verify constraints (e.g., title cannot be null or empty).

8.11.5 5. Testing the Book Repository

[Link] 5.1 Repository Overview
• BookRepository extends JpaRepository for CRUD operations.
• Includes custom query methods (e.g., findByTitle).
public interface BookRepository extends JpaRepository<Book, Long> {
Book findByTitle(String title); // Custom query
}

[Link] 5.2 Unit Test for BookRepository

• Uses @DataJpaTest to configure an in-memory database.
• Steps:
1. Save a Book entity.
2. Query it by title.
3. Assert retrieval correctness.
import [Link];
import static [Link].*;

@DataJpaTest
public class BookRepositoryTest {
@Autowired
private BookRepository bookRepository;

@Test
public void testFindByTitle() {
Book book = new Book("Clean Code");
[Link](book);
Book found = [Link]("Clean Code");
assertNotNull(found);
assertEquals("Clean Code", [Link]());
}
}

• Key Points:
– @DataJpaTest loads only JPA components (no full Spring context).
– Tests database interactions without external dependencies.

389
8.11.6 6. Testing the Book Service Implementation
[Link] 6.1 Service Layer Overview
• BookService contains business logic and interacts with BookRepository.
• Example Methods:
– addBook(Book book)
– findBookByTitle(String title)
@Service
public class BookService {
@Autowired
private BookRepository bookRepository;

public Book addBook(Book book) {

return [Link](book);
}

public Book findBookByTitle(String title) {

return [Link](title);
}
}

[Link] 6.2 Unit Test for BookService

• Mocks BookRepository to isolate service logic.
• Verifies:
– Correct method calls to the repository.
– Expected return values.
@ExtendWith([Link])
public class BookServiceTest {
@Mock
private BookRepository bookRepository;

@InjectMocks
private BookService bookService;

@Test
public void testAddBook() {
Book book = new Book("Design Patterns");
when([Link](book)).thenReturn(book);
Book result = [Link](book);
assertEquals(book, result);
verify(bookRepository).save(book);
}

@Test
public void testFindBookByTitle() {
when([Link]("Clean Code"))
.thenReturn(new Book("Clean Code"));
Book result = [Link]("Clean Code");

390
 assertNotNull(result);
verify(bookRepository).findByTitle("Clean Code");
}
}

• Key Assertions:
– assertEquals: Checks return values.
– verify: Ensures repository methods were called.

8.11.7 7. Setting Up the Testing Environment

[Link] 7.1 Dependencies

<dependencies>
<!-- JUnit 5 -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.8.2</version>
<scope>test</scope>
</dependency>
<!-- Mockito -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>mockito-core</artifactId>
<version>4.5.1</version>
<scope>test</scope>
</dependency>
<!-- Spring Boot Test -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

[Link].1 Maven ([Link])

dependencies {
testImplementation '[Link]:junit-jupiter-api:5.8.2'
testImplementation '[Link]:mockito-core:4.5.1'
testImplementation '[Link]:spring-boot-starter-test'
}

[Link].2 Gradle ([Link])

[Link] 7.2 Test Class Location

• Place tests in src/test/java/ (standard Maven/Gradle structure).

391
[Link] 7.3 Running Tests
• IDE: Right-click test class → “Run Tests” (e.g., VS Code, IntelliJ).
• Command Line:
– Maven: mvn test
– Gradle: gradle test

8.11.8 8. Analyzing Test Results

[Link] 8.1 IDE Output
• Passed Tests: Green checkmark ([OK]).
• Failed Tests: Red cross ([X]) with error details.

[Link] 8.2 Command Line Output

• Maven/Gradle provides a test report with:
– Number of tests run.
– Failures/errors (with stack traces).
– Execution time.

8.11.9 9. Summary of Key Concepts

[Link] 9.1 Unit Testing Principles
• Test individual components in isolation.
• Use JUnit for test structure and Mockito for dependencies.

[Link] 9.2 Testing Layers

Layer Focus Area Tools/Annotations

Entity Database mappings/constraints @Entity, @NotNull
Repository CRUD/custom queries @DataJpaTest, JpaRepository
Service Business logic + repository calls @Mock, @InjectMocks

[Link] 9.3 Best Practices

1. Isolate tests: Mock external dependencies.
2. Use assertions: Verify expected behavior (assertEquals, verify).
3. Follow naming conventions: testMethodName_Scenario_ExpectedResult.
4. Run tests frequently: Catch issues early.

8.11.10 10. Conclusion

• Unit testing ensures code reliability and early bug detection.
• JUnit + Mockito provide a robust framework for testing DAOs (Repositories) and Service methods.
• Key Takeaways:
– Write tests for entities, repositories, and services.
– Use mocking to isolate components.
– Analyze test results to improve code quality.

392
9 Module 9: Implementing the View Layer
9.1 Adding JavaScript to JSP Pages
9.1.1 Introduction to JavaScript in JSP
• Objective: By the end of this lecture, you will be able to:
– Create and add JavaScript files to a project.
– Use JavaScript in JSP pages.
– Create a simple JavaScript function.
• Focus Areas:
– Enhancing JSP with JavaScript.
– Creating and using JavaScript files.
– Integrating JavaScript with JSP pages.
– Writing simple JavaScript functions.

[Link] Role of JavaScript in Web Applications

• Definition: JavaScript is a powerful scripting language that enables dynamic behavior in web pages.
• Purpose in JSP:
– Adds interactivity and responsiveness to web applications.
– Handles user interactions (e.g., button clicks, form submissions).
– Manipulates HTML elements dynamically.
– Performs client-side validations (e.g., form input checks before submission).
• Advantage: Reduces server load by processing tasks on the client side.

9.1.2 Organizing JavaScript Files in a JSP Project

[Link] Project Structure Best Practices
• Recommended Location:
– JavaScript files should be placed in a dedicated js directory under the webapp folder of the project.
– Example structure:
project-root/
├── webapp/
│ ├── js/
│ │ └── [Link]
│ ├── WEB-INF/
│ └── [Link]
• Benefits of Organization:
– Keeps the project clean and maintainable.
– Simplifies script management (e.g., updates, debugging).
– Facilitates code reuse across multiple JSP pages.

[Link] Creating a JavaScript File

• Example: Create a file named [Link] in the js directory.
• Sample Code:
function showAlert() {
alert("Hello from JavaScript!");
}

393
 – This function displays a pop-up alert when called.
• Key Point: Save the file in the js directory to ensure it can be linked in JSP pages.

9.1.3 Including JavaScript in JSP Pages

[Link] Method 1: External JavaScript File (Recommended)
• Syntax:
<script src="js/[Link]"></script>

• Placement:
– Can be included in the <head> or <body> section of the JSP file.
• Example Usage:
<!DOCTYPE html>
<html>
<head>
<title>JavaScript in JSP</title>
<script src="js/[Link]"></script>
</head>
<body>
<button onclick="showAlert()">Click Me</button>
</body>
</html>

– The onclick event calls the showAlert() function from [Link].

[Link] Advantages of External JavaScript

• Separation of Concerns:
– Keeps HTML (JSP) and JavaScript separate.
– Improves code readability and maintainability.
• Reusability:
– The same .js file can be linked to multiple JSP pages.
• Performance:
– Browsers cache external scripts, reducing load times for subsequent visits.

[Link] Method 2: Inline JavaScript (For Small Scripts or Testing)

• Syntax:
<script>
function greetUser() {
alert("Welcome to the page!");
}
// Call the function when the page loads
[Link] = greetUser;
</script>

• Use Case:
– Suitable for small scripts or quick testing.

394
 – Not recommended for large-scale applications (leads to cluttered code).

[Link] Disadvantages of Inline JavaScript

• Maintenance Issues:
– Mixing JavaScript with JSP/HTML makes the code harder to debug and update.
• Reusability Limitations:
– Cannot be shared across multiple pages without duplication.

9.1.4 Embedding JSP Expressions in JavaScript

[Link] Bridging Server-Side and Client-Side Scripting
• Purpose: Pass server-side data (from JSP) to JavaScript variables.
• Syntax:
<script>
var username = '<%= [Link]("name") %>';
alert("Hello, " + username + "!");
</script>

• Example Scenario:
– A JSP page retrieves a user’s name from a request parameter and passes it to JavaScript.
– JavaScript then uses this value to dynamically generate content (e.g., personalized greetings).

[Link] Key Benefits

• Dynamic Content Generation:
– Enables real-time updates based on server data.
• Seamless Integration:
– Combines JSP (server-side) and JavaScript (client-side) logic.

9.1.5 Creating and Using Simple JavaScript Functions

[Link] Definition of a Function
• A reusable block of code that performs a specific task.
• Syntax:
function functionName(parameters) {
// Code to execute
}

[Link] Example: displayMessage Function

• Code:
function displayMessage(message) {
alert(message);
}

• Usage in JSP:

395
 <script src="js/[Link]"></script>
<button onclick="displayMessage('Button clicked!')">Test</button>

– The function is called via an event handler (e.g., onclick).

[Link] Advantages of Functions

• Modularity:
– Breaks down complex tasks into smaller, manageable pieces.
• Reusability:
– The same function can be called multiple times with different inputs.
• Maintainability:
– Easier to update or debug isolated functions.

9.1.6 Practical Implementation Steps

1. Create a js Directory:
• Locate the webapp folder in your project and add a js subdirectory.
2. Write JavaScript in [Link]:
• Define functions (e.g., showAlert(), displayMessage()).
3. Link the Script in JSP:
• Use <script src="js/[Link]"></script> in the JSP file.
4. Call Functions via Events:
• Attach functions to HTML events (e.g., onclick, onload).
5. Test Interactivity:
• Verify that JavaScript executes as expected (e.g., alerts appear on button clicks).

9.1.7 Summary of Key Concepts

Concept Description Example

External Scripts stored in .js files and linked via <script <script
JavaScript src="...">. src="js/[Link]"></script>
Inline JavaScript Scripts written directly in JSP using <script> tags. <script>alert("Hello");</script>
JSP Expressions Embedding server-side data into JavaScript variables. var name = '<%=
in JS [Link]() %>';
JavaScript Reusable code blocks for specific tasks. function greet() {
Functions alert("Hi!"); }
Event Handlers Triggering JavaScript functions via user actions (e.g., <button
clicks). onclick="greet()">Click</button>

9.1.8 Best Practices

1. Use External JavaScript Files:
• Avoid inline scripts for production code.
2. Organize Scripts Logically:
• Group related functions (e.g., [Link], [Link]).
3. Minimize Global Variables:
• Use local variables within functions to prevent conflicts.
4. Leverage JSP-JavaScript Integration:

396
 • Pass dynamic data from JSP to JavaScript for personalized experiences.
5. Test Cross-Browser Compatibility:
• Ensure JavaScript works across different browsers (Chrome, Firefox, Edge).

9.1.9 Conclusion
• JavaScript enhances JSP by adding interactivity, dynamic content, and client-side processing.
• Key Skills Acquired:
– Creating and linking external JavaScript files.
– Embedding JavaScript directly in JSP (for testing).
– Passing server-side data to JavaScript using JSP expressions.
– Writing and calling simple JavaScript functions.
• Application: These techniques are essential for building responsive, user-friendly web applications.

9.2 AJAX Calls with JQuery

9.2.1 1. Introduction to AJAX
[Link] 1.1 Definition and Purpose
• AJAX (Asynchronous JavaScript and XML) is a technology that enables asynchronous communication
between the client (browser) and the server.
• Allows sending and receiving data without reloading the entire page, improving user experience.
• Enables dynamic updates to parts of a web page (e.g., loading new data, submitting forms) while the user
continues interacting with other parts.

[Link] 1.2 Core Mechanism

• Uses the XmlHttpRequest (XHR) object to send requests to the server and receive responses.
• Supports multiple data formats:
– XML (originally designed for)
– JSON (most commonly used today)
– HTML (for partial page updates)

[Link] 1.3 Key Benefits

• Improved User Experience (UX):
– Faster load times (only necessary data is fetched).
– Smoother interactions (no full-page reloads).
• Efficiency:
– Reduces server load by minimizing data transfer.
– Enables real-time updates (e.g., live search, notifications).

9.2.2 2. Making AJAX Calls with JQuery

[Link] 2.1 Overview of JQuery’s AJAX Functionality
• JQuery simplifies AJAX calls with its $.ajax() function, providing a high-level interface for asynchronous
requests.
• Key components of $.ajax(): | Parameter | Description | |———–|————-| | url | Server endpoint (e.g.,
/books/updatecopies). | | type | HTTP method (GET, POST, PUT, DELETE, etc.). | | data | Data sent to the
server (typically in JSON format). | | success | Callback function executed on a successful response. | |
error | Callback function executed on a failed request. |

397
$.ajax({
url: "/books/updatecopies",
type: "POST",
data: [Link]({ bookId: 1, copies: 10 }),
success: function(response) {
[Link]("Update successful:", response);
},
error: function(xhr, status, error) {
[Link]("Error:", error);
}
});

[Link] 2.2 Basic Syntax Example

[Link] 2.3 Advantages of Using JQuery for AJAX

• Cross-browser compatibility (handles differences in XHR implementation).
• Simplified syntax compared to raw JavaScript.
• Built-in error handling and response processing.

9.2.3 3. Dynamic Content Updates with AJAX

[Link] 3.1 Use Case: Filtering Books by Category

[Link].1 3.1.1 Setup

• Frontend Components:
– A dropdown menu (for selecting book categories).
– A display area (to show filtered books).

[Link].2 3.1.2 Workflow

1. Event Listening:
• JQuery listens for changes in the dropdown ($( "#categoryDropdown" ).change()).
2. AJAX Request:
• On selection change, an AJAX call is made to fetch books of the selected category.
$( "#categoryDropdown" ).change(function() {
var category = $(this).val();
$.ajax({
url: "/books/filter",
type: "GET",
data: { category: category },
success: function(books) {
updateDisplayArea(books); // Refresh UI with new data
}
});
});
3. Dynamic Update:
• The display area is refreshed without a full page reload, improving responsiveness.

398
[Link].3 3.1.3 Benefits
• Real-time filtering enhances user interaction.
• Reduced server load (only fetches necessary data).

[Link] 3.2 Use Case: Updating Book Copies

[Link].1 3.2.1 Frontend (View Layer)

• HTML Structure:
– Input fields for:
* bookId (identifier for the book).
* copies (new quantity to update).
– An Update button to trigger the AJAX call.
<input type="text" id="bookId" placeholder="Book ID">
<input type="number" id="copies" placeholder="New Copies">
<button id="updateButton">Update</button>

[Link].2 3.2.2 JQuery Event Handling

• Captures the button click and sends an AJAX request:
$( "#updateButton" ).click(function() {
var bookId = $( "#bookId" ).val();
var copies = $( "#copies" ).val();
$.ajax({
url: "/books/updatecopies",
type: "POST",
data: [Link]({ bookId: bookId, copies: copies }),
contentType: "application/json",
success: function(response) {
alert("Update successful: " + [Link]);
},
error: function(xhr) {
alert("Error: " + [Link]);
}
});
});

[Link].3 3.2.3 Key Features

• Non-blocking UI: User can continue interacting while the request processes.
• Feedback: Success/error messages inform the user of the outcome.

9.2.4 4. Backend Architecture (MVC Pattern)

[Link] 4.1 Overview of Layers AJAX interacts with a Spring Boot backend structured using the MVC
(Model-View-Controller) pattern: 1. View Layer (Frontend) → Controller Layer → Service Layer →
Repository Layer → Database.

[Link] 4.2 Controller Layer

399
[Link].1 4.2.1 Role
• Acts as a bridge between the view and service/model layers.
• Handles incoming HTTP requests (e.g., from AJAX calls).

@RestController
@RequestMapping("/books")
public class BookController {

@Autowired
private BookService bookService;

@PostMapping("/updatecopies")
public ResponseEntity<String> updateCopies(@RequestBody Map<String, Object> payload) {
try {
int bookId = (int) [Link]("bookId");
int copies = (int) [Link]("copies");
[Link](bookId, copies);
return [Link]("{\"message\": \"Copies updated successfully\"}");
} catch (Exception e) {
return [Link]().body("{\"error\": \"" + [Link]() + "\"}");
}
}
}

[Link].2 4.2.2 Example: RestController for Book Updates

• Endpoint: /books/updatecopies (matches the AJAX url).
• Request Handling:
– Extracts bookId and copies from the request body.
– Calls the service layer to perform the update.
– Returns a success/error response (consumed by the AJAX success/error callbacks).

[Link] 4.3 Service Layer

[Link].1 4.3.1 Role

• Encapsulates business logic (e.g., validation, calculations).
• Ensures reusability and separation of concerns.

@Service
public class BookService {

@Autowired
private BookRepository bookRepository;

public void updateNumberOfCopies(int bookId, int copies) {

Book book = [Link](bookId)
.orElseThrow(() -> new RuntimeException("Book not found"));

400
 [Link](copies);
[Link](book);
}
}

[Link].2 4.3.2 Example: BookService Class

• Steps:
1. Retrieves the book using bookId.
2. Updates the copies field.
3. Saves changes via the repository layer.

[Link] 4.4 Model Layer

[Link].1 4.4.1 Role

• Represents data entities (e.g., Book) and their relationships.
• Mapped to database tables (e.g., via JPA annotations).

@Entity
@Table(name = "books")
public class Book {
@Id
private int id;
private String title;
private int copies;
// Getters and setters...
}

[Link].2 4.4.2 Example: Book Entity

• Annotations:
– @Entity: Marks the class as a JPA entity.
– @Table: Maps to the books table in the database.

[Link] 4.5 Repository Layer

[Link].1 4.5.1 Role

• Provides data access functionality (CRUD operations).
• Abstracts database interactions (no raw SQL required).

public interface BookRepository extends JpaRepository<Book, Integer> {

// Inherits methods like save(), findById(), delete(), etc.
}

[Link].2 **4.5.2 Example: BookRepository Interface

• Key Methods:
– save(entity): Updates or inserts a record.

401
 – findById(id): Retrieves a record by ID.
• Benefits:
– Clean code: No boilerplate SQL.
– Maintainability: Easy to extend or modify.

9.2.5 5. MVC Architecture Benefits

[Link] 5.1 Separation of Concerns
• View Layer: Handles UI and user interactions (AJAX calls).
• Controller Layer: Routes requests to the appropriate service.
• Service Layer: Implements business logic.
• Repository Layer: Manages data persistence.

[Link] 5.2 Advantages

• Modularity: Each layer can be developed/tested independently.
• Scalability: Easy to add new features (e.g., new endpoints).
• Collaboration: Frontend and backend teams can work in parallel.
• Maintainability: Clear structure reduces technical debt.

9.2.6 6. Summary of Key Concepts

[Link] 6.1 AJAX Fundamentals
• Enables asynchronous client-server communication.
• Uses XHR or modern alternatives like fetch().
• Supports JSON/XML/HTML data formats.

[Link] 6.2 JQuery AJAX

• Simplifies AJAX calls with $.ajax().
• Key parameters: url, type, data, success, error.
• Enables dynamic content updates without page reloads.

[Link] 6.3 Dynamic Updates

• Example 1: Filtering books by category (dropdown + AJAX).
• Example 2: Updating book copies (form + AJAX + backend).

[Link] 6.4 Backend Integration (Spring Boot MVC)

• Controller: Handles HTTP requests (@RestController).
• Service: Business logic (@Service).
• Repository: Data access (JpaRepository).
• Model: Data entities (@Entity).

[Link] 6.5 Benefits of MVC + AJAX

• Improved UX: Faster, smoother interactions.
• Efficient Data Transfer: Only necessary data is exchanged.
• Scalable Architecture: Clear separation of layers.

402
9.2.7 7. Conclusion
This lecture covered: 1. Introduction to AJAX: Asynchronous requests, XHR, and data formats. 2. JQuery
AJAX: Syntax, parameters, and dynamic updates. 3. Practical Examples: Filtering books and updating copies.
4. Backend Architecture: MVC layers (Controller, Service, Repository, Model). 5. Benefits: Separation of
concerns, scalability, and improved UX.
AJAX and JQuery enable real-time, interactive web applications by combining frontend dynamism with robust
backend processing.

9.3 Binding Data to JSP Pages

9.3.1 Introduction to Model Attributes in Spring Boot
• Objective: Use model attributes to bind data to JSP pages in a Spring Boot application.
• Key Concepts:
– Model attributes facilitate data transfer between controllers and JSP pages.
– The Model object in Spring Boot is used to pass attributes from the controller to the view.
– The primary method for adding data to the model is [Link]().
– The @ModelAttribute annotation can also bind form data to a model object.

9.3.2 Binding Data in a Spring Boot Controller

[Link] Controller Structure
• A @Controller annotation marks a class as a web controller.
• @GetMapping maps a URL path to a controller method (e.g., "/details" maps to getBookDetails()).

@Controller
public class BookController {

@GetMapping("/details")
public String getBookDetails(@RequestParam("id") Long id, Model model) {
Book book = [Link](id); // Retrieve book from service
[Link]("book", book); // Add book to the model
return "bookDetails"; // Return the view name
}
}

[Link] Example: Adding a Single Book to the Model

• Key Steps:
1. The controller retrieves a Book object based on an id parameter.
2. The [Link]("book", book) method binds the book object to the model with the key
“book”.
3. The method returns “bookDetails”, which resolves to a JSP file (e.g., [Link]).

[Link] View Resolution in Spring Boot

• The returned view name (“bookDetails”) is resolved by Spring Boot to locate the corresponding JSP file in:
– Default location: /src/main/webapp/WEB-INF/views/[Link]
– (Configuration may vary based on [Link] settings.)

403
9.3.3 Accessing Model Attributes in JSP
[Link] Expression Language (EL)
• Purpose: Access model attributes in JSP using dot notation.
• Syntax:
${[Link]}

• Example:
<p>Book ID: ${[Link]}</p>
<p>Title: ${[Link]}</p>

– ${[Link]} retrieves the id property of the book object.

– ${[Link]} retrieves the title property.

[Link] Advantages of Expression Language

• Keeps JSP code clean and maintainable.
• Avoids embedding Java scriptlets (e.g., <% %>), improving readability.

9.3.4 Displaying Data Safely with JSTL

[Link] JSTL (JSP Standard Tag Library)
• Purpose: Enhances JSP with tags for common tasks (e.g., iteration, conditional logic, output).
• Key Tag: <c:out> (from JSTL Core library) safely outputs data by escaping special characters (prevents
XSS attacks).

<%@ taglib prefix="c" uri="[Link] %>

<p>Title: <c:out value="${[Link]}" /></p>

[Link] Example: Safe Output with <c:out>

• value="${[Link]}" ensures the title is displayed without malicious scripts.

9.3.5 Binding a List of Books to the Model

@Controller
public class BookController {

@GetMapping("/list")
public String getBooksList(Model model) {
List<Book> books = [Link](); // Retrieve all books
[Link]("books", books); // Add list to the model
return "booksList"; // Return the view name
}
}

[Link] Controller Example: Adding a List of Books

404
 • Key Steps:
1. The controller retrieves a List<Book> from a service.
2. The list is added to the model with the key “books”.
3. The method returns “booksList”, resolving to [Link].

9.3.6 Displaying a List of Books in JSP

[Link] Using JSTL’s <c:forEach> Tag
• Purpose: Iterate over a collection (e.g., a list of books) and render each item.
• Syntax:
<c:forEach var="book" items="${books}">
<!-- Render each book -->
</c:forEach>

<%@ taglib prefix="c" uri="[Link] %>

<table>
<thead>
<tr>
<th>ID</th>
<th>Title</th>
<th>Author</th>
</tr>
</thead>
<tbody>
<c:forEach var="book" items="${books}">
<tr>
<td><c:out value="${[Link]}" /></td>
<td><c:out value="${[Link]}" /></td>
<td><c:out value="${[Link]}" /></td>
</tr>
</c:forEach>
</tbody>
</table>

[Link] Example: Table Display of Books

• Key Components:
– <c:forEach> iterates over the books list.
– Each book is assigned to the variable book (via var="book").
– <c:out> safely displays each property (id, title, author).
– The HTML table organizes data visually.

[Link] Advantages of This Approach

• Dynamic rendering: Automatically adjusts to the number of books.
• Separation of concerns: Logic (controller) is separate from presentation (JSP).
• Security: Prevents XSS via <c:out>.

405
9.3.7 Key Takeaways
1. Model Attributes:
• Used to pass data from controllers to JSP pages.
• Added via [Link]() or @ModelAttribute.
2. Expression Language (EL):
• Accesses model attributes in JSP (e.g., ${[Link]}).
3. JSTL:
• Provides tags like <c:out> (safe output) and <c:forEach> (iteration).
4. Binding Lists:
• Controllers can bind collections (e.g., List<Book>) to the model.
• JSP uses <c:forEach> to iterate and display lists dynamically.
5. Separation of Concerns:
• Controllers handle data retrieval/logic.
• JSP focuses on presentation.
6. Security:
• Always use <c:out> to escape output and prevent XSS.

9.3.8 Conclusion
• Spring Boot + JSP provides a robust framework for building web applications with:
– Clear separation between controllers (logic) and views (presentation).
– Dynamic data binding via model attributes.
– Secure and maintainable JSP code using JSTL and EL.
• Applications:
– Displaying single objects (e.g., a book’s details).
– Rendering lists (e.g., a table of books).
– Ensuring security and scalability in web applications.

9.4 Creating and Processing HTML Forms in JSP

9.4.1 Introduction to HTML Forms in JSP
[Link] Learning Objectives By the end of this lecture, learners will be able to: - Create HTML forms in JSP.
- Handle form submissions. - Process form data in a controller (Servlet).

[Link] Importance of HTML Forms

• HTML forms are essential for collecting user input in web applications.
• They enable interaction between users and the server by allowing data submission (e.g., login credentials,
book details, survey responses).

9.4.2 HTML Form Basics

[Link] Definition of an HTML Form
• A section of an HTML document containing interactive controls (input fields, buttons, etc.) to gather user
data.
• Encapsulated within the <form> tag, which acts as a container for all form elements.

[Link] Key Attributes of the <form> Tag

406
Attribute Description
action Specifies the URL where the form data will be sent upon
submission.
method Defines the HTTP method used to send data (typically
POST for secure submission or GET for visible URL
parameters).

[Link] Common Input Elements in HTML Forms Forms support various input types to collect different
kinds of data:

Input Type Description Example Usage

text Single-line text input. <input type="text" name="title">
password Hidden text input (for sensitive <input type="password"
data like passwords). name="pwd">
radio Single selection from a group of <input type="radio"
options. name="gender" value="male"> Male
checkbox Multiple selections from a group. <input type="checkbox"
name="hobbies" value="reading">
Reading
submit Button to trigger form <input type="submit"
submission. value="Submit">
email Validates email format. <input type="email"
name="user_email">
number Numeric input (with optional <input type="number" name="age">
min/max constraints).
date Date picker. <input type="date"
name="birthday">
file File upload control. <input type="file"
name="resume">

[Link] Role of the Submit Button

• The submit button is critical as it triggers the form submission.
• When clicked, the browser sends the form data to the server using the specified action URL and method.

9.4.3 Handling Form Submissions in JSP

[Link] Request and Response Objects
• When a form is submitted, the data is sent to the server via an HTTP request.
• Two key objects are involved:
1. HttpServletRequest – Retrieves submitted form data.
2. HttpServletResponse – Sends a response back to the client.

[Link] Key Methods for Processing Form Data

407
Method Description
getParameter(String name) Retrieves the value of a form field by its name attribute.
setContentType(String type) Sets the response content type (e.g., "text/html").
getWriter() Returns a PrintWriter object to send text responses.

9.4.4 Step-by-Step Implementation

[Link] 1. Creating an HTML Form in JSP ([Link]) Example: Book Submission Form
<form action="/submitForm" method="post">
<label for="title">Title:</label>
<input type="text" id="title" name="title" required><br>

<label for="author">Author:</label>
<input type="text" id="author" name="author" required><br>

<label for="category">Category:</label>
<select id="category" name="category">
<option value="fiction">Fiction</option>
<option value="non-fiction">Non-Fiction</option>
<option value="sci-fi">Science Fiction</option>
</select><br>

<input type="submit" value="Submit">

</form>

[Link].1 Key Points:

• The action attribute (/submitForm) specifies the Servlet URL that will process the form.
• The method is set to post for secure data submission (data is not visible in the URL).
• Each input field has a name attribute, which is used to retrieve the data in the Servlet.

[Link] 2. Configuring the Servlet in [Link] (Optional for Modern JSP) Traditional Approach (if not
using annotations):
<servlet>
<servlet-name>FormServlet</servlet-name>
<servlet-class>[Link]</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>FormServlet</servlet-name>
<url-pattern>/submitForm</url-pattern>
</servlet-mapping>

[Link].1 Key Points:

• Defines a Servlet named FormServlet.
• Maps it to the URL pattern /submitForm (matches the action in the form).

408
[Link] 3. Creating the Servlet ([Link]) Modern Approach (Using @WebServlet Annotation):
import [Link];
import [Link].*;
import [Link];
import [Link].*;

@WebServlet("/submitForm")
public class FormServlet extends HttpServlet {
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {

// Retrieve form data using getParameter()

String title = [Link]("title");
String author = [Link]("author");
String category = [Link]("category");

// Set attributes to pass data to JSP

[Link]("title", title);
[Link]("author", author);
[Link]("category", category);

// Forward the request to [Link]

RequestDispatcher dispatcher = [Link]("[Link]");
[Link](request, response);
}
}

[Link].1 Key Steps in the Servlet:

1. doPost Method – Handles POST requests (triggered by form submission).
2. getParameter – Retrieves form data using the name attributes from [Link].
3. setAttribute – Stores the data as request attributes for use in the JSP.
4. RequestDispatcher – Forwards the request to [Link] for display.

[Link] 4. Displaying Form Data in [Link] Using JSP Expression Language (EL):
<%@ page contentType="text/html;charset=UTF-8" %>
<html>
<head>
<title>Form Submission Result</title>
</head>
<body>
<h1>Book Submission Confirmation</h1>
<p><strong>Title:</strong> ${title}</p>
<p><strong>Author:</strong> ${author}</p>
<p><strong>Category:</strong> ${category}</p>
</body>
</html>

[Link].1 Key Points:

409
 • ${attributeName} – EL syntax to access request attributes set by the Servlet.
• Displays the submitted data dynamically.

9.4.5 Testing the Application

[Link] Step-by-Step Testing Process
1. Access the Form Page:
• Navigate to [Link] via the application URL (e.g., [Link]
2. Fill Out the Form:
• Enter data in all fields (e.g., Title: “The Great Gatsby”, Author: “F. Scott Fitzgerald”, Category:
“Fiction”).
3. Submit the Form:
• Click the Submit button.
4. Verify the Result:
• The [Link] page should display the submitted data, confirming successful processing.

9.4.6 Summary of Key Concepts

1. HTML Forms in JSP:
• Created using <form> with action and method attributes.
• Supports various input types (text, password, radio, checkbox, etc.).
2. Form Submission Handling:
• Data is sent to a Servlet specified in the action attribute.
• The doPost method in the Servlet processes the data.
3. Data Retrieval and Display:
• getParameter() retrieves form data in the Servlet.
• setAttribute() passes data to JSP.
• Expression Language (EL) (${...}) displays data in JSP.
4. Testing:
• Verify the flow: Form → Servlet → Result Page.

9.4.7 Conclusion
This lecture covered the end-to-end process of creating and processing HTML forms in JSP, including: - Designing
forms with appropriate input elements. - Configuring Servlets to handle submissions. - Processing and displaying
form data using JSP and EL. - Testing the application to ensure correct functionality.
This knowledge is foundational for building interactive web applications with user input handling.

9.5 Creating Basic HTML Templates

9.5.1 Introduction to JSP in HTML Templates
• Objective: By the end of this lecture, learners will be able to:
– Create a basic HTML template using JavaServer Pages (JSP).
– Use JSP tags and expressions effectively.
– Develop a simple JSP page with dynamic content.

9.5.2 JSP Scripting Elements

JSP scripting elements allow embedding Java code within HTML to create dynamic web content.

410
[Link] 1. Declarations (<%! ... %>)
• Used to declare variables and methods that can be reused throughout the JSP page.
• Syntax:
<%!
int counter = 0; // Variable declaration
void incrementCounter() { // Method declaration
counter++;
}
%>

• Purpose: Defines reusable components (e.g., counters, helper functions) accessible across multiple requests
to the same page.

[Link] 2. Scriptlets (<% ... %>)

• Contains Java code that executes every time the page is requested.
• Syntax:
<%
counter++; // Executes on each page load
%>

• Use Case: Performing logic (e.g., loops, conditionals) that generates dynamic content.

[Link] 3. Expressions (<%= ... %>)

• Outputs the result of a Java expression directly to the client’s browser.
• Syntax:
<%= counter %> <!-- Displays the current value of 'counter' -->

• Example:
<%!
int counter = 0;
%>
<%
counter++;
%>
Current count: <%= counter %> <!-- Outputs: "Current count: 1" -->

9.5.3 JSP Directives

Directives provide global instructions for the JSP page, influencing its behavior and properties.

[Link] 1. Page Directive (<%@ page ... %>)

• Sets page-specific attributes, such as:
– Language: Default is Java (language="java").
– Content Type: MIME type and character encoding (e.g., contentType="text/html;charset=UTF-
8").

411
 – Error Handling: Specifies error pages (errorPage="[Link]").
• Example:
<%@ page language="java" contentType="text/html;charset=UTF-8" %>

[Link] 2. Include Directive (<%@ include ... %>)

• Statically includes the content of another file (e.g., headers, footers) at compile time.
• Syntax:
<%@ include file="[Link]" %>

• Note: Changes to the included file require recompilation of the JSP.

[Link] 3. Taglib Directive (<%@ taglib ... %>)

• Declares a custom tag library (e.g., JSTL) for use in the JSP.
• Syntax:
<%@ taglib prefix="c" uri="[Link] %>

• Purpose: Enables the use of predefined tags (e.g., <c:out>, <c:forEach>) to simplify common tasks.

9.5.4 JSP Implicit Objects

JSP provides predefined objects (no declaration needed) representing key components of a web application.

Object Type Purpose

request HttpServletRequest Accesses client request data (e.g., parameters, headers).
response HttpServletResponse Manages the HTTP response (e.g., cookies, redirect).
session HttpSession Stores user-specific data across requests.
application ServletContext Shares application-wide data (e.g., configuration).
out JspWriter Writes output to the client (similar to PrintWriter).
config ServletConfig Provides servlet configuration (e.g., initialization parameters).
pageContext PageContext Central access to other implicit objects (e.g., request, session).
page Object (current JSP) Refers to the current JSP page instance.
exception Throwable Represents uncaught exceptions (only in error pages).

<!-- Retrieve a request parameter -->

User: <%= [Link]("username") %>

<!-- Store data in the session -->

<%
[Link]("userRole", "admin");
%>

[Link] Example Usage

412
9.5.5 JSP Standard Tag Library (JSTL)
JSTL simplifies common JSP tasks (e.g., iteration, conditionals) using standardized tags.

[Link] 1. Core JSTL Tags

Tag Purpose Example

<c:out> Safely outputs an expression (escapes <c:out value="${userInput}"
HTML/XML). default="Guest" />
<c:forEach>Iterates over a collection (e.g., arrays, lists). <c:forEach var="item"
items="${items}"> ${item} </c:forEach>
<c:if> Conditionally executes content. <c:if test="${userRole == 'admin'}">
Admin Panel </c:if>
<c:choose> Multi-way conditional (like switch-case). <c:choose> <c:when test="${score >=
90}">A</c:when> ... </c:choose>

[Link] 2. Using JSTL in a JSP Page

1. Declare the Taglib:
<%@ taglib prefix="c" uri="[Link] %>

2. Example: Displaying a List:

<c:forEach var="fruit" items="${fruits}">
<li><c:out value="${fruit}" /></li>
</c:forEach>

9.5.6 Creating a Simple JSP Page: Step-by-Step

Combine all concepts to build a dynamic JSP page.

<%@ page language="java" contentType="text/html;charset=UTF-8" %>

<%@ taglib prefix="c" uri="[Link] %>

[Link] 1. Set Up the Page

<html>
<head>
<title>Welcome Page</title>
</head>
<body>
<h1>Welcome to our website!</h1>
<p>Current date: <%= new [Link]() %></p>
</body>
</html>

[Link] 2. Display Static and Dynamic Content

413
<%
String[] items = {"Item 1", "Item 2", "Item 3"};
[Link]("itemsList", items); // Store in request scope
%>

[Link] 3. Use Scriptlets for Logic

<ul>
<c:forEach var="item" items="${itemsList}">
<li><c:out value="${item}" /></li>
</c:forEach>
</ul>

[Link] 4. Iterate with JSTL

<%@ page language="java" contentType="text/html;charset=UTF-8" %>

<%@ taglib prefix="c" uri="[Link] %>
<html>
<head>
<title>Dynamic JSP Example</title>
</head>
<body>
<h1>Welcome!</h1>
<p>Today is: <%= new [Link]() %></p>

<%
String[] fruits = {"Apple", "Banana", "Cherry"};
[Link]("fruits", fruits);
%>

<h2>Fruit List:</h2>
<ul>
<c:forEach var="fruit" items="${fruits}">
<li><c:out value="${fruit}" /></li>
</c:forEach>
</ul>
</body>
</html>

[Link] Full Example

[Link] Key Takeaways

• Declarations define reusable variables/methods.
• Scriptlets execute Java code per request.
• Expressions output dynamic values.
• Directives configure page behavior (e.g., includes, taglibs).
• Implicit objects (e.g., request, session) provide access to web components.

414
 • JSTL simplifies common tasks (e.g., loops, conditionals) with standardized tags.

9.5.7 Summary of Learning Outcomes

By the end of this lecture, learners can: 1. Create HTML templates with embedded JSP. 2. Use JSP scripting
elements (declarations, scriptlets, expressions). 3. Apply JSP directives (page, include, taglib). 4. Leverage
implicit objects (e.g., request, session). 5. Implement JSTL tags for dynamic content (e.g., <c:forEach>,
<c:if>). 6. Develop a complete JSP page combining static HTML and dynamic logic.

9.6 Enhancing Interactivity with JQuery

9.6.1 Introduction to jQuery
• Definition: jQuery is a lightweight JavaScript library designed to simplify:
– DOM (Document Object Model) manipulation
– Event handling
– AJAX operations
• Purpose: Streamlines complex JavaScript tasks, enabling developers to build interactive web applications
efficiently.
• Example Use Case: Book management system (used as a practical example in this lecture).

9.6.2 jQuery Syntax and Features

[Link] Basic Syntax
• Dollar Sign ($):
– Shorthand for jQuery.
– Used to select elements and perform actions on them.
• Example:
$("#bookList").append("<tr><td>New Book</td></tr>");

– Explanation:
* $("#bookList") selects the element with ID bookList.
* .append() inserts a new row (<tr>) into the table.

[Link] Selectors in jQuery

• Purpose: Target specific elements in the DOM.
• Types of Selectors:
1. Tag Selector: Selects all elements of a given tag.
– Example: $("button") → Selects all <button> elements.
2. ID Selector: Selects a single element by its ID.
– Example: $("#bookList") → Selects the element with id="bookList".
3. Class Selector: Selects all elements with a given class.
– Example: $(".bookTitle") → Selects all elements with class="bookTitle".
• Advantage: Simplifies DOM traversal and manipulation.

9.6.3 DOM Manipulation with jQuery

[Link] Methods for Content Manipulation

415
 • html(): Gets or sets the HTML content of an element.
• text(): Gets or sets the text content of an element.
• val(): Gets or sets the value of form elements (e.g., inputs).
• Example:
$("#bookTitle").text("New Title"); // Sets text content

[Link] Styling and Visibility

• css(): Modifies CSS properties of elements.
– Example: $("#bookList").css("color", "blue");
• Visibility Control:
– show(): Displays hidden elements.
– hide(): Hides visible elements.
– toggle(): Toggles visibility.
– Example:
$("#addBookForm").toggle(); // Toggles form visibility

9.6.4 Event Handling in jQuery

[Link] Key Concepts
• Purpose: Attach event listeners to DOM elements to trigger actions.
• Common Events:
– click (mouse click)
– submit (form submission)
– hover (mouse enter/exit)
– change (input value change)
– input (real-time input changes)

[Link] Syntax for Event Handling

• Basic Structure:
$(selector).event(function() {
// Action to perform
});

• Example (Click Event):

$("#toggleButton").click(function() {
$("#addBookForm").toggle(); // Shows/hides form on click
});

9.6.5 Dynamic Content Manipulation

[Link] Inserting Elements
• Methods:
– append(): Adds content inside an element, at the end.
– prepend(): Adds content inside an element, at the beginning.

416
 – after(): Adds content after an element.
– before(): Adds content before an element.
• Example (Adding a Book to a List):
$("#bookList").append("<tr><td>New Book</td><td>Author</td></tr>");

[Link] Animations for Smooth Transitions

• Methods:
– fadeIn() / fadeOut(): Gradually shows/hides elements.
– slideUp() / slideDown(): Slides elements up/down.
• Use Case: Enhances user experience with visually appealing transitions.

9.6.6 AJAX with jQuery

[Link] Definition
• AJAX (Asynchronous JavaScript and XML): Technique to update parts of a webpage without reloading
the entire page.
• jQuery AJAX Methods:
– $.ajax(): General-purpose AJAX request.
– $.get(): Sends a GET request to fetch data.
– $.post(): Sends a POST request to submit data.

$.get("/search", { query: "Database" }, function(data) {

// Process and display search results
});

[Link] Example (Fetching Search Results)

• Benefits:
– Faster interactions (no full page reload).
– More responsive user experience.

9.6.7 Enhancing Forms with jQuery

[Link] Key Enhancements
1. Client-Side Validation
2. Dynamic Fields
3. Real-Time Feedback
4. AJAX Form Submission

[Link] 1. Client-Side Validation

• Purpose: Validate form data before submission to ensure accuracy.
• Implementation:
$("#addBookForm").on("submit", function(event) {
if ($("#title").val() === "" || $("#author").val() === "") {
[Link](); // Stop submission

417
 alert("Title and Author are required!");
}
});

• Advantage: Prevents invalid data from being sent to the server.

[Link] 2. Dynamic Form Fields

• Purpose: Show/hide fields based on user input.
• Example (Category Dropdown):
$("#category").change(function() {
if ($(this).val() === "other") {
$("#customCategory").show(); // Show text field for "Other"
} else {
$("#customCategory").hide();
}
});

• Use Case: Improves usability by reducing clutter.

[Link] 3. Real-Time Feedback

• Purpose: Provide immediate responses to user input (e.g., formatting, validation).
• Example (ISBN Formatting):
$("#isbn").on("input", function() {
let formattedISBN = $(this).val()
.replace(/[^0-9]/g, "") // Remove non-numeric characters
.replace(/(\d{3})(\d{1,5})(\d{1,7})(\d{1,6})(\d{1})/, "$1-$2-$3-$4-$5");
$(this).val(formattedISBN); // Update input value
});

• Breakdown:
1. replace(/[^0-9]/g, ""): Removes all non-numeric characters.
2. Regular Expression:
– (\d{3}) → First 3 digits.
– (\d{1,5}) → Next 1–5 digits.
– (\d{1,7}) → Next 1–7 digits.
– (\d{1,6}) → Next 1–6 digits.
– (\d{1}) → Final digit.
3. $1-$2-$3-$4-$5: Inserts hyphens between groups.
4. $(this).val(formattedISBN): Updates the input field with the formatted ISBN.

[Link] 4. AJAX Form Submission

• Purpose: Submit forms asynchronously (no page reload).
• Implementation:

418
 $("#addBookForm").submit(function(event) {
[Link](); // Prevent default submission
let formData = $(this).serialize(); // Encode form data

$.ajax({
url: $(this).attr("action"), // Form action URL
type: "POST",
data: formData,
success: function(response) {
$("#successMessage").text("Book added successfully!");
$("#addBookForm")[0].reset(); // Reset form
},
error: function() {
$("#errorMessage").text("Failed to add book.");
}
});
});

• Key Components:
– [Link](): Stops the default form submission.
– $(this).serialize(): Encodes form data into a URL-encoded string.
– $.ajax(): Sends data to the server asynchronously.
– Success/Error Handling: Displays feedback to the user.

9.6.8 Conclusion
[Link] Summary of Key Learnings
1. jQuery Syntax:
• Uses $ as shorthand for jQuery.
• Simplifies DOM selection and manipulation.
2. DOM Manipulation:
• Methods like append(), html(), css(), show(), and hide().
3. Event Handling:
• Attach listeners for click, submit, change, etc.
4. Dynamic Content:
• Insert elements and apply animations (fadeIn, slideDown).
5. AJAX:
• Asynchronous data fetching/submission ($.get, $.post, $.ajax).
6. Form Enhancements:
• Validation, dynamic fields, real-time feedback, and AJAX submission.

[Link] Importance in Modern Web Development

• jQuery simplifies complex JavaScript tasks, making it easier to create:
– Dynamic web applications.
– User-friendly interfaces.
– Responsive interactions (e.g., no page reloads with AJAX).
• Practical Application: Book management system demonstrates real-world use cases.

419
[Link] Final Takeaways
• jQuery is essential for modern web development due to its:
– Efficiency (less code for complex tasks).
– Cross-browser compatibility.
– Extensive plugin ecosystem.
• Encouragement: Experiment with jQuery in projects to enhance interactivity and user experience.

9.7 Integrating CSS for Styling

9.7.1 1. Introduction to CSS
[Link] 1.1 Definition of CSS
• CSS (Cascading Style Sheets) is a stylesheet language used to describe the presentation of a document
written in HTML.
• It controls the visual appearance of web pages, including layout, colors, fonts, and spacing.

[Link] 1.2 Benefits of CSS CSS provides several key advantages: 1. Separation of Content from Presenta-
tion - HTML handles structure and content, while CSS handles styling and layout. - This separation improves
code maintainability and scalability. 2. Reusability Across Multiple Pages - A single CSS file can be applied
to multiple HTML/JSP pages, ensuring consistent styling. 3. Improved Accessibility - Proper CSS usage en-
hances readability and usability for users, including those with disabilities. 4. Enhanced Performance - External
CSS files are cached by browsers, reducing load times for subsequent page visits. 5. Design Flexibility - CSS
allows for responsive design, animations, and complex layouts without altering HTML structure.

[Link] 1.3 Role of CSS in Web Development

• Ensures web pages are visually appealing and user-friendly.
• Simplifies maintenance by centralizing style definitions.
• Enables consistent branding across an entire application.

9.7.2 2. CSS Syntax and Structure

[Link] 2.1 Basic CSS Rule Structure A CSS rule consists of: 1. Selector – Identifies the HTML element(s)
to be styled. 2. Declaration Block – Contains one or more declarations enclosed in curly braces {}. - Each
declaration consists of: - Property (e.g., color, font-size) - Value (e.g., blue, 16px) - Declarations are separated
by semicolons (;).

p {
color: blue;
}

[Link].1 Example:
• Selector: p (targets all <p> elements)
• Declaration Block: { color: blue; }
– Property: color
– Value: blue

420
9.7.3 3. CSS Selectors
Selectors determine which HTML elements are affected by a CSS rule. Below are the most common types:

[Link] 3.1 Element Selector

• Targets all instances of a specified HTML element.
• Syntax: element { ... }

p {
color: blue;
}

[Link].1 Example:
• Applies blue text color to all <p> elements in the document.

[Link] 3.2 Class Selector

• Targets elements with a specific class attribute.
• Syntax: .classname { ... }
• Useful for reusable styles across different elements.

.highlight {
background-color: yellow;
}

[Link].1 Example:
• Applies a yellow background to all elements with class="highlight".

[Link] 3.3 ID Selector

• Targets a single element with a specific ID attribute.
• Syntax: #idname { ... }
• IDs must be unique within a page.

#header {
font-size: 24px;
}

[Link].1 Example:
• Applies a font size of 24px to the element with id="header".

[Link] 3.4 Attribute Selector

• Targets elements with a specific attribute or attribute value.
• Syntax: element[attribute="value"] { ... }

421
input[type="text"] {
border: 1px solid #ccc;
}

[Link].1 Example:
• Applies a light gray border to all text input fields.

[Link] 3.5 Descendant Selector

• Targets elements that are descendants of a specified element.
• Syntax: ancestor descendant { ... }

div p {
margin: 10px;
}

[Link].1 Example:
• Applies a 10px margin to all <p> elements inside <div> elements.

[Link] 3.6 Pseudo-Classes and Pseudo-Elements

• Pseudo-classes target special states of an element (e.g., hover, focus).
• Syntax: selector:pseudo-class { ... }

a:hover {
color: red;
}

[Link].1 Example (Pseudo-Class):

• Changes link color to red when hovered.
• Pseudo-elements target specific parts of an element (e.g., first line, first letter).
• Syntax: selector::pseudo-element { ... }

p::first-line {
font-weight: bold;
}

[Link].2 Example (Pseudo-Element):

• Makes the first line of every paragraph bold.

9.7.4 4. Methods of Applying CSS to HTML/JSP

There are three primary ways to integrate CSS into a web page:

422
[Link] 4.1 Inline CSS
• Styles are applied directly to an HTML element using the style attribute.
• Use Case: Quick, one-off styling (not recommended for large-scale use).
• Drawback: Hard to maintain (mixes content with presentation).

<p style="color: blue;">This is a blue paragraph.</p>

[Link].1 Example:

[Link] 4.2 Internal CSS

• Styles are defined within a <style> tag in the <head> section of the HTML/JSP file.
• Use Case: Styling a single page without external files.
• Drawback: Not reusable across multiple pages.

<head>
<style>
p {
color: blue;
}
</style>
</head>

[Link].1 Example:

[Link] 4.3 External CSS

• Styles are defined in a separate .css file and linked to the HTML/JSP file.
• Use Case: Best practice for maintainability and reusability.
• Advantage: Single source of truth for styling across multiple pages.

[Link].1 Example:
1. CSS File ([Link]):
p {
color: blue;
}

2. HTML/JSP File:
<head>
<link rel="stylesheet" href="css/[Link]">
</head>

9.7.5 5. Best Practices for Adding CSS to JSP Pages

To ensure maintainable, performant, and scalable CSS in JSP applications, follow these guidelines:

423
[Link] 5.1 Use External CSS Whenever Possible
• Avoid inline and internal CSS for large applications.
• Centralize styles in .css files for reusability and easier updates.

[Link] 5.2 Organize CSS Files in a Dedicated Directory

• Store CSS files in a structured directory (e.g., /css/).
• Example:
/WebContent/
├── css/
│ └── [Link]
├── WEB-INF/
└── [Link]

[Link] 5.3 Minify CSS Files

• Minification removes whitespace and comments to reduce file size.
• Improves page load speed.
• Tools: CSSNano, UglifyCSS, or build tools like Webpack.

[Link] 5.4 Follow Consistent Naming Conventions

• Use meaningful class/ID names (e.g., .book-form, #header).
• Avoid generic names like .style1.

[Link] 5.5 Avoid !important Unless Necessary

• Overrides all other styles, making debugging difficult.
• Use specific selectors instead.

9.7.6 6. Styling JSP Pages with CSS

[Link] 6.1 Example 1: [Link] (Add Book Form)

[Link].1 Purpose:
• A form for adding new books (title, author, category).
• Styled using an external CSS file ([Link]).

<%@ page language="java" contentType="text/html; charset=UTF-8" %>

<html>
<head>
<title>Add Book</title>
<link rel="stylesheet" href="css/[Link]">
</head>
<body>
<h1>Add a New Book</h1>
<form action="addBook" method="post">
<div class="form-group">

424
 <label for="title">Title:</label>
<input type="text" id="title" name="title" required>
</div>
<div class="form-group">
<label for="author">Author:</label>
<input type="text" id="author" name="author" required>
</div>
<div class="form-group">
<label for="category">Category:</label>
<input type="text" id="category" name="category" required>
</div>
<button type="submit" class="submit-btn">Submit</button>
</form>
</body>
</html>

[Link].2 JSP Code ([Link]):

/* General Styles */
body {
font-family: Arial, sans-serif;
background-color: #f4f4f4;
margin: 0;
padding: 0;
}

/* Form Styling */
form {
background: white;
padding: 20px;
border-radius: 5px;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
max-width: 500px;
margin: 20px auto;
}

.form-group {
margin-bottom: 15px;
}

.form-group label {
display: block;
margin-bottom: 5px;
font-weight: bold;
}

.form-group input {
width: 100%;

425
 padding: 8px;
border: 1px solid #ddd;
border-radius: 4px;
}

.submit-btn {
background-color: #4CAF50;
color: white;
padding: 10px 15px;
border: none;
border-radius: 4px;
cursor: pointer;
}

.submit-btn:hover {
background-color: #45a049;
}

[Link].3 CSS Styling ([Link]):

[Link] 6.2 Example 2: [Link] (Book List Table)

[Link].1 Purpose:
• Displays a table of books (ID, title, author, category).
• Styled using external CSS for consistency.

<%@ page language="java" contentType="text/html; charset=UTF-8" %>

<html>
<head>
<title>Book List</title>
<link rel="stylesheet" href="css/[Link]">
</head>
<body>
<h1>Book List</h1>
<table>
<thead>
<tr>
<th>ID</th>
<th>Title</th>
<th>Author</th>
<th>Category</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>Book Title 1</td>
<td>Author 1</td>

426
 <td>Fiction</td>
</tr>
<tr>
<td>2</td>
<td>Book Title 2</td>
<td>Author 2</td>
<td>Non-Fiction</td>
</tr>
</tbody>
</table>
</body>
</html>

[Link].2 JSP Code ([Link]):

/* Table Styling */
table {
width: 100%;
border-collapse: collapse;
margin: 20px 0;
}

table th, table td {

padding: 12px;
text-align: left;
border-bottom: 1px solid #ddd;
}

table th {
background-color: #4CAF50;
color: white;
}

table tr:hover {
background-color: #f5f5f5;
}

[Link].3 CSS Styling ([Link]):

9.7.7 7. Key Styling Techniques Applied

[Link] 7.1 Form Styling
• Clean Layout:
– White background with shadow and rounded corners for a modern look.
– Padding and margins for spacing.
• Input Fields:
– Full-width inputs with consistent padding.
• Button Styling:
– Green background with hover effect for interactivity.

427
[Link] 7.2 Table Styling
• Full-Width Table:
– Ensures responsiveness.
• Header Styling:
– Green background with white text for clear distinction.
• Hover Effect:
– Highlights rows on hover for better user experience.
• Border Collapse:
– Ensures clean, unified borders.

[Link] 7.3 General Page Styling

• Font Family:
– Arial for readability.
• Background Color:
– Light gray (#f4f4f4) for a soft contrast.
• Spacing:
– Margins and padding for visual balance.

9.7.8 8. Summary of Key Concepts

1. CSS Fundamentals:
• Separates content (HTML) from presentation (CSS).
• Uses selectors and declaration blocks for styling.
2. Selector Types:
• Element, class, ID, attribute, descendant, pseudo-classes/elements.
3. CSS Integration Methods:
• Inline (avoid), internal (limited use), external (recommended).
4. Best Practices for JSP:
• Use external CSS, organize files, minify, and avoid !important.
5. Styling JSP Pages:
• Apply CSS to forms, tables, and general layout for consistency.
6. Interactive Elements:
• Use hover effects (:hover) for better UX.
End of Notes

9.8 Introduction to JavaScript and JQuery

9.8.1 1. Overview of JavaScript and jQuery
[Link] 1.1 Definitions
• JavaScript:
– A powerful scripting language used to create dynamic and interactive content on websites.
– Enables client-side execution, allowing real-time updates without page reloads.
– Core technology for modern web development alongside HTML (structure) and CSS (styling).
• jQuery:
– A fast, lightweight JavaScript library designed to simplify common JavaScript tasks.
– Key functionalities:
* DOM (Document Object Model) manipulation (selecting, modifying, and traversing HTML
elements).

428
 * Event handling (responding to user interactions like clicks, keypresses).
* AJAX (Asynchronous JavaScript and XML) requests (fetching data from servers asyn-
chronously).
* Animations and effects (e.g., fading, sliding).
– Cross-browser compatibility: Ensures consistent behavior across different browsers.

[Link] 1.2 Relationship Between JavaScript and jQuery

• jQuery is built on top of JavaScript—it is not a separate language but a library that abstracts complex
JavaScript operations.
• Purpose: Reduces code verbosity and simplifies development by providing pre-written, optimized func-
tions.

9.8.2 2. Fundamentals of JavaScript

[Link] 2.1 Variables and Data Types

[Link].1 Variable Declaration JavaScript provides three keywords for declaring variables: 1. let: - Block-
scoped (limited to the block {} where defined). - Can be reassigned but not redeclared in the same scope. -
Example: javascript let age = 25; age = 26; // Valid reassignment 2. const: -
Block-scoped. - Cannot be reassigned or redeclared after initialization. - Example: javascript const
PI = 3.14; PI = 3.14159; // Error: Assignment to constant variable 3. var (legacy, avoid in
modern JS): - Function-scoped (hoisted to the top of its function). - Can be reassigned and redeclared. - Example:
javascript var name = "Alice"; var name = "Bob"; // Valid redeclaration

[Link].2 Data Types JavaScript is dynamically typed (types are checked at runtime). Primary data types: 1.
Primitive Types: - String: Textual data ("Hello"). - Number: Numeric values (42, 3.14). - Boolean: Logical
values (true, false). - Undefined: Uninitialized variables (let x;). - Null: Intentional absence of value (let
y = null;). - Symbol (ES6): Unique identifiers. - BigInt (ES11): Large integers beyond Number limits. 2.
Non-Primitive (Reference) Types: - Object: Key-value pairs ({ key: "value" }). - Array: Ordered lists ([1,
2, 3]). - Function: Reusable code blocks.

[Link] 2.2 Functions Functions encapsulate reusable logic. Two syntaxes: 1. Function Declaration:
javascript function greet(name) { return `Hello, ${name}!`; } 2. Arrow Function
(ES6): - More concise syntax. - Does not bind its own this (lexical scoping). javascript const greet =
(name) => `Hello, ${name}!`;

[Link] 2.3 ConditionalsControl flow based on conditions: - if/else if/else: javascript if (age >=
18) { [Link]("Adult"); } else { [Link]("Minor"); } - Ternary Operator
(shorthand): javascript const status = (age >= 18) ? "Adult" : "Minor";

[Link] Repeat code blocks: 1. for Loop: javascript

2.4 Loops for (let i = 0; i < 5; i++) {
[Link](i); // 0, 1, 2, 3, 4 } 2. while Loop: javascript let i = 0; while (i <
5) { [Link](i); // 0, 1, 2, 3, 4 i++; }

[Link] 2.5 Events Events are browser actions (e.g., clicks, keypresses) that trigger JavaScript responses.
##### Event Listeners - Attach handlers to DOM elements using addEventListener: javascript docu-
[Link]("myButton").addEventListener("click", function() { alert("Button
clicked!"); }); - Breakdown: 1. [Link]("myButton"): Selects the element with

429
id="myButton". 2. .addEventListener("click", ...): Listens for a click event. 3. Anonymous function:
Executes when the event occurs (e.g., shows an alert).

9.8.3 3. Introduction to jQuery

[Link] 3.1 Core Features jQuery simplifies: 1. DOM Manipulation: Select, modify, or traverse HTML
elements. 2. Event Handling: Attach handlers with minimal code. 3. Animations: Pre-built effects (e.g., fade,
slide). 4. AJAX: Asynchronous server communication. 5. Cross-Browser Compatibility: Uniform behavior
across browsers.

[Link] 3.2 DOM Manipulation with jQuery

[Link].1 Selecting Elements

• Use CSS-style selectors with $():
$("p") // All <p> elements
$("#myDiv") // Element with id="myDiv"
$(".myClass") // Elements with class="myClass"

[Link].2 Common Methods

Method Description Example

.hide() Hides selected elements. $("p").hide();
.show() Shows hidden elements. $("#myDiv").show();
.css() Modifies CSS properties. $(".myClass").css("color",
"red");
.text() Gets/sets text content. $("h1").text("New Heading");
.html() Gets/sets HTML content. $("#container").html("<b>Bold</b>");

[Link].3 Document Ready Ensures code runs after the DOM is fully loaded:
$(document).ready(function() {
// Code here executes after DOM is ready
});

• Shorthand:
$(function() {
// Same as above
});

[Link] 3.3 Event Handling Attach handlers using jQuery methods:

$("#myButton").click(function() {
alert("Button clicked via jQuery!");
});

• Common Event Methods:

– .click(): Mouse click.
– .focus(): Input field focus.

430
 – .blur(): Input field loses focus.
– .submit(): Form submission.

[Link] 3.4 Animations Pre-built effects for smooth transitions:

$("#myDiv").fadeOut(); // Fade out
$("#myDiv").slideUp(); // Slide up
$("#myDiv").animate({ // Custom animation
opacity: 0.5,
height: "100px"
}, 1000);

[Link] 3.5 DOM Manipulation (Dynamic Content) Add/remove elements dynamically:

// Append a new list item
$("#myList").append("<li>New Item</li>");

// Remove the last list item

$("#myList li:last").remove();

[Link] 3.6 AJAX with jQuery Simplify asynchronous requests:

$.ajax({
url: "[Link]
method: "GET",
success: function(response) {
[Link]("Data received:", response);
},
error: function(xhr, status, error) {
[Link]("Error:", error);
}
});

• Key Parameters:
– url: Endpoint to fetch data from.
– method: HTTP method (GET, POST, etc.).
– success: Callback for successful responses.
– error: Callback for failures.

9.8.4 4. Benefits of JavaScript and jQuery

[Link] 4.1 JavaScript Benefits
• Interactivity: Enables dynamic content (e.g., form validation, real-time updates).
• Performance: Client-side execution reduces server load.
• Versatility: Works with backends ([Link]) and frontends (React, Angular).

[Link] 4.2 jQuery Benefits

1. Simplicity:
• Reduces boilerplate code (e.g., [Link] → $("#id")).
2. Cross-Browser Compatibility:

431
 • Handles browser inconsistencies automatically.
3. Efficiency:
• Optimized methods for common tasks (e.g., AJAX, animations).
4. Community Support:
• Extensive documentation, plugins, and Stack Overflow resources.
5. Performance:
• Lightweight (~30KB minified) with fast DOM traversal.

9.8.5 5. Setting Up jQuery in a Project

[Link] 5.1 Using a Content Delivery Network (CDN) Include jQuery via a CDN in your HTML <head> or
before closing </body>:
<script src="[Link]

• Advantages:
– Faster load times (cached by browsers).
– No local file management.

[Link] 5.2 Local Setup

1. Download jQuery:
• Get the latest version from [Link].
2. Place in Project:
• Save the file (e.g., [Link]) in your project directory (e.g., /js).
3. Include in HTML:
<script src="js/[Link]"></script>

[Link] 5.3 Verification Test jQuery is loaded:

$(document).ready(function() {
alert("jQuery is working!");
});

9.8.6 6. Summary of Key Concepts

Topic JavaScript jQuery

Purpose Core language for interactivity. Library to simplify JS tasks.
DOM Selection [Link]() $("#id")
Event Handling addEventListener() .click(), .focus()
AJAX fetch() or XMLHttpRequest $.ajax()
Animations Custom JS/CSS .fadeOut(), .slideUp()
Setup Native to browsers. CDN or local file inclusion.

9.8.7 7. Practical Examples

[Link] Example 1: Button Click Handler (JavaScript vs. jQuery)

432
[Link]("myButton").addEventListener("click", function() {
alert("Button clicked!");
});

[Link].1 JavaScript

$("#myButton").click(function() {
alert("Button clicked!");
});

[Link].2 jQuery

[Link] Example 2: Hiding/Showing Elements

[Link]("myDiv").[Link] = "none"; // Hide

[Link]("myDiv").[Link] = "block"; // Show

[Link].1 JavaScript

$("#myDiv").hide(); // Hide
$("#myDiv").show(); // Show

[Link].2 jQuery

[Link] Example 3: AJAX Request

fetch("[Link]
.then(response => [Link]())
.then(data => [Link](data))
.catch(error => [Link](error));

[Link].1 JavaScript (Fetch API)

$.get("[Link] function(data) {
[Link](data);
}).fail(function(error) {
[Link](error);
});

[Link].2 jQuery

433
9.9 Introduction to JSP (JavaServer Pages)
9.9.1 1. Overview of JSP (JavaServer Pages)
[Link] 1.1 Definition
• JavaServer Pages (JSP) is a server-side technology that enables the creation of dynamic web content
using Java.
• It allows developers to embed Java code directly into HTML pages, facilitating the generation of dynamic
content.

[Link] 1.2 Key Benefits of JSP

• Simplifies Development Process:
– Combines HTML (presentation layer) with Java (business logic) in a single file.
– Reduces the need for separate servlet coding for simple dynamic content.
• Separation of Concerns:
– Encourages modularity by separating presentation logic (HTML) from business logic (Java).
– Improves maintainability and scalability of web applications.
• Reusability:
– Supports custom tags and tag libraries (e.g., JSTL) to avoid repetitive code.
• Integration with Servlets:
– JSP pages are compiled into servlets at runtime, leveraging the power of Java servlets.
• Implicit Objects:
– Provides predefined objects (e.g., request, response, session) for easy access to web elements.

9.9.2 2. JSP Implicit Objects

[Link] 2.1 Definition
• Implicit objects are predefined variables available in JSP that simplify access to web-related components
without explicit declaration.

[Link] 2.2 Common Implicit Objects

Object Description
request Represents the HTTP request from the client. Used to access parameters, headers, and
attributes.
response Represents the HTTP response sent back to the client. Used to set headers, cookies, and
status codes.
session Maintains user-specific data across multiple requests (e.g., login sessions).
out Used to write output to the client (similar to PrintWriter in servlets).
application Represents the web application context (shared across all users).
pageContext Provides access to other implicit objects and JSP-specific features.
config Represents the servlet configuration (similar to ServletConfig).
page Refers to the current JSP page instance (equivalent to this in Java).
exception Available in error pages to handle exceptions.

[Link] 2.3 Use Cases

• request: Retrieve form data ([Link]("username")).

434
 • response: Redirect users ([Link]("[Link]")).
• session: Store user data ([Link]("user", userObject)).
• out: Dynamically generate HTML ([Link]("<h1>Hello</h1>")).

9.9.3 3. JSP Tags

JSP uses three types of tags to embed Java functionality into HTML:

[Link] 3.1 Directive Tags

• Purpose: Define page-level settings and include external resources.
• Syntax: <%@ directive attribute="value" %>
• Common Directives:
– page: Sets page-specific attributes (e.g., language, content type).
<%@ page language="java" contentType="text/html; charset=UTF-8" %>
– include: Includes content from another file at compile-time.
<%@ include file="[Link]" %>
– taglib: Declares a tag library (e.g., JSTL).
<%@ taglib prefix="c" uri="[Link] %>

[Link] 3.2 Scripting Tags

• Purpose: Embed Java code directly in JSP.
• Types:
– Scriptlets (<% ... %>): Contains Java statements.
<% String name = "John"; %>
– Expressions (<%= ... %>): Evaluates and outputs a value.
<%= new [Link]() %>
– Declarations (<%! ... %>): Declares methods/variables (global to the JSP).
<%! int count = 0; %>

[Link] 3.3 Action Tags

• Purpose: Perform runtime actions (e.g., forwarding, including dynamic content).
• Syntax: <jsp:action />
• Common Actions:
– jsp:forward: Redirects to another resource.
<jsp:forward page="[Link]" />
– jsp:include: Includes content from another resource at runtime.
<jsp:include page="[Link]" />
– jsp:useBean: Instantiates a JavaBean.
<jsp:useBean id="user" class="[Link]" />

435
9.9.4 4. JavaServer Pages Standard Tag Library (JSTL)
[Link] 4.1 Overview
• JSTL is a collection of custom tags that simplify common JSP tasks (e.g., iteration, conditional logic,
formatting).
• Reduces Java code in JSP, improving readability and maintainability.

[Link] 4.2 JSTL Core Tags

Tag Description Example

c:out Outputs a value (escapes HTML by default). <c:out value="${[Link]}" />
c:set Sets a variable in a given scope. <c:set var="message" value="Hello"
scope="session" />
c:if Conditional execution. <c:if
test="${[Link]}">Welcome!</c:if>
c:choose Multi-way conditional (like switch). <c:choose><c:when test="${role ==
'admin'}">...</c:when></c:choose>
c:forEach Iterates over a collection. <c:forEach var="item"
items="${[Link]}">${[Link]}</c:forEach>

[Link] 4.3 Other JSTL Libraries

Library Prefix Purpose

Formatting fmt Formats dates, numbers, and internationalization.
SQL sql Executes SQL queries (not recommended for production).
XML x Processes XML data.
Functions fn Provides string manipulation functions (e.g., fn:length()).

<%@ taglib prefix="c" uri="[Link] %>

<ul>
<c:forEach var="item" items="${productList}">
<li>${[Link]} - $${[Link]}</li>
</c:forEach>
</ul>

[Link] 4.4 Example: Iterating with c:forEach

9.9.5 5. Setting Up JSP in a Spring Boot Application

[Link] 5.1 Prerequisites
• Spring Boot project (created via [Link]).
• Maven (for dependency management).

[Link] 5.2 Step-by-Step Setup

436
<dependencies>
<!-- Spring Boot Web Starter -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- Tomcat Jasper (JSP Support) -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>tomcat-embed-jasper</artifactId>
<scope>provided</scope>
</dependency>
<!-- JSTL (Standard Tag Library) -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>jstl</artifactId>
</dependency>
</dependencies>

[Link].1 5.2.1 Add Dependencies ([Link])

# JSP View Resolver Configuration

[Link]=/WEB-INF/views/
[Link]=.jsp

[Link].2 5.2.2 Configure [Link]

[Link].3 5.2.3 Directory Structure

src/
├── main/
│ ├── java/ # Java source code
│ ├── resources/ # Static resources (CSS, JS)
│ └── webapp/ # Web application root
│ └── WEB-INF/
│ └── views/ # JSP files
│ ├── [Link]
│ └── ...

<%@ page language="java" contentType="text/html; charset=UTF-8" %>

<html>
<head>
<title>JSP Example</title>
</head>
<body>
<h1>Welcome to JSP!</h1>

437
 <p>Current time: <%= new [Link]() %></p>
</body>
</html>

[Link].4 5.2.4 Create a Basic JSP Page ([Link])

[Link].5 5.2.5 Run the Application

1. Execute the main method in [Link].
2. Open a browser and navigate to:
[Link]

9.9.6 6. JSP Execution Flow

1. Client Request:
• User requests a JSP page (e.g., [Link]).
2. Translation:
• The JSP engine converts the JSP into a servlet (.java file).
3. Compilation:
• The servlet is compiled into bytecode (.class file).
4. Execution:
• The servlet executes, generating dynamic HTML.
5. Response:
• The HTML response is sent back to the client’s browser.
Key Point: - The translation and compilation happen only once (unless the JSP is modified). - Subsequent
requests reuse the compiled servlet, improving performance.

9.9.7 7. Example: Using JSTL to Iterate Over a Collection

@Controller
public class ProductController {
@GetMapping("/products")
public String showProducts(Model model) {
List<Product> products = [Link](
new Product("Laptop", 999.99),
new Product("Phone", 699.99)
);
[Link]("products", products);
return "products";
}
}

[Link] 7.1 Controller (Spring Boot)

<%@ taglib prefix="c" uri="[Link] %>

<html>
<body>

438
 <h1>Product List</h1>
<ul>
<c:forEach var="product" items="${products}">
<li>${[Link]} - $${[Link]}</li>
</c:forEach>
</ul>
</body>
</html>

[Link] 7.2 JSP Page ([Link])

<ul>
<li>Laptop - $999.99</li>
<li>Phone - $699.99</li>
</ul>

[Link] 7.3 Output

9.9.8 8. Summary of Key Concepts

1. JSP enables dynamic web content by embedding Java in HTML.
2. Implicit objects (request, response, session) simplify web development.
3. JSP tags (directive, scripting, action) control page behavior.
4. JSTL reduces Java code in JSP with standardized tags (e.g., c:forEach).
5. Spring Boot + JSP requires:
• Dependencies: spring-boot-starter-web, tomcat-embed-jasper, jstl.
• Configuration: [Link] and suffix in [Link].
• Directory structure: WEB-INF/views/ for JSP files.
6. Execution flow: JSP → Servlet → Bytecode → HTML Response.
7. Best Practices:
• Use JSTL instead of scriptlets for cleaner code.
• Separate business logic (Java) from presentation (JSP).
• Avoid SQL tags in production (use Spring Data JPA instead).

9.10 Validating Forms with JavaScript and JQuery

9.10.1 Introduction to Form Validation
[Link] Learning Objectives By the end of this lecture, learners will be able to: - Understand the principles
of form validation using JavaScript and JQuery. - Create validation rules for HTML forms. - Implement
client-side validation using JQuery.

[Link] Importance of Form Validation Form validation ensures that user input meets specific criteria be-
fore form submission. Key benefits include: - Error prevention: Reduces incorrect or malformed data submis-
sions. - Enhanced security: Mitigates risks such as SQL injection or invalid data processing. - Improved user
experience (UX): Provides immediate feedback, reducing frustration and submission errors.

[Link] Client-Side vs. Server-Side Validation

• Client-side validation occurs in the user’s browser before data is sent to the server.

439
 – Advantages:
* Immediate feedback (users see errors without waiting for server response).
* Reduces server load (invalid submissions are caught early).
* Enhances data integrity (only properly formatted data is submitted).
– Limitations:
* Can be bypassed (malicious users may disable JavaScript).
* Should always be supplemented with server-side validation.
• Server-side validation occurs on the server after submission.
– Essential for security and data integrity.
– Slower due to round-trip communication.

9.10.2 Key Concepts and Terminology

[Link] Document Object Model (DOM)
• A programming interface for HTML and XML documents.
• Represents the structure of a webpage as a tree of objects (nodes).
• Allows JavaScript/JQuery to access, modify, and manipulate HTML elements dynamically.

[Link] Event Handling

• Events are user actions (e.g., clicks, form submissions, keystrokes).
• Event handlers are functions that respond to events.
• Example: Preventing form submission if validation fails.

9.10.3 Establishing Validation Rules

Validation rules define what constitutes valid input for each form field. Common rules include: - Non-empty
fields (required input). - Format constraints (e.g., email, phone number, ISBN). - Length requirements (e.g.,
password minimum length). - Logical constraints (e.g., publication date cannot be in the future).

[Link] Example: Registration Form Validation Rules Consider a registration form with the following
fields: 1. Username - Must be alphanumeric (letters and numbers only). - Must not be empty. 2. Password -
Must be at least 8 characters long. 3. Email - Must follow a standard email format (e.g., user@[Link]).

[Link] Example: Book Submission Form Validation Rules Consider a book submission form with the
following fields: 1. Title - Must not be empty. 2. Author - Must not be empty. 3. ISBN (International
Standard Book Number) - Must be a 13-digit number (modern ISBN standard). 4. Publication Date - Must not
be a future date.

9.10.4 HTML Form Structure

<form id="registrationForm">
<label for="username">Username:</label>
<input type="text" id="username" name="username" required>

<label for="password">Password:</label>
<input type="password" id="password" name="password" required>

<label for="email">Email:</label>

440
 <input type="email" id="email" name="email" required>

<button type="submit">Register</button>
</form>

[Link] Registration Form Example

• Key Attributes:
– required: Ensures the field is not empty (basic HTML5 validation).
– type="email": Provides built-in email format validation (but additional JavaScript validation is
recommended).

<form id="bookForm">
<label for="title">Title:</label>
<input type="text" id="title" name="title" required>

<label for="author">Author:</label>
<input type="text" id="author" name="author" required>

<label for="isbn">ISBN:</label>
<input type="text" id="isbn" name="isbn" required>

<label for="publicationDate">Publication Date:</label>

<input type="date" id="publicationDate" name="publicationDate" required>

<button type="submit">Submit</button>
</form>

[Link] Book Submission Form Example

• Notes:
– required ensures basic non-empty validation.
– Additional JavaScript/JQuery validation is needed for ISBN format and date logic.

9.10.5 Implementing Validation with JQuery

[Link] General Approach
1. Listen for the form’s submit event.
2. Prevent default submission if validation fails.
3. Check each field against validation rules.
4. Display error messages if validation fails.
5. Allow submission only if all checks pass.

$(document).ready(function() {
$('#registrationForm').submit(function(event) {
// Prevent default form submission
[Link]();

// Get form values

441
 const username = $('#username').val();
const password = $('#password').val();
const email = $('#email').val();

// Validation flags
let isValid = true;

// Username validation (alphanumeric and non-empty)

if (!username || !/^[a-zA-Z0-9]+$/.test(username)) {
$('#username').after('<span class="error">Username must be alphanumeric and non-empty.</span>')
isValid = false;
}

// Password validation (minimum 8 characters)

if (!password || [Link] < 8) {
$('#password').after('<span class="error">Password must be at least 8 characters.</span>');
isValid = false;
}

// Email validation (standard format)

if (!email || !/^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email)) {
$('#email').after('<span class="error">Please enter a valid email address.</span>');
isValid = false;
}

// Submit form if valid

if (isValid) {
[Link]();
}
});
});

[Link] Registration Form Validation with JQuery

• Key Steps:
1. [Link](): Stops the form from submitting immediately.
2. Field validation:
– Username: Checks for alphanumeric characters using a regular expression (/^[a-zA-Z0-
9]+$/).
– Password: Ensures length >= 8.
– Email: Validates format using a regex (/^[^\s@]+@[^\s@]+\.[^\s@]+$/).
3. Error handling: Appends error messages (<span class="error">) after invalid fields.
4. Conditional submission: Only submits if isValid remains true.

$(document).ready(function() {
$('#bookForm').submit(function(event) {
[Link]();

// Get form values

442
 const title = $('#title').val();
const author = $('#author').val();
const isbn = $('#isbn').val();
const publicationDate = new Date($('#publicationDate').val());
const today = new Date();

// Validation flags
let isValid = true;

// Title and author validation (non-empty)

if (!title) {
$('#title').after('<span class="error">Title is required.</span>');
isValid = false;
}

if (!author) {
$('#author').after('<span class="error">Author is required.</span>');
isValid = false;
}

// ISBN validation (13 digits)

if (!isbn || !/^\d{13}$/.test(isbn)) {
$('#isbn').after('<span class="error">ISBN must be a 13-digit number.</span>');
isValid = false;
}

// Publication date validation (not in the future)

if (publicationDate > today) {
$('#publicationDate').after('<span class="error">Publication date cannot be in the future.</spa
isValid = false;
}

// Submit form if valid

if (isValid) {
[Link]();
}
});
});

[Link] Book Submission Form Validation with JQuery

• Key Steps:
1. Date validation:
– Compares publicationDate with today to ensure it is not a future date.
2. ISBN validation:
– Uses regex (/^\d{13}$/) to enforce exactly 13 digits.
3. Error display: Appends messages dynamically.

443
9.10.6 Best Practices for Form Validation
1. Combine Client-Side and Server-Side Validation
• Client-side validation improves UX but must not replace server-side validation.
2. Provide Clear Error Messages
• Errors should be specific (e.g., “Password must be 8+ characters”) and visible (e.g., near the relevant
field).
3. Use HTML5 Attributes for Basic Validation
• required, type="email", minlength provide fallback validation if JavaScript is disabled.
4. Sanitize Input on the Server
• Even with client-side validation, always sanitize and validate data server-side to prevent security
risks.
5. Test Edge Cases
• Validate with empty inputs, invalid formats, and unexpected data types.

9.10.7 Summary of Key Takeaways

1. Form validation is critical for data integrity, security, and user experience.
2. Client-side validation (JavaScript/JQuery) provides immediate feedback but should be paired with server-
side validation.
3. Validation rules define constraints for each field (e.g., non-empty, format, length).
4. JQuery simplifies validation by:
• Listening to form events (e.g., submit).
• Preventing default submission ([Link]()).
• Dynamically checking fields and displaying errors.
5. HTML5 attributes (required, type) offer basic validation but are not sufficient alone.
6. Regular expressions (regex) are powerful for pattern-based validation (e.g., emails, ISBNs).

9.10.8 Conclusion
This lecture covered: - The importance and methods of form validation. - Creating validation rules for HTML
forms. - Implementing validation using JQuery for both registration and book submission forms. - Best prac-
tices to ensure robust, user-friendly validation.
Next Steps: - Experiment with custom validation rules (e.g., password strength meters). - Explore JQuery
validation plugins (e.g., jQuery Validation Plugin) for advanced features. - Integrate server-side validation
(e.g., PHP, [Link]) to complement client-side checks.

444
10 Module 10: Implementing the Controller Layer
10.1 Creating a Controller for the Home Page
10.1.1 1. Introduction to Controllers in Spring Boot
[Link] 1.1 Definition of a Controller
• A controller is a class in Spring Boot that:
– Handles HTTP requests (e.g., GET, POST, PUT, DELETE).
– Returns responses (e.g., HTML views, JSON data).
• It is a key component of the Model-View-Controller (MVC) design pattern in Spring Boot.

[Link] 1.2 Role of the @Controller Annotation

• The @Controller annotation is used to mark a class as a controller.
• This tells Spring Boot to:
– Manage the class as a Spring bean.
– Use it to handle web requests.

[Link] 1.3 Controller Methods

• Within a controller class, methods are defined to handle specific HTTP requests.
• Each method is mapped to a URL and determines what response to return.

10.1.2 2. Defining a Controller Method for the Homepage

[Link] 2.1 Using @GetMapping for HTTP GET Requests
• The @GetMapping annotation is used to:
– Map HTTP GET requests to a specific method.
– Specify the URL pattern that the method should handle.
• Example:
@Controller
public class HomeController {

@GetMapping("/") // Maps the root URL to this method

public String homePage() {
return "home"; // Returns the view name
}
}

– Explanation:
* @GetMapping("/") ensures that when a user visits the base URL (e.g., [Link]
this method is invoked.
* The method returns "home", which is the name of the view to be rendered.

[Link] 2.2 Returning a View Name

• The view name returned by the controller method is used by the view resolver to:
– Locate the actual JSP/HTML file that should be rendered.

445
 – Example: Returning "home" tells Spring to look for a file named [Link] (or [Link], depending
on configuration).

10.1.3 3. URL Mapping in Spring Boot

[Link] 3.1 Purpose of URL Mapping
• URL mapping links specific URLs to controller methods.
• Ensures that when a user visits a URL, the correct method is executed.

[Link] 3.2 How @GetMapping Works

• @GetMapping specifies the URL pattern a method should handle.
• Example:
@GetMapping("/home") // Maps to "[Link]
public String homePage() {
return "home";
}

– If a user visits /home, the homePage() method is called.

[Link] 3.3 Mapping the Root URL (/)

• Mapping the root URL (/) ensures that the homepage is displayed when users visit the base application
URL.
• Example:
@GetMapping("/") // Maps to "[Link]
public String homePage() {
return "home";
}

10.1.4 4. View Resolution in Spring Boot

[Link] 4.1 Role of the View Resolver
• The view resolver is responsible for:
– Locating the view file (e.g., JSP, HTML) based on the view name returned by the controller.
– Rendering the view to the user.

[Link] 4.2 Configuring the View Resolver

• Configuration is typically done in [Link] (or [Link]).
• Example properties:
[Link]=/WEB-INF/jsp/
[Link]=.jsp

– Explanation:
* [Link] defines the directory where view files are stored (/WEB-INF/jsp/).
* [Link] defines the file extension (.jsp).
* If the controller returns "home", the view resolver looks for /WEB-INF/jsp/[Link].

446
[Link] 4.3 Default View Resolution Behavior
• If no custom configuration is provided, Spring Boot uses default settings:
– Views are expected in src/main/resources/templates/.
– Supports Thymeleaf, FreeMarker, or JSP (if properly configured).

10.1.5 5. Creating the JSP File for the Homepage

[Link] 5.1 Location of JSP Files
• JSP files should be placed in:
src/main/webapp/WEB-INF/jsp/

• Example structure:
src/
└── main/
└── webapp/
└── WEB-INF/
└── jsp/
└── [Link]

<%@ page contentType="text/html;charset=UTF-8" language="java" %>

<html>
<head>
<title>Home Page</title>
</head>
<body>
<h1>Welcome to the Home Page!</h1>
</body>
</html>

[Link] 5.2 Example of a Simple [Link] File

• Key Points:
– The filename ([Link]) must match the view name returned by the controller ("home").
– Contains HTML content that will be displayed to the user.

10.1.6 6. Putting It All Together: Full Workflow

[Link] 6.1 Step-by-Step Process
1. Define the Controller Class:
• Annotate with @Controller.
• Example:
@Controller
public class HomeController {
// ...
}
2. Map the URL to a Method:
• Use @GetMapping to link a URL to a method.

447
 • Example:
@GetMapping("/")
public String homePage() {
return "home";
}
3. Configure the View Resolver (if using JSP):
• Set prefix and suffix in [Link].
4. Create the JSP File:
• Place [Link] in /WEB-INF/jsp/.
5. User Accesses the URL:
• When a user visits [Link] Spring:
– Invokes homePage().
– Resolves "home" to /WEB-INF/jsp/[Link].
– Renders the JSP file as the response.

[Link] 6.2 Visual Representation of the Flow

User Request (GET /)
↓
Spring DispatcherServlet
↓
[Link]() → returns "home"
↓
View Resolver → /WEB-INF/jsp/[Link]
↓
Rendered HTML Sent to User

10.1.7 7. Handling Different HTTP Methods

[Link] 7.1 Overview of HTTP Method Annotations

Annotation HTTP Method Purpose

@GetMapping GET Retrieve data (e.g., display a page).
@PostMapping POST Submit data (e.g., form submission).
@PutMapping PUT Update existing data.
@DeleteMapping DELETE Remove data.

@Controller
public class BookController {

// Handles GET request to fetch and display books

@GetMapping("/books")
public String getBooks() {
// Logic to retrieve books from database
return "books"; // Renders [Link]
}

// Handles POST request to add a new book

448
 @PostMapping("/books")
public String addBook() {
// Logic to save a new book to database
return "redirect:/books"; // Redirects to GET /books
}
}

[Link] 7.2 Example: BookController with Multiple Mappings

• Key Points:
– @GetMapping("/books") → Displays a list of books.
– @PostMapping("/books") → Adds a new book and redirects.
– Best Practice: Use redirects after POST to avoid duplicate submissions.

10.1.8 8. Summary of Key Concepts

[Link] 8.1 What You Learned
1. Defining a Controller:
• Use @Controller to mark a class as a Spring MVC controller.
2. Mapping URLs to Methods:
• Use @GetMapping, @PostMapping, etc., to link URLs to methods.
3. Returning Views:
• Controller methods return a view name, which the view resolver converts to a JSP/HTML file.
4. View Resolution Configuration:
• Configure [Link] to specify JSP locations and extensions.
5. JSP File Placement:
• Store JSP files in /WEB-INF/jsp/ (or src/main/resources/templates/ for Thymeleaf).
6. Handling Different HTTP Methods:
• Use appropriate annotations (@GetMapping, @PostMapping, etc.) for different request types.

[Link] 8.2 Practical Example Recap

• Controller:
@Controller
public class HomeController {
@GetMapping("/")
public String homePage() {
return "home";
}
}

• JSP File ([Link]):

<html>
<body>
<h1>Welcome to the Home Page!</h1>
</body>
</html>

• Result:
– Visiting [Link] displays the content of [Link].

449
10.2 Creating a Controller Method to Add a Book
10.2.1 Introduction
This lecture covers the process of creating a controller method to add new books in a database application. The
method handles: - Displaying a form for user input. - Processing form submissions. - Validating data. - Saving the
book to the database via the service layer. - Redirecting the user after successful submission.

10.2.2 Purpose of the Add Book Method

The addBook method serves three primary functions: 1. Displaying the Form - Shows a form where users can
enter book details. 2. Processing Submissions - Handles submitted form data. - Performs validation. - Executes
business logic (e.g., saving the book). 3. Integration with the Service Layer - Delegates the saving operation to
the service layer. - Redirects the user to another page (e.g., book list).

10.2.3 Mapping URLs to Controller Methods

To map HTTP requests to controller methods, Spring uses the following annotations:

Annotation Purpose Example

@GetMapping Handles HTTP GET requests (e.g., displaying a form). @GetMapping("/addBook")
@PostMapping Handles HTTP POST requests (e.g., processing form @PostMapping("/addBook")
submissions).

[Link] Example Mapping

• GET Request (@GetMapping)
– Returns the addBookForm view.
– Example:
@GetMapping("/addBook")
public String showAddBookForm(Model model) {
[Link]("book", new Book());
return "addBookForm";
}
• POST Request (@PostMapping)
– Processes submitted form data.
– Example:
@PostMapping("/addBook")
public String addBook(@Valid @ModelAttribute("book") Book book, BindingResult result) {
// Validation & saving logic
}

10.2.4 Business Logic of the addBook Method

The method follows a structured workflow:

[Link] 1. Displaying the Form (GET Request)

• Initializes a new Book object (empty form).
• Adds the Book object to the model for binding in the view.

450
 • Returns the view name (e.g., [Link]).
• User Interaction:
– Users enter book details (title, author, ISBN, etc.).
– The form submits data via HTTP POST.

[Link] 2. Processing Submissions (POST Request)

[Link].1 a. Data Binding

• The submitted form data is bound to a Book object using @ModelAttribute.
• Example:
@PostMapping("/addBook")
public String addBook(@ModelAttribute("book") Book book, BindingResult result) {
// ...
}

[Link].2 b. Validation
• Validation Annotations ensure data integrity:
– @Valid – Triggers validation.
– @NotNull – Ensures a field is not empty.
– @Size(min=2, max=100) – Restricts string length.
– @Pattern(regexp="...") – Enforces format (e.g., ISBN).
• BindingResult captures validation errors.
• Error Handling:
– If errors exist, the method returns the form view for corrections.
– Example:
if ([Link]()) {
return "addBookForm"; // Re-display form with errors
}

[Link].3 c. Saving the Book

• If validation passes:
1. The service layer (BookService) is called to save the book.
2. The book is persisted to the database via the repository layer.
3. The user is redirected to avoid form resubmission.

10.2.5 Integration with the Service Layer

[Link] Role of the Service Layer
• Encapsulates business logic (e.g., validation rules, transactions).
• Interacts with the repository layer for database operations.
• Example Service Method:
@Service
public class BookService {
@Autowired

451
 private BookRepository bookRepository;

public void saveBook(Book book) {

[Link](book);
}
}

[Link] Controller-Service Interaction

• The controller delegates saving to BookService.
• Example:
@PostMapping("/addBook")
public String addBook(@Valid @ModelAttribute("book") Book book, BindingResult result) {
if ([Link]()) {
return "addBookForm";
}
[Link](book); // Calls service layer
return "redirect:/books"; // Redirects to book list
}

10.2.6 Repository Layer (Data Access)

[Link] Role of Repositories
• Handles CRUD operations (Create, Read, Update, Delete).
• Extends JpaRepository (Spring Data JPA) for automatic method implementation.
• Example Repository Interface:
public interface BookRepository extends JpaRepository<Book, Long> {
// Custom query methods (if needed)
List<Book> findByAuthor(String author);
}

• Automatic Methods:
– save(), findById(), delete(), etc.

10.2.7 Redirecting After Submission

[Link] Why Redirect?
• Prevents duplicate form submissions (e.g., refreshing the page).
• Ensures users see the updated book list.

[Link] Implementation
• Use redirect: prefix in the return statement.
• Example:

452
 return "redirect:/books"; // Redirects to the book list page

10.2.8 Summary of Key Concepts

1. Controller Methods
• @GetMapping → Displays the form.
• @PostMapping → Processes submissions.
2. Data Binding & Validation
• @ModelAttribute binds form data to an object.
• @Valid triggers validation.
• BindingResult captures errors.
3. Service Layer
• Handles business logic (e.g., saving books).
• Uses repositories for database operations.
4. Repository Layer
• Extends JpaRepository for CRUD operations.
5. Redirects
• Prevents resubmission and ensures updated data display.

10.2.9 Key Takeaways

• A well-structured addBook method separates concerns:
– Presentation (form display).
– Validation (data integrity).
– Business Logic (saving via service).
– Data Access (repository operations).
• Annotations (@GetMapping, @PostMapping, @Valid, @ModelAttribute) streamline development.
• Redirects improve user experience by preventing duplicate submissions.
By following this structure, developers can create robust, maintainable database applications with proper separa-
tion of layers.

10.3 Creating a Controller Method to Delete a Book

10.3.1 Introduction to Deleting Records
• Importance of Deletion:
– Deleting outdated or unnecessary data is essential for maintaining a clean and efficient database.
– Ensures optimal performance and reduces storage overhead.
• CRUD Operations:
– Deletion is one of the four fundamental CRUD (Create, Read, Update, Delete) operations in database
management.
– Each operation plays a critical role in data lifecycle management.

10.3.2 Designing the Deletion Method

[Link] 1. Repository Layer Implementation
• JPA Repository Interface:
– Spring Data JPA provides built-in methods for CRUD operations.
– The deleteById method is used to remove a book by its unique identifier.
– Key Features:

453
 Abstracts SQL operations, allowing focus on higher-level logic.
*
Directly affects the database by executing a DELETE operation.
*
– Example Definition:
public interface BookRepository extends JpaRepository<Book, Long> {
// deleteById is inherited from JpaRepository
}

[Link] 2. Service Layer Implementation

• Business Logic Encapsulation:
– The service layer method should:
1. Verify the existence of the book before deletion.
2. Use the @Transactional annotation to manage transactions.
– Purpose of @Transactional:
* Ensures atomicity: The operation either completes fully or rolls back in case of errors.
* Maintains database consistency.
– Example Implementation:
@Service
@Transactional
public class BookService {
private final BookRepository bookRepository;

public BookService(BookRepository bookRepository) {

[Link] = bookRepository;
}

public void deleteBook(Long id) {

if () {
throw new BookNotFoundException("Book with ID " + id + " not found.");
}
[Link](id);
}
}

10.3.3 Handling HTTP Requests in the Controller

[Link] 1. Controller Method Setup
• @DeleteMapping Annotation:
– Maps HTTP DELETE requests to a specific controller method.
– Used to define the request URL and extract the book ID via @PathVariable.
• Error Handling:
– The controller should return meaningful error messages if deletion fails (e.g., book not found).
• Example Code:
@Controller
@RequestMapping("/books")
public class BookController {

454
 private final BookService bookService;

public BookController(BookService bookService) {

[Link] = bookService;
}

@DeleteMapping("/{id}")
public ResponseEntity<String> deleteBook(@PathVariable Long id) {
try {
[Link](id);
return [Link]("Book deleted successfully.");
} catch (BookNotFoundException e) {
return [Link](HttpStatus.NOT_FOUND).body([Link]());
}
}
}

10.3.4 Database Operations and Transaction Management

[Link] 1. Repository Operations
• deleteById Method:
– Executes the deletion in the database.
– Spring Data JPA handles the underlying SQL, simplifying implementation.
• Impact on Database:
– Permanently removes the record with the specified ID.

[Link] 2. Transaction Management

• Atomicity Guarantee:
– The @Transactional annotation ensures the deletion is atomic:
* Success: The book is removed, and the transaction commits.
* Failure: The transaction rolls back, preserving database integrity.
• Consistency:
– Prevents partial updates or corrupt states in the database.

[Link] 3. Error Handling

• Common Scenarios:
– Book Not Found: Return a 404 (Not Found) response.
– Database Errors: Return a 500 (Internal Server Error) with details.
• User Feedback:
– Clear error messages improve user experience and debugging.

10.3.5 Frontend Integration (JSP)

[Link] 1. Displaying Books with Delete Options
• JSP Page Structure:
– Lists all books in a table format using JSTL (JSP Standard Tag Library).
– Each row includes a Delete button.

455
 • Example JSP Code:
<table>
<thead>
<tr>
<th>ID</th>
<th>Title</th>
<th>Author</th>
<th>Action</th>
</tr>
</thead>
<tbody>
<c:forEach var="book" items="${books}">
<tr>
<td>${[Link]}</td>
<td>${[Link]}</td>
<td>${[Link]}</td>
<td>
<form action="/books/${[Link]}" method="post">
<input type="hidden" name="_method" value="DELETE" />
<button type="submit">Delete</button>
</form>
</td>
</tr>
</c:forEach>
</tbody>
</table>

[Link] 2. Simulating DELETE Requests

• HTML Form Limitation:
– Standard HTML forms only support GET and POST methods.
– Workaround: Use a hidden field (_method=DELETE) to simulate a DELETE request.
• Server-Side Handling:
– Spring MVC recognizes the _method parameter and treats the request as DELETE.

[Link] 3. User Interaction Flow

1. User clicks the Delete button in the JSP table.
2. A DELETE request is sent to the server (/books/{id}).
3. The server processes the request:
• Deletes the book if found.
• Returns a success/error response.
4. The client-side updates (e.g., refreshes the book list).

10.3.6 Summary of Key Concepts

1. Deletion in CRUD: Essential for maintaining database hygiene.
2. Repository Layer: Uses deleteById from JPA for database operations.
3. Service Layer: Encapsulates business logic and transaction management.
4. Controller Layer: Handles HTTP DELETE requests and error responses.

456
 5. Frontend Integration: JSP pages enable user interaction with delete functionality.
6. Transaction Management: Ensures atomicity and consistency.
7. Error Handling: Provides meaningful feedback for failed operations.

10.4 Creating a Controller Method to View Books

10.4.1 Introduction
• Objective: By the end of this lecture, students will be able to:
– Create a method to retrieve and display books.
– Identify the use of the service layer to fetch data.
– Examine how to bind data to the view.
• Focus: Understanding how to retrieve and display data using the controller, service, and view layers in a
Spring MVC application.

10.4.2 Role of the Controller in Spring MVC

• Definition: The controller is the entry point for handling HTTP requests in the Spring MVC framework.
• Responsibilities:
– Processes user inputs.
– Orchestrates responses.
– Maps requests to specific handler methods.
• Analogy: Acts as the “front door” of the application, managing incoming and outgoing data flows.

10.4.3 Creating the Book Controller Class

[Link] Annotations Used
1. @Controller
• Indicates that the class is a Spring MVC controller.
• Marks the class as a request-handling component.
2. @Autowired
• Used for dependency injection of the BookService.
• Ensures the controller has access to business logic services.
3. @GetMapping
• Maps HTTP GET requests to a specific method (e.g., viewBooks).
• Defines the URL path (e.g., /books) that triggers the method.

[Link] Structure of the viewBooks Method

• Purpose: Handles requests to view the list of books.
• Steps:
1. Calls getAllBooks() from the BookService to fetch the list of books from the database.
2. Adds the list to the model using [Link]("books", bookList).
– Makes the data accessible in the view layer.
3. Returns the view name ("view-books"), instructing the view resolver to render the corresponding
JSP page.

[Link] Key Components in the Controller

457
Component Description
@Controller Declares the class as a Spring MVC controller.
@Autowired BookService Injects the BookService dependency.
@GetMapping("/books") Maps GET requests for /books to viewBooks().
Model An interface that holds model attributes (data to be passed
to the view).
[Link]() Binds data (e.g., book list) to the model for view access.
Return Value ("view-books") Specifies the JSP page to be rendered.

10.4.4 Understanding the Service Layer

[Link] Role of the Service Layer
• Acts as an intermediary between the controller and repository layers.
• Encapsulates business logic and operations.
• Ensures the controller remains clean and focused on HTTP request handling.
• Promotes separation of concerns, improving maintainability and scalability.

[Link] Creating the BookService Class

• Annotations:
– @Service: Indicates the class provides business services.
– @Autowired BookRepository: Injects the data access layer (repository).
• Key Method: getAllBooks()
– Purpose: Retrieves all book records from the database.
– Implementation:
* Calls findAll() from the BookRepository.
* Acts as a bridge between the data layer and controller.
– Business Logic: Can include validation, filtering, or transformations before passing data to the
controller.

[Link] Functionality of the Service Layer

• Retrieves data from the repository.
• Performs necessary business logic (e.g., sorting, filtering).
• Passes processed data to the controller.

[Link] Key Components in the Service Layer

Component Description
@Service Marks the class as a business service provider.
@Autowired BookRepository Injects the JPA repository for database operations.
getAllBooks() Fetches all books via [Link]().
Business Logic Encapsulates rules (e.g., access control, data validation).

10.4.5 Understanding the Repository Layer

[Link] Role of the BookRepository

458
 • Extends JpaRepository<Book, Long>, providing CRUD operations (e.g., findAll(), save(),
delete()).
• Abstracts database access, allowing focus on business logic.
• Efficiently interacts with the database using JPA (Java Persistence API).

[Link] Key Features

• Predefined Methods: findAll(), findById(), save(), delete().
• Custom Queries: Can define custom methods (e.g., findByAuthor()).
• Database Agnostic: Works with any relational database (MySQL, PostgreSQL, etc.).

10.4.6 Data Binding in Spring MVC

[Link] Definition
• The process of associating data from the model with elements in the view.
• Ensures seamless data flow between the controller and view.

[Link] How It Works

1. Controller adds data to the model (e.g., [Link]("books", bookList)).
2. View (JSP) accesses the data using expression language (EL) or JSTL tags.
3. View Resolver renders the JSP page with the bound data.

[Link] Benefits
• Separation of Concerns: Keeps application logic separate from presentation logic.
• Maintainability: Changes in the view do not affect the controller or service layer.
• Reusability: The same model data can be used across multiple views.

10.4.7 Creating the View (JSP Page)

[Link] File: [Link]
• Purpose: Displays the list of books in a tabular format.

[Link] Key Components

1. Tag Libraries
• Spring Tags: <%@ taglib prefix="c" uri="[Link] %> (for
iteration).
• JSTL Core: Used for looping (<c:forEach>) and conditional logic.
2. HTML Structure
• Table Layout: Each row represents a book.
• Columns: ID, Title, Author, Category, Number of Copies.
3. Data Binding in JSP
• <c:forEach>: Iterates over the books list from the model.
<c:forEach var="book" items="${books}">
<tr>
<td>${[Link]}</td>
<td>${[Link]}</td>
<td>${[Link]}</td>

459
 <td>${[Link]}</td>
<td>${[Link]}</td>
</tr>
</c:forEach>
• Expression Language (EL): Accesses book attributes (e.g., ${[Link]}).

[Link] Benefits of This Approach

• Dynamic Rendering: Automatically updates when the model data changes.
• Clean Separation: Presentation logic (JSP) is separate from business logic (Service/Controller).
• User-Friendly: Provides a structured, readable interface for users.

10.4.8 Summary of Key Concepts

[Link] Controller Layer
• @Controller: Marks the class as a Spring MVC controller.
• @GetMapping: Maps HTTP GET requests to a method.
• Model: Holds data to be passed to the view.
• [Link](): Binds data to the model.

[Link] Service Layer

• @Service: Indicates business logic services.
• @Autowired BookRepository: Injects the repository for database operations.
• getAllBooks(): Fetches books via findAll().

[Link] Repository Layer

• JpaRepository: Provides CRUD operations.
• findAll(): Retrieves all records from the database.

[Link] View Layer (JSP)

• <c:forEach>: Iterates over the book list.
• ** ${books}**: Accesses model data in JSP.
• Table Structure: Displays book attributes in rows/columns.

[Link] Data Flow

1. Request → Controller (@GetMapping).
2. Controller → Service (getAllBooks()).
3. Service → Repository (findAll()).
4. Repository → Database (fetches data).
5. Database → Repository → Service → Controller.
6. Controller → Model (addAttribute).
7. Model → View (JSP) (renders data).

10.4.9 Conclusion
• Learned How To:
– Create a controller method to retrieve and display books.

460
 – Use the service layer to fetch data from the repository.
– Bind data to the view using Spring MVC’s model and JSP.
• Key Takeaways:
– Separation of concerns improves maintainability.
– Spring MVC simplifies data binding between layers.
– JPA Repository abstracts database operations, reducing boilerplate code.

10.5 Displaying Validation Errors

10.5.1 Introduction to Form Validation in Spring MVC
• Purpose of Form Validation:
– Ensures integrity and security of user input.
– Acts as a gatekeeper to verify that submitted data meets application requirements before processing.
• Client-Side vs. Server-Side Validation:
– Client-side validation (e.g., JavaScript) improves user experience by providing immediate feedback.
– Server-side validation is essential because:
* Protects against users bypassing client-side checks (e.g., disabling JavaScript).
* Ensures data consistency and security at the backend.
• Spring Boot Validation Framework:
– Uses the Java Bean Validation (JSR 380) specification.
– Provides annotations and APIs to define validation rules declaratively in entity classes.

10.5.2 Key Validation Annotations in Spring MVC

Validation annotations are applied to fields in entity classes to enforce constraints.

[Link] 1. @NotBlank
• Purpose: Ensures a field is not null and not empty (trims whitespace).
• Use Case: Required for fields like usernames, emails, or any mandatory text input.
• Example:
@NotBlank(message = "Title is required")
private String title;

– If submitted blank, displays: “Title is required”.

[Link] 2. @Size
• Purpose: Specifies minimum and maximum length for a string field.
• Use Case: Validating passwords, usernames, or descriptions.
• Example:
@Size(min = 1, message = "At least one copy is required")
private int numberOfCopies;

– Ensures numberOfCopies is >= 1.

461
[Link] 3. @Min and @Max
• Purpose: Sets lower (@Min) and upper (@Max) bounds for numeric fields.
• Use Case: Age, quantity, or price validation.
• Example:
@Min(value = 0, message = "Price cannot be negative")
private double price;

[Link] 4. @Email
• Purpose: Validates that a string field contains a well-formed email address.
• Use Case: Email input fields.
• Example:
@Email(message = "Invalid email format")
private String email;

[Link] Advantages of Annotations

• Declarative: Rules are defined directly in the model class.
• Reusable: Annotations can be applied across multiple fields/classes.
• Readable: Makes validation logic explicit and easy to maintain.

10.5.3 Creating a Form in JSP for User Input

The JSP form binds input fields to the model attributes and includes error message placeholders.

[Link] Form Structure

• Field Binding:
– Each input field (e.g., title, author) is bound to a corresponding attribute in the Book model.
– Example:
<input type="text" name="title" value="${[Link]}" />
• Error Message Placeholders:
– Errors are displayed adjacent to the relevant field using Spring’s <form:errors> tag.
– Example:
<form:input path="title" />
<form:errors path="title" cssClass="error" />

* If title is blank, the error “Title is required” appears next to the field.

[Link] Purpose of Error Feedback

• Immediate Guidance: Helps users correct mistakes without submitting the form repeatedly.
• User Experience: Reduces frustration by clearly indicating what went wrong.

10.5.4 Handling Form Submissions in the Controller

The controller validates the submitted data and processes errors or proceeds with business logic.

462
[Link] Key Components
1. @Valid Annotation:
• Triggers validation for the submitted object (e.g., Book).
• Example:
@PostMapping("/addBook")
public String addBook(@Valid @ModelAttribute("book") Book book, BindingResult bindingResult)
// ...
}
2. BindingResult Interface:
• Captures validation errors after @Valid is processed.
• Must be declared immediately after the validated object.
3. Error Handling Flow:
• If errors exist:
– Return to the form view with error messages.
– Example:
if ([Link]()) {
return "bookForm"; // Re-renders the form with errors
}
• If no errors:
– Proceed with business logic (e.g., saving to database).
– Example:
[Link](book);
return "redirect:/success";

[Link] Why This Approach?

• Separation of Concerns: Validation logic is decoupled from business logic.
• Consistency: Ensures all submissions are validated before processing.

10.5.5 Displaying Validation Errors in JSP

Errors are retrieved from the BindingResult and rendered in the JSP to provide user feedback.

[Link] Mechanism
1. Error Retrieval:
• Spring automatically populates the BindingResult with validation errors.
2. Error Display:
• Use <form:errors> to render messages next to the corresponding fields.
• Example:
<form:form modelAttribute="book" action="addBook" method="post">
<label>Title:</label>
<form:input path="title" />
<form:errors path="title" cssClass="error" /><br>

<label>Author:</label>
<form:input path="author" />

463
 <form:errors path="author" cssClass="error" /><br>
</form:form>
3. Styling Errors:
• Apply CSS classes (e.g., error) to highlight errors visually.

[Link] User Experience Benefits

• Contextual Feedback: Errors appear next to the problematic field, not as a generic alert.
• Actionable: Users know exactly what to fix (e.g., “Author is required”).

10.5.6 Best Practices for Server-Side Validation

1. Always Implement Server-Side Validation:
• Client-side validation can be bypassed (e.g., via API tools or disabled JavaScript).
• Server-side validation is the final defense against invalid data.
2. Combine with Client-Side Validation:
• Use both for a robust and user-friendly experience.
• Example: JavaScript for instant feedback + Spring Validation for security.
3. Clear Error Messages:
• Messages should be specific (e.g., “Password must be at least 8 characters”) and helpful.
4. Consistent Validation Rules:
• Ensure rules are uniform across client and server sides to avoid conflicts.

10.5.7 Summary of Key Concepts

Concept Description
Server-Side Validation Critical for security; prevents invalid/malicious data from being processed.
Java Bean Validation Standard (JSR 380) for declarative validation using annotations.
@Valid Triggers validation in the controller.
BindingResult Captures validation errors for display.
<form:errors> JSP tag to render error messages next to fields.
Best Practice Always validate on the server, even with client-side checks.

10.5.8 Conclusion
• Server-side validation is non-negotiable for data integrity and security.
• Spring MVC simplifies validation with annotations (@NotBlank, @Size, etc.) and BindingResult.
• JSP forms bind fields to model attributes and display errors contextually.
• Effective error handling improves user experience by providing clear, actionable feedback.
By implementing these techniques, applications can ensure robust data validation while maintaining a smooth
user workflow.

10.6 Handling Exceptions in Controllers

10.6.1 1. Introduction to Exception Handling in Spring MVC
[Link] 1.1 Definition and Importance
• Exception handling in Spring MVC refers to the mechanisms used to manage runtime errors gracefully,
ensuring application stability and improving user experience.

464
 • Key Objectives:
– Prevent application crashes due to unhandled exceptions.
– Provide meaningful feedback to users.
– Maintain application robustness and maintainability.

[Link] 1.2 Core Mechanisms in Spring MVC Spring MVC offers multiple approaches to handle exceptions:
1. @ControllerAdvice – Global exception handling across the entire application. 2. @ExceptionHandler –
Handles exceptions within specific controllers. 3. ResponseStatusException – Flexible handling of HTTP
status codes and custom error messages. 4. Custom Error Pages – Configurable error pages for different HTTP
status codes.

10.6.2 2. Global Exception Handling with @ControllerAdvice

[Link] 2.1 Definition and Purpose
• @ControllerAdvice is a global exception handler that centralizes exception management for the entire
application.
• Advantages:
– Centralization: Avoids repetitive exception-handling code in multiple controllers.
– Separation of Concerns: Isolates error-handling logic from business logic.
– Maintainability: Easier to update and manage exception responses.

[Link] 2.2 Implementation Steps

1. Define a class annotated with @ControllerAdvice:
@ControllerAdvice
public class GlobalExceptionHandler {
// Exception-handling methods go here
}

2. Use @ExceptionHandler to handle specific exceptions:

@ExceptionHandler([Link])
public ResponseEntity<ErrorResponse> handleResourceNotFound(ResourceNotFoundException ex) {
ErrorResponse error = new ErrorResponse("Resource Not Found", [Link]());
return new ResponseEntity<>(error, HttpStatus.NOT_FOUND);
}

3. Handle general exceptions (fallback):

@ExceptionHandler([Link])
public ResponseEntity<ErrorResponse> handleGeneralException(Exception ex) {
ErrorResponse error = new ErrorResponse("Internal Server Error", [Link]());
return new ResponseEntity<>(error, HttpStatus.INTERNAL_SERVER_ERROR);
}

@ControllerAdvice
public class GlobalExceptionHandler {

@ExceptionHandler([Link])
public ResponseEntity<ErrorResponse> handleResourceNotFound(ResourceNotFoundException ex) {

465
 ErrorResponse error = new ErrorResponse("Resource Not Found", [Link]());
return new ResponseEntity<>(error, HttpStatus.NOT_FOUND);
}

@ExceptionHandler([Link])
public ResponseEntity<ErrorResponse> handleAllExceptions(Exception ex) {
ErrorResponse error = new ErrorResponse("Internal Server Error", "An unexpected error occurre
return new ResponseEntity<>(error, HttpStatus.INTERNAL_SERVER_ERROR);
}
}

[Link] 2.3 Example: Global Exception Handler

[Link] 2.4 Benefits

• Consistency: Uniform error responses across the application.
• Reusability: Single handler for multiple controllers.
• Scalability: Easy to add new exception-handling logic.

10.6.3 3. Custom Exception Handlers

[Link] 3.1 Definition and Need
• Custom exceptions allow developers to define application-specific errors (e.g., ResourceNotFoundExcep-
tion, InvalidInputException).
• Purpose:
– Provide meaningful error messages tailored to business logic.
– Improve debugging with structured error responses.

[Link] 3.2 Implementation Steps

1. Define a custom exception class (extend RuntimeException or Exception):
public class ResourceNotFoundException extends RuntimeException {
public ResourceNotFoundException(String message) {
super(message);
}
}

2. Create an ErrorResponse class to structure error messages:

public class ErrorResponse {
private String error;
private String message;

// Constructor, getters, and setters

public ErrorResponse(String error, String message) {
[Link] = error;
[Link] = message;
}
// Getters and setters...
}

466
@Service
public class ProductService {
public Product getProductById(Long id) {
Product product = [Link](id)
.orElseThrow(() -> new ResourceNotFoundException("Product not found with id: " + id));
return product;
}
}

[Link] 3.3 Example: Custom Exception Usage

[Link] 3.4 Advantages

• Clarity: Exceptions are self-documenting (e.g., ResourceNotFoundException clearly indicates the issue).
• Consistency: Standardized error responses across the application.
• Extensibility: Easy to add new custom exceptions as needed.

10.6.4 4. Controller-Level Exception Handling with @ExceptionHandler

[Link] 4.1 Definition and Use Case
• @ExceptionHandler within a controller handles exceptions specific to that controller.
• Use Case:
– When exceptions are controller-specific and not applicable globally.
– When different controllers require different error responses for the same exception.

[Link] 4.2 Implementation Steps

1. Define an exception-handling method inside the controller:
@Controller
public class ProductController {

@GetMapping("/products/{id}")
public String getProduct(@PathVariable Long id, Model model) {
try {
Product product = [Link](id);
[Link]("product", product);
return "product-details";
} catch (ResourceNotFoundException ex) {
[Link]("error", [Link]());
return "error-page";
}
}

@ExceptionHandler([Link])
public String handleResourceNotFound(ResourceNotFoundException ex, Model model) {
[Link]("error", [Link]());
return "custom-error";
}
}

467
@Controller
@RequestMapping("/products")
public class ProductController {

@Autowired
private ProductService productService;

@GetMapping("/{id}")
public String viewProduct(@PathVariable Long id, Model model) {
Product product = [Link](id);
[Link]("product", product);
return "product-page";
}

@ExceptionHandler([Link])
public String handleNotFound(ResourceNotFoundException ex, Model model) {
[Link]("errorMessage", [Link]());
return "error/not-found";
}
}

[Link] 4.3 Example: Handling Exceptions in a Controller

[Link] 4.4 When to Use Controller-Level Handling

• Controller-specific logic: When errors are unique to a controller’s functionality.
• Different error views: When different controllers require different error pages.
• Avoiding global overhead: When exceptions are not relevant to the entire application.

10.6.5 5. Handling Validation Errors

[Link] 5.1 Importance of Validation
• Ensures user input meets application constraints (e.g., non-null fields, valid formats).
• Prevents invalid data from corrupting the system.

[Link] 5.2 Spring MVC Validation Mechanisms

1. @Valid Annotation:
• Applied to method parameters to trigger validation.
• Works with JSR-303 annotations (e.g., @NotNull, @Size).
2. BindingResult:
• Captures validation errors.
• Allows checking for errors before processing.

[Link] **5.3 Implementation Steps

1. Annotate the model with validation constraints:
public class Product {
@NotNull(message = "Name cannot be null")

468
 @Size(min = 2, max = 50, message = "Name must be between 2 and 50 characters")
private String name;

@Min(value = 0, message = "Price must be positive")

private BigDecimal price;
// Getters and setters...
}

2. Use @Valid and BindingResult in the controller:

@PostMapping("/products")
public String addProduct(@Valid @ModelAttribute("product") Product product,
BindingResult bindingResult) {
if ([Link]()) {
return "product-form"; // Return to form with errors
}
[Link](product);
return "redirect:/products";
}

@Controller
public class ProductController {

@GetMapping("/add-product")
public String showAddProductForm(Model model) {
[Link]("product", new Product());
return "product-form";
}

@PostMapping("/add-product")
public String addProduct(@Valid @ModelAttribute("product") Product product,
BindingResult bindingResult) {
if ([Link]()) {
return "product-form"; // Re-render form with errors
}
[Link](product);
return "redirect:/products/success";
}
}

[Link] **5.4 Example: Validation Error Handling

<form th:action="@{/add-product}" th:object="${product}" method="post">

<div>
<label>Name:</label>
<input type="text" th:field="*{name}" />
<span th:if="${#[Link]('name')}" th:errors="*{name}" style="color: red;"></span>
</div>

469
 <div>
<label>Price:</label>
<input type="number" th:field="*{price}" />
<span th:if="${#[Link]('price')}" th:errors="*{price}" style="color: red;"></span>
</div>
<button type="submit">Submit</button>
</form>

[Link] **5.5 Displaying Errors in Thymeleaf

10.6.6 6. Custom Error Pages for HTTP Status Codes

[Link] 6.1 Purpose
• Replace default Spring error pages with user-friendly custom pages.
• Improve user experience by providing clear error messages.

[Link] 6.2 Implementation Steps

1. Disable default error handling in [Link]:
[Link]=false

2. Define custom error paths:

[Link]=/error

3. Create a custom error controller:

@Controller
public class CustomErrorController {

@RequestMapping("/error")
public String handleError(HttpServletRequest request, Model model) {
Object status = [Link](RequestDispatcher.ERROR_STATUS_CODE);
if (status != null) {
int statusCode = [Link]([Link]());
[Link]("errorCode", statusCode);
[Link]("errorMessage", getErrorMessage(statusCode));
}
return "error";
}

private String getErrorMessage(int statusCode) {

switch (statusCode) {
case 404: return "Page Not Found";
case 500: return "Internal Server Error";
default: return "An Error Occurred";
}
}
}

4. Create a Thymeleaf error template ([Link]):

470
 <!DOCTYPE html>
<html xmlns:th="[Link]
<head>
<title>Error</title>
</head>
<body>
<h1 th:text="${errorCode}">Error</h1>
<p th:text="${errorMessage}">An error occurred.</p>
</body>
</html>

[Link] **6.3 Example: Custom 404 and 500 Pages

• [Link] (for “Not Found” errors):
<h1>404 - Page Not Found</h1>
<p>The page you requested does not exist.</p>

• [Link] (for “Internal Server Error”):

<h1>500 - Internal Server Error</h1>
<p>Something went wrong on our end. Please try again later.</p>

10.6.7 7. Best Practices for Exception Handling

[Link] 7.1 Use Meaningful Error Messages
• Do:
– Provide clear, actionable messages (e.g., “Product ID 123 not found”).
– Avoid technical jargon for end-users.
• Don’t:
– Expose sensitive information (e.g., stack traces in production).

[Link] 7.2 Log Exceptions for Debugging

• Use logging frameworks (e.g., SLF4J, Log4j) to record exceptions:
@ExceptionHandler([Link])
public ResponseEntity<ErrorResponse> handleException(Exception ex, WebRequest request) {
[Link]("Error occurred: ", ex); // Log the full exception
ErrorResponse error = new ErrorResponse("Internal Server Error", "An unexpected error occurr
return new ResponseEntity<>(error, HttpStatus.INTERNAL_SERVER_ERROR);
}

[Link] 7.3 Handle Specific Exceptions

• Avoid generic Exception handlers where possible.
• Example:
@ExceptionHandler({
[Link],
[Link]

471
 })
public ResponseEntity<ErrorResponse> handleKnownExceptions(Exception ex) {
if (ex instanceof ResourceNotFoundException) {
return new ResponseEntity<>(new ErrorResponse("Not Found", [Link]()), HttpStatus.
} else {
return new ResponseEntity<>(new ErrorResponse("Bad Request", [Link]()), HttpStatu
}
}

[Link] 7.4 Use HTTP Status Codes Appropriately

Status Code Usage

200 OK Successful request
400 Bad Request Invalid user input
404 Not Found Resource does not exist
500 Internal Server Error Unexpected server error

[Link] 7.5 Separate Error Handling from Business Logic

• Do:
– Use @ControllerAdvice for global exceptions.
– Keep controllers clean by delegating error handling.
• Don’t:
– Mix error handling with business logic in controllers.

10.6.8 8. Summary of Key Concepts

Concept Description Example

@ControllerAdvice Global exception handler @ControllerAdvice class
with @ExceptionHandler
methods
Custom Exceptions Application-specific errors ResourceNotFoundException
extends RuntimeException
@ExceptionHandler Controller-specific exception handling @ExceptionHandler inside a
@Controller
Validation Ensures valid user input @Valid + BindingResult
Custom Error Pages User-friendly HTTP error pages [Link] with Thymeleaf
Best Practices Log errors, use meaningful messages [Link]("Error: ", ex)

10.6.9 9. Conclusion
• Exception handling is critical for robust, user-friendly applications.
• Spring MVC provides multiple mechanisms:
– Global handling (@ControllerAdvice).
– Controller-specific handling (@ExceptionHandler).
– Custom exceptions and validation.
– Custom error pages for HTTP status codes.
• Best practices ensure maintainable, secure, and efficient error management.

472
10.7 Handling Form Submissions (Add Book)
10.7.1 1. Introduction to Form Handling in Database Applications
[Link] 1.1 Learning Objectives By the end of this lecture, learners will be able to: - Create an HTML form
for adding books to a database. - Handle form submissions in a Spring Boot controller. - Save form data to a
database using the service layer and repository layer.

[Link] 1.2 Key Concepts

• Form Submission: The process of sending user-input data from an HTML form to a server.
• Controller Layer: Handles HTTP requests, processes form data, and interacts with the service layer.
• Service Layer: Contains business logic and interacts with the repository to persist data.
• Repository Layer: Provides an interface for database operations (CRUD: Create, Read, Update, Delete).

10.7.2 2. Creating an HTML Form for Adding Books

[Link] 2.1 Purpose of the Form The form captures essential book details, including: - Title - Author - Cat-
egory - Number of copies

[Link] 2.2 HTML Form Structure The form is built using HTML and includes: - Input fields for each book
attribute. - A submit button to send data to the server. - Action URL (/books/add) – The endpoint where the
form data is submitted. - HTTP Method (POST) – Ensures data is sent securely (not visible in the URL).

<form action="/books/add" method="post">

<label for="title">Title:</label>
<input type="text" id="title" name="title" required>

<label for="author">Author:</label>
<input type="text" id="author" name="author" required>

<label for="category">Category:</label>
<input type="text" id="category" name="category" required>

<label for="copies">Number of Copies:</label>

<input type="number" id="copies" name="copies" required>

<button type="submit">Add Book</button>

</form>

[Link].1 Example HTML Form Code

[Link].2 Key Attributes in the Form

Attribute Purpose
action="/books/add" Specifies the URL where the form data is sent.
method="post" Ensures data is submitted via HTTP POST
(secure, not visible in URL).
name="title" (and others) Binds input fields to form parameters for
server-side processing.

473
Attribute Purpose
required Ensures the field must be filled before
submission.

10.7.3 3. Handling Form Submissions in the Spring Boot Controller

[Link] 3.1 Role of the Controller The controller: - Displays the form when the user navigates to
/books/add. - Processes submitted data by: - Binding form parameters to a Book object. - Calling the service
layer to save the data. - Returning an appropriate response (e.g., success message, redirect).

[Link] 3.2 Controller Class Structure

@Controller
@RequestMapping("/books")
public class BookController {

private final BookService bookService;

public BookController(BookService bookService) {

[Link] = bookService;
}

// Displays the "Add Book" form

@GetMapping("/add")
public String showAddBookForm(Model model) {
[Link]("book", new Book());
return "add-book"; // Refers to the HTML template (e.g., [Link])
}

// Processes the submitted form data

@PostMapping("/add")
public String addBook(@ModelAttribute Book book) {
[Link](book);
return "redirect:/books"; // Redirects to a list of books after saving
}
}

[Link].1 Example Controller Code

[Link] 3.3 Key Annotations and Methods

Annotation/Method Purpose
@Controller Marks the class as a Spring MVC
controller.
@RequestMapping("/books") Maps all methods in this controller
to /books base path.

474
Annotation/Method Purpose
@GetMapping("/add") Handles GET requests to
/books/add (displays the form).
@PostMapping("/add") Handles POST requests to
/books/add (processes form
submission).
@ModelAttribute Binds form data to a Book object
automatically.
[Link]("book", new Book()) Adds a new Book object to the
model for the form to bind to.

[Link] 3.4 Benefits of @ModelAttribute

• Automatic Data Binding: Maps form fields (e.g., title, author) to the Book object’s properties.
• Simplifies Controller Logic: Reduces boilerplate code for manual parameter extraction.
• Encourages Clean Code: Promotes separation of concerns by delegating business logic to the service layer.

10.7.4 4. Implementing the Service Layer

[Link] 4.1 Purpose of the Service Layer
• Encapsulates business logic (e.g., validation, data processing).
• Interacts with the repository to persist data.
• Keeps the controller lean by handling complex operations separately.

[Link] 4.2 BookService Class

@Service
public class BookService {

private final BookRepository bookRepository;

public BookService(BookRepository bookRepository) {

[Link] = bookRepository;
}

// Saves a book to the database

public void saveBook(Book book) {
[Link](book);
}
}

[Link].1 Example Service Class Code

[Link].2 Key Components

475
Component Purpose
@Service Marks the class as a Spring service (manages
business logic).
BookRepository Injected dependency for database operations.
saveBook(Book book) Calls the repository’s save() method to persist
the book.

[Link] 4.3 Separation of Concerns

• Controller: Handles HTTP requests/responses.
• Service: Handles business logic (e.g., validation, transactions).
• Repository: Handles database operations (CRUD).

10.7.5 5. Saving Form Data to the Database

[Link] 5.1 Required Components
1. Book Entity: Represents the book in the application (maps to a database table).
2. Book Repository: Provides CRUD operations (extends JpaRepository).
3. Database Configuration: Defined in [Link] (e.g., database URL, credentials).

[Link] 5.2 Assumptions from Previous Modules

• The Book entity, BookRepository, and database configuration were covered in earlier sessions.
• Example entity structure (for reference):
@Entity
public class Book {
@Id
@GeneratedValue(strategy = [Link])
private Long id;
private String title;
private String author;
private String category;
private int copies;
// Getters and setters
}

• Example repository interface:

public interface BookRepository extends JpaRepository<Book, Long> {
}

[Link] 5.3 Data Flow Summary

1. User submits form → Data sent to /books/add (POST).
2. Controller receives data, binds it to a Book object via @ModelAttribute.
3. Service layer processes the Book object (e.g., validation).
4. Repository saves the Book to the database.
5. Controller redirects to a confirmation page or book list.

476
10.7.6 6. Summary of Key Steps
[Link] 6.1 Workflow Recap
1. Create the HTML Form
• Define input fields for book attributes.
• Set action="/books/add" and method="post".
2. Configure the Controller
• Use @GetMapping to display the form.
• Use @PostMapping and @ModelAttribute to process submissions.
3. Implement the Service Layer
• Define saveBook() method in BookService.
• Inject BookRepository to interact with the database.
4. Persist Data
• Ensure Book entity and BookRepository are properly configured.
• Verify database settings in [Link].

[Link] 6.2 Best Practices

• Separation of Concerns: Keep controllers thin; delegate logic to services.
• Use @ModelAttribute: Simplifies form data binding.
• Redirect After POST: Prevents duplicate submissions (Post-Redirect-Get pattern).
• Validation: Add validation in the service layer (e.g., check for empty fields).

10.7.7 7. Conclusion
[Link] 7.1 Key Takeaways
• Form Creation: HTML forms with POST method ensure secure data submission.
• Controller Handling: @ModelAttribute simplifies binding form data to objects.
• Service Layer: Encapsulates business logic and interacts with the repository.
• Database Persistence: Requires proper entity, repository, and configuration setup.

[Link] 7.2 Next Steps

• Refer to previous modules for:
– Entity and Repository creation.
– Database configuration in [Link].
• Extend the application by adding:
– Form validation (e.g., using @Valid).
– Error handling (e.g., duplicate book entries).
– Confirmation messages after successful submission.

10.8 Introduction to Spring MVC Controllers

10.8.1 1. Overview of Spring MVC Controllers
[Link] 1.1 Context in Database Application Development
• In the current stage of the project:
– A database has been designed.
– A model has been implemented to interact with and update the database.
– A view has been added to interface with the user.

477
 • The next step is developing the controller layer.
• The focus of this lecture is on how controllers handle user requests.

[Link] 1.2 Learning Objectives By the end of this lecture, students will be able to: 1. Understand MVC
controllers and their benefits. 2. Create a simple controller class. 3. Map URLs to controller methods.

10.8.2 2. Introduction to Spring MVC

[Link] 2.1 Definition
• Spring MVC is a module within the Spring Framework designed for building web applications.
• It follows the Model-View-Controller (MVC) pattern, which separates application concerns into three
distinct components:
1. Model – Represents the data and business logic.
2. View – Handles the presentation of data to users (e.g., JSP, Thymeleaf, HTML).
3. Controller – Processes user requests, interacts with the model, and returns the appropriate view to
the user.

10.8.3 3. Benefits of Spring MVC Controllers

Spring MVC controllers provide several advantages in web application development:

[Link] 3.1 Separation of Concerns

• Clearly separates:
– Business logic (Model)
– User interface (View)
– Request handling (Controller)
• Results in a cleaner, more organized codebase.

[Link] 3.2 Flexibility and Extensibility

• Supports integration with various view technologies (e.g., JSP, Thymeleaf, FreeMarker).
• Developers can choose the best-suited technology for their project.

[Link] 3.3 Ease of Testing

• Controllers handle specific responsibilities, allowing them to be tested independently from the rest of the
application.
• Facilitates unit testing and integration testing.

[Link] 3.4 Annotation-Based Configuration

• Uses annotations (e.g., @Controller, @RequestMapping, @GetMapping) to simplify configuration.
• Reduces boilerplate code and improves readability.

10.8.4 4. Creating a Simple Controller in Spring Boot

[Link] 4.1 Basic Controller Structure
• A controller class is annotated with @Controller to indicate that it handles web requests.

478
 • Example:
@Controller
public class GreetingController {

@GetMapping("/hello")
public String sayHello(Model model) {
[Link]("message", "Hello, Spring MVC!");
return "hello"; // Refers to a view (e.g., [Link] or [Link])
}
}

[Link] 4.2 Key Components in the Example

1. @Controller
• Tells Spring that this class is a controller and should process web requests.
2. @GetMapping("/hello")
• Maps HTTP GET requests for the URL /hello to the sayHello method.
3. Model model
• Allows adding attributes (data) that can be used in the view.
4. [Link]("message", "Hello, Spring MVC!")
• Adds a key-value pair (message = "Hello, Spring MVC!") to the model.
5. return "hello"
• Specifies the view name ([Link] or [Link]) to be rendered.

10.8.5 5. Mapping URLs to Controller Methods

Spring MVC provides multiple annotations to map URLs to controller methods.

[Link] 5.1 @RequestMapping (General-Purpose Annotation)

• Can handle multiple HTTP methods (GET, POST, PUT, DELETE, etc.).
• Example:
@RequestMapping(value = "/books", method = [Link])
public String getAllBooks(Model model) {
// Logic to fetch books
return "books";
}

[Link] 5.2 @GetMapping (Shortcut for GET Requests)

• Specifically handles HTTP GET requests.
• Used for retrieving data (e.g., displaying a list of books).
• Example:
@GetMapping("/books")
public String getAllBooks(Model model) {
[Link]("books", [Link]());
return "books";
}

479
[Link] 5.3 @PostMapping (Shortcut for POST Requests)
• Specifically handles HTTP POST requests.
• Used for submitting data (e.g., adding a new book).
• Example:
@PostMapping("/addBook")
public String addBook(@ModelAttribute Book book) {
[Link](book);
return "redirect:/books"; // Redirects to the book list
}

[Link] 5.4 @PutMapping and @DeleteMapping

• @PutMapping – Handles HTTP PUT requests (updating existing resources).
• @DeleteMapping – Handles HTTP DELETE requests (removing resources).
• Example:
@PutMapping("/books/{id}")
public String updateBook(@PathVariable Long id, @ModelAttribute Book book) {
[Link](id, book);
return "redirect:/books";
}

@DeleteMapping("/books/{id}")
public String deleteBook(@PathVariable Long id) {
[Link](id);
return "redirect:/books";
}

[Link] 5.5 Benefits of Using Specific Annotations

• Improves code readability by clearly indicating the HTTP method each method handles.
• Reduces boilerplate compared to using @RequestMapping with method parameters.
• Enforces RESTful conventions by separating concerns based on HTTP methods.

10.8.6 6. Practical Example: Book Management System

@Controller
public class BookController {

@GetMapping("/books") // Handles GET request to display books

public String getAllBooks(Model model) {
[Link]("books", [Link]());
return "books"; // Renders [Link] or [Link]
}

@PostMapping("/addBook") // Handles POST request to add a book

public String addBook(@ModelAttribute Book book) {

480
 [Link](book);
return "redirect:/books"; // Redirects to avoid duplicate submissions
}
}

[Link] 6.1 Controller for Displaying and Adding Books

[Link] 6.2 Key Observations

1. /books (GET) – Retrieves and displays a list of books.
2. /addBook (POST) – Submits a new book and redirects to the book list to prevent duplicate form submis-
sions.
3. redirect:/books – Ensures the user sees an updated list after adding a book.

10.8.7 7. Summary of Key Concepts

Concept Description
Spring MVC A module in the Spring Framework for building web applications
using the MVC pattern.
MVC Components Model (data/logic), View (presentation), Controller (request
handling).
@Controller Marks a class as a Spring MVC controller.
@GetMapping Maps HTTP GET requests to a method.
@PostMapping Maps HTTP POST requests to a method.
@PutMapping Maps HTTP PUT requests to a method.
@DeleteMapping Maps HTTP DELETE requests to a method.
@RequestMapping General-purpose mapping for any HTTP method.
Model Used to pass data from the controller to the view.
View Resolution The returned string (e.g., "hello") corresponds to a view
template (e.g., [Link]).

10.8.8 8. Conclusion
[Link] 8.1 Key Takeaways
1. Spring MVC controllers enable separation of concerns by handling user requests independently of the
model and view.
2. Annotations (@GetMapping, @PostMapping, etc.) simplify URL-to-method mapping and improve code
clarity.
3. Controllers can be easily tested due to their focused responsibilities.
4. Flexibility in view technologies allows developers to choose the best presentation layer for their application.

[Link] 8.2 Next Steps

• Experiment with creating custom controllers in a Spring Boot application.
• Explore form handling and data validation in controllers.
• Learn about RESTful controllers using @RestController for API development.

481
10.9 Redirecting and Forwarding Requests
10.9.1 Introduction
This lecture explores two fundamental mechanisms in Spring Boot MVC for managing web requests: 1. Redi-
recting requests 2. Forwarding requests
Understanding these concepts is essential for: - Managing user interactions in web applications. - Navigating
between different parts of an application. - Maintaining a seamless user experience while handling internal routing.

10.9.2 1. Redirecting Requests

[Link] 1.1 Definition
• A redirect occurs when the server sends a response instructing the client’s browser to make a new request
to a different URL.
• The browser receives a new URL and performs a fresh HTTP request to that location.
• The browser’s address bar updates to reflect the new URL.

[Link] 1.2 Key Characteristics

• HTTP Status Code: Typically 302 (Found) or 301 (Moved Permanently).
• Client-Side Operation: The browser initiates a new request.
• Use Cases:
– After form submissions (e.g., redirecting to a success page).
– When guiding users to different parts of the application (e.g., login → dashboard).
– When the URL must change (e.g., after an action is completed).

[Link] 1.3 Implementation in Spring Boot MVC Spring Boot MVC provides two primary ways to imple-
ment redirects: 1. Using the RedirectView class. 2. Using the redirect: prefix in controller method return
values.

@Controller
public class RedirectController {

@GetMapping("/redirectExample")
public String redirectExample() {
return "redirect:/newPage"; // Redirects to "/newPage"
}
}

[Link].1 1.3.1 Example: Redirecting with redirect: Prefix

• Explanation:
– The method returns a string with the redirect: prefix.
– Spring Boot automatically sends a 302 redirect response to the client.
– The browser makes a new request to /newPage.

[Link].2 1.3.2 Example: Redirecting After Form Submission Scenario: - A user submits a form (e.g., a
contact form). - After processing, the server redirects them to a success page.
Controller Code:

482
@Controller
public class FormController {

@PostMapping("/submitForm")
public String handleFormSubmission(@ModelAttribute("formData") FormData formData) {
// Process form data (e.g., save to database)
return "redirect:/success"; // Redirects to "/success"
}
}

• User Experience:
1. User submits the form at /submitForm.
2. Server processes the data.
3. Browser receives a 302 redirect to /success.
4. Browser makes a new GET request to /success.
5. The success page is displayed with a confirmation message.
Advantages: - Prevents duplicate form submissions (since a new request is made). - Ensures the URL updates
to reflect the new state (e.g., /success).

10.9.3 2. Forwarding Requests

[Link] 2.1 Definition
• A forward is a server-side operation where the server internally routes a request to another resource
without the client knowing.
• The browser’s URL does not change.
• The server processes the request as if it was originally directed to the new resource.

[Link] 2.2 Key Characteristics

• No New HTTP Request: The server handles the forwarding internally.
• URL Remains Unchanged: The user sees the original URL in the browser.
• Use Cases:
– Including internal resources (e.g., JSP pages, fragments).
– Modularizing views without exposing internal routing to the client.
– Reusing components (e.g., headers, footers) across multiple pages.

[Link] 2.3 Implementation in Spring Boot MVC Forwarding is achieved using the forward: prefix in
controller method return values.

@Controller
public class ForwardController {

@GetMapping("/displayPage")
public String displayPage() {
return "forward:/WEB-INF/views/[Link]"; // Forwards to JSP
}
}

483
[Link].1 2.3.1 Example: Forwarding to a JSP Page
• Explanation:
– The method returns a string with the forward: prefix.
– The server internally processes the request as if it was originally sent to /WEB-INF/views/[Link].
– The browser URL remains /displayPage.

[Link].22.3.2 Example: Forwarding for Internal Resource Management Scenario: - A user navigates to
/dashboard. - The server forwards the request to a JSP page that renders the dashboard.

Controller Code:
@Controller
public class DashboardController {

@GetMapping("/dashboard")
public String showDashboard() {
return "forward:/WEB-INF/views/[Link]"; // Forwards internally
}
}

• User Experience:
1. User visits /dashboard.
2. Server forwards the request to [Link].
3. The URL remains /dashboard.
4. The JSP page is rendered without a new request.
Advantages: - Seamless navigation (user does not see internal paths). - Efficient resource management (no
extra HTTP requests).

10.9.4 3. Key Differences Between Redirecting and Forwarding

Feature Redirecting Forwarding

Operation Client-side (browser makes new request) Server-side (internal routing)
HTTP Status Code 302 (Found) or 301 (Moved Permanently) No new status code (same request)
URL Change Yes (browser updates URL) No (URL remains the same)
Use Cases Form submissions, navigation changes Internal resource inclusion, JSP rendering
Performance Impact Additional HTTP request No additional request (faster)
Security Exposes new URL to client Hides internal routing from client

10.9.5 4. When to Use Redirecting vs. Forwarding

[Link] 4.1 Use Redirecting When:
• You need to change the URL in the browser (e.g., after login, form submission).
• You want to prevent duplicate form submissions (Post-Redirect-Get pattern).
• The user should see a new page (e.g., /success after an action).
Example Scenarios: - After submitting a payment form, redirect to /confirmation. - After deleting a record,
redirect to /list to show updated data.

484
[Link] 4.2 Use Forwarding When:
• You want to reuse internal resources (e.g., JSP fragments).
• The URL should not change (e.g., rendering a view without exposing its path).
• You need server-side includes (e.g., embedding a header in multiple pages).
Example Scenarios: - Forwarding /home to [Link] while keeping the URL as /home. - Including a common
footer across multiple pages without extra requests.

10.9.6 5. Practical Examples Summary

[Link] 5.1 Redirect Example (After Form Submission) Controller:
@PostMapping("/submit")
public String submitForm(@ModelAttribute("user") User user) {
[Link](user);
return "redirect:/success"; // Redirects to success page
}

User Flow: 1. User submits form → /submit (POST). 2. Server processes data → redirects to /success (GET).
3. Browser shows /success with confirmation.

[Link] 5.2 Forward Example (Rendering a JSP) Controller:

@GetMapping("/profile")
public String showProfile() {
return "forward:/WEB-INF/views/[Link]"; // Forwards internally
}

User Flow: 1. User visits /profile. 2. Server forwards to [Link] without URL change. 3. Browser still
shows /profile.

10.9.7 6. Best Practices

1. Use Redirects for State Changes:
• After POST requests (e.g., form submissions), always redirect to avoid resubmission on refresh.
2. Use Forwards for Internal Views:
• When rendering JSPs or Thymeleaf templates, forward to keep URLs clean.
3. Avoid Exposing Internal Paths:
• Never forward to paths outside WEB-INF (security risk).
4. Prefer Redirects for Public Navigation:
• If the user should see a new page (e.g., /login → /dashboard), use redirect.

10.9.8 7. Conclusion
• Redirecting and forwarding are essential for controlling navigation in Spring Boot MVC.
• Redirects are client-side, change the URL, and are ideal for post-action navigation.
• Forwards are server-side, preserve the URL, and are best for internal resource management.
• Choosing the right mechanism depends on whether the URL should change and where the processing
happens.
By mastering these concepts, you can build more efficient, user-friendly, and secure web applications.

485
10.10 Updating Book Information
10.10.1 Introduction
• Objective: By the end of this lecture, you will be able to:
– Create a method to update book details.
– Handle update requests in the controller.
– Save updated book information to the database.
• Context: Updating book information is a fundamental feature in any library management system, ensur-
ing:
– Accurate and up-to-date records.
– Effective inventory management.
– Data consistency across the application.

10.10.2 Approach Overview

• The process involves:
1. Service Layer: Encapsulates business logic for fetching and updating book details.
2. Controller Layer: Manages user interactions and data flow between the UI and backend.
3. Repository Layer: Handles database persistence (saving updated data).
4. View Layer: Provides the user interface (JSP form) for inputting updates.

10.10.3 Service Layer: Business Logic Implementation

[Link] Responsibilities
• Encapsulates business logic for:
– Fetching current book details (getBookForUpdate).
– Updating book details (updateBook).
• Separation of concerns ensures modularity and maintainability.

[Link] Key Methods in BookService Class

1. getBookForUpdate(Long id)
• Purpose: Fetches the existing book details for editing.
• Returns: A BookUpdateDTO (Data Transfer Object) containing the book’s current data.
2. updateBook(BookUpdateDTO bookUpdateDTO)
• Purpose: Saves the updated book details to the database.
• Process:
– The DTO contains updated fields (title, author, category, numberOfCopies).
– The method maps these fields to a Book entity.
– Calls [Link](book) to persist changes.

[Link] Data Transfer Object (DTO)

• BookUpdateDTO:
– Used to transfer data between the controller and view layer.
– Contains fields with setter methods (setId, setTitle, setAuthor, setCategory, setNumberOf-
Copies).
– Ensures clean separation between the presentation layer and business logic.

486
public class BookService {
@Autowired
private BookRepository bookRepository;

public BookUpdateDTO getBookForUpdate(Long id) {

Book book = [Link](id).orElseThrow();
BookUpdateDTO dto = new BookUpdateDTO();
[Link]([Link]());
[Link]([Link]());
[Link]([Link]());
[Link]([Link]());
[Link]([Link]());
return dto;
}

public void updateBook(BookUpdateDTO bookUpdateDTO) {

Book book = new Book();
[Link]([Link]());
[Link]([Link]());
[Link]([Link]());
[Link]([Link]());
[Link]([Link]());
[Link](book); // Persists updates to the database
}
}

[Link] Code Example (Service Layer)

10.10.4 Controller Layer: Handling User Requests

[Link] Responsibilities
• Manages data flow between the user interface and backend.
• Provides two key methods:
1. showUpdateForm: Loads the update form with existing book data.
2. updateBook: Processes the submitted updates.

[Link] Key Annotations

Annotation Purpose
@GetMapping Maps HTTP GET requests to the update form page.
@PostMapping Handles form submission (HTTP POST) and updates the book.
@RequestParam Captures the book ID from the URL.
@ModelAttribute Binds form data to a model attribute (e.g., BookUpdateDTO).

[Link] Method Breakdown

1. showUpdateForm(@RequestParam Long id, Model model)
• Purpose: Displays the update form with pre-populated book details.

487
 • Steps:
– Retrieves the book’s current data via [Link](id).
– Adds the BookUpdateDTO to the model for rendering in the JSP.
• Example:
@GetMapping("/books/update")
public String showUpdateForm(@RequestParam Long id, Model model) {
BookUpdateDTO bookUpdateDTO = [Link](id);
[Link]("bookUpdateDTO", bookUpdateDTO);
return "updateBook"; // Renders [Link]
}
2. updateBook(@ModelAttribute BookUpdateDTO bookUpdateDTO)
• Purpose: Processes the submitted form and updates the book.
• Steps:
– Validates the input (if applicable).
– Calls [Link](bookUpdateDTO) to save changes.
– Redirects to a confirmation page or book list.
• Example:
@PostMapping("/books/update")
public String updateBook(@ModelAttribute BookUpdateDTO bookUpdateDTO) {
[Link](bookUpdateDTO);
return "redirect:/books"; // Redirects to the book list
}

10.10.5 Repository Layer: Database Persistence

[Link] Responsibilities
• Handles all database operations (CRUD).
• Extends JpaRepository, which provides built-in methods (e.g., save, findById).

[Link] Key Features

• BookRepository Interface:
– Inherits save() from JpaRepository.
– Automatically generates boilerplate code for database interactions.
– Ensures data integrity and consistency.

public interface BookRepository extends JpaRepository<Book, Long> {

// Inherits save(), findById(), etc.
}

[Link] Example (Repository Layer)

[Link] How save() Works

• Input: A Book entity (with updated fields).
• Action:
– If the entity exists, updates the record.
– If the entity is new, inserts a new record.

488
 • Output: Returns the saved entity.

10.10.6 View Layer: User Interface (JSP)

[Link] Responsibilities
• Provides the user interface for updating book details.
• Uses JSP (JavaServer Pages) with Spring Form Tags for binding data.

[Link] Key Components

1. [Link]
• Contains a form for editing book details.
• Bound to BookUpdateDTO (pre-populated with current data).
• Includes validation error handling.
2. Spring Form Tags
• <form:form>: Binds the form to a model attribute.
• <form:input>: Renders input fields (e.g., title, author).
• <form:errors>: Displays validation messages.

<%@ taglib uri="[Link] prefix="form" %>

<form:form modelAttribute="bookUpdateDTO" method="POST" action="/books/update">
<form:hidden path="id"/> <!-- Hidden field for book ID -->

<label>Title:</label>
<form:input path="title"/><br>
<form:errors path="title"/><br>

<label>Author:</label>
<form:input path="author"/><br>
<form:errors path="author"/><br>

<label>Category:</label>
<form:input path="category"/><br>
<form:errors path="category"/><br>

<label>Number of Copies:</label>
<form:input path="numberOfCopies"/><br>
<form:errors path="numberOfCopies"/><br>

<input type="submit" value="Update"/>

</form:form>

[Link] JSP Code Example

[Link] Features of the Update Form

• Pre-populated fields: Displays current book data for editing.
• Error handling: Shows validation messages if input is invalid.
• Hidden ID field: Ensures the correct book is updated.

489
10.10.7 Navigation to the Update Form
[Link] Implementation in Book List View
• An Update button is added next to each book in the list view.
• Functionality:
– Triggers a GET request to /books/update?id={bookId}.
– Passes the book ID as a URL parameter.

<%@ taglib uri="[Link] prefix="c" %>

<c:forEach var="book" items="${books}">
<tr>
<td>${[Link]}</td>
<td>${[Link]}</td>
<td>
<form action="/books/update" method="GET">
<input type="hidden" name="id" value="${[Link]}"/>
<input type="submit" value="Update"/>
</form>
</td>
</tr>
</c:forEach>

[Link] JSP Code for Update Button

[Link] How It Works

1. User clicks “Update”:
• A GET request is sent to /books/update?id={bookId}.
2. Controller handles request:
• showUpdateForm() fetches the book data.
• Renders [Link] with the book’s current details.
3. User submits changes:
• A POST request is sent to /books/update.
• updateBook() processes and saves the updates.

10.10.8 Summary of Key Concepts

Concept Description
Service Layer Encapsulates business logic (fetching/updating books).
Controller Layer Handles user requests (@GetMapping, @PostMapping).
Repository Layer Persists data to the database ([Link]()).
View Layer (JSP) Provides the UI for updates (Spring Form Tags, error handling).
DTO (Data Transfer Transfers data between layers (BookUpdateDTO).
Object)
Navigation Update buttons in the book list trigger GET requests to the update form.

10.10.9 Key Takeaways

1. Modular Design:

490
 • Separation of concerns (Service, Controller, Repository, View) improves maintainability.
2. Data Flow:
• GET request → Load form with current data.
• POST request → Save updates to the database.
3. User Experience:
• Pre-populated forms and error handling ensure a smooth update process.
4. Database Integrity:
• [Link]() ensures consistent and accurate data persistence.

491
11 Module 11: Testing, Debugging, and Deployment
11.1 Configuring Application Properties for Different Environments
11.1.1 1. Introduction to Environment-Specific Configurations
[Link] 1.1 Definition and Importance
• Environment-specific configurations refer to distinct settings applied to an application depending on the
deployment environment (e.g., development, testing, staging, production).
• Purpose:
– Ensures consistent behavior across all stages of deployment.
– Enhances security by isolating sensitive configurations (e.g., database credentials, API keys).
– Maintains consistency in application performance and logging.
• Each environment serves a unique role:
– Development: Detailed logging, local databases, and debugging tools.
– Testing/Staging: Simulates production but with controlled data.
– Production: Optimized performance, minimal logging, and strict security.

[Link] 1.2 Practical Example: Database Connections

• Development Environment:
– Uses a local database (e.g., H2, MySQL on localhost).
– Allows frequent schema changes and detailed logging for debugging.
• Production Environment:
– Connects to a secure, high-performance database server (e.g., AWS RDS, Azure SQL).
– Implements stricter access controls and minimal logging for performance.

11.1.2 2. Key Concepts in Environment-Specific Configurations

[Link] 2.1 Profiles
• Definition: Mechanisms to group and manage configurations for different environments.
• Function:
– Enable switching between environments without modifying code.
– Isolate environment-specific settings (e.g., database URLs, logging levels).

[Link] 2.2 Configuration Files

• Structure:
– Default configuration file ([Link]/[Link]): Contains common
properties shared across all environments.
– Environment-specific files (e.g., [Link], [Link]):
Override or extend default settings for a specific environment.

[Link] 2.3 Property Overriding

• Definition: The process of replacing default property values with environment-specific values.
• Mechanism:
– Properties in environment-specific files take precedence over default properties.
– Example: A [Link] in [Link] overrides the same property in ap-
[Link].

492
11.1.3 3. Setting Up Configuration Files
[Link] 3.1 Default Configuration File ([Link])
• Contents:
– Common properties applicable to all environments (e.g., application name, server port).
– Example:
[Link]=my-app
[Link]=8080

[Link] 3.2 Environment-Specific Configuration Files

• Naming Convention: application-{profile}.properties (e.g., [Link],
[Link]).
• Contents:
– Development ([Link]):
[Link]=jdbc:h2:mem:devdb
[Link]=sa
[Link]=
[Link]=DEBUG
– Production ([Link]):
[Link]=jdbc:postgresql://[Link]/mydb
[Link]=admin
[Link]=securepassword
[Link]=WARN

11.1.4 4. Activating Profiles

[Link] 4.1 Command Line Activation
• Method: Use the --[Link] flag when running the application.
• Example:
java -jar [Link] --[Link]=dev

[Link] 4.2 Environment Variables

• Method: Set the SPRING_PROFILES_ACTIVE environment variable.
• Example (Linux/macOS):
export SPRING_PROFILES_ACTIVE=prod
java -jar [Link]

• Example (Windows):
set SPRING_PROFILES_ACTIVE=prod
java -jar [Link]

[Link] 4.3 Configuration File Activation

• Method: Define the active profile in the default [Link]:

493
 [Link]=test

11.1.5 5. Maven Profiles for Environment-Specific Builds

[Link] 5.1 Definition
• Maven Profiles: Allow defining different build configurations in the [Link] file.
• Use Case: Set environment-specific properties (e.g., database URLs, credentials) during the build process.

[Link] 5.2 Configuration in [Link]

• Example:
<profiles>
<profile>
<id>dev</id>
<properties>
<[Link]>jdbc:h2:mem:devdb</[Link]>
<[Link]>sa</[Link]>
<[Link]></[Link]>
</properties>
</profile>
<profile>
<id>prod</id>
<properties>
<[Link]>jdbc:postgresql://[Link]/mydb</[Link]>
<[Link]>admin</[Link]>
<[Link]>securepassword</[Link]>
</properties>
</profile>
</profiles>

[Link] 5.3 Activating Maven Profiles

[Link].1 5.3.1 Command Line Activation

• Method: Use the -P flag to specify the profile.
• Example:
mvn clean install -Pdev

[Link].2 5.3.2 Activation via [Link]

• Method: Define active profiles in Maven’s [Link]:
<activeProfiles>
<activeProfile>prod</activeProfile>
</activeProfiles>

[Link] 5.4 Resource Filtering

494
 • Definition: Maven replaces placeholders in resource files (e.g., [Link]) with profile-
specific values during the build.
• Example:
– In [Link]:
[Link]=${[Link]}
[Link]=${[Link]}
[Link]=${[Link]}
– Maven replaces ${[Link]} with the value defined in the active profile.

11.1.6 6. Spring Boot Configuration Classes

[Link] 6.1 Overview
• Purpose: Define beans (e.g., data sources) dynamically based on the active profile.
• Mechanism: Use @Profile annotation to conditionally load configurations.

[Link] 6.2 Example: Development Configuration

• Class Definition:
@Configuration
@Profile("dev") // Only active when 'dev' profile is enabled
public class DevConfig {

@Bean
public DataSource dataSource() {
return [Link]()
.url("jdbc:h2:mem:devdb")
.username("sa")
.password("")
.build();
}
}

• Activation: The DevConfig class is loaded only when the dev profile is active.

[Link] 6.3 Example: Production Configuration

• Class Definition:
@Configuration
@Profile("prod") // Only active when 'prod' profile is enabled
public class ProdConfig {

@Bean
public DataSource dataSource() {
return [Link]()
.url("jdbc:postgresql://[Link]/mydb")
.username("admin")
.password("securepassword")
.build();

495
 }
}

[Link] 6.4 Profile Activation Methods

• Command Line:
java -jar [Link] --[Link]=dev

• Environment Variable:
export SPRING_PROFILES_ACTIVE=prod

• Configuration File:
[Link]=test

11.1.7 7. Summary of Key Takeaways

1. Environment-Specific Configurations:
• Isolate settings for development, testing, staging, and production.
• Ensure security, consistency, and performance optimization.
2. Configuration Files:
• Default ([Link]) + environment-specific files (application-{profile}.properties).
• Property overriding ensures environment-specific values take precedence.
3. Profile Activation:
• Command line (--[Link]).
• Environment variables (SPRING_PROFILES_ACTIVE).
• Configuration file ([Link]).
4. Maven Profiles:
• Define build-time configurations in [Link].
• Activate via -P flag or [Link].
• Resource filtering replaces placeholders with profile-specific values.
5. Spring Boot Configuration Classes:
• Use @Profile to conditionally load beans.
• Example: DevConfig for development, ProdConfig for production.

11.2 Debugging Common Issues in Database Applications

11.2.1 1. Introduction to Debugging Database Applications
[Link] 1.1 Learning Objectives By the end of this lecture, learners will be able to: - Identify common issues
in database applications. - Learn debugging techniques and tools. - Debug common issues with practical examples.
- Understand how to resolve various database issues effectively.

[Link] 1.2 Importance of Debugging Effective debugging is critical for: - Application Reliability: Ensures
applications run smoothly and reliably. - Data Integrity: Prevents data corruption and maintains accuracy. -
Performance Optimization: Ensures queries execute efficiently and resources are utilized effectively. - User
Experience: Delivers a high-quality experience, maintaining user trust and satisfaction.

11.2.2 2. Common Database Issues

Five primary categories of database issues are frequently encountered:

496
[Link] 2.1 Connection Problems
• Definition: Issues that prevent the application from interacting with the database.
• Subtypes:
– Connectivity Errors:
* Caused by misconfigured data source properties (e.g., in [Link] or applica-
[Link]).
* Common issues:
ꞏ Incorrect database URLs.
ꞏ Invalid credentials (username/password).
ꞏ Network problems (firewalls, unreachable servers).
– Connection Pool Exhaustion:
* Occurs when all available connections in the pool are in use, leading to timeouts or degraded
performance.

[Link] 2.2 SQL Query Issues

• Definition: Problems arising from SQL queries, including:
– Syntax Errors: Missing or incorrect syntax in queries (e.g., JPQL queries).
– Logical Errors: Queries that execute but return incorrect results.
– Performance Issues: Slow-executing queries due to inefficient plans or missing indexes.
• Example: A QuerySyntaxException may occur if a JPQL query has incorrect syntax.

[Link] 2.3 Data Integrity Concerns

• Definition: Issues that compromise the accuracy and consistency of data.
• Subtypes:
– Constraint Violations:
* Occur when data does not adhere to database rules (e.g., unique constraints, foreign key con-
straints).
* Example: DataIntegrityViolationException when inserting duplicate values in a unique col-
umn.
– Data Type Mismatches:
* Occur when incompatible data types are used (e.g., inserting a string into an integer column).
[Link] 2.4 Transaction and Concurrency Problems
• Definition: Issues arising from multiple transactions interacting with the database simultaneously.
• Subtypes:
– Deadlocks:
* Occur when two or more transactions are waiting indefinitely for resources locked by each other.
– Lost Updates:
* Occur when concurrent transactions overwrite each other’s changes, leading to data inconsistency.
[Link] 2.5 Configuration Errors
• Definition: Issues stemming from incorrect database or environment settings.
• Examples:
– Incorrect JDBC URL, username, or password in configuration files (e.g., [Link]).
– Environment-specific discrepancies (e.g., differences between development and production settings).

497
11.2.3 3. Debugging Techniques and Tools
[Link] 3.1 Logging and Monitoring Tools
• Purpose: Track application and database behavior to identify anomalies.
• Examples:
– Application logs (e.g., Spring Boot logs).
– Database logs (e.g., MySQL error logs, PostgreSQL logs).

[Link] 3.2 Database Profiling

• Purpose: Analyze query performance and resource usage.
• Tools:
– EXPLAIN: Analyzes the execution plan of a query to identify bottlenecks.
– IDE Debugging Features: Step-through debugging for application code interacting with the database.

[Link] 3.3 Exception Handling

• Best Practices:
– Implement global exception handlers to catch and log database-related exceptions.
– Use custom exception classes for specific database errors (e.g., ConstraintViolationException).

[Link] 3.4 Testing Strategies

• Unit Testing: Test individual components (e.g., repository methods) in isolation.
• Integration Testing: Test interactions between the application and the database.
• Tools: JUnit, Testcontainers (for database integration tests).

[Link] 3.5 Regular Database Health Checks

• Purpose: Proactively identify and resolve performance issues.
• Tasks:
– Monitor query performance.
– Optimize indexes.
– Perform routine maintenance (e.g., vacuuming in PostgreSQL, analyzing tables in MySQL).

11.2.4 4. Practical Debugging Examples

[Link] 4.1 Example 1: Connection Pool Exhaustion

[Link].1 Scenario:
• Application experiences timeouts or slow performance due to exhausted connection pool.

[Link].2 Debugging Steps:

1. Check Connection Pool Configuration:
• Verify settings in [Link] or [Link]:
[Link]-pool-size=10
[Link]-timeout=30000
2. Analyze Connection Usage:
• Monitor active connections and usage patterns (e.g., using database monitoring tools).

498
 3. Adjust Pool Size:
• Increase maximum-pool-size if necessary.
• Ensure connections are properly released (e.g., using try-with-resources in Java).

[Link] 4.2 Example 2: Slow Query Performance

[Link].1 Scenario:
• A query fetching book details is executing slowly.

[Link].2 Debugging Steps:

1. Profile the Query:
• Use EXPLAIN to analyze the query execution plan:
EXPLAIN SELECT * FROM books WHERE title LIKE '%Database%';
2. Check Indexes:
• Ensure appropriate indexes exist for filtered columns (e.g., title).
• Add indexes if missing:
CREATE INDEX idx_books_title ON books(title);
3. Modify the Query:
• Rewrite the query for efficiency (e.g., avoid SELECT *, use pagination).

[Link] 4.3 Example 3: Data Integrity Violation

[Link].1 Scenario:
• An attempt to insert a book fails due to a constraint violation (e.g., duplicate ISBN).

[Link].2 Debugging Steps:

1. Check Constraints:
• Verify database constraints (e.g., UNIQUE, FOREIGN KEY).
• Example:
ALTER TABLE books ADD CONSTRAINT unique_isbn UNIQUE (isbn);
2. Inspect Error Messages:
• Review the exception message to identify the violated constraint (e.g., Duplicate entry '12345'
for key 'unique_isbn').
3. Correct the Data or Constraints:
• Either:
– Modify the data to comply with constraints.
– Adjust constraints if they are overly restrictive.

[Link] 4.4 Example 4: Deadlocks

[Link].1 Scenario:
• Transactions frequently encounter deadlocks.

499
[Link].2 Debugging Steps:
1. Analyze Deadlock Logs:
• Review database logs for deadlock information (e.g., MySQL deadlock logs, PostgreSQL pg_locks).
2. Optimize Transactions:
• Ensure transactions acquire locks in a consistent order to prevent circular waits.
3. Implement Retry Logic:
• Use retry mechanisms to handle deadlock exceptions gracefully:
@Retryable(value = { [Link] }, maxAttempts = 3)
public void updateBook(Book book) {
// Transactional code
}

11.2.5 5. Summary of Key Takeaways

1. Common Issues:
• Connection problems, SQL query issues, data integrity concerns, transaction/concurrency problems,
and configuration errors.
2. Debugging Techniques:
• Logging, profiling (e.g., EXPLAIN), exception handling, testing, and regular health checks.
3. Practical Examples:
• Resolving connection pool exhaustion, slow queries, data integrity violations, and deadlocks.
4. Best Practices:
• Proactive monitoring, proper exception handling, and efficient query optimization.

11.3 Deploying the Application to a Web Server (Tomcat)

11.3.1 Introduction to Web Server Deployment
• Definition: Deploying a Spring Boot application refers to the process of packaging and placing it on a web
server to make it accessible to users.
• Key Objectives:
– Introduce web server deployment concepts.
– Deploy a Spring Boot application to Apache Tomcat.
– Verify successful deployment on the server.

11.3.2 Packaging Formats for Deployment

Two primary packaging formats are used for deploying Java applications:

[Link] 1. JAR (Java Archive) Files

• Definition: A self-contained archive that includes:
– Application classes.
– Dependencies.
– Resources.
• Use Case:
– Designed for standalone applications.
– Can be executed directly using java -jar.
• Characteristics:
– Not web-specific (lacks servlet/JSP support).

500
 – Typically used for command-line or desktop applications.

[Link] 2. WAR (Web Application Archive) Files

• Definition: A web-specific archive containing:
– Servlets.
– JSP (JavaServer Pages) files.
– Static web resources (HTML, CSS, JS).
– Configuration files (e.g., [Link]).
• Use Case:
– Designed for web applications.
– Deployed on servlet containers (e.g., Tomcat, Jetty).
• Characteristics:
– Requires a servlet container to run.
– Enables scalability and centralized management in web environments.

[Link] Comparison: JAR vs. WAR

Feature JAR WAR

Purpose Standalone applications Web applications
Execution Direct (java -jar) Requires a servlet container
Contents Classes + dependencies Servlets, JSPs, web resources
Deployment Target Local machine Web servers (Tomcat, etc.)
Example Use Case CLI tools, batch processing Web apps, REST APIs

11.3.3 Benefits of Deploying an Application

Deploying an application to a web server provides several advantages: 1. Scalability: - Web servers (e.g., Tomcat)
can handle multiple concurrent requests. - Supports load balancing and clustering. 2. Centralized Manage-
ment: - Easier to monitor, update, and maintain applications in a single location. - Simplifies version control and
rollbacks. 3. Enhanced Security: - Web servers provide built-in security features (e.g., HTTPS, authentication).
- Isolates applications from the underlying OS.

11.3.4 Prerequisites for Deploying to Tomcat

Before deploying a Spring Boot application to Tomcat, ensure the following:
1. Apache Tomcat Installed and Running:
• Download and install Tomcat from the official website.
• Verify Tomcat is running (default port: 8080).
2. Application Packaged as WAR:
• If the application is currently a JAR, it must be repackaged as a WAR.
• Requires modifications to:
– The main application class (extend SpringBootServletInitializer).
– The [Link] file (Maven configuration).

11.3.5 Repackaging a Spring Boot Application as a WAR

[Link] Step 1: Extend SpringBootServletInitializer To convert a JAR-based Spring Boot app into a
WAR, the main class must extend SpringBootServletInitializer and override the configure method.

501
import [Link];
import [Link];

public class MyApplication extends SpringBootServletInitializer {

@Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder builder) {
return [Link]([Link]);
}

public static void main(String[] args) {

[Link]([Link], args);
}
}

[Link].1 Code Example:

• Purpose:
– Allows the application to run in a servlet container (e.g., Tomcat).
– The configure method specifies the main application class.

[Link] Step 2: Modify [Link] for WAR Packaging Three key changes are required in the Maven [Link]
file:
1. Change Packaging Type to WAR:
<packaging>war</packaging>

2. Add spring-boot-starter-tomcat with provided Scope:

<dependency>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<scope>provided</scope>
</dependency>

• Why provided?
– Indicates that Tomcat will provide the servlet container libraries at runtime.
– Prevents dependency conflicts between embedded Tomcat (in Spring Boot) and the external Tom-
cat server.
3. Include spring-boot-maven-plugin:
<build>
<plugins>
<plugin>
<groupId>[Link]</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>

• Ensures the application is correctly packaged as a WAR.

502
[Link] Step 3: Build the WAR File After modifying the code and [Link], build the WAR file using Maven:
1. Run the Maven Command:
mvn clean package

• Actions Performed:
– Cleans previous builds (clean).
– Compiles the application.
– Packages it into a WAR file.
2. Locate the WAR File:
• The generated WAR file will be in the target/ directory.
• Example filename: [Link].

11.3.6 Deploying the WAR File to Tomcat

[Link] Step 1: Copy the WAR File to Tomcat’s webapps Directory
1. Navigate to Tomcat’s installation directory.
2. Copy the WAR file to:
{Tomcat Installation Directory}/webapps/

• Example (Linux/macOS):
cp target/[Link] /opt/tomcat/webapps/

• Example (Windows):
copy target\[Link] C:\Tomcat\webapps\

[Link] Step 2: Start Tomcat

1. Linux/macOS:
./[Link] start

2. Windows:
[Link] start

• Tomcat will automatically deploy the WAR file when started.

• The application will be extracted into a subdirectory under webapps/ (e.g., myapp-0.0.1-SNAPSHOT).

[Link] Step 3: Access the Deployed Application

• Default URL Format:
[Link]

– Example:
[Link]

• Custom Context Path:

– If a custom context path is configured (e.g., /myapp), the URL becomes:

503
 [Link]

11.3.7 Verifying the Deployment

Ensure the application is deployed correctly by performing the following checks:

[Link] 1. Check Tomcat Logs

• Location:
{Tomcat Installation Directory}/logs/[Link]

• What to Look For:

– No errors during startup.
– Logs indicating the application started successfully (e.g., Started MyApplication in X seconds).
– HTTP status 200 for the root endpoint.

[Link] 2. Test the Application in a Web Browser

1. Open a browser and navigate to the deployment URL (e.g., [Link]
2. Verify:
• The homepage loads without errors.
• API endpoints (if applicable) return expected responses.
• Static resources (CSS, JS, images) are accessible.

[Link] 3. Monitor Server Performance

• Use tools like JConsole (Java Monitoring and Management Console) to:
– Check memory usage.
– Monitor thread activity.
– Detect potential bottlenecks.

[Link] 4. Debugging Deployment Issues If the application fails to deploy: 1. Review [Link] for er-
ror stacks. 2. Common Issues: - Missing dependencies (check provided scope in [Link]). - Port conflicts (en-
sure Tomcat’s port 8080 is free). - Incorrect context path (verify WAR filename or [Link]-
path in [Link]). 3. Redeploy After Fixes: - Stop Tomcat (./[Link] stop). - Delete
the old WAR and extracted directory from webapps/. - Copy the updated WAR and restart Tomcat.

11.3.8 Summary of Key Steps

1. Repackage the Application:
• Extend SpringBootServletInitializer.
• Modify [Link] (WAR packaging, provided scope for Tomcat).
2. Build the WAR:
• Run mvn clean package.
3. Deploy to Tomcat:
• Copy WAR to webapps/.
• Start Tomcat.
4. Verify Deployment:
• Check logs ([Link]).
• Test in a browser.
• Monitor performance.

504
11.3.9 Conclusion
• Deploying a Spring Boot application to Tomcat involves:
– Repackaging as a WAR.
– Configuring Maven for servlet container compatibility.
– Copying the WAR to Tomcat’s webapps/ directory.
– Verifying through logs and browser testing.
• Best Practices:
– Always test locally before deploying to production.
– Monitor logs for runtime errors.
– Use tools like JConsole for performance insights.

11.4 Final Project Review and Next Steps

11.4.1 1. Introduction and Lecture Objectives
By the end of this lecture, students will be able to: - Review their completed Spring Boot-based project and its core
functionalities. - Discuss potential improvements and next steps for project enhancement. - Explore additional
learning resources for further skill development.

11.4.2 2. Project Overview

[Link] 2.1 Project Context
• The course focused on developing a Spring Boot-based application, specifically a Library Management
System (simplified to a Book Management System).
• The system manages:
– Book titles (e.g., names, authors, genres).
– Book details (e.g., publication year, ISBN, availability status).

[Link] 2.2 Student Projects

• All students developed their own Spring Boot applications with similar objectives:
– Primary Goal: Effectively manage a book database using modern web technologies.

11.4.3 3. Technologies Used

The project leveraged the following key technologies:

Technology Purpose
Spring Boot Backend framework for building the Java-based web application.
MySQL Relational database for storing book records.
JPA/Hibernate Object-Relational Mapping (ORM) for database interactions and persistence.
JSP (JavaServer Server-side rendering to dynamically generate HTML pages.
Pages)
CSS Styling web pages for improved visual presentation.
jQuery Client-side scripting for enhanced interactivity (e.g., dynamic updates).

11.4.4 4. Core Functionalities

[Link] 4.1 CRUD Operations

505
 • The application implements Create, Read, Update, Delete (CRUD) operations for book records.
– Create: Add new book entries.
– Read: Retrieve and display book details.
– Update: Modify existing book information.
– Delete: Remove book records from the database.

[Link] 4.2 User Interface (UI) Components The UI consists of two primary sections:

[Link].1 4.2.1 Dashboard

• Provides an overview of books categorized by genre.
• Includes visual summaries (e.g., charts, counters) for quick insights.

[Link].2 4.2.2 Management Pages

• Forms for:
– Adding new books.
– Updating existing book details.
• Tables for:
– Viewing all book records.
– Sorting/filtering data.

[Link] 4.3 Data Binding

• JSP pages handle:
– Displaying book data retrieved from the database.
– Inputting new or updated book information via forms.

[Link] 4.4 Styling and Interactivity

• CSS: Applied for consistent and visually appealing UI design.
• jQuery: Used for:
– Dynamic content updates (e.g., real-time validation feedback).
– Enhanced interactivity (e.g., dropdown menus, modal dialogs).

[Link] 4.5 Validation and Error Handling

• Ensures user input accuracy through:
– Client-side validation (e.g., checking required fields).
– Server-side validation (e.g., preventing duplicate ISBNs).
• Provides clear error feedback (e.g., alert messages for invalid inputs).

11.4.5 5. Potential Improvements

The project can be enhanced in multiple dimensions:

[Link] 5.1 Performance Optimization

• Database Indexing:
– Add indexes to frequently queried columns (e.g., book_title, author) to speed up search operations.
• Caching:

506
 – Implement caching mechanisms (e.g., Redis, Spring Cache) to reduce database load and improve re-
sponse times.

[Link] 5.2 User Experience (UX) Enhancements

• Responsiveness:
– Optimize the UI for mobile and tablet devices using responsive design frameworks (e.g., Bootstrap).
• Advanced Search:
– Add filtering options (e.g., by genre, publication year, author).
– Implement full-text search for book titles/descriptions.

[Link] 5.3 Functionality Extensions

• Book Recommendations:
– Use collaborative filtering or content-based algorithms to suggest similar books.
• Export/Import Features:
– Allow users to:
* Export book data (e.g., CSV, Excel).
* Import bulk book records (e.g., from a spreadsheet).
• External API Integration:
– Fetch additional book data from third-party APIs (e.g., Google Books API, Open Library API).
– Automatically populate fields like cover images or author biographies.

11.4.6 6. Learning Resources for Further Development

To deepen knowledge and expand the project, explore the following resources:

[Link] 6.1 Spring Boot and Backend Development

• Official Documentation: Spring Boot Docs
• Books:
– Spring Boot in Action by Craig Walls.
– Pro Spring 5 by Iuliana Cosmina et al.
• Courses:
– Udemy: Spring & Hibernate for Beginners (Chad Darby).
– Coursera: Building Scalable Java Microservices with Spring Boot (Google Cloud).

[Link] 6.2 Database and ORM

• MySQL Optimization:
– High Performance MySQL by Baron Schwartz et al.
– MySQL Official Documentation on Indexing.
• JPA/Hibernate:
– Java Persistence with Hibernate by Christian Bauer et al.
– Baeldung’s Hibernate Tutorials.

[Link] 6.3 Frontend and UX

• CSS Frameworks:
– Bootstrap for responsive design.
– Tailwind CSS for utility-first styling.
• jQuery and JavaScript:

507
 – jQuery Documentation.
– Eloquent JavaScript by Marijn Haverbeke (for deeper JS knowledge).
• UX Design:
– Nielsen Norman Group’s Usability Heuristics.
– Don’t Make Me Think by Steve Krug.

[Link] 6.4 Advanced Features

• Caching:
– Redis Documentation: [Link].
– Spring Cache Guide: Baeldung’s Spring Cache.
• API Integration:
– Google Books API: [Link]/books.
– Open Library API: [Link]/developers/api.
• Deployment:
– Deploying Spring Boot Apps on AWS/Heroku (official guides).
– Docker for containerization: Docker Docs.

11.4.7 7. Summary and Next Steps

[Link] 7.1 Review of Key Takeaways
• The project successfully implemented a Book Management System using Spring Boot, MySQL, JPA, JSP,
CSS, and jQuery.
• Core functionalities include CRUD operations, data validation, and a user-friendly dashboard.
• Potential improvements span performance, UX, and extended features (e.g., recommendations, API in-
tegrations).

[Link] 7.2 Action Items for Students

1. Assess Current Project:
• Identify strengths and weaknesses in the existing implementation.
2. Prioritize Enhancements:
• Select 1–2 improvements (e.g., caching, mobile responsiveness) to implement first.
3. Explore Learning Resources:
• Use the provided materials to fill knowledge gaps (e.g., Spring Cache, API integration).
4. Plan for Deployment:
• Consider deploying the application to a cloud platform (e.g., Heroku, AWS) for real-world testing.

[Link] 7.3 Final Remarks

• The project serves as a foundation for building more complex database applications.
• Continuous learning and iteration are key to mastering full-stack development.

11.5 Introduction to Testing Spring Boot Applications

11.5.1 1. Importance of Testing in Spring Boot Applications
Testing is a critical process in software development that evaluates a system or its components to verify whether
they meet specified requirements.

508
[Link] 1.1 Key Benefits of Testing
• Ensures Correct Behavior: Confirms that the application behaves as expected under various conditions.
• Improves Code Quality: Helps identify and fix bugs early, ensuring the code functions as intended.
• Facilitates Continuous Integration and Delivery (CI/CD):
– Automated tests enable frequent code integration and deployment.
– Reduces manual testing efforts, allowing faster releases.
• Enhances Maintainability and Scalability:
– Ensures future code changes do not break existing functionality (regression testing).
– Simplifies debugging and refactoring.
• Boosts Developer Confidence:
– Well-tested code increases trust in application stability.
– Reduces fear of introducing new bugs during development.

11.5.2 2. Types of Testing in Spring Boot

Different testing strategies are employed based on the scope and depth of evaluation required.

[Link] 2.1 Unit Testing

• Definition: Tests individual components (e.g., methods, classes) in isolation from other parts of the system.
• Example:
– Testing a single method in a Service class (e.g., calculateSum(int a, int b)).
– Verifying that a repository method returns the correct data when queried.

[Link] 2.2 Integration Testing

• Definition: Tests interactions between multiple components to ensure they work together correctly.
• Example:
– Testing a Service class and its interaction with a Repository.
– Verifying that a Controller correctly processes input and delegates to a Service.

[Link] 2.3 End-to-End (E2E) Testing

• Definition: Tests the entire application workflow, including all integrated components and external depen-
dencies (e.g., databases, APIs).
• Example:
– Simulating a complete user workflow (e.g., logging in, adding an item to a cart, and checking out).
– Validating the entire request-response cycle in a web application.

11.5.3 3. Setting Up Testing Dependencies in Spring Boot

To begin testing, the necessary dependencies must be included in the project.

[Link] 3.1 Required Dependencies (Maven - [Link]) Spring Boot provides built-in support for testing with
the following key libraries: - JUnit: A widely used testing framework for Java. - Mockito: A mocking framework
for creating and managing mock objects. - Spring Test: Spring-specific testing utilities.

<dependencies>
<!-- JUnit 5 -->

509
 <dependency>
<groupId>[Link]</groupId>
<artifactId>junit-jupiter-api</artifactId>
<scope>test</scope>
</dependency>
<!-- Mockito -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
<!-- Spring Boot Test -->
<dependency>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

[Link].1 Example Maven Dependencies

11.5.4 4. Basics of Writing Tests

[Link] 4.1 JUnit Annotations JUnit provides annotations to structure and control test execution.

Annotation Purpose
@Test Marks a method as a test case.
@BeforeEach Runs before each test method (used for setup,
e.g., initializing objects).
@AfterEach Runs after each test method (used for cleanup,
e.g., resetting state).

[Link] 4.2 Common Assertions Assertions verify expected outcomes in test cases.

Assertion Method Description

assertEquals(expected, actual) Checks if two values are equal.
assertTrue(condition) Verifies that a condition is true.
assertFalse(condition) Verifies that a condition is false.
assertNull(object) Checks if an object is null.
assertNotNull(object) Checks if an object is not null.

import [Link];
import static [Link];

public class CalculatorTest {

@Test

510
 public void testAdd() {
Calculator calculator = new Calculator();
int result = [Link](2, 3);
assertEquals(5, result); // Verifies that 2 + 3 = 5
}
}

[Link].1 Example: Unit Test for a Sum Method

11.5.5 5. Mocking with Mockito

Mockito is used to create mock objects that simulate real dependencies, allowing isolated unit testing.

[Link] 5.1 Key Mockito Concepts

Concept Description
Mocking Creating a fake object that mimics the behavior of a real
dependency.
Stubbing Defining the behavior of a mock object’s methods (e.g., returning
a specific value).
Verification Checking whether certain methods were called as expected.

[Link] 5.2 Mockito Methods

Method Purpose
[Link](Class) Creates a mock object of the given class.
when([Link]()).thenReturn(value) Stubs a method to return a specific value.
verify(mock).method() Verifies that a method was called.

import [Link];
import static [Link].*;
import static [Link].*;

public class UserServiceTest {

@Test
public void testFindUserById() {
// 1. Create a mock repository
UserRepository userRepository = mock([Link]);

// 2. Stub the findById method to return a specific user

User expectedUser = new User(1L, "John Doe");
when([Link](1L)).thenReturn(expectedUser);

// 3. Test the service method

UserService userService = new UserService(userRepository);
User actualUser = [Link](1L);

511
 // 4. Assert the result
assertEquals(expectedUser, actualUser);

// 5. Verify the repository method was called

verify(userRepository).findById(1L);
}
}

[Link].1 Example: Mocking a User Repository

11.5.6 6. Writing Unit Tests for Spring Boot Components

[Link] 6.1 Testing a Service Class Unit tests for services focus on business logic, often mocking dependencies
like repositories.

import [Link];
import static [Link].*;
import static [Link].*;

public class BookServiceTest {

@Test
public void testFindBookById() {
// 1. Create a mock repository
BookRepository bookRepository = mock([Link]);

// 2. Stub the repository method

Book expectedBook = new Book(1L, "Effective Java");
when([Link](1L)).thenReturn(expectedBook);

// 3. Initialize the service with the mock repository

BookService bookService = new BookService(bookRepository);

// 4. Call the service method

String actualTitle = [Link](1L);

// 5. Assert the result

assertEquals("Effective Java", actualTitle);
}
}

[Link].1 Example: Testing a Book Service

[Link] 6.2 Key Components of a Mockito Stub

Component Description Example

Mock Object The fake dependency created using BookRepository
mock(). bookRepository =
mock([Link]);

512
Component Description Example
Method Call The method on the mock object being [Link](1L)
stubbed.
Stubbed Value The predefined return value for the new Book(1L, "Effective
method. Java")
when().thenReturn() Defines the stubbed behavior. when([Link](1L)).

11.5.7 7. Summary of Key Learnings

By the end of this lecture, the following concepts were covered: 1. Importance of Testing: - Ensures correctness,
improves quality, and supports CI/CD. - Enhances maintainability and developer confidence. 2. Types of Testing: -
Unit Testing: Individual components in isolation. - Integration Testing: Interactions between components. - End-
to-End Testing: Complete system workflows. 3. Setting Up Dependencies: - JUnit for test execution. - Mockito
for mocking. - Spring Test for Spring-specific utilities. 4. Writing Basic Tests: - Using JUnit annotations (@Test,
@BeforeEach, @AfterEach). - Assertions (assertEquals, assertTrue). 5. Mocking with Mockito: - Creating
mock objects. - Stubbing methods (when().thenReturn()). - Verifying interactions (verify()). 6. Unit Testing
Services: - Isolating business logic by mocking dependencies. - Validating expected behavior with assertions.

11.6 Monitoring and Maintaining the Application

11.6.1 1. Introduction to Application Monitoring and Maintenance
[Link] 1.1 Importance of Monitoring and Maintenance
• Purpose: Ensures application performance, reliability, and user satisfaction.
• Key Benefits:
– Early Issue Detection: Identifies problems before they escalate, improving user experience.
– Cost Reduction: Proactive monitoring minimizes downtime and repair costs.
– Efficiency: Regular maintenance keeps the application running optimally.
– User Expectations: Ensures the application meets evolving user needs.

11.6.2 2. Tools and Techniques for Monitoring Applications

[Link] 2.1 Application Performance Monitoring (APM) Tools
• Definition: Tools that track real-time application performance metrics.
• Key Tools:
– New Relic: Provides detailed insights into performance bottlenecks.
– Dynatrace: Offers AI-driven monitoring for complex applications.
– AppDynamics: Specializes in end-to-end application performance tracking.
• Use Case: Essential for Spring Boot applications to maintain health and responsiveness.

[Link] 2.2 Logging Tools

• Definition: Systems that collect, aggregate, and analyze log data.
• Key Tools:
– ELK Stack (Elasticsearch, Logstash, Kibana): Enables centralized log management.
– Splunk: Advanced log analysis and visualization.
• Use Case: Tracks user interactions and system events in a library management web application.

513
[Link] 2.3 Infrastructure Monitoring Tools
• Definition: Monitors servers, databases, and supporting infrastructure.
• Key Tools:
– Nagios: Alerts on infrastructure failures.
– Prometheus: Tracks performance metrics of distributed systems.
– Datadog: Provides cloud-scale monitoring and analytics.
• Use Case: Ensures robustness of the book web application’s backend.

[Link] 2.4 Synthetic Monitoring Tools

• Definition: Simulates user interactions to test performance and availability.
• Key Tools:
– Pingdom: Monitors uptime and response times.
– UptimeRobot: Proactively checks application health.
• Use Case: Identifies issues before they impact real users in a book data management system.

11.6.3 3. Maintenance Activities for Application Health

[Link] 3.1 Regular Maintenance Tasks
• Dependency Updates: Ensures libraries and frameworks are up-to-date.
• Database Maintenance: Optimizes queries, indexes, and storage.
• Performance Optimization: Refines code and infrastructure for efficiency.

[Link] 3.2 Role of Monitoring in Maintenance

• New Relic: Identifies performance bottlenecks.
• ELK Stack: Analyzes log data for troubleshooting.
• Prometheus: Tracks infrastructure health.
• Pingdom: Ensures optimal user interactions.

11.6.4 4. Case Study: Spring Boot Library Management Web Application

[Link] 4.1 Monitoring Setup
• Tools Used:
– ELK Stack: For log analysis.
– Pingdom: For synthetic monitoring.
– Prometheus: For infrastructure tracking.

[Link] 4.2 Maintenance Practices Implemented

• Dependency Updates: Kept third-party libraries current.
• Database Optimization: Improved query performance.
• Performance Refinements: Enhanced application responsiveness.

[Link] 4.3 Outcomes

• Result: Application remained responsive and reliable for library database management.
• User Impact: Ensured smooth access to book data and services.

514
11.6.5 5. Summary of Key Takeaways
• Monitoring Importance: Critical for performance, reliability, and cost efficiency.
• Tools Covered:
– APM: New Relic, Dynatrace, AppDynamics.
– Logging: ELK Stack, Splunk.
– Infrastructure: Nagios, Prometheus, Datadog.
– Synthetic: Pingdom, UptimeRobot.
• Maintenance Best Practices:
– Update dependencies.
– Optimize databases.
– Continuously refine performance.
• Case Study Insight: Practical application of tools in a Spring Boot library management system.

11.7 Packaging the Application (JARWAR)

11.7.1 Introduction to Application Packaging
[Link] Definition of Packaging
• Packaging is the process of bundling all necessary components of an application—including code, depen-
dencies, and resources—into a single archive file.
• This archive can be easily transported and deployed across different environments.

[Link] Purpose and Benefits of Packaging

• Portability: Ensures the application runs consistently across different environments (development, testing,
production).
• Simplified Deployment: Reduces complexity by bundling everything into a single file.
• Dependency Isolation: Prevents conflicts by embedding required dependencies within the package.
• Environment Consistency: Minimizes issues arising from differences in environment configurations.

[Link] Common Packaging Formats

1. JAR (Java ARchive)
• Default packaging format for Spring Boot applications.
• Contains executable code, dependencies, and resources.
• Includes an embedded server (e.g., Tomcat, Jetty) for standalone execution.
2. WAR (Web Application ARchive)
• Used for traditional web applications deployed on external servlet containers (e.g., Apache Tomcat,
JBoss).
• Contains servlets, JSPs, static resources, and configuration files.
• Does not include an embedded server; relies on the container’s servlet support.

11.7.2 Packaging in Spring Boot

[Link] Default Packaging Format
• Spring Boot applications are packaged as executable JAR files by default.
• The JAR includes:
– Compiled application code.
– Embedded server (e.g., Tomcat).
– All dependencies (libraries, frameworks).

515
 – Configuration files (e.g., [Link]).

[Link] Significance of Packaging in Deployment

• Ensures consistent behavior across different deployment environments.
• Simplifies distribution (single file deployment).
• Reduces environment-specific issues (e.g., missing dependencies, version conflicts).

11.7.3 Creating a JAR File in Spring Boot

[Link] Steps to Package as a JAR
1. Add the Spring Boot Plugin
• Required for Maven and Gradle to handle packaging.
• Maven: Include the spring-boot-maven-plugin in [Link].
<build>
<plugins>
<plugin>
<groupId>[Link]</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
• Gradle: Include the [Link] plugin in [Link].
plugins {
id '[Link]' version 'x.x.x'
}
2. Build the Application
• Maven Command:
mvn clean package
– Compiles code.
– Runs tests.
– Packages the application into a JAR file in target/ directory.
• Gradle Command:
./gradlew build
– Generates the JAR in build/libs/.
3. Run the JAR File
• Execute using the java -jar command:
java -jar target/[Link]
• Behavior:
– Launches the embedded server.
– Starts the application with all dependencies included.

11.7.4 Creating a WAR File in Spring Boot

[Link] When to Use WAR Packaging
• Required for deployment on external servlet containers (e.g., Tomcat, JBoss).

516
 • Useful when:
– The organization mandates traditional servlet containers.
– The application must share a container with other web apps.

[Link] Steps to Package as a WAR

1. Modify Packaging Type
• Maven: Set <packaging>war</packaging> in [Link].
<packaging>war</packaging>
• Gradle: Configure the war plugin in [Link].
apply plugin: 'war'
2. Extend SpringBootServletInitializer
• Required to boot the application in a servlet container.
• Modify the main application class:
@SpringBootApplication
public class MyApplication extends SpringBootServletInitializer {
@Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder builder) {
return [Link]([Link]);
}

public static void main(String[] args) {

[Link]([Link], args);
}
}
3. Build the WAR File
• Use the same build commands as for JAR:
– Maven:
mvn clean package
– Gradle:
./gradlew build
• Output: WAR file generated in target/ (Maven) or build/libs/ (Gradle).
4. Deploy the WAR File
• Copy the WAR file to the servlet container’s deployment directory:
– Apache Tomcat: Place in webapps/ directory.
– The container automatically detects and deploys the application.

11.7.5 Customizing Packaging Settings

[Link] 1. Customizing the Manifest File
• The [Link] file contains metadata (e.g., main class, version).
• Customization Methods:
– Maven: Use the maven-jar-plugin to specify attributes.
<plugin>
<groupId>[Link]</groupId>
<artifactId>maven-jar-plugin</artifactId>

517
 <configuration>
<archive>
<manifest>
<mainClass>[Link]</mainClass>
<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
</manifest>
</archive>
</configuration>
</plugin>
– Gradle: Configure the jar task.
jar {
manifest {
attributes(
'Main-Class': '[Link]',
'Implementation-Version': '1.0.0'
)
}
}

[Link] 2. Excluding Dependencies

• Prevents unnecessary dependencies from being bundled (e.g., servlet API for WAR files).
• Methods:
– Maven: Set dependency scope to provided.
<dependency>
<groupId>[Link]</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<scope>provided</scope>
</dependency>
– Gradle: Use compileOnly or providedCompile.
compileOnly '[Link]:spring-boot-starter-tomcat'

[Link] 3. Changing the Output Directory

• Default locations:
– Maven: target/
– Gradle: build/libs/
• Customization:
– Maven: Set <directory> in the build section.
<build>
<directory>${[Link]}/custom-output</directory>
</build>
– Gradle: Configure the destinationDirectory in the bootJar or bootWar task.
bootJar {
destinationDirectory = file("${projectDir}/custom-output")
}

518
11.7.6 Key Takeaways
1. Packaging Formats:
• JAR: Default for Spring Boot (standalone, embedded server).
• WAR: For traditional servlet containers (external server required).
2. JAR Creation Process:
• Requires Spring Boot plugin.
• Built using mvn clean package or ./gradlew build.
• Executed via java -jar.
3. WAR Creation Process:
• Requires SpringBootServletInitializer.
• Packaging type set to war.
• Deployed to servlet container’s webapps/.
4. Customization Options:
• Manifest file: Modify metadata (main class, version).
• Exclude dependencies: Use provided scope.
• Output directory: Redirect build artifacts to a custom location.
5. Deployment Benefits:
• Consistency: Eliminates environment-specific issues.
• Portability: Single file for easy distribution.
• Isolation: Bundled dependencies prevent conflicts.

11.8 Using Postman to Test API Endpoints

11.8.1 Introduction to Postman for API Testing
• Objective: By the end of this lecture, you will be able to:
– Introduce Postman for API testing.
– Create and run test requests in Postman.
– Verify API responses using Postman.
• Focus: Testing the BookController API from a Spring Boot application.
• Key Features of Postman:
– Intuitive user interface.
– Support for various HTTP methods (GET, POST, PUT, DELETE).
– Environment management (storing variables for different environments).
– Automated testing (writing and running test scripts).

11.8.2 Setting Up Postman

[Link] Installation and Initial Setup
1. Download and Install:
• Obtain Postman from the official website.
• Install the application on your system.
2. Launch Postman:
• Open the application to begin creating API requests.

[Link] Creating a New Request

1. Steps to Create a Request:
• Click “New” in the top-left corner.
• Select “Request”.

519
 • Name the request (e.g., “GET All Books”).
• Organize into a collection (group related requests for better management).
2. Purpose of Collections:
• Keeps related API requests organized.
• Allows for easy reuse and sharing.

11.8.3 Configuring API Requests in Postman

[Link] Request Configuration Components To configure a request, specify the following: 1. HTTP
Method (GET, POST, PUT, DELETE). 2. Endpoint URL (e.g., [Link] 3. Parame-
ters (query parameters, if applicable). 4. Headers (e.g., Content-Type: application/json). 5. Request Body
(for POST/PUT requests, typically in JSON format).

[Link] Example: BookController API Endpoints The BookController in the Spring Boot application ex-
poses the following endpoints: | HTTP Method | Endpoint | Description | |—————–|—————————
-|————————————-| | GET | /books | Retrieve all books. | | POST | /books | Create a new book. | |
PUT | /books/{id} | Update an existing book by ID. | | DELETE | /books/{id} | Delete a book by ID. |

11.8.4 Testing API Endpoints in Postman

[Link] 1. Testing the GET /books Endpoint

[Link].1 Steps:
1. Set the HTTP method to GET.
2. Enter the URL: [Link]
3. Click “Send”.

[Link].2 Expected Response:

• Status Code: 200 OK.
• Response Body: A list of books in JSON format.
[
{
"id": 1,
"title": "Book Title 1",
"author": "Author 1",
"publicationYear": 2020
},
{
"id": 2,
"title": "Book Title 2",
"author": "Author 2",
"publicationYear": 2021
}
]

[Link].3 Verification:
• Ensure the response matches the expected JSON structure.

520
 • Confirm the status code is 200 OK.

[Link] 2. Testing the POST /books Endpoint

[Link].1 Steps:
1. Set the HTTP method to POST.
2. Enter the URL: [Link]
3. Navigate to the “Body” tab.
• Select “raw” and choose “JSON” format.
4. Enter the book details in JSON format:
{
"title": "New Book",
"author": "New Author",
"publicationYear": 2023
}

5. Click “Send”.

[Link].2 Expected Response:

• Status Code: 201 Created.
• Response Body: The newly created book in JSON format.
{
"id": 3,
"title": "New Book",
"author": "New Author",
"publicationYear": 2023
}

[Link].3 Verification:
• Check that the response includes the new book with an auto-generated id.
• Confirm the status code is 201 Created.

[Link] 3. Testing the PUT /books/{id} Endpoint

[Link].1 Steps:
1. Set the HTTP method to PUT.
2. Enter the URL: [Link] (replace {id} with an existing book ID, e.g., 1).
3. Navigate to the “Body” tab.
• Select “raw” and choose “JSON” format.
4. Enter the updated book details:

521
 {
"title": "Updated Book Title",
"author": "Updated Author",
"publicationYear": 2024
}

5. Click “Send”.

[Link].2 Expected Response:

• Status Code: 200 OK.
• Response Body: The updated book details in JSON format.
{
"id": 1,
"title": "Updated Book Title",
"author": "Updated Author",
"publicationYear": 2024
}

[Link].3 Verification:
• Ensure the response reflects the updated fields.
• Confirm the status code is 200 OK.

[Link] 4. Testing the DELETE /books/{id} Endpoint

[Link].1 Steps:
1. Set the HTTP method to DELETE.
2. Enter the URL: [Link] (replace {id} with an existing book ID, e.g., 3).
3. Click “Send”.

[Link].2 Expected Response:

• Status Code: 204 No Content (indicates successful deletion).
• Response Body: Empty (no content returned).

[Link].3 Verification:
• Confirm the status code is 204 No Content.
• Verify the book no longer exists by sending a GET request for the same ID.

11.8.5 Writing Automated Tests in Postman

[Link] Purpose of Automated Tests
• Validate API responses automatically.
• Ensure the API functions as expected without manual intervention.
• Improve efficiency and reliability in testing.

522
[Link] Example Test Script Postman allows writing test scripts in the “Tests” tab using JavaScript. Exam-
ple:
// Check if the status code is 200
[Link]("Status code is 200", function () {
[Link](200);
});

// Verify response time is below 200ms

[Link]("Response time is less than 200ms", function () {
[Link]([Link]).[Link](200);
});

// Ensure the response body contains expected data

[Link]("Response body contains expected book title", function () {
var jsonData = [Link]();
[Link](jsonData[0].title).[Link]("Book Title 1");
});

[Link] Key Test Assertions

Test Case Postman Syntax

Check status code [Link](200)
Verify response time [Link]([Link]).[Link](200)
Validate response body content [Link]([Link]).[Link]("value")

11.8.6 Analyzing Test Results

[Link] Reviewing Test Outcomes
1. Test Results Tab:
• Displays passed/failed tests.
• Provides detailed logs for debugging.
2. Identifying Issues:
• Examine failed tests to pinpoint errors.
• Check response bodies and status codes for discrepancies.
3. Debugging and Refactoring:
• Use test feedback to debug API code.
• Adjust the API to meet expected functionality.
• Re-run tests after making changes.

[Link] Example Workflow:

1. Run automated tests in Postman.
2. Review the “Test Results” tab.
3. If a test fails:
• Investigate the error logs.
• Modify the API code (e.g., fix a bug in the BookController).
• Re-test until all tests pass.

523
11.8.7 Summary of Key Concepts
[Link] Postman Features for API Testing

Feature Description
Intuitive UI User-friendly interface for sending requests.
HTTP Methods Support Supports GET, POST, PUT, DELETE, etc.
Environment Store variables (e.g., base URLs) for different environments (dev, prod).
Management
Automated Testing Write and run test scripts to validate API responses.
Collections Organize related API requests for easy access and reuse.

[Link] BookController API Testing Workflow

1. GET /books → Retrieve all books (200 OK).
2. POST /books → Create a new book (201 Created).
3. PUT /books/{id} → Update a book (200 OK).
4. DELETE /books/{id} → Delete a book (204 No Content).

[Link] Best Practices

• Organize requests into collections.
• Use automated tests to ensure API reliability.
• Analyze test results to debug and improve the API.
• Validate both status codes and response bodies.

11.8.8 Conclusion
• Postman is a powerful tool for testing API endpoints efficiently.
• It simplifies sending requests, analyzing responses, and automating tests.
• By following the steps outlined, you can thoroughly test the BookController API and ensure it meets
functional requirements.

11.9 Writing Integration Tests

11.9.1 1. Introduction to Integration Testing
[Link] 1.1 Definition and Purpose
• Integration testing validates the interaction between different components of a software application.
• Ensures that components work together as expected in a real-world scenario.
• Unlike unit testing, which isolates individual components, integration testing checks the complete flow,
including:
– All involved layers (e.g., controllers, services, repositories).
– External systems (e.g., databases, APIs, third-party services).

[Link] 1.2 Key Differences from Unit Testing A structured comparison between unit testing and integration
testing:

524
Aspect Unit Testing Integration Testing
Purpose Tests individual components in isolation. Tests interactions between components.
Scope Single function, method, or class. Multiple components or entire
workflows.
Dependencies Mocks or stubs replace real Uses real dependencies (e.g., databases,
dependencies. APIs).
Speed Fast (minimal setup). Slower (requires full environment).
Tools JUnit, Mockito, AssertJ. Spring Boot Test, Testcontainers,
MockMvc.
Example Scenario Testing a service method with mocked Testing a REST API endpoint that
dependencies. interacts with a database.

11.9.2 2. Writing Integration Tests in Spring Boot

[Link] 2.1 Core Annotations for Integration Testing To write integration tests in Spring Boot, use the fol-
lowing annotations:
1. @SpringBootTest
• Purpose: Loads the full application context (simulates the real application environment).
• Effect: All beans, configurations, and dependencies are initialized.
• Usage:
@SpringBootTest
public class BookIntegrationTest {
// Test methods here
}
2. @AutoConfigureMockMvc
• Purpose: Sets up MockMvc for simulating HTTP requests without starting a full server.
• Effect: Allows testing controllers and their interactions with services.
• Usage:
@SpringBootTest
@AutoConfigureMockMvc
public class BookControllerIntegrationTest {
@Autowired
private MockMvc mockMvc;
}

[Link] 2.2 Testing End-to-End Functionality

• Integration tests verify complete application flows (e.g., from API call to database persistence).
• Example: Testing a book creation endpoint:
@Test
public void testAddBook() throws Exception {
[Link](post("/books")
.contentType(MediaType.APPLICATION_JSON)
.content("{\"title\":\"Effective Java\",\"author\":\"Joshua Bloch\"}"))
.andExpect(status().isCreated())
.andExpect(jsonPath("$.title").value("Effective Java"));
}

525
 – Steps:
1. Sends a POST request to /books with JSON payload.
2. Verifies the HTTP status is 201 Created.
3. Checks the response body contains the correct title.

11.9.3 3. Managing Test Data

[Link] 3.1 Setup and Teardown Methods
• Purpose: Ensure a consistent starting state for each test by:
– Setting up test data before execution (@BeforeEach).
– Cleaning up after execution (@AfterEach).
• Example:
@SpringBootTest
public class BookIntegrationTest {
@Autowired
private BookRepository bookRepository;

@BeforeEach
public void setup() {
// Insert test data before each test
[Link](new Book("Test Book", "Test Author"));
}

@AfterEach
public void teardown() {
// Delete test data after each test
[Link]();
}
}

[Link] 3.2 Test Containers for External Systems

• Purpose: Simulate real external systems (e.g., databases, message brokers) using Docker containers.
• Advantages:
– Tests run against a real database (e.g., PostgreSQL, MySQL) without manual setup.
– Ensures environment consistency across different machines.
• Example with Testcontainers:
@SpringBootTest
@Testcontainers
public class BookRepositoryIntegrationTest {
@Container
static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:13");

@DynamicPropertySource
static void configureProperties(DynamicPropertyRegistry registry) {

526
 [Link]("[Link]", postgres::getJdbcUrl);
[Link]("[Link]", postgres::getUsername);
[Link]("[Link]", postgres::getPassword);
}
}

11.9.4 4. Advanced Integration Testing Techniques

[Link] 4.1 Parameterized Tests
• Purpose: Run the same test logic with different inputs to cover multiple scenarios.
• Use Case: Testing validation rules (e.g., invalid book titles, empty fields).
• Example:
@ParameterizedTest
@MethodSource("provideInvalidBooks")
public void testAddBook_InvalidInput_ReturnsBadRequest(String title, String author) throws Excep
[Link](post("/books")
.contentType(MediaType.APPLICATION_JSON)
.content([Link]("{\"title\":\"%s\",\"author\":\"%s\"}", title, author)))
.andExpect(status().isBadRequest());
}

static Stream<Arguments> provideInvalidBooks() {

return [Link](
[Link]("", "Author"), // Empty title
[Link](null, "Author"), // Null title
[Link]("Title", "") // Empty author
);
}

11.9.5 5. Summary of Key Concepts

[Link] 5.1 What You Learned
1. Definition of Integration Testing:
• Validates interactions between components (vs. unit testing’s isolation).
2. Spring Boot Integration Testing:
• Use @SpringBootTest to load the full context.
• Use @AutoConfigureMockMvc for HTTP request testing.
3. Test Data Management:
• Setup/Teardown (@BeforeEach, @AfterEach) for consistent test states.
• Testcontainers for real database testing.
4. Advanced Techniques:
• Parameterized tests to cover multiple input scenarios.

[Link] 5.2 When to Use Integration Tests

• End-to-end validation (e.g., API → Service → Database).
• External system interactions (e.g., databases, third-party APIs).

527
 • Regression testing to ensure changes don’t break existing flows.

11.10 Writing Unit Tests for Controllers

11.10.1 Introduction to Unit Testing for Controllers
• Definition of Unit Testing:
– Testing individual units or components of a system in isolation from other components.
– Ensures each unit functions correctly on its own.
• Focus on Controllers:
– Controllers handle incoming HTTP requests and generate responses.
– Testing involves:
* Request handling logic.
* Response generation.
* Interactions with service layers (e.g., BookService).
– Goal: Verify controller logic behaves correctly without external dependencies.

11.10.2 Mockito for Mocking Dependencies

[Link] Key Concepts
• Mocking:
– Creating simulated objects that mimic real dependencies (e.g., services, repositories).
– Allows testing controllers without relying on actual implementations.
• Annotations:
– @Mock: Creates a mock instance of a dependency.
* Example:
@Mock
private BookService bookService; // Mock instance of BookService
– @InjectMocks: Injects mock dependencies into the controller under test.
* Example:
@InjectMocks
private BookController bookController; // BookService mock is injected here
– Purpose:
* Isolate the controller by replacing real dependencies with controlled mocks.
11.10.3 MockMvc: Testing the Web Layer
[Link] Overview
• Definition:
– Part of the Spring Test framework.
– Simulates HTTP requests and validates responses in Spring MVC applications.
– Runs in-memory (no need for a live server).
• Key Features:
– Tests the web layer (controllers, filters, etc.) in isolation.
– Validates:
* Status codes (e.g., 200 OK, 404 Not Found).
* Response content (e.g., JSON fields).
* Headers (e.g., Content-Type).
– Integrates with Spring’s testing framework for comprehensive testing.

528
[Link] Core Methods of MockMvc
1. perform(RequestBuilder):
• Executes an HTTP request (GET, POST, etc.) based on the provided RequestBuilder.
• Returns a ResultActions object for assertions.
• Example:
[Link](get("/books/1")) // Performs a GET request to /books/1
2. andExpect(ResultMatcher):
• Asserts expectations on the response (e.g., status code, content).
• Example:
.andExpect(status().isOk()) // Verifies status code is 200 OK
.andExpect(jsonPath("$.title").value("Spring Boot")) // Checks JSON field
3. andDo(ResultHandler):
• Performs additional actions on the response (e.g., logging, custom handling).
• Example:
.andDo(print()) // Prints the request/response details
4. andReturn():
• Returns the full response (including headers, body) for manual inspection.
• Example:
MvcResult result = [Link](get("/books/1")).andReturn();

[Link](get("/books/1")) // Perform GET request

.andExpect(status().isOk()) // Assert status is 200 OK
.andExpect(jsonPath("$.id").value(1)) // Assert ID field
.andExpect(jsonPath("$.title").value("Spring Boot")); // Assert title field

[Link] Example: Testing a GET Request

• Breakdown:
– get("/books/1"): Targets the endpoint for fetching a book with ID 1.
– status().isOk(): Verifies HTTP status is 200 OK.
– jsonPath("$.title"): Extracts the title field from the JSON response.
– .value("Spring Boot"): Asserts the title matches the expected value.

11.10.4 Testing a Spring MVC Controller Method

[Link] Controller Setup
• Annotations:
– @RestController: Marks the class as a RESTful web service controller.
* Automatically serializes returned objects to JSON.
– @RequestMapping("/books"): Maps all requests starting with /books to this controller.
– @GetMapping("/{id}"): Handles GET requests to /books/{id} (e.g., /books/1).
• Example Controller Method:
@RestController
@RequestMapping("/books")

529
 public class BookController {
private final BookService bookService;

public BookController(BookService bookService) {

[Link] = bookService;
}

@GetMapping("/{id}")
public ResponseEntity<Book> getBook(@PathVariable Long id) {
Book book = [Link](id); // Mocked in tests
return [Link](book); // Returns 200 OK with book data
}
}

– Key Points:
* [Link](book): Wraps the Book object in a 200 OK response.

[Link] Test Class Setup

• Annotations:
– @WebMvcTest([Link]):
* Configures Spring Boot to test only the BookController.
* Loads only the web layer (excludes service/repository layers).
– @Autowired private MockMvc mockMvc:
* Injects MockMvc to simulate HTTP requests and validate responses.
• Example Test Method:
@WebMvcTest([Link])
public class BookControllerTest {
@Autowired
private MockMvc mockMvc;

@MockBean
private BookService bookService; // Mocked service

@Test
public void testGetBook() throws Exception {
// Arrange: Mock the service response
Book mockBook = new Book(1L, "Spring Boot", "Author");
when([Link](1L)).thenReturn(mockBook);

// Act & Assert: Perform request and validate response

[Link](get("/books/1"))
.andExpect(status().isOk())
.andExpect(jsonPath("$.id").value(1))
.andExpect(jsonPath("$.title").value("Spring Boot"));
}
}

– Breakdown:

530
 1. Arrange:
* Create a mock Book object.
* Use when(...).thenReturn(...) to define the mock service behavior.
2. Act & Assert:
* Perform a GET request to /books/1.
* Assert the response has:
ꞏ Status 200 OK.
ꞏ JSON fields id=1 and title="Spring Boot".

11.10.5 Best Practices for Controller Unit Tests

1. Isolate Tests:
• Use mocking (@Mock, @MockBean) to avoid external dependencies (e.g., databases, real services).
2. Avoid External Dependencies:
• Tests should run in-memory without requiring a live server or database.
3. Cover Different Scenarios:
• Test success cases (e.g., valid IDs).
• Test error cases (e.g., invalid IDs, missing fields).
• Test edge cases (e.g., empty responses, null values).
4. Use Descriptive Assertions:
• Clearly specify expected outcomes (e.g., status codes, JSON fields).
5. Leverage Spring Test Annotations:
• @WebMvcTest: Focuses on the web layer.
• @MockBean: Replaces real beans with mocks.
• @Autowired: Injects MockMvc for HTTP simulations.

11.10.6 Summary of Key Takeaways

• Unit Testing Controllers:
– Verify request handling, response generation, and service interactions in isolation.
• Mockito:
– Use @Mock and @InjectMocks to mock dependencies and inject them into controllers.
• MockMvc:
– Simulate HTTP requests and validate responses without a live server.
– Key methods: perform(), andExpect(), andDo(), andReturn().
• Testing Workflow:
1. Mock dependencies (e.g., BookService).
2. Define test scenarios (success/error cases).
3. Perform requests using MockMvc.
4. Assert responses (status, content, headers).
• Best Practices:
– Isolate tests, avoid external dependencies, and cover all scenarios.

531