Module 12 Building Database Apps With PostgreSQL & Python
Module 12 Building Database Apps With PostgreSQL & Python
Python
Welcome to Module 12! This module will introduce you to the world of databases,
specifically PostgreSQL, and teach you how to build robust applications by connecting your
Python code to a powerful relational database. Data is at the heart of most modern
applications and understanding how to manage it is a critical skill.
Definition: Data refers to raw facts, figures, statistics, or items of information that, in
their initial form, may not convey much meaning on their own.
Examples: A single number like 25, a word like Hyderabad, a date like 2025-07-02,
or a name like Alice.
Data vs. Information: When data is processed, organized, structured, or presented in
a given context to make it meaningful or useful, it becomes information.
o Data: Alice, 30, Hyderabad
o Information: "Alice is 30 years old and lives in Hyderabad."
1. Structured Data:
o Highly organized and formatted in a way that makes it easily searchable and
manageable.
o Fits neatly into a fixed schema (like rows and columns in a table).
o Examples: Data in relational databases (SQL databases), spreadsheets
(Excel), CSV files.
o Characteristics: Defined data types, clear relationships.
2. Unstructured Data:
o Lacks a predefined format or organization. It cannot be easily put into a
traditional row-and-column database.
o Examples: Text documents, emails, social media posts, audio files, video
files, images.
o Characteristics: Difficult to search, requires advanced analytics tools.
3. Semi-structured Data:
o Has some organizational properties, but it's not strictly rigid like structured
data. It contains tags or other markers to identify distinct elements and impose
hierarchy.
o Examples: XML, JSON files, NoSQL databases (like MongoDB documents).
1
o Characteristics: Flexible schema, easier to process than unstructured data,
but less rigid than structured.
In today's digital world, data is constantly being generated. Effective data management is
crucial because it ensures:
Managing large amounts of data manually (e.g., in text files or simple spreadsheets) quickly
becomes impractical. This is where databases come in.
While there are many types, the most common distinction is between:
2
Adhere to ACID properties (Atomicity, Consistency, Isolation, Durability)
o
for transaction reliability.
o Examples: PostgreSQL, MySQL, Oracle, SQL Server, SQLite.
2. NoSQL Databases (Non-relational Databases):
o Designed for specific data models and have flexible schemas for handling
modern applications' varying data access patterns.
o Do not primarily use SQL.
o Examples: Document databases (MongoDB), Key-value stores (Redis),
Column-family stores (Cassandra), Graph databases (Neo4j).
Now that we understand databases, let's look at the specific RDBMS we'll be using:
PostgreSQL.
3
It has been under active development for more than 35 years and is backed by a large,
dedicated community of developers.
Open Source: Free to use, modify, and distribute, making it very popular for startups
and large enterprises alike.
Robustness & Reliability: Adheres strictly to ACID (Atomicity, Consistency,
Isolation, Durability) properties, ensuring data integrity even in the event of system
failures.
Extensibility: Highly extensible; you can define your own data types, functions,
operators, and even programming languages.
Advanced Features: Supports complex queries, full-text search, JSON, XML,
geographic data (PostGIS), and more.
Concurrency Control: Uses Multi-Version Concurrency Control (MVCC) to allow
many users to work with the database concurrently without interfering with each
other.
Community Support: A large and active community provides extensive
documentation, forums, and support.
Cross-Platform: Runs on all major operating systems, including Linux, Windows,
and macOS.
Web Applications: Backend for dynamic websites and web services (often with
frameworks like Django, Flask).
Data Warehousing & Analytics: Its robust features make it suitable for large-scale
data storage and complex analytical queries.
Geographic Information Systems (GIS): With PostGIS extension, it's a leading
database for spatial data.
Financial Applications: Its strong ACID compliance is critical for transactional
systems.
Scientific Data: Used for managing complex datasets in research.
Before we can interact with PostgreSQL, we need to install it on your system. The
installation process varies slightly depending on your operating system.
4
2. Installer: Download the "EDB Interactive installer" (or similar graphical installer for
your OS).
3. Run Installer:
o Installation Directory: Choose where to install PostgreSQL.
o Select Components: Make sure to select "PostgreSQL Server" and
"pgAdmin 4" (the graphical administration tool). You might also want
"Command Line Tools."
o Data Directory: Choose where your database files will be stored.
o Password for postgres user: This is crucial! You will be asked to set a
password for the default postgres superuser. Remember this password!
o Port: The default port is 5432. You can usually leave this as default unless it
conflicts with other software.
o Locale: Choose your preferred language/region settings.
4. Finish Installation: The installer will complete the setup and start the PostgreSQL
server service.
Bash
psql -U postgres
oIt will ask for the password you set during installation.
oIf successful, you'll see a postgres=# prompt. This means you are connected
to the postgres database as the postgres user.
o Type \q and press Enter to exit psql.
3. Use pgAdmin 4 (Graphical Tool):
o Launch pgAdmin 4 (usually available in your applications menu).
o It will likely prompt you to set a master password for pgAdmin itself.
o You should see a "Servers" section. Expand it, and you'll likely see
"PostgreSQL x.x". Double-click it.
o It will ask for the postgres user's password. Enter it.
o If successful, you can browse your databases, tables, etc., graphically.
5
Chapter 5: Creating a Database (SQL)
Once PostgreSQL is installed, your first step is to create a new database where your
application's data will reside. We'll do this using SQL commands via the psql command-line
tool.
To execute SQL commands, you first need to connect to the PostgreSQL server.
Bash
Syntax:
SQL
oDatabase names are typically lowercase and can contain letters, numbers, and
underscores.
o SQL commands usually end with a semicolon ;.
Example: Let's create a database for our Python application, named myapp_db.
SQL
To confirm your database has been created, you can list all databases on the server.
psql Meta-command:
6
SQL
\l
# or
\list
Once a database is created, you usually want to connect to it specifically to work with its
tables.
psql Meta-command:
SQL
\c myapp_db
# or
\connect myapp_db
You should see output like You are now connected to database "myapp_db" as
user "postgres". Your prompt will change to myapp_db=#, indicating you're now
working within that database.
Exit psql:
SQL
\q
Sometimes, you might need to remove an existing database. This is done using the DROP
DATABASE command.
This command permanently deletes a database and all its contents. Use with extreme caution!
Syntax:
SQL
Example: Let's assume you created a test database called test_db and now want to
remove it.
7
o First, connect to a different database (e.g., postgres or any other database)
because you cannot drop a database you are currently connected to.
Bash
psql -U postgres
# password prompt
# Now you are connected to postgres=#
SQL
You CANNOT drop a database if you are currently connected to it. If you try,
you'll get an error like "ERROR: cannot drop the currently open database".
Always connect to a different database (like the default postgres database) before
attempting to drop the one you intend to remove.
Dropping a database is irreversible and will delete all tables, data, and other objects
within it.
Now that we have a database (myapp_db), let's create a table within it and populate it with
some initial data.
Before creating tables, ensure you are connected to the correct database.
Bash
This command defines the structure of your table, including column names, data types, and
constraints.
8
o INT: Integer numbers.
o SERIAL: Auto-incrementing integer, commonly used for primary keys.
o VARCHAR(n): Variable-length string, n is the maximum length.
o TEXT: Variable-length string, no explicit length limit.
o BOOLEAN: TRUE or FALSE.
o DATE: Date (e.g., 'YYYY-MM-DD').
o TIMESTAMP: Date and time.
o NUMERIC(p, s) or DECIMAL(p, s): Exact numeric, p is precision (total
digits), s is scale (digits after decimal point).
Common Constraints:
o PRIMARY KEY: Uniquely identifies each row; implies NOT NULL and UNIQUE.
o NOT NULL: Ensures a column cannot have NULL values.
o UNIQUE: Ensures all values in a column are different.
o DEFAULT value: Provides a default value if none is specified during insertion.
Example: Creating a Users table Let's create a table called Users to store user
information.
SQL
SQL
\dt
This meta-command lists tables in the current database. You should see users. To see
the table structure: \d users
SQL
9
VALUES (value1, value2, ...);
oYou can omit the column list if you provide values for all columns in the exact
order they appear in the table definition.
Example: Adding data to Users table
SQL
SQL
SQL
Chapter 8: Retrieving Data from Database and Deleting Contents in the Table
(SQL)
Now that we have data, let's learn how to view it and how to remove it.
The SELECT statement is used to retrieve data from one or more tables. It's the most
frequently used SQL command.
SQL
This will retrieve all columns (*) for all rows from the Users table.
SQL
10
SELECT username, email FROM Users;
This will retrieve only the username and email columns for all rows.
Filtering Data with WHERE Clause The WHERE clause is used to specify a condition to
filter the rows returned by the SELECT statement.
SQL
This will retrieve all columns for users whose age is greater than 28.
SQL
This retrieves username and email for the user with that specific username.
The DELETE FROM statement is used to remove existing rows from a table.
Deleting Specific Rows: To delete specific rows, you must use a WHERE clause.
Without it, you will delete all rows!
SQL
This will delete the row where the username is 'bob_jones'. You'll see DELETE 1 if
one row was deleted.
Verify Deletion:
SQL
Deleting All Contents (but keeping table structure): To remove all data from a
table without deleting the table itself, you use DELETE FROM without a WHERE clause.
SQL
This will delete all rows from the Users table. You'll see DELETE X where X is the
number of rows deleted.
11
Warning: This is a powerful command. Be absolutely sure before executing it!
Alternative (faster for full table clear): TRUNCATE TABLE If you want to quickly
remove all rows from a table and reset auto-incrementing counters, TRUNCATE TABLE
is often faster than DELETE FROM without a WHERE clause. It does not allow a WHERE
clause.
SQL
Exit psql:
SQL
\q
Before we install Python libraries for database interaction, it's a best practice to set up a
virtual environment.
venv is now built into Python 3.3 and later, so you usually don't need to install virtualenv
as a separate package. However, if you are on an older Python 3 version or prefer the
virtualenv package for its extra features, you can install it globally:
Bash
(If you have Python 3.3+, you can skip this step and use python -m venv directly)
12
9.3 Creating a Virtual Environment
Navigate to your project directory (or create a new one for this module).
Bash
Now, create the virtual environment. Let's call it venv (a common convention).
Bash
This command creates a directory named venv inside your current directory. This
directory contains a minimal Python installation and a pip installer just for this
environment.
Before you can install packages into your virtual environment, you must activate it.
On Windows:
Bash
.\venv\Scripts\activate
or
Bash
venv\Scripts\[Link]
On macOS / Linux:
Bash
source venv/bin/activate
After activation, your terminal prompt will change to include the name of your virtual
environment (e.g., (venv) C:\...\DatabaseApp>). This indicates that you are now
operating within the isolated environment. Any pip install commands will now install
packages into this venv directory.
13
9.5 Deactivating the Virtual Environment
When you're done working on your project, you can exit the virtual environment.
Bash
deactivate
Now that your virtual environment is set up and activated, we can install the necessary
Python library to interact with PostgreSQL.
10.2 Installation
Make sure your virtual environment is activated before running the installation command.
Bash
10.3 Verification
Bash
(venv) python
14
Python
Python
>>> exit()
This is where the real integration begins! We'll write Python code to establish a connection to
your PostgreSQL database.
The first line in your Python script will be to import the library.
Python
import psycopg2
11.2 Connection Parameters
dbname: The name of the database you want to connect to (e.g., myapp_db).
user: The username to connect with (e.g., postgres).
password: The password for that user.
host: The database server's address (usually localhost if on your machine).
port: The port the database server is listening on (default is 5432).
11.3 [Link]()
Python
conn = [Link](
dbname="myapp_db",
user="postgres",
password="your_postgres_password", # IMPORTANT: Replace with your
actual password
host="localhost",
port="5432"
)
15
11.4 Cursor Object
Once you have a connection object, you need a cursor object. The cursor allows you to
execute SQL commands and fetch results from the database.
Python
cur = [Link]()
11.5 Handling Connections (try...finally or with statement)
It's crucial to always close your database connections and cursors to free up resources. The
try...finally block or the with statement are excellent for this.
Using try...finally:
Python
try:
conn = [Link](...)
cur = [Link]()
# Your database operations go here
print("Database connection successful!")
except [Link] as e:
print(f"Error connecting to database: {e}")
finally:
if 'cur' in locals() and cur: # Check if cursor was created
[Link]()
if 'conn' in locals() and conn: # Check if connection was created
[Link]()
print("Database connection closed.")
Using with statement (Recommended for cleaner code):
psycopg2 connection and cursor objects can be used with Python's with statement, which
automatically handles closing them even if errors occur.
Python
try:
with [Link](dbname="myapp_db", user="postgres",
password="your_postgres_password", host="localhost", port="5432") as conn:
with [Link]() as cur:
# Your database operations go here
print("Database connection successful!")
except [Link] as e:
print(f"Error connecting to database: {e}")
print("Database connection closed (handled by 'with' statement).")
11.6 Example Code: Basic Connection and Disconnection
Python
# connect_db.py
import psycopg2
16
def test_db_connection():
conn = None
cur = None
try:
# Replace with your actual PostgreSQL password
conn = [Link](
dbname="myapp_db",
user="postgres",
password="your_postgres_password",
host="localhost",
port="5432"
)
cur = [Link]()
print("Successfully connected to the database!")
# You can execute a simple query to confirm
[Link]("SELECT version();")
db_version = [Link]()[0]
print(f"PostgreSQL version: {db_version}")
except [Link] as e:
print(f"Database connection error: {e}")
finally:
if cur:
[Link]()
if conn:
[Link]()
print("Database connection closed.")
if __name__ == "__main__":
test_db_connection()
To run this:
Expected Output:
Now that we can connect, let's use Python to execute a CREATE TABLE SQL command. This
is useful for programmatically setting up your database schema.
You'll define your SQL CREATE TABLE statement as a multi-line Python string.
17
12.2 [Link]()
12.3 [Link]()
For SQL commands that modify the database (like CREATE TABLE, INSERT, UPDATE,
DELETE), these changes are initially staged in a transaction.
To make these changes permanent in the database, you must call [Link](). If
you don't commit, the changes will not be saved.
For SELECT queries, commit() is not strictly necessary, but it doesn't hurt.
Let's create a new table called Products with product_id, name, price, and
stock_quantity.
Python
# create_table.py
import psycopg2
def create_products_table():
conn = None
try:
conn = [Link](
dbname="myapp_db",
user="postgres",
password="your_postgres_password", # Replace
host="localhost",
port="5432"
)
cur = [Link]()
create_table_sql = """
CREATE TABLE IF NOT EXISTS Products (
product_id SERIAL PRIMARY KEY,
name VARCHAR(100) UNIQUE NOT NULL,
price NUMERIC(10, 2) NOT NULL,
stock_quantity INT DEFAULT 0
);
"""
[Link](create_table_sql)
[Link]() # Commit the transaction to make changes permanent
except [Link] as e:
print(f"Error creating table: {e}")
finally:
if conn:
[Link]()
if __name__ == "__main__":
create_products_table()
18
To run this:
Now, let's insert some data into our Products table using Python.
When inserting data from Python, it's crucial to use parameterized queries to prevent SQL
Injection vulnerabilities and properly handle data types. psycopg2 uses %s as a placeholder
for parameters.
Syntax:
SQL
13.3 [Link]()
# insert_data.py
import psycopg2
19
conn = None
try:
conn = [Link](
dbname="myapp_db",
user="postgres",
password="your_postgres_password", # Replace
host="localhost",
port="5432"
)
cur = [Link]()
sql_insert = """
INSERT INTO Products (name, price, stock_quantity)
VALUES (%s, %s, %s);
"""
[Link](sql_insert, (name, price, quantity))
[Link]()
print(f"Successfully inserted '{name}'.")
except [Link] as e:
print(f"Error inserting single product: {e}")
finally:
if conn:
[Link]()
def insert_multiple_products(products_list):
conn = None
try:
conn = [Link](
dbname="myapp_db",
user="postgres",
password="your_postgres_password", # Replace
host="localhost",
port="5432"
)
cur = [Link]()
sql_insert_many = """
INSERT INTO Products (name, price, stock_quantity)
VALUES (%s, %s, %s);
"""
# products_list is a list of tuples: [('name', price, quantity),
...]
[Link](sql_insert_many, products_list)
[Link]()
print(f"Successfully inserted {len(products_list)} products.")
except [Link] as e:
print(f"Error inserting multiple products: {e}")
finally:
if conn:
[Link]()
if __name__ == "__main__":
# Insert a single product
insert_single_product("Laptop", 1200.00, 10)
20
("Monitor", 300.00, 15)
]
insert_multiple_products(new_products)
To run this:
Reading data from the database is fundamental. This is done using SELECT queries with
Python.
14.2 [Link](sql)
Execute the SELECT query using [Link](). No commit() is needed for SELECT
operations.
After executing a SELECT query, the results are stored in the cursor object. You use fetch
methods to retrieve them:
[Link](): Retrieves the next row of a query result set, returning a single
tuple, or None when no more data is available.
[Link](): Retrieves all (remaining) rows of a query result set, returning a
list of tuples. An empty list is returned if no rows are available.
[Link](size): Retrieves the next set of rows (up to size) of a query
result, returning a list of tuples.
If using fetchall(), you typically loop through the list of tuples. If using fetchone(), you
might loop until None is returned.
21
# extract_data.py
import psycopg2
def get_all_products():
conn = None
try:
conn = [Link](
dbname="myapp_db",
user="postgres",
password="your_postgres_password", # Replace
host="localhost",
port="5432"
)
cur = [Link]()
if products:
print("\n--- All Products ---")
for product in products:
print(f"ID: {product[0]}, Name: {product[1]}, Price:
{product[2]}, Stock: {product[3]}")
else:
print("\nNo products found.")
except [Link] as e:
print(f"Error retrieving all products: {e}")
finally:
if conn:
[Link]()
def get_product_by_name(product_name):
conn = None
try:
conn = [Link](
dbname="myapp_db",
user="postgres",
password="your_postgres_password", # Replace
host="localhost",
port="5432"
)
cur = [Link]()
if product:
print(f"\n--- Product '{product_name}' Found ---")
print(f"ID: {product[0]}, Name: {product[1]}, Price:
{product[2]}, Stock: {product[3]}")
else:
print(f"\nProduct '{product_name}' not found.")
except [Link] as e:
print(f"Error retrieving product by name: {e}")
22
finally:
if conn:
[Link]()
if __name__ == "__main__":
get_all_products()
get_product_by_name("Keyboard")
get_product_by_name("Webcam") # This product does not exist
To run this:
Finally, let's combine user input with our database operations, allowing users to interact with
your data.
You can use Python's built-in input() function for simple command-line user interaction, or
integrate it with a Tkinter GUI for more complex applications (as you learned in Module 9).
For simplicity, we'll use input() here.
It's paramount to always use parameterized queries (%s placeholders) when inserting or
updating data based on user input. NEVER use f-strings or direct string concatenation to
build SQL queries with user input. This prevents a major security vulnerability known as
SQL Injection.
# user_input_db_app.py
import psycopg2
def get_db_connection():
23
"""Helper function to create and return a database connection."""
return [Link](
dbname="myapp_db",
user="postgres",
password="your_postgres_password", # Replace
host="localhost",
port="5432"
)
def add_new_product():
name = input("Enter product name: ")
try:
price = float(input("Enter price: "))
quantity = int(input("Enter stock quantity: "))
except ValueError:
print("Invalid price or quantity. Please enter numbers.")
return
conn = None
try:
conn = get_db_connection()
cur = [Link]()
sql = "INSERT INTO Products (name, price, stock_quantity) VALUES
(%s, %s, %s);"
[Link](sql, (name, price, quantity))
[Link]()
print(f"Product '{name}' added successfully!")
except [Link] as e:
print(f"Error adding product: {e}")
finally:
if conn:
[Link]()
def find_product_by_name():
name = input("Enter product name to search: ")
conn = None
try:
conn = get_db_connection()
cur = [Link]()
sql = "SELECT product_id, name, price, stock_quantity FROM Products
WHERE name = %s;"
[Link](sql, (name,))
product = [Link]()
if product:
print(f"\n--- Product Details ---")
print(f"ID: {product[0]}, Name: {product[1]}, Price:
{product[2]}, Stock: {product[3]}")
else:
print(f"Product '{name}' not found.")
except [Link] as e:
print(f"Error searching for product: {e}")
finally:
if conn:
[Link]()
def delete_product_by_name():
name = input("Enter product name to delete: ")
conn = None
try:
conn = get_db_connection()
24
cur = [Link]()
sql = "DELETE FROM Products WHERE name = %s;"
[Link](sql, (name,))
[Link]()
if [Link] > 0: # Check if any rows were affected
print(f"Product '{name}' deleted successfully!")
else:
print(f"Product '{name}' not found for deletion.")
except [Link] as e:
print(f"Error deleting product: {e}")
finally:
if conn:
[Link]()
def view_all_products():
conn = None
try:
conn = get_db_connection()
cur = [Link]()
[Link]("SELECT product_id, name, price, stock_quantity FROM
Products ORDER BY product_id;")
products = [Link]()
if products:
print("\n--- All Products in Database ---")
for p in products:
print(f"ID: {p[0]}, Name: {p[1]}, Price: {p[2]}, Stock:
{p[3]}")
else:
print("\nNo products found in the database.")
except [Link] as e:
print(f"Error viewing products: {e}")
finally:
if conn:
[Link]()
def main_menu():
while True:
print("\n--- Product Management System ---")
print("1. Add New Product")
print("2. Search Product by Name")
print("3. Delete Product by Name")
print("4. View All Products")
print("5. Exit")
choice = input("Enter your choice: ")
if choice == '1':
add_new_product()
elif choice == '2':
find_product_by_name()
elif choice == '3':
delete_product_by_name()
elif choice == '4':
view_all_products()
elif choice == '5':
print("Exiting application. Goodbye!")
break
else:
print("Invalid choice. Please try again.")
25
if __name__ == "__main__":
main_menu()
To run this:
Expected Interaction: The script will present a menu. You can choose options to add
products (it will prompt for name, price, quantity), search for products, delete products, or
view all existing products in your myapp_db.
26