0% found this document useful (0 votes)
22 views7 pages

Postgres Practical DBA Guide

The document is a comprehensive playbook for PostgreSQL database administration, covering installation, configuration, database creation, user management, backup and restore procedures, and replication. It provides detailed steps for setting up PostgreSQL on a Linux server, managing roles and privileges, and performing daily operational checks. Additionally, it includes best practices for security hardening, performance diagnostics, and useful scripts for automation.

Uploaded by

ujjinenisubbu04
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
22 views7 pages

Postgres Practical DBA Guide

The document is a comprehensive playbook for PostgreSQL database administration, covering installation, configuration, database creation, user management, backup and restore procedures, and replication. It provides detailed steps for setting up PostgreSQL on a Linux server, managing roles and privileges, and performing daily operational checks. Additionally, it includes best practices for security hardening, performance diagnostics, and useful scripts for automation.

Uploaded by

ujjinenisubbu04
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

PostgreSQL: From Scratch — Practical DBA Playbook

A step-by-step, hands-on guide for installation, configuration, database creation, users &
roles, tablespaces, backup & restore, replication (standby), and daily operational checks.

1. Prerequisites & Environment


Assumptions:

• Linux server (RHEL/CentOS/Ubuntu).

• You have root or sudo privileges to install packages and create OS users.

• PostgreSQL binaries available (example uses PostgreSQL 16 paths where applicable).

Paths used in examples:

• PGDATA: /var/lib/pgsql/16/data

• PGHOME: /usr/pgsql-16

2. Install PostgreSQL (example: RHEL/CentOS)


Install repository and PostgreSQL server:

# CentOS/RHEL (example)
sudo dnf install -y [Link]
x86_64/[Link]
sudo dnf module disable -y postgresql
sudo dnf install -y postgresql16-server postgresql16-contrib

# Initialize database cluster (as root or via sudo)


sudo /usr/pgsql-16/bin/postgresql-16-setup initdb

Start and enable service:

sudo systemctl enable --now postgresql-16


sudo systemctl status postgresql-16

3. Create postgres OS user (if needed)


# Create group and user
sudo groupadd -f postgres
sudo useradd -m -g postgres -s /bin/bash postgres
sudo passwd postgres

Important: Use 'su - postgres' to become postgres user and use psql or initdb as postgres.

4. Initialize a new DB cluster (initdb)


Example: initialize a data directory at /var/lib/pgsql/16/data:

sudo -u postgres /usr/pgsql-16/bin/initdb -D /var/lib/pgsql/16/data -E UTF8 --


locale=en_US.UTF-8
# or if packaged: sudo /usr/pgsql-16/bin/postgresql-16-setup initdb

5. Configuration Files & Logs (locations & quick checks)


Important files inside PGDATA:

• [Link] — main server configuration (listen_addresses, port, wal settings,


memory).

• pg_hba.conf — client authentication rules (host/hostssl, database, user, address, method).

• pg_ident.conf — optional user mapping file.

• pg_wal/ (or pg_xlog) — write-ahead logs (WAL).

Logging:

• If using systemd packaged Postgres, main logs go to journalctl:

sudo journalctl -u postgresql-16 -f


# or check PGDATA/log (if configured to file)
tail -n 200 /var/lib/pgsql/16/data/log/postgresql-<date>.log

6. Create Database, Roles, and Privileges


Create a database and role (user) and grant privileges:

# As postgres OS user
sudo -u postgres psql

-- create role (login user)


CREATE ROLE app_user LOGIN PASSWORD 'S3cr3tP@ss' CREATEDB NOREPLICATION
CONNECTION LIMIT 5;

-- create database owned by user


CREATE DATABASE appdb OWNER app_user ENCODING 'UTF8' LC_COLLATE='en_US.UTF-8'
LC_CTYPE='en_US.UTF-8';
-- connect to DB and create schema objects
\c appdb
CREATE SCHEMA app AUTHORIZATION app_user;
-- create table example
CREATE TABLE [Link] (
id serial PRIMARY KEY,
username text NOT NULL,
created_ts timestamptz DEFAULT now()
);
GRANT SELECT, INSERT, UPDATE, DELETE ON [Link] TO app_user;

Role attributes and using groups:

• ROLE vs GROUP: In Postgres, roles can act as users or groups. Use CREATE ROLE for group
roles without LOGIN. Use GRANT to membership:

CREATE ROLE dba_group NOLOGIN;


GRANT dba_group TO app_user;
-- grant privileges to role
GRANT CREATE, CONNECT ON DATABASE appdb TO dba_group;

7. Create Tablespace
Steps: create OS directory, set ownership, then CREATE TABLESPACE pointing to directory:

sudo mkdir -p /u01/pg_tblspc/tbs_data1


sudo chown -R postgres:postgres /u01/pg_tblspc/tbs_data1
# inside psql as postgres or superuser:
CREATE TABLESPACE tbs_data1 LOCATION '/u01/pg_tblspc/tbs_data1';
-- create table in tablespace
CREATE TABLE app.big_table (id bigserial PRIMARY KEY, data text) TABLESPACE
tbs_data1;

8. Resource Controls & Profiles


Postgres has role attributes to limit connections and can use external OS mechanisms
(cgroups) for CPU/IO. Common controls:

-- connection limit
ALTER ROLE app_user CONNECTION LIMIT 10;

-- limit creation of new objects using privileges (do not grant CREATE on
schema/db)
REVOKE CREATE ON DATABASE appdb FROM public;
9. Backups: Logical (pg_dump) and Physical (pg_basebackup)
Logical backup (schema or table or full DB):

# schema + data for one DB


sudo -u postgres pg_dump -Fc -f /tmp/appdb_full.dump appdb

# table only
sudo -u postgres pg_dump -t [Link] -f /tmp/app_users.sql appdb

Restore logical dump:

# restore custom format


sudo -u postgres pg_restore -d appdb /tmp/appdb_full.dump
# or restore tables from dump
sudo -u postgres psql appdb < /tmp/app_users.sql

Physical base backup for replica or full physical backup:

# as postgres user
sudo -u postgres pg_basebackup -D /var/lib/pgsql/16/standby_data -Fp -Xs -P -R -
h primary_host -U replicator
# -R writes [Link]/[Link] for streaming

10. PITR and WAL Archiving


Enable archiving in [Link]:

# in [Link]
wal_level = replica
archive_mode = on
archive_command = 'test ! -f /var/lib/pgsql/wal_archive/%f && cp %p
/var/lib/pgsql/wal_archive/%f'
max_wal_senders = 5

To perform PITR (restore to time T):

• Restore base backup to PGDATA


• Put recovery signal and restore_command in [Link] or [Link] (old
versions)
• Start server and use recovery_target_time or target LSN.

11. Streaming Replication (Standby) Steps


1) Create replicator role on primary:
-- on primary psql
CREATE ROLE replicator WITH REPLICATION LOGIN PASSWORD 'replica_pass';

2) Configure primary ([Link] & pg_hba.conf):

# [Link] on primary
listen_addresses = '*'
wal_level = replica
max_wal_senders = 5
archive_mode = on
archive_command = 'cp %p /var/lib/pgsql/wal_archive/%f'

# pg_hba.conf on primary - allow replication connection from standby


host replication replicator [Link]/24 md5

3) Take base backup and start standby:

# on standby
sudo -u postgres pg_basebackup -h primary_host -D /var/lib/pgsql/16/data -U
replicator -P -R
# ensure correct permissions and start service on standby
sudo systemctl start postgresql-16
# check replication status on primary
sudo -u postgres psql -c "SELECT client_addr, state, sync_priority FROM
pg_stat_replication;"

12. Day-to-Day Operational Checks (commands & queries)


Quick checks to run daily or per shift:

• Service status:

sudo systemctl status postgresql-16


sudo journalctl -u postgresql-16 -n 200

• Connections and long-running queries:

sudo -u postgres psql -c "SELECT pid, usename, application_name, client_addr,


state, query_start, now()-query_start AS duration, query FROM pg_stat_activity
WHERE state <> 'idle' ORDER BY duration DESC LIMIT 20;"

• Replication:
sudo -u postgres psql -c "SELECT pid, application_name, client_addr, state,
sync_priority, sync_state FROM pg_stat_replication;"

• Database-wide stats and bloat:

sudo -u postgres psql -c "SELECT datname, numbackends, xact_commit,


xact_rollback, blks_hit, blks_read FROM pg_stat_database ORDER BY blks_read
DESC;"
# check bloat using pgstattuple or extensions like pg_repack

• Autovacuum and stats:

sudo -u postgres psql -c "SELECT relname, last_autovacuum, last_autoanalyze FROM


pg_stat_user_tables ORDER BY last_autovacuum NULLS FIRST;"

• Disk usage:

df -h /var/lib/pgsql
du -sh /var/lib/pgsql/16/data/*

13. Performance Diagnostics (AWR equivalents)


Install and use pg_stat_statements extension for SQL-level stats:

# enable extension in [Link]: shared_preload_libraries =


'pg_stat_statements'
# restart, then in each DB:
CREATE EXTENSION pg_stat_statements;
SELECT * FROM pg_stat_statements ORDER BY total_time DESC LIMIT 20;

Use EXPLAIN ANALYZE to tune slow queries. Capture plans and add indexes or rewrite
queries.

14. Backup Verification & Restore Testing


Always test restores in a sandbox. Validate logical dumps and physical base backups.
Example restore from pg_dump:

# create new DB and restore


sudo -u postgres createdb test_restore
sudo -u postgres pg_restore -d test_restore /tmp/appdb_full.dump
15. Security Hardening & pg_hba.conf
Use specific CIDR addresses, prefer scram-sha-256 or md5; disable trust. Example:

# only allow specific hosts for application


host appdb app_user [Link]/24 scram-sha-256
# allow replication
host replication replicator [Link]/24 scram-sha-256

16. Useful Scripts & Automation


• Health check script: combine pg_isready, pg_stat_activity, disk usage, replication status
and send email.

• Scheduled vacuum and analyze jobs for each DB during low hours.

• Use cron or systemd timers for WAL archive cleanup and backup retention policies.

Appendix: Quick Commands Reference


• Start/Stop: sudo systemctl start|stop|restart postgresql-16

• Check logs: sudo journalctl -u postgresql-16 -f

• Connect psql: sudo -u postgres psql

• Create DB: createdb -O owner dbname

• Dump DB: pg_dump -Fc -f /tmp/[Link] dbname

• Restore DB: pg_restore -d dbname /tmp/[Link]

• Base backup: pg_basebackup -D /path -h primary -U replicator -P -R

• Show active queries: SELECT * FROM pg_stat_activity;

• Replication status: SELECT * FROM pg_stat_replication;

Common questions

Powered by AI

The 'pg_hba.conf' file in PostgreSQL plays a critical role in database security by controlling client authentication. This configuration file specifies which hosts are allowed to connect to which databases, the authentication method to use, and the specific user details . It contributes to security by allowing PostgreSQL admins to define precise access controls, such as using 'scram-sha-256' or 'md5' for secure password transmission, and restricting connections to specific CIDR address ranges, which reduces the risk of unauthorized access . Additionally, configuring 'pg_hba.conf' ensures that only specified roles, like 'replicator' for replication connections, have permissions to connect, adding another layer of security to protect the database system from unauthorized data access and manipulation .

A PostgreSQL admin can manage resources and handle connection limits effectively using a few strategies: setting role attributes that limit the number of connections, such as 'ALTER ROLE app_user CONNECTION LIMIT 10;', controls excessive connections and conserves system resources . Additionally, external OS mechanisms like cgroups can further manage CPU and IO resources dedicated to PostgreSQL processes . Revoking unnecessary privileges, such as using 'REVOKE CREATE ON DATABASE appdb FROM public;', prevents users from creating new objects which might strain the database system . Monitoring resource usage through diagnostic tools, like viewing active connections via 'pg_stat_activity', and using extensions like 'pg_stat_statements' can also help identify and mitigate resource-heavy queries .

In PostgreSQL, a tablespace provides a mechanism to store database objects on specific storage devices, allowing for the efficient management of data storage, I/O optimization, and disk space utilization. To create a new tablespace, first, establish an OS directory, ensuring it is owned by the 'postgres' user for security purposes . From within psql as the 'postgres' user or another superuser, execute 'CREATE TABLESPACE tbs_data1 LOCATION '/u01/pg_tblspc/tbs_data1';', which maps the tablespace to the specified location . Associating tables or indexes with specific tablespaces, as illustrated by creating 'CREATE TABLE app.big_table TABLESPACE tbs_data1;', allows targeted data allocation . This organization can improve data retrieval performance by aligning physical storage attributes like RAID or SSD storage with the workload demands of specific database objects .

Verifying and testing the backup and restore process in PostgreSQL involves several recommended practices. Admins are advised to perform restores in a sandbox environment to ensure backups are complete and functional without risking production data . For logical backups, like those from 'pg_dump', restoring to a test database verifies the integrity of the dump file . For physical backups, using 'pg_basebackup', admins should periodically simulate a failover to validate that all files are present and WAL logs replay successfully . Testing should include scenarios like Point-In-Time Recovery (PITR) to ensure historical data recovery capabilities . This process is crucial for maintaining database integrity as it confirms that backups are reliable, minimizes data loss risk, and ensures business continuity in the event of unforeseen data corruption or system failure .

Enhancing the security of a PostgreSQL database involves several strategies that mitigate common vulnerabilities. Configuring 'pg_hba.conf' to specify precise connection rules using strong authentication methods like 'scram-sha-256' instead of weaker ones such as 'trust', mitigates unauthorized access risks . Limiting user privileges to only necessary access, such as revoking CREATE privileges from public roles or schemas, reduces the attack surface area . Enabling encryption, applying network security measures, regular updates, and patch management further protect against exploits. Furthermore, logging and monitoring using tools like 'pg_audit' and reviewing access logs aid in early detection of security breaches. These strategies help harden PostgreSQL instances against common threats, ensuring more robust and secure database operations .

A PostgreSQL DBA should perform several daily operational checks to ensure database performance and health: 1) Checking service status and logs using 'systemctl' and 'journalctl' to identify any service anomalies . 2) Monitoring active connections and long-running queries with 'pg_stat_activity' to detect and address potential performance bottlenecks . 3) Reviewing replication status through 'pg_stat_replication' to confirm standby servers are in sync with the primary server . 4) Observing database-wide statistics like transaction commits, rollbacks, and bloat using 'pg_stat_database' . 5) Checking for recent autovacuum processes to ensure they are occurring regularly and efficiently . These tasks help detect early signs of issues, maintain optimal performance, and ensure database reliability.

Using both logical and physical backups in PostgreSQL is recommended due to their complementary strengths and limitations. Logical backups, implemented via 'pg_dump', allow for exporting complete databases, specific tables, or schemas in a portable format, providing flexibility and ease of restoration to different PostgreSQL versions . However, they can be slower and resource-intensive for large databases because they require reading all data into memory, and they do not capture transaction logs, which are essential for complete point-in-time recovery . Physical backups, like those generated by 'pg_basebackup', capture the entire data directory, including WAL files, enabling exact restoration of a database to a prior state, which is ideal for quickly restoring large databases without data loss . They are storage-intensive and less portable across different PostgreSQL versions or platforms. Combining both methods allows an admin to leverage logical backups for data portability and flexibility, while physical backups provide fast, comprehensive recovery of the entire database environment .

Setting up streaming replication in PostgreSQL involves several critical steps to ensure data consistency: 1) Creating a replicator role on the primary server with REPLICATION privileges, which allows the standby to connect and replicate data . 2) Configuring the primary server's 'postgresql.conf' to enable replication settings such as 'wal_level = replica', 'max_wal_senders', 'archive_mode', and specifying 'archive_command', which manages WAL archiving for replication . Additionally, configuring 'pg_hba.conf' to allow replication connections from the standby server adds a security layer by specifying permitted IP addresses . 3) Taking a base backup of the primary server's data using 'pg_basebackup' and initializing the standby server with this data, ensuring it starts in the correct state . 4) Setting up the Standby server, including starting the PostgreSQL service and ensuring correct permissions on the data directory . These steps ensure that the standby server is an exact replica of the primary server, maintaining data consistency through synchronized WAL logs and allowing for a failover scenario.

Performing a Point-In-Time Recovery (PITR) in PostgreSQL involves restoring a base backup followed by reapplying archived WAL logs to recover the database to a specific point in time. This process starts with enabling archiving in 'postgresql.conf', ensuring 'wal_level = replica' and 'archive_mode = on', with an appropriate 'archive_command' to store WAL logs . During recovery, restore the base backup to PGDATA, and place a recovery signal (e.g., 'standby.signal' or appropriate configuration in 'recovery.conf') to guide PostgreSQL on which WAL files to apply . The server is then started, and recovery is directed to a target time ('recovery_target_time') or target LSN. This feature enhances data recovery capabilities by allowing precise restoration to a desired time before an erroneous transaction, minimizing data loss and ensuring database integrity following unexpected events or human error .

PostgreSQL administrators can use the 'pg_stat_statements' extension to diagnose performance issues by tracking executed SQL queries and their execution metrics. To enable this extension, the 'shared_preload_libraries' configuration in 'postgresql.conf' must include 'pg_stat_statements', requiring a database restart . Within each database, running 'CREATE EXTENSION pg_stat_statements;' allows access to detailed statistics such as query execution times, frequency, and efficiency . Analyzing these metrics helps identify the most resource-intensive queries, facilitating performance optimization through query tuning, rewriting inefficient statements, or adding indexes . Sorting query data by 'total_time' highlights candidates for the most significant performance gains, making 'pg_stat_statements' invaluable for ongoing query optimization efforts .

You might also like