# PDF Processing Advanced Reference
This document contains advanced PDF processing features, detailed
examples, and additional libraries not covered in the main skill
instructions.
## pypdfium2 Library (Apache/BSD License)
### Overview
pypdfium2 is a Python binding for PDFium (Chromium's PDF library). It's
excellent for fast PDF rendering, image generation, and serves as a
PyMuPDF replacement.
### Render PDF to Images
```python
import pypdfium2 as pdfium
from PIL import Image
# Load PDF
pdf = [Link]("[Link]")
# Render page to image
page = pdf[0] # First page
bitmap = [Link](
scale=2.0, # Higher resolution
rotation=0 # No rotation
)
# Convert to PIL Image
img = bitmap.to_pil()
[Link]("page_1.png", "PNG")
# Process multiple pages
for i, page in enumerate(pdf):
bitmap = [Link](scale=1.5)
img = bitmap.to_pil()
[Link](f"page_{i+1}.jpg", "JPEG", quality=90)
```
### Extract Text with pypdfium2
```python
import pypdfium2 as pdfium
pdf = [Link]("[Link]")
for i, page in enumerate(pdf):
text = page.get_text()
print(f"Page {i+1} text length: {len(text)} chars")
```
## JavaScript Libraries
### pdf-lib (MIT License)
pdf-lib is a powerful JavaScript library for creating and modifying PDF
documents in any JavaScript environment.
#### Load and Manipulate Existing PDF
```javascript
import { PDFDocument } from 'pdf-lib';
import fs from 'fs';
async function manipulatePDF() {
// Load existing PDF
const existingPdfBytes = [Link]('[Link]');
const pdfDoc = await [Link](existingPdfBytes);
// Get page count
const pageCount = [Link]();
[Link](`Document has ${pageCount} pages`);
// Add new page
const newPage = [Link]([600, 400]);
[Link]('Added by pdf-lib', {
x: 100,
y: 300,
size: 16
});
// Save modified PDF
const pdfBytes = await [Link]();
[Link]('[Link]', pdfBytes);
}
```
#### Create Complex PDFs from Scratch
```javascript
import { PDFDocument, rgb, StandardFonts } from 'pdf-lib';
import fs from 'fs';
async function createPDF() {
const pdfDoc = await [Link]();
// Add fonts
const helveticaFont = await
[Link]([Link]);
const helveticaBold = await
[Link]([Link]);
// Add page
const page = [Link]([595, 842]); // A4 size
const { width, height } = [Link]();
// Add text with styling
[Link]('Invoice #12345', {
x: 50,
y: height - 50,
size: 18,
font: helveticaBold,
color: rgb(0.2, 0.2, 0.8)
});
// Add rectangle (header background)
[Link]({
x: 40,
y: height - 100,
width: width - 80,
height: 30,
color: rgb(0.9, 0.9, 0.9)
});
// Add table-like content
const items = [
['Item', 'Qty', 'Price', 'Total'],
['Widget', '2', '$50', '$100'],
['Gadget', '1', '$75', '$75']
];
let yPos = height - 150;
[Link](row => {
let xPos = 50;
[Link](cell => {
[Link](cell, {
x: xPos,
y: yPos,
size: 12,
font: helveticaFont
});
xPos += 120;
});
yPos -= 25;
});
const pdfBytes = await [Link]();
[Link]('[Link]', pdfBytes);
}
```
#### Advanced Merge and Split Operations
```javascript
import { PDFDocument } from 'pdf-lib';
import fs from 'fs';
async function mergePDFs() {
// Create new document
const mergedPdf = await [Link]();
// Load source PDFs
const pdf1Bytes = [Link]('[Link]');
const pdf2Bytes = [Link]('[Link]');
const pdf1 = await [Link](pdf1Bytes);
const pdf2 = await [Link](pdf2Bytes);
// Copy pages from first PDF
const pdf1Pages = await [Link](pdf1,
[Link]());
[Link](page => [Link](page));
// Copy specific pages from second PDF (pages 0, 2, 4)
const pdf2Pages = await [Link](pdf2, [0, 2, 4]);
[Link](page => [Link](page));
const mergedPdfBytes = await [Link]();
[Link]('[Link]', mergedPdfBytes);
}
```
### pdfjs-dist (Apache License)
[Link] is Mozilla's JavaScript library for rendering PDFs in the browser.
#### Basic PDF Loading and Rendering
```javascript
import * as pdfjsLib from 'pdfjs-dist';
// Configure worker (important for performance)
[Link] = './[Link]';
async function renderPDF() {
// Load PDF
const loadingTask = [Link]('[Link]');
const pdf = await [Link];
[Link](`Loaded PDF with ${[Link]} pages`);
// Get first page
const page = await [Link](1);
const viewport = [Link]({ scale: 1.5 });
// Render to canvas
const canvas = [Link]('canvas');
const context = [Link]('2d');
[Link] = [Link];
[Link] = [Link];
const renderContext = {
canvasContext: context,
viewport: viewport
};
await [Link](renderContext).promise;
[Link](canvas);
}
```
#### Extract Text with Coordinates
```javascript
import * as pdfjsLib from 'pdfjs-dist';
async function extractText() {
const loadingTask = [Link]('[Link]');
const pdf = await [Link];
let fullText = '';
// Extract text from all pages
for (let i = 1; i <= [Link]; i++) {
const page = await [Link](i);
const textContent = await [Link]();
const pageText = [Link]
.map(item => [Link])
.join(' ');
fullText += `\n--- Page ${i} ---\n${pageText}`;
// Get text with coordinates for advanced processing
const textWithCoords = [Link](item => ({
text: [Link],
x: [Link][4],
y: [Link][5],
width: [Link],
height: [Link]
}));
}
[Link](fullText);
return fullText;
}
```
#### Extract Annotations and Forms
```javascript
import * as pdfjsLib from 'pdfjs-dist';
async function extractAnnotations() {
const loadingTask = [Link]('[Link]');
const pdf = await [Link];
for (let i = 1; i <= [Link]; i++) {
const page = await [Link](i);
const annotations = await [Link]();
[Link](annotation => {
[Link](`Annotation type: ${[Link]}`);
[Link](`Content: ${[Link]}`);
[Link](`Coordinates:
${[Link]([Link])}`);
});
}
}
```
## Advanced Command-Line Operations
### poppler-utils Advanced Features
#### Extract Text with Bounding Box Coordinates
```bash
# Extract text with bounding box coordinates (essential for structured
data)
pdftotext -bbox-layout [Link] [Link]
# The XML output contains precise coordinates for each text element
```
#### Advanced Image Conversion
```bash
# Convert to PNG images with specific resolution
pdftoppm -png -r 300 [Link] output_prefix
# Convert specific page range with high resolution
pdftoppm -png -r 600 -f 1 -l 3 [Link] high_res_pages
# Convert to JPEG with quality setting
pdftoppm -jpeg -jpegopt quality=85 -r 200 [Link] jpeg_output
```
#### Extract Embedded Images
```bash
# Extract all embedded images with metadata
pdfimages -j -p [Link] page_images
# List image info without extracting
pdfimages -list [Link]
# Extract images in their original format
pdfimages -all [Link] images/img
```
### qpdf Advanced Features
#### Complex Page Manipulation
```bash
# Split PDF into groups of pages
qpdf --split-pages=3 [Link] output_group_%[Link]
# Extract specific pages with complex ranges
qpdf [Link] --pages [Link] 1,3-5,8,10-end -- [Link]
# Merge specific pages from multiple PDFs
qpdf --empty --pages [Link] 1-3 [Link] 5-7 [Link] 2,4 --
[Link]
```
#### PDF Optimization and Repair
```bash
# Optimize PDF for web (linearize for streaming)
qpdf --linearize [Link] [Link]
# Remove unused objects and compress
qpdf --optimize-level=all [Link] [Link]
# Attempt to repair corrupted PDF structure
qpdf --check [Link]
qpdf --fix-qdf [Link] [Link]
# Show detailed PDF structure for debugging
qpdf --show-all-pages [Link] > [Link]
```
#### Advanced Encryption
```bash
# Add password protection with specific permissions
qpdf --encrypt user_pass owner_pass 256 --print=none --modify=none --
[Link] [Link]
# Check encryption status
qpdf --show-encryption [Link]
# Remove password protection (requires password)
qpdf --password=secret123 --decrypt [Link] [Link]
```
## Advanced Python Techniques
### pdfplumber Advanced Features
#### Extract Text with Precise Coordinates
```python
import pdfplumber
with [Link]("[Link]") as pdf:
page = [Link][0]
# Extract all text with coordinates
chars = [Link]
for char in chars[:10]: # First 10 characters
print(f"Char: '{char['text']}' at x:{char['x0']:.1f}
y:{char['y0']:.1f}")
# Extract text by bounding box (left, top, right, bottom)
bbox_text = page.within_bbox((100, 100, 400, 200)).extract_text()
```
#### Advanced Table Extraction with Custom Settings
```python
import pdfplumber
import pandas as pd
with [Link]("complex_table.pdf") as pdf:
page = [Link][0]
# Extract tables with custom settings for complex layouts
table_settings = {
"vertical_strategy": "lines",
"horizontal_strategy": "lines",
"snap_tolerance": 3,
"intersection_tolerance": 15
}
tables = page.extract_tables(table_settings)
# Visual debugging for table extraction
img = page.to_image(resolution=150)
[Link]("debug_layout.png")
```
### reportlab Advanced Features
#### Create Professional Reports with Tables
```python
from [Link] import SimpleDocTemplate, Table, TableStyle,
Paragraph
from [Link] import getSampleStyleSheet
from [Link] import colors
# Sample data
data = [
['Product', 'Q1', 'Q2', 'Q3', 'Q4'],
['Widgets', '120', '135', '142', '158'],
['Gadgets', '85', '92', '98', '105']
]
# Create PDF with table
doc = SimpleDocTemplate("[Link]")
elements = []
# Add title
styles = getSampleStyleSheet()
title = Paragraph("Quarterly Sales Report", styles['Title'])
[Link](title)
# Add table with advanced styling
table = Table(data)
[Link](TableStyle([
('BACKGROUND', (0, 0), (-1, 0), [Link]),
('TEXTCOLOR', (0, 0), (-1, 0), [Link]),
('ALIGN', (0, 0), (-1, -1), 'CENTER'),
('FONTNAME', (0, 0), (-1, 0), 'Helvetica-Bold'),
('FONTSIZE', (0, 0), (-1, 0), 14),
('BOTTOMPADDING', (0, 0), (-1, 0), 12),
('BACKGROUND', (0, 1), (-1, -1), [Link]),
('GRID', (0, 0), (-1, -1), 1, [Link])
]))
[Link](table)
[Link](elements)
```
## Complex Workflows
### Extract Figures/Images from PDF
#### Method 1: Using pdfimages (fastest)
```bash
# Extract all images with original quality
pdfimages -all [Link] images/img
```
#### Method 2: Using pypdfium2 + Image Processing
```python
import pypdfium2 as pdfium
from PIL import Image
import numpy as np
def extract_figures(pdf_path, output_dir):
pdf = [Link](pdf_path)
for page_num, page in enumerate(pdf):
# Render high-resolution page
bitmap = [Link](scale=3.0)
img = bitmap.to_pil()
# Convert to numpy for processing
img_array = [Link](img)
# Simple figure detection (non-white regions)
mask = [Link](img_array != [255, 255, 255], axis=2)
# Find contours and extract bounding boxes
# (This is simplified - real implementation would need more
sophisticated detection)
# Save detected figures
# ... implementation depends on specific needs
```
### Batch PDF Processing with Error Handling
```python
import os
import glob
from pypdf import PdfReader, PdfWriter
import logging
[Link](level=[Link])
logger = [Link](__name__)
def batch_process_pdfs(input_dir, operation='merge'):
pdf_files = [Link]([Link](input_dir, "*.pdf"))
if operation == 'merge':
writer = PdfWriter()
for pdf_file in pdf_files:
try:
reader = PdfReader(pdf_file)
for page in [Link]:
writer.add_page(page)
[Link](f"Processed: {pdf_file}")
except Exception as e:
[Link](f"Failed to process {pdf_file}: {e}")
continue
with open("batch_merged.pdf", "wb") as output:
[Link](output)
elif operation == 'extract_text':
for pdf_file in pdf_files:
try:
reader = PdfReader(pdf_file)
text = ""
for page in [Link]:
text += page.extract_text()
output_file = pdf_file.replace('.pdf', '.txt')
with open(output_file, 'w', encoding='utf-8') as f:
[Link](text)
[Link](f"Extracted text from: {pdf_file}")
except Exception as e:
[Link](f"Failed to extract text from {pdf_file}:
{e}")
continue
```
### Advanced PDF Cropping
```python
from pypdf import PdfWriter, PdfReader
reader = PdfReader("[Link]")
writer = PdfWriter()
# Crop page (left, bottom, right, top in points)
page = [Link][0]
[Link] = 50
[Link] = 50
[Link] = 550
[Link] = 750
writer.add_page(page)
with open("[Link]", "wb") as output:
[Link](output)
```
## Performance Optimization Tips
### 1. For Large PDFs
- Use streaming approaches instead of loading entire PDF in memory
- Use `qpdf --split-pages` for splitting large files
- Process pages individually with pypdfium2
### 2. For Text Extraction
- `pdftotext -bbox-layout` is fastest for plain text extraction
- Use pdfplumber for structured data and tables
- Avoid `pypdf.extract_text()` for very large documents
### 3. For Image Extraction
- `pdfimages` is much faster than rendering pages
- Use low resolution for previews, high resolution for final output
### 4. For Form Filling
- pdf-lib maintains form structure better than most alternatives
- Pre-validate form fields before processing
### 5. Memory Management
```python
# Process PDFs in chunks
def process_large_pdf(pdf_path, chunk_size=10):
reader = PdfReader(pdf_path)
total_pages = len([Link])
for start_idx in range(0, total_pages, chunk_size):
end_idx = min(start_idx + chunk_size, total_pages)
writer = PdfWriter()
for i in range(start_idx, end_idx):
writer.add_page([Link][i])
# Process chunk
with open(f"chunk_{start_idx//chunk_size}.pdf", "wb") as output:
[Link](output)
```
## Troubleshooting Common Issues
### Encrypted PDFs
```python
# Handle password-protected PDFs
from pypdf import PdfReader
try:
reader = PdfReader("[Link]")
if reader.is_encrypted:
[Link]("password")
except Exception as e:
print(f"Failed to decrypt: {e}")
```
### Corrupted PDFs
```bash
# Use qpdf to repair
qpdf --check [Link]
qpdf --replace-input [Link]
```
### Text Extraction Issues
```python
# Fallback to OCR for scanned PDFs
import pytesseract
from pdf2image import convert_from_path
def extract_text_with_ocr(pdf_path):
images = convert_from_path(pdf_path)
text = ""
for i, image in enumerate(images):
text += pytesseract.image_to_string(image)
return text
```
## License Information
- **pypdf**: BSD License
- **pdfplumber**: MIT License
- **pypdfium2**: Apache/BSD License
- **reportlab**: BSD License
- **poppler-utils**: GPL-2 License
- **qpdf**: Apache License
- **pdf-lib**: MIT License
- **pdfjs-dist**: Apache License