4.

MACHINE LEARNING IMPLEMENTATION

4.1 CNN Architecture Design

A custom Convolutional Neural Network (CNN) was designed from scratch and trained on the
CASIA2 dataset to classify document images as genuine (0) or tampered (1). The architecture
is optimized for document forensics with progressive feature extraction through four
convolutional blocks followed by fully connected layers.

CNN Architecture Specification

Layer Operation Output Shape Purpose
Input — 128 × 128 × 3 RGB document image
Conv2d(3→32) + BN +
Conv Block 1 64 × 64 × 32 Detect basic edges
ReLU + MaxPool
Conv2d(32→64) + BN
Conv Block 2 32 × 32 × 64 Detect textures
+ ReLU + MaxPool
Conv2d(64→128) + BN
Conv Block 3 16 × 16 × 128 Detect artifacts
+ ReLU + MaxPool
Conv2d(128→256) +
Conv Block 4 8 × 8 × 256 Detect forgery patterns
BN + ReLU + MaxPool
Flatten Reshape 3D → 1D 16,384 Format for FC layers
Linear + ReLU + Learn feature
FC Layer 1 1024
Dropout(0.5) combinations
Linear + ReLU +
FC Layer 2 512 Refine decision
Dropout(0.3)
0.0=Genuine,
Output Linear + Sigmoid 1
1.0=Tampered

The architecture employs batch normalization after each convolutional layer to stabilize training
and accelerate convergence. ReLU activation functions introduce non-linearity, enabling the
network to learn complex tampering patterns. Max pooling layers progressively reduce spatial
dimensions while retaining important features.

4.2 Dataset: CASIA2

The CASIA2 (Chinese Academy of Sciences Image Splicing Detection Evaluation Database
v2.0) is the standard benchmark dataset for image forgery detection research. Although the
dataset contains natural images (animals, nature, people) rather than document-specific
content, the low-level pixel patterns that indicate tampering such as compression artifacts, noise
inconsistencies, and splicing boundaries are universal across image types.
CASIA2 Dataset Composition
Property Details Statistics
CASIA Image Tampering Detection
Dataset Name Standard benchmark
Evaluation Database v2.0
Au folder - authentic, unmodified
Genuine Images 7,492 images
images
Tampered Images Tp folder - tampered/forged images 5,125 images
Total Images Combined dataset 12,617 images
Supported Formats Multiple image formats JPG, JPEG, PNG, TIF, TIFF
Training Split 80% for model training 10,093 images
Validation Split 20% for model validation 2,524 images
Tamper Types Types of forgery in dataset Splicing, Copy-move
Mask images showing tampered
Ground Truth Available but unused in training
regions

Despite the domain mismatch between natural images and document images, the CNN
successfully learns transferable features. The low-level pixel patterns that characterize
tampering—such as JPEG compression artifacts at splicing boundaries, noise inconsistencies
from pasting edited regions, and boundary artifacts—are independent of the document type and
are present across all image domains.

4.3 Training Configuration and Hyperparameters

The model was trained using PyTorch on Google Colab with GPU acceleration. The training
pipeline employed industry-standard techniques for optimization and regularization to achieve
robust convergence and prevent overfitting.

Training Hyperparameters
Parameter Value Justification
Framework PyTorch Better control over training loop
Appropriate for binary classification
Loss Function Binary Cross Entropy (BCELoss)
with sigmoid output
Adaptive learning rate, faster
Optimizer Adam with LR = 0.001
convergence
Halves LR if validation accuracy
LR Scheduler ReduceLROnPlateau
stagnates for 3 epochs
Balances gradient stability and
Batch Size 32
memory usage
Sufficient resolution for document
Image Input Size 128 × 128 pixels
forensics
 Max Epochs 50 Early stopping with patience = 5
Free GPU acceleration for faster
Training Platform Google Colab (GPU)
training

Data Augmentation Strategy

Data augmentation was applied only to the training set to increase diversity and improve model
generalization. The validation and test sets remained unaugmented to provide accurate
performance metrics.
• Random horizontal flip: Accounts for document orientation variability
• Random rotation ±10 degrees with white fill: Simulates slightly rotated documents
• Normalization: Pixels mapped to [−1, +1] range using mean=[0.5, 0.5, 0.5], std=[0.5, 0.5,
0.5]

Learning Rate Progression During Training

Phase Learning Rate Explanation
Standard Adam starting rate for
Initial 0.001
stable convergence
Reduced by scheduler when
Mid-training 0.0005
validation plateaus
Further reduction for fine-tuning
Final 0.00025
and convergence

4.4 Training Results and Model Performance

Final Model Metrics

Metric Value Interpretation
Best Validation Model correctly classified 78.4% of
78.4%
Accuracy validation documents
Training set accuracy after
Final Training Accuracy 82.8%
convergence
Healthy generalization with minimal
Train/Val Gap 4.4%
overfitting
Model achieved best performance at
Best Epoch 50 / 50
final epoch
Reduced from 0.001 by adaptive
Final Learning Rate 0.00025
scheduler
Model complexity suitable for
Total Parameters ~4.2 million
forensic analysis
The train/validation gap of only 4.4% indicates healthy generalization without significant
overfitting. The model converged smoothly through all 50 epochs, with the scheduler
successfully reducing the learning rate to fine-tune the model during later training phases.

4.5 CNN Inference Pipeline

During inference, the trained model weights are loaded once at pipeline startup and reused for
all subsequent document analysis requests. This approach minimizes computational overhead
and ensures consistent classification across multiple documents.

Inference Procedure
• Load model weights: Restore trained weights from stage3_model.pth into the same
architecture
• Read image: Load normalized PNG using [Link]()
• Color conversion: Convert BGR→RGB to match training normalization
• Resize: Scale image to 128×128 pixels using [Link]()
• Normalize: Apply identical normalization as training: mean=[0.5, 0.5, 0.5], std=[0.5, 0.5,
0.5]
• Forward pass: Pass through model with torch.no_grad() (no gradient tracking needed)
• Sigmoid output: Obtain probability score from 0.0 (genuine) to 1.0 (tampered)
• Scale score: Multiply sigmoid output (0.0–1.0) by 100 to produce final CNN score (0–
100)

4.6 Image Forensics Detectors (Stage 2)

Stage 2 implements three classical computer vision detectors that analyze low-level pixel
statistics to identify signs of tampering. These detectors work independently and their outputs
are combined using weighted fusion.

Image Forensics Detector Summary

Detector Technique Detection Target Weight
Compression artifacts at
ELA JPEG re-save + pixel diff 35 pts
edit boundaries
Pasted regions with
Noise Inconsistency Laplacian + 16-tile std 25 pts
different noise patterns
Duplicated regions at
Copy-Move Cosine similarity blocks 35 pts max
different locations

Error Level Analysis (ELA)

ELA detects regions that have a different compression history than the rest of the image. When
an image is edited and saved, the edited regions typically have different JPEG compression
artifacts than the original image.

Algorithm: The normalized PNG image is re-saved as JPEG at 90% quality. The absolute pixel
difference between the original PNG and re-saved JPEG is computed using [Link]().
Regions previously edited appear as brighter spots because they were not subject to the
original JPEG compression. The mean brightness of the difference map is the ELA score. A
PNG is intentionally used as input (rather than JPEG) because PNG is lossless—re-saving a
PNG produces zero difference, ensuring fair comparison.

Noise Inconsistency Detection

Genuine documents contain uniform noise characteristics. When regions are pasted from
another image, the noise pattern changes abruptly at boundaries, creating detectable
inconsistencies.

Algorithm: A Laplacian filter ([Link]) is applied to amplify edges and grain. The resulting
noise map is divided into a 4×4 grid (16 tiles). Standard deviation is computed for each tile to
measure local noise level. The standard deviation of all 16 tile standard deviations measures
global noise inconsistency. A high value indicates pasted or edited regions with different noise
characteristics.

Copy-Move Detection
Copy-move forgery involves copying a region from one part of an image and pasting it to
another location. This creates two identical regions that should not exist in genuine documents.

Algorithm: The image is divided into 16×16 pixel blocks. Blocks with very low variance (std <
3.0) are skipped to avoid false positives from blank paper regions. Each block is converted to a
1D vector and normalized to unit length. Cosine similarity is computed between all block pairs
using efficient matrix multiplication. Block pairs with similarity ≥ 0.995 are flagged as copy-move
matches. Each match pair contributes 7 points to the forensics score, with a maximum cap at 35
points.

Combined Forensic Heatmap

For visualization and comprehensive analysis, ELA and noise maps are normalized
independently and blended 50/50. Copy-move matched block locations are marked as bright
white. A COLORMAP_JET color map is applied (blue indicating clean regions, red indicating
suspicious regions) and overlaid on the original image at 50% opacity using [Link]().

Forensics score formula:

(ELA/30 × 35) + min(clone_count × 7, 35) + (Noise/16 × 25)
Total capped at 100 points

4.7 Machine Learning Technology Stack

Library Version Purpose Reason

CNN model training and
PyTorch Latest Research standard
inference
Image transforms and Official PyTorch
torchvision Latest
augmentation utilities
 Image forensics and PNG Industry standard for
OpenCV (cv2) Latest
conversion vision
Efficient matrix
NumPy Latest Array operations in forensics
operations
Supports Hindi +
EasyOCR Latest Text extraction from documents
confidence scores
Pure Python, low-
pypdf Latest PDF reading and forensics
level access
Pillow (PIL) Latest Image loading in CNN dataset Standard image I/O

4.8 Model Deployment and Serving

The trained model (stage3_model.pth) is saved using PyTorch's state_dict() format, which
contains only the learned weights and biases. This lightweight format enables fast loading
without storing the full computational graph.

Model Loading at Pipeline Startup:

• The model is loaded once during pipeline initialization
• Loaded using [Link]() with map_location='cpu' for CPU inference
• Model set to evaluation mode with [Link]() to disable dropout and batch norm
training behaviors
• Reused for all subsequent document classifications, eliminating reload overhead

4.9 Machine Learning Integration Summary

The machine learning implementation combines three complementary approaches: classical
image forensics (ELA, noise analysis, copy-move detection), deep learning (CNN-based learned
features), and structural analysis (PDF forensics). The CNN component achieves 78.4%
validation accuracy on the CASIA2 dataset, while the forensic detectors provide explainable
signals that indicate specific types of tampering. Together, these components create a robust,
multi-signal detection pipeline that quantifies the suspicion level of submitted documents on a
0–100 risk scale.