VISVESVARAYA TECHNOLOGICAL UNIVERSITY
"Jnana Sangama", Belagavi: 590 018
Subject: Deep Learning (BAI701)
Mini Project report on
“Image Colorization using Convolutional Neural Networks”
Submitted in partial fulfilment of the requirement for the award of Degree of
BACHELOR OF ENGINEERING
IN
ARTIFICIAL INTELLIGENCE & MACHINE LEARNING
By
Pavan Kumar V 1AY22AI063
Under the guidance of
Dr. Vijayashekhar S S
DEPARTMENT OF ARTIFICIAL INTELLIGENCE & MACHINE
LEARNING
ACHARYA INSTITUTE OF TECHNOLOGY
(Affiliated to Visvesvaraya Technological University, Belagavi)
2025-2026
CHAPTER 1:
Abstract
This project explores the application of Deep Learning techniques, specifically Convolutional Neural
Networks (CNNs), to the challenging task of automatic image colorization. The primary objective is to
predict the missing color channels (Red, Green, and Blue) of an image given only its single-channel
grayscale version. Using the CIFAR-10 dataset as a benchmark, we trained a sequential CNN model to
learn the complex mapping between grayscale luminance and chromaticity. The model demonstrates a
strong capability to learn semantic color distributions, successfully identifying that sky is typically blue
and vegetation is green, and reconstructs plausible, recognizable color images from black-and-white
inputs. This report details the architectural decisions, training methodology, and an analysis of the
model's performance and limitations.
CHAPTER 2:
Introduction
Image colorization is the process of estimating and filling in plausible color information for
monochrome images. Historically, this was a manual, labor-intensive task requiring expert artists to
hand-paint frames, often taking hours or days for a single image. Today, it is an active and exciting
area of research in Computer Vision with applications in legacy photo restoration, video compression,
and artistic rendering.
The problem of colorization is mathematically described as "ill-posed" and "multimodal." This is
because the mapping from a color image to a grayscale image is deterministic (a simple formula), but
the inverse,mapping grayscale to color,is one-to-many. For example, a grayscale shape could
plausibly be a red apple, a green apple, or a yellow tennis ball. There is no single "correct" answer
based solely on pixel intensity. Therefore, the model must learn to recognize objects, textures, and
context (e.g., "grass texture implies green," "ocean waves imply blue") to hallucinate the most
probable colors. This project implements a data-driven approach where a neural network learns these
statistical priors from a large dataset of images.
CHAPTER 3:
Dataset
We utilized the CIFAR-10 dataset, a standard benchmark in computer vision research. While often
used for classification, its diversity makes it an interesting candidate for generative tasks like
colorization.
● Content: The dataset consists of 60,000 color images distributed across 10 distinct classes,
including biological subjects (cats, dogs, birds, frogs) and mechanical objects (airplanes,
automobiles, ships, trucks).
● Dimensions: Each image is 32×32 pixels with 3 color channels. While low resolution, this
allows for rapid training and prototyping of deep learning architectures.
● Split: The data is partitioned into 50,000 training images (used to update model weights) and
10,000 test images (used exclusively for final evaluation).
CHAPTER 4:
Methodology
4.1 Data Preprocessing
Data preprocessing is a critical step to ensure the neural network receives data in a format optimized
for learning.
1. Normalization: Raw pixel intensities range from 0 to 255. We scaled these values to the
floating-point range$$0, 1$$
. This normalization is crucial because it keeps the input values small and centered, preventing
exploding gradients and allowing the optimizer to converge faster and more stably.
2. Grayscale Conversion: The original RGB images served as the "Ground Truth" (Y). To
generate the input data (X), we converted these RGB images to grayscale using the standard
luminance formula .This mimics the real-world scenario where only the luminance channel is
available.
4.2 Model Architecture
We designed a custom Convolutional Neural Network (CNN) using the TensorFlow/Keras Sequential
API. The architecture follows a simplified "Autoencoder" philosophy—it extracts features and then
reconstructs the output—but maintains the spatial dimensions throughout to avoid loss of fine detail.
● Input Layer: Receives a (32, 32, 1) grayscale tensor.
● Feature Extraction (Hidden Layers):
○ Layer 1: Conv2D (64 filters, 3x3 kernel, ReLU). This layer acts as the initial feature
detector, identifying simple edges, corners, and gradients in the black-and-white image.
○ Layer 2: Conv2D (64 filters, 3x3 kernel, ReLU). This layer combines the low-level features
to detect textures and simple patterns.
○ Layer 3: Conv2D (128 filters, 3x3 kernel, ReLU). By doubling the number of filters, we
increase the "capacity" of the network, allowing it to capture more complex semantic
information (e.g., distinguishing between fur and water).
● Reconstruction (Output Layer):
○ Layer 4: Conv2D (3 filters, 3x3 kernel, Sigmoid activation). This final layer projects the
high-dimensional feature maps back into the 3-channel RGB color space. We use the
Sigmoid activation function specifically because our target data is normalized between [0,
1]. Sigmoid guarantees the model's output pixels also fall strictly within this valid range.
Padding Strategy: We employed padding='same' for all convolutional layers. This ensures that the
output feature maps have the same height and width (32x32) as the input, eliminating the need for
complex upsampling layers in this mini-project.
4.3 Training Configuration
● Optimizer: We used the Adam (Adaptive Moment Estimation) optimizer. Adam is preferred for
its ability to adapt the learning rate for each parameter individually, handling the sparse gradients
and non-convex loss landscapes typical in image generation tasks.
● Loss Function: The model was trained using Mean Squared Error (MSE). In this context,
MSE calculates the average squared difference between the predicted pixel color and the actual
pixel color. By minimizing this value, the model learns to output colors that are, on average,
closest to the true colors.
● Hyperparameters:
○ Epochs: 20 (iterations over the entire dataset).
○ Batch Size: 128 (number of samples processed before updating weights).
○ Validation Split: 10% (45,000 images for training, 5,000 for validation).
CHAPTER 5:
Implementation Details
The project was implemented in a Python environment utilizing the Google TensorFlow framework
(v2.x) with the Keras high-level API. The [Link].rgb_to_grayscale utility provided a consistent
method for generating inputs. Matplotlib was extensively used for visualizing the training progress
and comparing the Input (Grayscale), Predicted (Colorized), and Ground Truth (Original) images
side-by-side.
CHAPTER 6:
Results and Discussion
6.1 Quantitative Results
The training process demonstrated steady learning dynamics over the course of 20 epochs.
● Convergence: The loss started at approximately 0.0179 and decayed rapidly, stabilizing around
0.0054.
● Generalization: The final Training Loss (0.0054) and Validation Loss (0.0054) were nearly
identical. This is a strong indicator that the model is not overfitting; it has learned general rules
about coloring rather than memorizing the specific images in the training set.
6.2 Qualitative Results (Visual Analysis)
Testing the model on unseen data from the test set revealed interesting behavioral patterns:
● Semantic Success: The model exhibits "semantic awareness." For instance, images of frogs
were consistently colored green, and images featuring the horizon were colored with blue tops
(sky) and darker bottoms (ground/sea). This proves the network isn't just guessing randomly; it is
classifying objects implicitly to assign color.
● The "Sepia" Effect (Limitations): While the colors are plausible, they are often desaturated or
"washed out" compared to the ground truth. This is a known side-effect of using the MSE Loss
function. Because multiple colors are valid for a single object (e.g., a car can be red, blue, or
white), MSE encourages the model to predict the "average" of all possibilities to minimize the
error penalty.
Figure 6.1: Results of the colorization
CHAPTER 7:
Conclusion and Future Scope
This mini-project successfully demonstrated the feasibility of using a lightweight CNN for automatic
image colorization. The model effectively learned to associate low-level grayscale textures with high-
level color concepts.
However, the inherent ambiguity of grayscale-to-color mapping limits the vibrancy of the output
when using simple regression (MSE). To overcome these limitations, future iterations of this project
could explore:
1. U-Net Architecture: Implementing skip connections to preserve high-frequency details from
the input layers directly to the output layers.
2. Generative Adversarial Networks (GANs): Using a "Discriminator" network (e.g., Pix2Pix) to
penalize unrealistic, desaturated colors and force the generator to produce vibrant, photorealistic
outputs.
3. Perceptual Loss: Replacing MSE with loss functions based on human perception or pre-trained
feature maps (e.g., VGG loss) to prioritize structural similarity over pixel-perfect accuracy.