Table of Contents
Sr. No Content Page No.
1. Broad Area of Work 1
2. Introduction 2
Literature Survey
/ Work Done in
3. 3
the Field of
Proposed Work
4. Existing Gaps 4
Objectives of the
5. Proposed Work 5
Proposed
6. Methodologies 6
Expected
7. Outcome 7
of the Proposed
Work Future
8 Scope of the 8
. Work 9
References
9
Broad Area of
Work
The proposed work lies in the domain of Artificial Intell
(AI), with a focus on Computer Vision and Natural Lang
Processing (NLP). The project integrates these two sub
develop an AI Image Caption Generator that can autom
describe the content of an image in human-readable tex
Introduction
In the modern digital age, billions of images are
shared daily across various platforms. Interpreting and
describing these images manually is inefficient. An AI
Image Caption Generator aims to automate this
process by analyzing visual features and generating
meaningful captions. This project will employ deep
learning models that mimic human perception and
linguistic understanding to generate natural,
descriptive sentences corresponding to the visual
content of an image.
Literature Survey / Work Done in the
Field of Proposed Work
Early works in image captioning combined
Convolutional Neural Networks (CNNs) for image
feature extraction and Recurrent Neural Networks
(RNNs) for sentence generation. Notable research
includes: Show and Tell: A Neural Image Caption
Generator (Google, 2015) Show, Attend and Tell (Xuet
al., 2016) introducing attention mechanisms
Transformers and Vision-Language Models (e.g., CLIP,
BLIP, ViLT, and GPT-4V) that have improved the
contextual understanding of images. These models
demonstrate the effectiveness of combining visual and
textual representations but still face challenges in
understanding abstract or complex visual scenes.
A
Project Based Learning
Title of the PBL
Submitted to
For the partial fulfillment of
Award of the degree
BACHELOR OF TECHNOLOGY
IN
COMPUTER SCIENCE & ENGINEERING
By
name : Arshil khan BETN1AI25034
Ishant sharma BETN1AI25035
Abhishek singh bhadoriya BETN1AI25036
Under the Supervision of
Mr. Shubham Dhakarey
Assistant Professor
Department of CSA, SOET
ITM University
Objectives of the
Proposed Work
Todesignand implement a system capableof
automatically generating accurate and meaningful
captions for images. To utilize deep learning
techniques that integrate image recognition
(CNN) and text generation
(RNN/LSTM/Transformer). To improve
accessibility for visually impaired users by
converting visual content into text. To evaluate
system performance using standardized metrics
such as BLEU, METEOR, and CIDEr.
Proposed Methodologies
Dataset Selection: Use publicly available datasets like
MS COCO, Flickr8k, or Flickr30k, which contain images
with multiple human-written captions. Feature
Extraction: Apply CNN architectures such as ResNet-50
or InceptionV3 to extract meaningful visual features.
Text Generation: Use an LSTM or Transformer-based
language model to convert extracted features into
coherent sentences. Model Training and Optimization:
Combine the CNN and language model using an
encoder– decoder architecture and train using paired
image-caption data. Evaluation: Validate results using
automated caption evaluation metrics and human
judgment for qualitative analysis.
Expected Outcome of
the Proposed Work
A fully functional model that generates human-like cap
for images.
Enhanced understanding of the integration of vision and
language models.
Potential real-world applications in social media, digita
marketing, image retrieval, and assistive technologies.
Future Scope of the
Work
Integrating speech output to convert generated caption
audiofor visuallyimpairedusers.
Extending the model to handle video captioning and
multimodal analysis.
Improving contextual understanding using multilingual s
and knowledge graphs.
Deploying the system as a web or mobile application for
time use.
References
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015). S
Tell: A Neural Image Caption [Link].
Xu, K. et al. (2016). Show, Attend and Tell: Neural Image C
Generation with Visual [Link].
Radford, A. et al. (2021). Learning Transferable Visual Mod
Natural Language Supervision (CLIP).OpenAI.
Li, J. et al. (2022). BLIP: Bootstrapped Language–Image Pre
Training for Unified Vision–Language Understanding and
Generation.
Datasets: MS COCO, Flickr8k, Flickr30k (available from
respective open research sources).