Deep Learning (BCS714A)
Module 1: Introduction to Deep
Learning
7th Semester | VTU
Lecturer: [Your Name]
Learning Objectives
• Understand biological vs machine vision
• Explore early CNN models: Neocognitron,
LeNet-5, AlexNet
• Learn about ImageNet revolution
• Understand NLP basics: one-hot, embeddings,
word2vec
• Hands-on demos: TensorFlow Playground &
embeddings
Biological Vision – Inspiration
• Human visual cortex (Hubel & Wiesel, 1959)
• Edge detection, hierarchical feature learning
• Analogy → Biological neurons vs artificial
neurons
Machine Vision – Early Models
• Neocognitron (1980, Fukushima) → precursor
to CNN
• LeNet-5 (1998, LeCun) → digit recognition
• Hierarchical feature extraction
Traditional ML vs Deep Learning
• Traditional ML: Handcrafted features +
Classifier
• Deep Learning: End-to-end feature learning +
classification
The ImageNet Revolution (2012)
• ImageNet dataset (14M+ labeled images)
• ILSVRC competition → breakthrough with
AlexNet
• Key innovations: ReLU, dropout, GPU training
• Error rates dropped sharply post-2012
TensorFlow Playground
• Interactive NN training demo
• Adjust layers, activations, learning rate
• Visualize decision boundaries
• Link: [Link]
Deep Learning for Language
• NLP requires converting text to numbers
• One-hot encoding: simple but sparse
• Does not capture relationships between words
Word Vectors (Embeddings)
• word2vec (Mikolov et al., 2013)
• Captures semantic meaning
• Example: king – man + woman ≈ queen
Localist vs Distributed
Representations
• Localist: one concept = one node
• Distributed: concept = pattern across multiple
nodes
• Distributed more powerful for NLP
Summary
• Biological → Machine vision inspiration
• Key models: Neocognitron → LeNet-5 →
AlexNet
• ImageNet revolutionized DL
• NLP embeddings outperform one-hot
• Practical demos reinforce concepts
Lab & Assignment
• Lab 1: Train LeNet-5 on MNIST
• Lab 2: Explore TensorFlow Playground
• Lab 3: Implement word2vec with gensim
• Assignment 1: Compare one-hot vs word2vec
embeddings for classification