Overview of Deep Learning
Rowel Atienza, PhD
University of the Philippines
[Link]/roatienza
2023
Artificial Intelligence (AI)
Machines exhibiting animal or human intelligence
Ants bridging algo
[Link]
Rowel Atienza Deep Learning, University of the Philippines 2
Intelligence
• A very general mental capability that among other things involves the
ability to:
• Reason
• Plan
• Solve problems
• Think abstractly
• Comprehend complex ideas
• Learning quickly and learning from experience
Journal of Intelligence 1997 Vol 24 No 1
Rowel Atienza Deep Learning, University of the Philippines 3
AI, Machine Learning and Deep Learning
NVIDIA
Rowel Atienza Deep Learning, University of the Philippines 4
Artificial Intelligence, Machine Learning and
Deep Learning on Face Detection
YOLO, SSD, RCNN [>2012]
[Viola & Jones 2001]
Rule-based (AI):
Detect facial
Machine Learning: Deep Learning:
features based on
Use Haar Cascade Train a network by
color/template
Classifier showing thousands of
Apply if-else-if-else
Hand-crafted feature labelled region of faces
detection Automatic feature
detection
Rowel Atienza Deep Learning, University of the Philippines 5
Deep Learning Moment - 2012
• AlexNet – 650,000-neuron deep neural network won 2012
ImageNet1k competition with 15% Top-5 error rate compared to 2nd
place with 26%. It’s top 1 accuracy is 63.3%.
Rowel Atienza Deep Learning, University of the Philippines 6
Deep Learning - what made it work?
• Concepts of artificial neural network (ANN) and convolutional neural
network (CNN) are old
• Neurons in perceptron (1-layer NN) – 1958
• Neocognitron (1980) and CNN (1989)
• Backpropagation (1986)
• What’s new?
• Computing power – Massive number of GPU CUDA cores
• Data – from the Internet
Perceptron is a binary classifier
Rowel Atienza Deep Learning, University of the Philippines 7
CPU (AMD RyZen) vs GPU (RTX 3090)
64 3.7GHz super fast cores 10,496 1.4GHz fast cores
6.9 TFLOPS 35.6 TFLOPS
Rowel Atienza Deep Learning, University of the Philippines 8
1D vs 2D Tensor Operation
Rowel Atienza Deep Learning, University of the Philippines 9
The Rest is History
Rowel Atienza Deep Learning, University of the Philippines 10
Barely scratching
the surface of
Artificial General
Intelligence (AGI) 𝐼𝑚𝑎𝑔𝑖𝑛𝑖𝑛𝑔
Starting to
move here 𝐷𝑜𝑖𝑛𝑔 ∶ 𝑝 𝒚 𝑑𝑜 𝒙
We are here 𝑆𝑒𝑒𝑖𝑛𝑔: 𝑝 𝒚 𝒙
Pearl, Book of Why
Rowel Atienza Deep Learning, University of the Philippines 11
Does ChatGPT (or LLMs in
general) exhibit AGI?
Rowel Atienza Deep Learning, University of the Philippines 12
Do Large Language Models
Perform Reasoning in Problem
Solving Tasks?
Rowel Atienza Deep Learning, University of the Philippines 13
Comprehensive Tests
Rowel Atienza Deep Learning, University of the Philippines 14
GPT4 Reasoning Test
Rowel Atienza Deep Learning, University of the Philippines 15
Failed!
Rowel Atienza Deep Learning, University of the Philippines 16
How Humans use Reasoning to Solve Tasks
• Break a problem into sub-tasks
• A sub-task is a node
• Solving a sub-task transitions it into a new sub-task
• Repeat until all sub-tasks are completely solved
Computation Graph!
Sub-task - Vertex
Solution – Edge or Operator
Rowel Atienza Deep Learning, University of the Philippines 17
Computation Graph for Multiplication Algo
Reasoning Depth
Reasoning Width
Rowel Atienza Deep Learning, University of the Philippines 18
GPT4 Zero-shot Multiplication
No longer applies as demonstrated in the previous slides!
Rowel Atienza Deep Learning, University of the Philippines 19
Relative Information Gain (RIG)
𝑌! ∶ output
𝑋 ∶ input random variables
𝐻 𝑌 = −𝔼 log 𝑝 𝑌 : entropy
Rowel Atienza Deep Learning, University of the Philippines 20
Observation
• LLMs break down a problem into a computational graph
• LLMs can solve problems where the RIG is high between sub-tasks
• When the RIG is low, the LLM hallucinates
𝑌!"# 𝑌!
𝑌!"$ …
Rowel Atienza Deep Learning, University of the Philippines 21
2 Critical Points
• Ability to break down a problem into a correct computation graph
• High RIG between sub-tasks
Rowel Atienza Deep Learning, University of the Philippines 22
When do LLMs fail?
• In-correct computation graph
• A presence of low RIG between sub-tasks
• Deep reasoning graphs can amplify errors due to error propagation
Rowel Atienza Deep Learning, University of the Philippines 23
Incorrect computation graphs
• Instruction-based tuning
• Prompt engineering
• Etc
Rowel Atienza Deep Learning, University of the Philippines 24
Low RIG between sub-tasks
• Higher data quality
Rowel Atienza Deep Learning, University of the Philippines 25
Improving Reasoning by
Grounding the Language
Modality
Rowel Atienza Deep Learning, University of the Philippines 26
RT-2: Vision-Language-Action
Models Transfer Web
Knowledge to Robotic Control
[Link]
Rowel Atienza Deep Learning, University of the Philippines 27
Rowel Atienza Deep Learning, University of the Philippines 28
Pre-train on Internet
scale vision-language
data
Rowel Atienza Deep Learning, University of the Philippines 29
Co-fine-tune with real
robot data.
The robot language is
used during co-fine-
tuning.
Rowel Atienza Deep Learning, University of the Philippines 30
Rowel Atienza Deep Learning, University of the Philippines 31
LINGO-1: Exploring Natural
Language for Autonomous
Driving
[Link]
Rowel Atienza Deep Learning, University of the Philippines 32
Rowel Atienza Deep Learning, University of the Philippines 33
LINGO-1 Architecture
Rowel Atienza Deep Learning, University of the Philippines 34
Lingo-1: Driver Commentator for
Autonomous Driving
• slowing down for a lead vehicle or a change in traffic lights,
• changing lanes to follow a route,
• accelerating to the speed limit,
• noticing other cars coming onto the road or stopped at an intersection
• approaching hazards such as roundabouts and Give Way signs,
• parked cars, traffic lights or schools,
• actions other road users are taking, such as changing lanes or overtaking
parked vehicles,
• cyclists and pedestrians waiting at zebra crossings or coming up from
behind the car in a cycle lane.
Rowel Atienza Deep Learning, University of the Philippines 35
Rowel Atienza Deep Learning, University of the Philippines 36
LINGO-1 Contributions
• Improved Reasoning due to instructions from driver commentator
• Querying next actions improve model explainability
Rowel Atienza Deep Learning, University of the Philippines 37
References
Strickland, E. The Turbulent Past and Uncertain Future of Artificial
IntelligenceIs there a way out of AI's boom-and-bust cycle?
[Link]
[Link]/c/s/[Link]/amp/history-of-ai-
2655064200
Liu, Zhuang, et al. "A ConvNet for the 2020s." arXiv preprint
arXiv:2201.03545 (2022).
[Link]
[Link]
Dziri, Nouha, et al. "Faith and Fate: Limits of Transformers on
Compositionality." arXiv preprint arXiv:2305.18654 (2023).
Rowel Atienza Deep Learning, University of the Philippines 38
End