0% found this document useful (0 votes)
4 views19 pages

Gradient-Based Learning Algorithms

Chapter 10 discusses gradient-based learning algorithms, focusing on the optimization of parameters to minimize a cost function J(✓). It emphasizes the importance of finding the optimal parameters that lead to the best performance in machine learning models. The chapter is part of a broader text on the foundations of computer vision.

Uploaded by

balkrishna.sehra
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
4 views19 pages

Gradient-Based Learning Algorithms

Chapter 10 discusses gradient-based learning algorithms, focusing on the optimization of parameters to minimize a cost function J(✓). It emphasizes the importance of finding the optimal parameters that lead to the best performance in machine learning models. The chapter is part of a broader text on the foundations of computer vision.

Uploaded by

balkrishna.sehra
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Chapter 10

Gradient-Based
Learning Algorithms J(✓)
<latexit sha1_base64="a/Lk8rEB5YqTpm5iqH/sBpa7bWw=">AAACiHicfVHbahRBEO0db3G8JfqYl8FFiCLLjAjRtxB9kIAkgewmsj2Emt6a2SZ9Gbpr1GWYr/BVP8y/sWezgdywoOH0qVNddbqKWklPafp3EN25e+/+g7WH8aPHT54+W994PvG2cQLHwirrTgrwqKTBMUlSeFI7BF0oPC7OPvX54+/ovLTmiBY15hoqI0spgAL1bW+L0xwJXp+uD9NRuozkJshWYMhWcXC6MZB8ZkWj0ZBQ4P00S2vKW3AkhcIu5o3HGsQZVDgN0IBGn7fLibvkVWBmSWldOIaSJXu5ogXt/UIXQamB5v56ridvy00bKj/krTR1Q2jEeaOyUQnZpLefzKRDQWoRAAgnw6yJmIMDQeGT4ittikJ3gfmMwZ7Dr6HVfo0OyLo3LQdXafjZBbsVf9uj/wmluRAGFJ40+ENYrcHMWm6s0900y1uusCSuJuhomHEnqzlx19+6OCwnu76Km2DybpSlo+zw/XBnd7WmNbbJXrItlrFttsO+sAM2ZoJp9ov9Zn+iOEqj7ejjuTQarGpesCsR7f4DaXjHOg==</latexit>


<latexit sha1_base64="t5duRUGQiNnGS0onDa5D0jhO93E=">AAAChXicfVHbahRBEO2dRBPXW2IefRmyCCK6zEhQ3xKiD75IImQ3ge0h1PTWzHbSl6G7xrgM8w++xj/zb+zZbMBcsKDh9KlTXXW68kpJT0nypxetrD54uLb+qP/4ydNnzzc2X4y9rZ3AkbDKupMcPCppcESSFJ5UDkHnCo/z889d/vgHOi+tOaJ5hZmG0shCCqBAjTnNkOB0Y5AMk0XEd0G6BAO2jMPTzZ7kUytqjYaEAu8naVJR1oAjKRS2fV57rECcQ4mTAA1o9FmzGLeNXwVmGhfWhWMoXrD/VjSgvZ/rPCg10MzfznXkfblJTcWnrJGmqgmNuGpU1ComG3fe46l0KEjNAwDhZJg1FjNwICj8UP9GmzzXbWC+YLDn8FtodVChA7LuTcPBlRp+tsFuyd926H9Caa6FAYUnDV4IqzWYacONdbqdpFnDFRbE1RgdDVLuZDkj7rpb2w/LSW+v4i4Yvx+myTD9vjPY21+uaZ29ZNvsNUvZR7bHvrJDNmKCnbFf7JL9jtaid9FO9OFKGvWWNVvsRkS7fwGr9MaB</latexit>
<latexit

J
<latexit sha1_base64="Ro1vbEsDMxSDmIPyW68eIg0SCUY=">AAACgHicfVHbahRBEO0dL4nrLdFHXwYXIYis0yIYfArqgwhiAu4msD2Emt6a2SZ9GbprTJZhviCv8eP8G3s2K5hELGg4fepUV52uotYqUJb9GiS3bt+5u7F5b3j/wcNHj7e2n0yDa7zEiXTa+aMCAmplcUKKNB7VHsEUGg+Lk499/vAH+qCc/U7LGnMDlVWlkkCROvhyvDXKxtkq0puAr8GIrWP/eHugxNzJxqAlqSGEGc9qylvwpKTGbiiagDXIE6hwFqEFgyFvV5N26YvIzNPS+XgspSv274oWTAhLU0SlAVqE67me/Fdu1lC5m7fK1g2hlZeNykan5NLedjpXHiXpZQQgvYqzpnIBHiTFzxleaVMUpovMJ4z2PH6Nrb7V6IGcf9kK8JWBsy7arcSrHv1PqOwfYUTxSYun0hkDdt4K67zpZjxvhcaShJ6ipxEXXlULEr6/dcO4HH59FTfB9M2YZ2N+8Ha092G9pk32jD1nO4yzd2yPfWb7bMIkQ3bOLtjPJEl2ktcJv5Qmg3XNU3Ylkve/AV23xCc=</latexit>
<latexit
r✓ J(✓)
<latexit sha1_base64="6vNmN7cUeCZNCT1FkkaAoLRvGUw=">AAACmXicfZFtaxNBEMc351OND031jdA3h0GoIuFOBPuyPrwoBbGiSQvZI8xt5i5L9+HYnauG4/w0vtXv47dxL4lgW3Fg4bf/mdnZmckrJT0lya9edO36jZu3tm7379y9d397sPNg4m3tBI6FVdad5uBRSYNjkqTwtHIIOld4kp+97fwn5+i8tOYzLSvMNJRGFlIABWk2eMQN5ApmDacFErRHe2t4OhsMk1GysvgqpBsYso0dz3Z6ks+tqDUaEgq8n6ZJRVkDjqRQ2PZ57bECcQYlTgMa0OizZtVCGz8JyjwurAvHULxS/85oQHu/1HmI1EALf9nXif/yTWsq9rNGmqomNGJdqKhVTDbu5hHPpUNBahkAhJPhr7FYgANBYWr9C2XyXLdBeYehPYfvQ6kPFTog6541HFyp4Wsb2i35847+FyjNn8BA4UmDX4TVGsy84cY63U7TrOEKC+Jqgo6GKXeyXBB33a3th+Wkl1dxFSYvRmkySj++HB682axpi+2yx2yPpewVO2CH7JiNmWDf2Hf2g/2MdqPX0WF0tA6Nepuch+yCRZ9+A3yLziQ=</latexit>

H✓ J(✓)
<latexit sha1_base64="0MJeSNq7QzKBxeL2/Ryd673gPRg=">AAACpXicjZHbahRBEIZ7x1McTxtz6c3gIkaRZUaE5DKoF0EIKmQ3ge1hqOmt2W3Sh6G7Rl2GeRZvfRrvfRt7dlfQRMSChq+r/uru+ruslfSUpj8G0bXrN27e2rkd37l77/6D4e7DqbeNEzgRVll3XoJHJQ1OSJLC89oh6FLhWXnxpq+ffULnpTWntKox17AwspICKKSK4d5x0XJaIkH3bn8Dz4rhKB2n60iuQraFEdvG/8mL3cGEz61oNBoSCryfZWlNeQuOpFDYxbzxWIO4gAXOAhrQ6PN2PWKXPAmZeVJZF5ahZJ39vaMF7f1Kl0GpgZb+cq1P/q02a6g6zFtp6obQiM1FVaMSsknvVzKXDgWpVQAQToa3JmIJDgQFV+OYv8UwjMOTcPD7Gh2Qdc9bDm6h4UsXhlvwFz39SyjNL2GgmBv8LKzWYOYtN9bpbpblLVdYEVdTdDTKuJOLJXHX77pgf3bZ7KswfTnO0nH28dXo6PX233bYI/aY7bOMHbAjdsw+sAkTbMW+sm/se/Q0OolOo+lGGg22PXvsj4iKnzwrzBs=</latexit>
sha1_base64="vJ0CpofJC5BPp/GuCIJRZlKEDqc=">AAACgHicfZFbi9NAFMen8bbGy3bdR1+CRagiNRFB8WlRHxZBXMF2FzqhnExP0mHnEmZO1BLyWXzVj+S3cdJW0F3xwMBv/uecmXMpaiU9penPQXTl6rXrN/Zuxrdu37m7Pzy4N/O2cQKnwirrzgrwqKTBKUlSeFY7BF0oPC3O3/T+08/ovLTmE61rzDVURpZSAAVpMTw8XrScVkjQvRtv4dFiOEon6caSy5DtYMR2drI4GEz50opGoyGhwPt5ltaUt+BICoVdzBuPNYhzqHAe0IBGn7eb6rvkYVCWSWldOIaSjfpnRgva+7UuQqQGWvmLvl78l2/eUPkyb6WpG0Ijth+VjUrIJv0okqV0KEitA4BwMtSaiBU4EBQGFsf8LYZmHL4PD3+o0QFZ97jl4CoNX7vQXMWf9PS/QGl+BwaKucEvwmoNZtlyY53u5lnecoUlcTVDR6OMO1mtiLv+1oVFZBfHfhlmzyZZOsk+Ph8dvd6tZI/dZw/YmGXsBTtix+yETZlga/aNfWc/oigaR0+jbBsaDXY5h+wvi179AnAkxKI=</latexit>

Update

Foundations of Computer Vision Torralba, Isola, Freeman 2024


Optimization

Params


<latexit sha1_base64="t5duRUGQiNnGS0onDa5D0jhO93E=">AAAChXicfVHbahRBEO2dRBPXW2IefRmyCCK6zEhQ3xKiD75IImQ3ge0h1PTWzHbSl6G7xrgM8w++xj/zb+zZbMBcsKDh9KlTXXW68kpJT0nypxetrD54uLb+qP/4ydNnzzc2X4y9rZ3AkbDKupMcPCppcESSFJ5UDkHnCo/z889d/vgHOi+tOaJ5hZmG0shCCqBAjTnNkOB0Y5AMk0XEd0G6BAO2jMPTzZ7kUytqjYaEAu8naVJR1oAjKRS2fV57rECcQ4mTAA1o9FmzGLeNXwVmGhfWhWMoXrD/VjSgvZ/rPCg10MzfznXkfblJTcWnrJGmqgmNuGpU1ComG3fe46l0KEjNAwDhZJg1FjNwICj8UP9GmzzXbWC+YLDn8FtodVChA7LuTcPBlRp+tsFuyd926H9Caa6FAYUnDV4IqzWYacONdbqdpFnDFRbE1RgdDVLuZDkj7rpb2w/LSW+v4i4Yvx+myTD9vjPY21+uaZ29ZNvsNUvZR7bHvrJDNmKCnbFf7JL9jtaid9FO9OFKGvWWNVvsRkS7fwGr9MaB</latexit>
<latexit

J J(✓) ✓ = arg min J(✓)



<latexit sha1_base64="Ro1vbEsDMxSDmIPyW68eIg0SCUY=">AAACgHicfVHbahRBEO0dL4nrLdFHXwYXIYis0yIYfArqgwhiAu4msD2Emt6a2SZ9GbprTJZhviCv8eP8G3s2K5hELGg4fepUV52uotYqUJb9GiS3bt+5u7F5b3j/wcNHj7e2n0yDa7zEiXTa+aMCAmplcUKKNB7VHsEUGg+Lk499/vAH+qCc/U7LGnMDlVWlkkCROvhyvDXKxtkq0puAr8GIrWP/eHugxNzJxqAlqSGEGc9qylvwpKTGbiiagDXIE6hwFqEFgyFvV5N26YvIzNPS+XgspSv274oWTAhLU0SlAVqE67me/Fdu1lC5m7fK1g2hlZeNykan5NLedjpXHiXpZQQgvYqzpnIBHiTFzxleaVMUpovMJ4z2PH6Nrb7V6IGcf9kK8JWBsy7arcSrHv1PqOwfYUTxSYun0hkDdt4K67zpZjxvhcaShJ6ipxEXXlULEr6/dcO4HH59FTfB9M2YZ2N+8Ha092G9pk32jD1nO4yzd2yPfWb7bMIkQ3bOLtjPJEl2ktcJv5Qmg3XNU3Ylkve/AV23xCc=</latexit>
<latexit

<latexit sha1_base64="a/Lk8rEB5YqTpm5iqH/sBpa7bWw=">AAACiHicfVHbahRBEO0db3G8JfqYl8FFiCLLjAjRtxB9kIAkgewmsj2Emt6a2SZ9Gbpr1GWYr/BVP8y/sWezgdywoOH0qVNddbqKWklPafp3EN25e+/+g7WH8aPHT54+W994PvG2cQLHwirrTgrwqKTBMUlSeFI7BF0oPC7OPvX54+/ovLTmiBY15hoqI0spgAL1bW+L0xwJXp+uD9NRuozkJshWYMhWcXC6MZB8ZkWj0ZBQ4P00S2vKW3AkhcIu5o3HGsQZVDgN0IBGn7fLibvkVWBmSWldOIaSJXu5ogXt/UIXQamB5v56ridvy00bKj/krTR1Q2jEeaOyUQnZpLefzKRDQWoRAAgnw6yJmIMDQeGT4ittikJ3gfmMwZ7Dr6HVfo0OyLo3LQdXafjZBbsVf9uj/wmluRAGFJ40+ENYrcHMWm6s0900y1uusCSuJuhomHEnqzlx19+6OCwnu76Km2DybpSlo+zw/XBnd7WmNbbJXrItlrFttsO+sAM2ZoJp9ov9Zn+iOEqj7ejjuTQarGpesCsR7f4DaXjHOg==</latexit>

r✓ J(✓)
<latexit sha1_base64="AYItQoyunNkE46+v9OudQ3G4+AY=">AAACpnicfZFtaxNBEMc3Vx9qfErrS98shkIsEu6KYN8UivpCBLWCSQrZM+5t5i5L9+HYnbMNx30CP41v9ZP4bdxLIthWHFj47X9mdnZmslJJj3H8qxNt3bh56/b2ne7de/cfPOzt7I69rZyAkbDKutOMe1DSwAglKjgtHXCdKZhkZ69a/+QrOC+t+YTLElLNCyNzKTgGadbbY7gA5J/36RFl3BVamlm91hr6drCmp7NePx7GK6PXIdlAn2zsZLbTkWxuRaXBoFDc+2kSl5jW3KEUCpouqzyUXJzxAqYBDdfg03rVT0P3gjKnuXXhGKQr9e+MmmvvlzoLkZrjwl/1teK/fNMK88O0lqasEIxYF8orRdHSdjh0Lh0IVMsAXDgZ/krFgjsuMIywe6lMlukmKK8htOfgXSj1oQTH0br9ejVIftGEdgv2rKX/BUrzJzBQeNLAubBaczOvmbFON9MkrZmCHJkag8N+wpwsFshce2u6YTnJ1VVch/HBMImHycfn/eOXmzVtk8fkCRmQhLwgx+QNOSEjIsg38p38ID+jQfQ+GkWTdWjU2eQ8Ipcs+vIbi5PSvQ==</latexit>

<latexit sha1_base64="6vNmN7cUeCZNCT1FkkaAoLRvGUw=">AAACmXicfZFtaxNBEMc351OND031jdA3h0GoIuFOBPuyPrwoBbGiSQvZI8xt5i5L9+HYnauG4/w0vtXv47dxL4lgW3Fg4bf/mdnZmckrJT0lya9edO36jZu3tm7379y9d397sPNg4m3tBI6FVdad5uBRSYNjkqTwtHIIOld4kp+97fwn5+i8tOYzLSvMNJRGFlIABWk2eMQN5ApmDacFErRHe2t4OhsMk1GysvgqpBsYso0dz3Z6ks+tqDUaEgq8n6ZJRVkDjqRQ2PZ57bECcQYlTgMa0OizZtVCGz8JyjwurAvHULxS/85oQHu/1HmI1EALf9nXif/yTWsq9rNGmqomNGJdqKhVTDbu5hHPpUNBahkAhJPhr7FYgANBYWr9C2XyXLdBeYehPYfvQ6kPFTog6541HFyp4Wsb2i35847+FyjNn8BA4UmDX4TVGsy84cY63U7TrOEKC+Jqgo6GKXeyXBB33a3th+Wkl1dxFSYvRmkySj++HB682axpi+2yx2yPpewVO2CH7JiNmWDf2Hf2g/2MdqPX0WF0tA6Nepuch+yCRZ9+A3yLziQ=</latexit>

H✓ (J(✓))
<latexit sha1_base64="Dp56a+zglYeLw/xxC6sFCv+6E2c=">AAAClnicfZFtaxNBEMc351OND00VQfDNYRBSkXAngn1Vig9YBDGCSQvZEOY2c5el+3DszqnhuA/jW/1Efhv3kgi2FQcWfvufmZ2ZnaxU0lOS/OpEV65eu35j52b31u07d3d7e/cm3lZO4FhYZd1pBh6VNDgmSQpPS4egM4Un2dnr1n/yBZ2X1nymVYkzDYWRuRRAQZr3HhzPa05LJGgG7wcb2t+f9/rJMFlbfBnSLfTZ1kbzvY7kCysqjYaEAu+naVLSrAZHUihsurzyWII4gwKnAQ1o9LN63X8TPwnKIs6tC8dQvFb/zqhBe7/SWYjUQEt/0deK//JNK8oPZrU0ZUVoxKZQXqmYbNx+RryQDgWpVQAQToZeY7EEB4LCl3XPlcky3QTlDYbxHH4IpT6W6ICse1pzcIWGb00Yt+DPWvpfoDR/AgOFJw1+FVZrMIuaG+t0M01nNVeYE1cTdNRPuZPFkrhrb003LCe9uIrLMHk+TJNh+ulF/+jVdk077BF7zAYsZS/ZETtmIzZmgtXsO/vBfkYPo8PobfRuExp1tjn32TmLRr8BOYjMRQ==</latexit>

• What’s the knowledge we have about J?

– We can evaluate J(✓) <latexit sha1_base64="a/Lk8rEB5YqTpm5iqH/sBpa7bWw=">AAACiHicfVHbahRBEO0db3G8JfqYl8FFiCLLjAjRtxB9kIAkgewmsj2Emt6a2SZ9Gbpr1GWYr/BVP8y/sWezgdywoOH0qVNddbqKWklPafp3EN25e+/+g7WH8aPHT54+W994PvG2cQLHwirrTgrwqKTBMUlSeFI7BF0oPC7OPvX54+/ovLTmiBY15hoqI0spgAL1bW+L0xwJXp+uD9NRuozkJshWYMhWcXC6MZB8ZkWj0ZBQ4P00S2vKW3AkhcIu5o3HGsQZVDgN0IBGn7fLibvkVWBmSWldOIaSJXu5ogXt/UIXQamB5v56ridvy00bKj/krTR1Q2jEeaOyUQnZpLefzKRDQWoRAAgnw6yJmIMDQeGT4ittikJ3gfmMwZ7Dr6HVfo0OyLo3LQdXafjZBbsVf9uj/wmluRAGFJ40+ENYrcHMWm6s0900y1uusCSuJuhomHEnqzlx19+6OCwnu76Km2DybpSlo+zw/XBnd7WmNbbJXrItlrFttsO+sAM2ZoJp9ov9Zn+iOEqj7ejjuTQarGpesCsR7f4DaXjHOg==</latexit>


Gradient Black box optimization

– We can evaluate J(✓) and r✓ J(✓) <latexit sha1_base64="a/Lk8rEB5YqTpm5iqH/sBpa7bWw=">AAACiHicfVHbahRBEO0db3G8JfqYl8FFiCLLjAjRtxB9kIAkgewmsj2Emt6a2SZ9Gbpr1GWYr/BVP8y/sWezgdywoOH0qVNddbqKWklPafp3EN25e+/+g7WH8aPHT54+W994PvG2cQLHwirrTgrwqKTBMUlSeFI7BF0oPC7OPvX54+/ovLTmiBY15hoqI0spgAL1bW+L0xwJXp+uD9NRuozkJshWYMhWcXC6MZB8ZkWj0ZBQ4P00S2vKW3AkhcIu5o3HGsQZVDgN0IBGn7fLibvkVWBmSWldOIaSJXu5ogXt/UIXQamB5v56ridvy00bKj/krTR1Q2jEeaOyUQnZpLefzKRDQWoRAAgnw6yJmIMDQeGT4ittikJ3gfmMwZ7Dr6HVfo0OyLo3LQdXafjZBbsVf9uj/wmluRAGFJ40+ENYrcHMWm6s0900y1uusCSuJuhomHEnqzlx19+6OCwnu76Km2DybpSlo+zw/XBnd7WmNbbJXrItlrFttsO+sAM2ZoJp9ov9Zn+iOEqj7ejjuTQarGpesCsR7f4DaXjHOg==</latexit>

<latexit sha1_base64="6vNmN7cUeCZNCT1FkkaAoLRvGUw=">AAACmXicfZFtaxNBEMc351OND031jdA3h0GoIuFOBPuyPrwoBbGiSQvZI8xt5i5L9+HYnauG4/w0vtXv47dxL4lgW3Fg4bf/mdnZmckrJT0lya9edO36jZu3tm7379y9d397sPNg4m3tBI6FVdad5uBRSYNjkqTwtHIIOld4kp+97fwn5+i8tOYzLSvMNJRGFlIABWk2eMQN5ApmDacFErRHe2t4OhsMk1GysvgqpBsYso0dz3Z6ks+tqDUaEgq8n6ZJRVkDjqRQ2PZ57bECcQYlTgMa0OizZtVCGz8JyjwurAvHULxS/85oQHu/1HmI1EALf9nXif/yTWsq9rNGmqomNGJdqKhVTDbu5hHPpUNBahkAhJPhr7FYgANBYWr9C2XyXLdBeYehPYfvQ6kPFTog6541HFyp4Wsb2i35847+FyjNn8BA4UmDX4TVGsy84cY63U7TrOEKC+Jqgo6GKXeyXBB33a3th+Wkl1dxFSYvRmkySj++HB682axpi+2yx2yPpewVO2CH7JiNmWDf2Hf2g/2MdqPX0WF0tA6Nepuch+yCRZ9+A3yLziQ=</latexit>
First order optimization

– We can evaluate J(✓) , r✓ J(✓) , and H✓ (J(✓)) <latexit sha1_base64="a/Lk8rEB5YqTpm5iqH/sBpa7bWw=">AAACiHicfVHbahRBEO0db3G8JfqYl8FFiCLLjAjRtxB9kIAkgewmsj2Emt6a2SZ9Gbpr1GWYr/BVP8y/sWezgdywoOH0qVNddbqKWklPafp3EN25e+/+g7WH8aPHT54+W994PvG2cQLHwirrTgrwqKTBMUlSeFI7BF0oPC7OPvX54+/ovLTmiBY15hoqI0spgAL1bW+L0xwJXp+uD9NRuozkJshWYMhWcXC6MZB8ZkWj0ZBQ4P00S2vKW3AkhcIu5o3HGsQZVDgN0IBGn7fLibvkVWBmSWldOIaSJXu5ogXt/UIXQamB5v56ridvy00bKj/krTR1Q2jEeaOyUQnZpLefzKRDQWoRAAgnw6yJmIMDQeGT4ittikJ3gfmMwZ7Dr6HVfo0OyLo3LQdXafjZBbsVf9uj/wmluRAGFJ40+ENYrcHMWm6s0900y1uusCSuJuhomHEnqzlx19+6OCwnu76Km2DybpSlo+zw/XBnd7WmNbbJXrItlrFttsO+sAM2ZoJp9ov9Zn+iOEqj7ejjuTQarGpesCsR7f4DaXjHOg==</latexit>

<latexit sha1_base64="6vNmN7cUeCZNCT1FkkaAoLRvGUw=">AAACmXicfZFtaxNBEMc351OND031jdA3h0GoIuFOBPuyPrwoBbGiSQvZI8xt5i5L9+HYnauG4/w0vtXv47dxL4lgW3Fg4bf/mdnZmckrJT0lya9edO36jZu3tm7379y9d397sPNg4m3tBI6FVdad5uBRSYNjkqTwtHIIOld4kp+97fwn5+i8tOYzLSvMNJRGFlIABWk2eMQN5ApmDacFErRHe2t4OhsMk1GysvgqpBsYso0dz3Z6ks+tqDUaEgq8n6ZJRVkDjqRQ2PZ57bECcQYlTgMa0OizZtVCGz8JyjwurAvHULxS/85oQHu/1HmI1EALf9nXif/yTWsq9rNGmqomNGJdqKhVTDbu5hHPpUNBahkAhJPhr7FYgANBYWr9C2XyXLdBeYehPYfvQ6kPFTog6541HFyp4Wsb2i35847+FyjNn8BA4UmDX4TVGsy84cY63U7TrOEKC+Jqgo6GKXeyXBB33a3th+Wkl1dxFSYvRmkySj++HB682axpi+2yx2yPpewVO2CH7JiNmWDf2Hf2g/2MdqPX0WF0tA6Nepuch+yCRZ9+A3yLziQ=</latexit>
<latexit sha1_base64="Dp56a+zglYeLw/xxC6sFCv+6E2c=">AAAClnicfZFtaxNBEMc351OND00VQfDNYRBSkXAngn1Vig9YBDGCSQvZEOY2c5el+3DszqnhuA/jW/1Efhv3kgi2FQcWfvufmZ2ZnaxU0lOS/OpEV65eu35j52b31u07d3d7e/cm3lZO4FhYZd1pBh6VNDgmSQpPS4egM4Un2dnr1n/yBZ2X1nymVYkzDYWRuRRAQZr3HhzPa05LJGgG7wcb2t+f9/rJMFlbfBnSLfTZ1kbzvY7kCysqjYaEAu+naVLSrAZHUihsurzyWII4gwKnAQ1o9LN63X8TPwnKIs6tC8dQvFb/zqhBe7/SWYjUQEt/0deK//JNK8oPZrU0ZUVoxKZQXqmYbNx+RryQDgWpVQAQToZeY7EEB4LCl3XPlcky3QTlDYbxHH4IpT6W6ICse1pzcIWGb00Yt+DPWvpfoDR/AgOFJw1+FVZrMIuaG+t0M01nNVeYE1cTdNRPuZPFkrhrb003LCe9uIrLMHk+TJNh+ulF/+jVdk077BF7zAYsZS/ZETtmIzZmgtXsO/vBfkYPo8PobfRuExp1tjn32TmLRr8BOYjMRQ==</latexit>
Second order optimization

Hessian
Foundations of Computer Vision Torralba, Isola, Freeman 2024
Gradient Descent

N
X
⇤ (i) (i)
✓ = arg min L(f✓ (x ), y )
<latexit sha1_base64="Q10CJJ1ondCg8EG7JnRaFON1ooM=">AAAC3nicfVFNbxMxEPUuLZTlK4UjF4sIKa2qaBchwaVSBRw48FEkklSKk8XrzCZW/bGyvdDI2is3xJU/xoHfwgVvmiBoESNZen7zxp55U1SCW5emP6L4ytb21Ws715MbN2/dvtPZvTu0ujYMBkwLbU4KakFwBQPHnYCTygCVhYBRcfq8zY8+grFcq/duWcFE0rniJWfUBSrvaCKpWxSlJ24BjjbTfXyICTVzyVW+ITGxtcw9P8ya6Ru8qmBU+FdNr/yt6W0eOmumvsf3mr0DvKGWGwoneaeb9tNV4MsgW4MuWsdxvhuNyEyzWoJyTFBrx1lauYmnxnEmoElIbaGi7JTOYRygohLsxK+cafDDwMxwqU04yuEV+2eFp9LapSyCsu3VXsy15L9y49qVTyeeq6p2oNj5R2UtsNO4tRnPuAHmxDIAygwPvWK2oIYyF5aRJETBJ6alpGrmidJGNuNs4omA0hExBOO6GTF8vnDEtLcw5QsI0xt4HTp5W4GhTpt9v1oUPWuCG3Ny0KL/CbnaCANK2lVkF42/DIaP+lnaz9497h49Wy9lB91HD1APZegJOkIv0TEaIIa+o5/RVrQdf4g/x1/ir+fSOFrX3EN/RfztF0Oj6Ec=</latexit>
✓ i=1

<latexit sha1_base64="owEByfgSfqYY+gqWmMd3gcCbpjg=">AAACh3icfVHbahRBEO0db8l6S/TRl8ZFEJF1JojJY4w++CJGcDcL20Oo6a2ZbdKXobsmugzzE77qj/k39mxWMIlY0HD61KmuOl1FrVWgNP01SG7cvHX7ztb28O69+w8e7uw+mgbXeIkT6bTzswICamVxQoo0zmqPYAqNJ8XZuz5/co4+KGe/0KrG3EBlVakkUKRm4khVFRft6c4oHafr4NdBtgEjtonj092BEgsnG4OWpIYQ5llaU96CJyU1dkPRBKxBnkGF8wgtGAx5ux64488is+Cl8/FY4mv274oWTAgrU0SlAVqGq7me/Fdu3lB5kLfK1g2hlReNykZzcrx3zxfKoyS9igCkV3FWLpfgQVL8o+GlNkVhusi8x2jP48fY6lONHsj5F60AXxn41kW7lXjZo/8Jlf0jjCg+afGrdMaAXbTCOm+6eZa3QmNJQk/R0ygTXlVLEr6/dcO4nOzqKq6D6d44S8fZ59ejw6PNmrbYE/aUPWcZ22eH7AM7ZhMmmWbf2Q/2M9lOXiVvkoMLaTLY1DxmlyJ5+xu4Ycbv</latexit>
J(✓)
<latexit sha1_base64="a/Lk8rEB5YqTpm5iqH/sBpa7bWw=">AAACiHicfVHbahRBEO0db3G8JfqYl8FFiCLLjAjRtxB9kIAkgewmsj2Emt6a2SZ9Gbpr1GWYr/BVP8y/sWezgdywoOH0qVNddbqKWklPafp3EN25e+/+g7WH8aPHT54+W994PvG2cQLHwirrTgrwqKTBMUlSeFI7BF0oPC7OPvX54+/ovLTmiBY15hoqI0spgAL1bW+L0xwJXp+uD9NRuozkJshWYMhWcXC6MZB8ZkWj0ZBQ4P00S2vKW3AkhcIu5o3HGsQZVDgN0IBGn7fLibvkVWBmSWldOIaSJXu5ogXt/UIXQamB5v56ridvy00bKj/krTR1Q2jEeaOyUQnZpLefzKRDQWoRAAgnw6yJmIMDQeGT4ittikJ3gfmMwZ7Dr6HVfo0OyLo3LQdXafjZBbsVf9uj/wmluRAGFJ40+ENYrcHMWm6s0900y1uusCSuJuhomHEnqzlx19+6OCwnu76Km2DybpSlo+zw/XBnd7WmNbbJXrItlrFttsO+sAM2ZoJp9ov9Zn+iOEqj7ejjuTQarGpesCsR7f4DaXjHOg==</latexit>

Foundations of Computer Vision Torralba, Isola, Freeman 2024


Gradient Descent

J(✓)
<latexit sha1_base64="a/Lk8rEB5YqTpm5iqH/sBpa7bWw=">AAACiHicfVHbahRBEO0db3G8JfqYl8FFiCLLjAjRtxB9kIAkgewmsj2Emt6a2SZ9Gbpr1GWYr/BVP8y/sWezgdywoOH0qVNddbqKWklPafp3EN25e+/+g7WH8aPHT54+W994PvG2cQLHwirrTgrwqKTBMUlSeFI7BF0oPC7OPvX54+/ovLTmiBY15hoqI0spgAL1bW+L0xwJXp+uD9NRuozkJshWYMhWcXC6MZB8ZkWj0ZBQ4P00S2vKW3AkhcIu5o3HGsQZVDgN0IBGn7fLibvkVWBmSWldOIaSJXu5ogXt/UIXQamB5v56ridvy00bKj/krTR1Q2jEeaOyUQnZpLefzKRDQWoRAAgnw6yJmIMDQeGT4ittikJ3gfmMwZ7Dr6HVfo0OyLo3LQdXafjZBbsVf9uj/wmluRAGFJ40+ENYrcHMWm6s0900y1uusCSuJuhomHEnqzlx19+6OCwnu76Km2DybpSlo+zw/XBnd7WmNbbJXrItlrFttsO+sAM2ZoJp9ov9Zn+iOEqj7ejjuTQarGpesCsR7f4DaXjHOg==</latexit>

✓2
<latexit sha1_base64="NpiJT5/KEODczN77QRTuqpv3m44=">AAACh3icfVFNb9NAEN2YFtpQaAvHXqxGSFWFgl0h6LF8HLhUFImkkbJWNN6MnVX3w9odA5HlP8EV/hj/hnUaJNoiRlrp7Zs3O/N28kpJT0nyqxfd29i8/2Bru/9w59Hj3b39J2NvaydwJKyybpKDRyUNjkiSwknlEHSu8DK/etflL7+g89Kaz7SsMNNQGllIARSoCacFEsxOZnuDZJisIr4L0jUYsHVczPZ7ks+tqDUaEgq8n6ZJRVkDjqRQ2PZ57bECcQUlTgM0oNFnzWrgNn4WmHlcWBeOoXjF/l3RgPZ+qfOg1EALfzvXkf/KTWsqTrNGmqomNOK6UVGrmGzcuY/n0qEgtQwAhJNh1lgswIGg8Ef9G23yXLeBeY/BnsPz0OpjhQ7IuuOGgys1fGuD3ZI/79D/hNL8EQYUnjT4VVitwcwbbqzT7TTNGq6wIK7G6GiQcifLBXHX3dp+WE56exV3wfhkmCbD9NPLwdnb9Zq22AE7ZEcsZa/ZGfvALtiICabYd/aD/Yy2oxfRq+j0Whr11jVP2Y2I3vwGLj/HJg==</latexit>

✓1
<latexit sha1_base64="dRSr4rXm+n73Ts3IpBaIU3XH5Gg=">AAACh3icfVHbbhMxEHWWS9twa+GRlxUREkIorBGifSyXB14QRSJppHgVzTqzG6u+rOxZIFrtT/AKP8bf4E2DRFvESJaOz5zxzPEUtVaBsuzXILl2/cbNnd294a3bd+7e2z+4Pw2u8RIn0mnnZwUE1MrihBRpnNUewRQaT4uzt33+9Av6oJz9TOsacwOVVaWSQJGaCVohwYIv9kfZONtEehXwLRixbZwsDgZKLJ1sDFqSGkKY86ymvAVPSmrshqIJWIM8gwrnEVowGPJ2M3CXPo7MMi2dj8dSumH/rmjBhLA2RVQaoFW4nOvJf+XmDZVHeats3RBaed6obHRKLu3dp0vlUZJeRwDSqzhrKlfgQVL8o+GFNkVhusi8w2jP44fY6mONHsj5p60AXxn41kW7lXjWo/8Jlf0jjCg+afGrdMaAXbbCOm+6Oc9bobEkoafoacSFV9WKhO9v3TAuh19exVUwfTHm2Zh/ejk6frNd0y57yB6xJ4yzQ3bM3rMTNmGSafad/WA/k73kefIqOTqXJoNtzQN2IZLXvwEsI8cl</latexit>


✓ = arg min J(✓)
<latexit sha1_base64="AYItQoyunNkE46+v9OudQ3G4+AY=">AAACpnicfZFtaxNBEMc3Vx9qfErrS98shkIsEu6KYN8UivpCBLWCSQrZM+5t5i5L9+HYnbMNx30CP41v9ZP4bdxLIthWHFj47X9mdnZmslJJj3H8qxNt3bh56/b2ne7de/cfPOzt7I69rZyAkbDKutOMe1DSwAglKjgtHXCdKZhkZ69a/+QrOC+t+YTLElLNCyNzKTgGadbbY7gA5J/36RFl3BVamlm91hr6drCmp7NePx7GK6PXIdlAn2zsZLbTkWxuRaXBoFDc+2kSl5jW3KEUCpouqzyUXJzxAqYBDdfg03rVT0P3gjKnuXXhGKQr9e+MmmvvlzoLkZrjwl/1teK/fNMK88O0lqasEIxYF8orRdHSdjh0Lh0IVMsAXDgZ/krFgjsuMIywe6lMlukmKK8htOfgXSj1oQTH0br9ejVIftGEdgv2rKX/BUrzJzBQeNLAubBaczOvmbFON9MkrZmCHJkag8N+wpwsFshce2u6YTnJ1VVch/HBMImHycfn/eOXmzVtk8fkCRmQhLwgx+QNOSEjIsg38p38ID+jQfQ+GkWTdWjU2eQ8Ipcs+vIbi5PSvQ==</latexit>

Foundations of Computer Vision Torralba, Isola, Freeman 2024


proportional to the gradient magnitude. This algorithm is de
Gradient Descent

Algorithm

10.1:
N
X Gradient
(i)
descent
(i)
(GD).
✓ = arg min L(f✓ (x ), y )
Algorithm 10.1: ✓ i=1
0
1 Input: objective function J, initial parameter vector ✓ , l
<latexit sha1_base64="Q10CJJ1ondCg8EG7JnRaFON1ooM=">AAAC3nicfVFNbxMxEPUuLZTlK4UjF4sIKa2qaBchwaVSBRw48FEkklSKk8XrzCZW/bGyvdDI2is3xJU/xoHfwgVvmiBoESNZen7zxp55U1SCW5emP6L4ytb21Ws715MbN2/dvtPZvTu0ujYMBkwLbU4KakFwBQPHnYCTygCVhYBRcfq8zY8+grFcq/duWcFE0rniJWfUBSrvaCKpWxSlJ24BjjbTfXyICTVzyVW+ITGxtcw9P8ya6Ru8qmBU+FdNr/yt6W0eOmumvsf3mr0DvKGWGwoneaeb9tNV4MsgW4MuWsdxvhuNyEyzWoJyTFBrx1lauYmnxnEmoElIbaGi7JTOYRygohLsxK+cafDDwMxwqU04yuEV+2eFp9LapSyCsu3VXsy15L9y49qVTyeeq6p2oNj5R2UtsNO4tRnPuAHmxDIAygwPvWK2oIYyF5aRJETBJ6alpGrmidJGNuNs4omA0hExBOO6GTF8vnDEtLcw5QsI0xt4HTp5W4GhTpt9v1oUPWuCG3Ny0KL/CbnaCANK2lVkF42/DIaP+lnaz9497h49Wy9lB91HD1APZegJOkIv0TEaIIa+o5/RVrQdf4g/x1/ir+fSOFrX3EN/RfztF0Oj6Ec=</latexit>

imizing a cost function

<latexit sha1_base64="owEByfgSfqYY+gqWmMd3gcCbpjg=">AAACh3icfVHbahRBEO0db8l6S/TRl8ZFEJF1JojJY4w++CJGcDcL20Oo6a2ZbdKXobsmugzzE77qj/k39mxWMIlY0HD61KmuOl1FrVWgNP01SG7cvHX7ztb28O69+w8e7uw+mgbXeIkT6bTzswICamVxQoo0zmqPYAqNJ8XZuz5/co4+KGe/0KrG3EBlVakkUKRm4khVFRft6c4oHafr4NdBtgEjtonj092BEgsnG4OWpIYQ5llaU96CJyU1dkPRBKxBnkGF8wgtGAx5ux64488is+Cl8/FY4mv274oWTAgrU0SlAVqGq7me/Fdu3lB5kLfK1g2hlReNykZzcrx3zxfKoyS9igCkV3FWLpfgQVL8o+GlNkVhusi8x2jP48fY6lONHsj5F60AXxn41kW7lXjZo/8Jlf0jjCg+afGrdMaAXbTCOm+6eZa3QmNJQk/R0ygTXlVLEr6/dcO4nOzqKq6D6d44S8fZ59ejw6PNmrbYE/aUPWcZ22eH7AM7ZhMmmWbf2Q/2M9lOXiVvkoMLaTLY1DxmlyJ5+xu4Ycbv</latexit>
: ✓ ! R by descending steps K J(✓) <latexit sha1_base64="a/Lk8rEB5YqTpm5iqH/sBpa7bWw=">AAACiHicfVHbahRBEO0db3G8JfqYl8FFiCLLjAjRtxB9kIAkgewmsj2Emt6a2SZ9Gbpr1GWYr/BVP8y/sWezgdywoOH0qVNddbqKWklPafp3EN25e+/+g7WH8aPHT54+W994PvG2cQLHwirrTgrwqKTBMUlSeFI7BF0oPC7OPvX54+/ovLTmiBY15hoqI0spgAL1bW+L0xwJXp+uD9NRuozkJshWYMhWcXC6MZB8ZkWj0ZBQ4P00S2vKW3AkhcIu5o3HGsQZVDgN0IBGn7fLibvkVWBmSWldOIaSJXu5ogXt/UIXQamB5v56ridvy00bKj/krTR1Q2jEeaOyUQnZpLefzKRDQWoRAAgnw6yJmIMDQeGT4ittikJ3gfmMwZ7Dr6HVfo0OyLo3LQdXafjZBbsVf9uj/wmluRAGFJ40+ENYrcHMWm6s0900y1uusCSuJuhomHEnqzlx19+6OCwnu76Km2DybpSlo+zw/XBnd7WmNbbJXrItlrFttsO+sAM2ZoJp9ov9Zn+iOEqj7ejjuTQarGpesCsR7f4DaXjHOg==</latexit>

K
the gradient r✓ J. 2 Output: trained parameter vector ✓ = ✓

One iteration of gradient descent:
3 for k = 0, . . . , K – 1 do
k+1 k k
4 ✓ ✓ – ⌘r✓ J(✓ )

learning rate
This algorithm has two hyperparameters, the learning r
size (learning rate times gradient magnitude), and the num
Foundations of Computer Vision Torralba, Isola, Freeman 2024 0
Stochastic Gradient Descent (SGD)
• Want to minimize overall loss function J, which is sum of individual losses over each
example.
• In Stochastic gradient descent, compute gradient on sub-set (batch) of data.
• If batchsize=1 then θ is updated after each example.
• If batchsize=N (full set) then this is standard gradient descent.
• Gradient direction is noisy, relative to average over all examples (standard gradient descent).
• Advantages
• Faster: approximates total gradient with small sample
• Implicit regularizer
• Disadvantages
• High variance, unstable updates

Foundations of Computer Vision Torralba, Isola, Freeman 2024


391, 459]. Momentum makes the analogy to skiing even more precise: momentu
Momentum
e inertia of Algorithm
the skier, carrying
1.3: GD+momentum.
them over the little bumps and imperfections in th
and increasing
• A 1heavy
1.3: Gradient their
ball speed
rolling as
downtheya descend
hill, gains along
speed.0
a straight path. In math, mome
h momentum. Input: objective function J, initial parameter vector ✓ , learning rate ⌘, momentum
k+1
eans that we set the
µ, number parameter
of steps K
• Gradient steps biased to continue
update to be a direction
in direction of previous update: v , given by a weig
K
2 Output: trained parameter vector ✓ = ✓

k
nation of the 0previous update direction, v , plus the current negative gradient:
3 v =0Current update
4 for k = 0, . . . , K – 1 do k+1 k k
5
k+1 k
v = µv – ⌘r✓ J(✓ ) kv = µv – ⌘r✓ J(✓ ) (
6 ✓k+1 ✓k + vk+1 Previous update
eight µ in this combination is a new hyperparameter, sometimes simply calle
• Can help or hurt. Strength of momentum is a hyperparameter.
ntum.
e 1.2: (left)The
A full algorithm is given in algorithm 10.3.
loss function
ure 10.2
abs(✓). (right) shows how momentum affects gradient descent for a simple obje
s(✓) (absolute
ization trajec-
hree different
value of ✓). As can be seen in the figure, some momentum can
gence
momentum
ndicates value
rate
µ. (figure 10.2, µ = 0.5) but too much momentum will cause the trajecto
ameter at each
optimization,
ing at top and
ing to bottom.
Which are differentiable?
J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>
J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>
J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

Foundations of Computer Vision Torralba, Isola, Freeman 2024


Which are have defined gradients in Pytorch?
J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>
J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>
J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

Foundations of Computer Vision Torralba, Isola, Freeman 2024


Which will be hard to optimize?
J J J
Local minima Vanishing gradient
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit> <latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

Vanishing gradient J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

Exploding gradient J
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
<latexit sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

Foundations of Computer Vision Torralba, Isola, Freeman 2024


Simple case:
• Convex
• Single minimum
• Gradients point toward it J
<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>
J
<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

everywhere
• Gradient gracefully goes to zero
as minimum is approached


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

Foundations of Computer Vision Torralba, Isola, Freeman 2024


Discontinuous:
• But well-defined one-sided J
<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>
J
<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

derivatives
• Not a problem for Pytorch


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

Foundations of Computer Vision Torralba, Isola, Freeman 2024


Vanishing gradient:
J J
• progress is slow, noise may
<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

dominate


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

Foundations of Computer Vision Torralba, Isola, Freeman 2024


Zero gradient:
• Gradient is completely
J J
uninformative as to how to make <latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

progress
• Low loss region is never reached


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

Foundations of Computer Vision Torralba, Isola, Freeman 2024


Exploding gradient:
• Gradient goes to infinity as J J
minimizer is approached
<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

• Unstable updates, overshoots


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

Foundations of Computer Vision Torralba, Isola, Freeman 2024


Multiple local minima:
• Where you initialize matters
• Gradient descent does not
J J
guarantee reaching global
<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

minimizer
• Reaches a local minimizer


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

Foundations of Computer Vision Torralba, Isola, Freeman 2024


hm 10.4: J : ✓ ! R by evolution standard deviation , number of samples M, number o
Evolution
function
Strategies
Input:
strategies,
1 objective
i.e., sampling function J, initial parameter vector
2 Output: trained parameter vector ✓ = ✓
✓ 0
, learning
⇤ Krate ⌘, samp
different values
standard for ✓ and
deviation
volution• Gradient-like: finds a locally , 3number
for k = of
0, samples
. . . , K – 1M,
do number of steps K
loss-minimizing direction in parameter space
taking a step toward
ampling 2 Output: trained parameter the vector K
✓ .=. .✓, M do

4 for i = 1,
values small
or ✓ and• Sample that work best.
perturbations of θ and move toward perturbations that
3 for k = 0, . . . , K – 1 do5 ✏i ⇠ N (0, I)
ward the achieved lower loss
4 for i = 1, . . . , M do6 si = J(✓ + ✏i )
ork best.
5 ✏i ⇠ N (0, I) k+1 k 1
P M
7 ✓ ✓ – ⌘ M i=1 si ✏i
6 si = J(✓ + ✏i )
k+1 k 1
P M
7 ✓ ✓ – ⌘ M i=1 s ✏
As shown
i i in figure 10.4, this algorithm can succe
figure 10.3(c).
As shown in figure 10.4, this algorithm can successfully minimize the f
Successfully minimizes this function: J J
figure 10.3(c).
Figure 10.4: Using
<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

ES (algorithm 10.4) to
4: Using minimize a nondiffer-
10.4) to entiable (zero-gradient) ✓ ✓
J
<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit> <latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

Foundations of Computer Vision <latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>


<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

Torralba, Isola, Freeman 2024


Algorithm 10.5: GD+clip.
Gradient clipping
0
Input: objective function J, initial parameter vector ✓ , learning rate ⌘, number
1
• If gradients
steps K, exceed
max gradienta magnitude m,mscale them to magnitude m
magnitude
K
2 Output: trained parameter vector ✓ = ✓

• Useful, and commonly used hack
3 for k = 0, . . . , K – 1 do
k
4 v = r✓ J(✓ )
T
5 ✓ k+1
✓ – ⌘[clip(v1 , –m, m), …, clip(vM , –m, m)]
k

This algorithm indeed successfully minimizes our example of the exploding gra
can be seen in figure 10.5.
Successfully minimizes this function: J J
<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>

<latexit sha1_base64="hGDJ/rdfhLukv/n4ugVFVQo+G90=">AAAB8XicbVA9SwNBEN2LXzF+RS1tFoNgddyJoGXQRizEoPmQ5Ah7m0myZG/v2J0TwpFfYGtjI2Lrv7Hz37hJrtDog4HHezPMzAsTKQx63pdTWFpeWV0rrpc2Nre2d8q7ew0Tp5pDnccy1q2QGZBCQR0FSmglGlgUSmiGo8up33wEbUSs7nGcQBCxgRJ9wRlaqXbdLVc815uB/iV+Tiokx8PNnWtQCzXolj87vZinESjkkhnT9r0Eg4xpFFzCpNRJDSSMj9gA2pYqFoEJstnFE3pklR7tx9qWQjpTf05kLDJmHIW2M2I4NIveVPzPa6fYPw8yoZIUQfH5on4qKcZ0+j7tCQ0c5dgSxrWwt1I+ZJpxtCGVbBr+4u9/SePE9T3Xr51Wqhd5LkVyQA7JMfHJGamSK3JL6oQTIE/kmbw6xnlx3pz3eWvByWf2yS84H9+7fZBz</latexit>
<latexit sha1_base64="gC1TdVZAitJIwcbY3vXysa0Vu3s=">AAACDXicjVC7SgNBFL0bXzG+opY2g0GwCrsiaBm0ESsD5gHJEmYnd5Mhs7PLzKwQlnyBhY2/YiNia2/n3zhJttDEwgMDh3PO5c49QSK4Nq775RRWVtfWN4qbpa3tnd298v5BU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj66nfekCleSzvzThBP6IDyUPOqLFS/bZXrrhVdwayTLycVCDH/+K98me3H7M0QmmYoFp3PDcxfkaV4UzgpNRNNSaUjegAO5ZKGqH2s9k1E3JilT4JY2WfNGSm/pzIaKT1OApsMqJmqBe9qfiX10lNeOlnXCapQcnmi8JUEBOTaTWkzxUyI8aWUKa4/SthQ6ooM7bAkj3dWzx0mTTPqp5b9ernldpV3lkRjuAYTsGDC6jBDdxBAxggPMIzvDpPzovz5rzPowUnnzmEX3A+vgEAp5RH</latexit>
sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit>
sha1_base64="+1LsSFmFNQIH2LXy9i7M3znPKFI=">AAAB3XicbZBLSwMxFIXv1FetVatbN8EiuCozbnQpuBFXLdgHtEPJpHfa2ExmSO4IpfQXuHGhiH/Lnf/G9LHQ1gOBj3MScu+JMiUt+f63V9ja3tndK+6XDsqHR8eVk3LLprkR2BSpSk0n4haV1NgkSQo7mUGeRArb0fhunref0ViZ6keaZBgmfKhlLAUnZzUe+pWqX/MXYpsQrKAKK9X7la/eIBV5gpqE4tZ2Az+jcMoNSaFwVurlFjMuxnyIXYeaJ2jD6WLQGbtwzoDFqXFHE1u4v19MeWLtJInczYTTyK5nc/O/rJtTfBNOpc5yQi2WH8W5YpSy+dZsIA0KUhMHXBjpZmVixA0X5LopuRKC9ZU3oXVVC/xa0PChCGdwDpcQwDXcwj3UoQkCEF7gDd69J+/V+1jWVfBWvZ3CH3mfP43Vi3s=</latexit>
sha1_base64="gw8Pa1ixMp2vvNZp4PrpWM+AVAM=">AAAB6HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5sTBm0EasEzAckR9jbzCVr9vaO3T0hHPkFNhaK2PqT7Pw3bpIrNPHBwOO9GWbmBYng2rjut1PY2Nza3inulvb2Dw6PyscnbR2nimGLxSJW3YBqFFxiy3AjsJsopFEgsBNMbud+5wmV5rF8MNME/YiOJA85o8ZKzftBueJW3QXIOvFyUoEcjUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4tAZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSas+RmXSWpQsuWiMBXExGT+NRlyhcyIqSWUKW5vJWxMFWXGZlOyIXirL6+T9lXVc6te063Ub/I4inAG53AJHlxDHe6gAS1ggPAMr/DmPDovzrvzsWwtOPnMKfyB8/kDn0GMyg==</latexit>
sha1_base64="sUw7tiBtF+H8EqwBLDH9JL5vMNI=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi3hqwX5AG8pmO2nXbjZhdyOU0F/gxYMiXv1J3vw3btsctPXBwOO9GWbmBYng2rjut1NYW9/Y3Cpul3Z29/YPyodHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKjft+ueJW3TnIKvFyUoEc9X75qzeIWRqhNExQrbuemxg/o8pwJnBa6qUaE8rGdIhdSyWNUPvZ/NApObPKgISxsiUNmau/JzIaaT2JAtsZUTPSy95M/M/rpia89jMuk9SgZItFYSqIicnsazLgCpkRE0soU9zeStiIKsqMzaZkQ/CWX14lrYuq51a9xmWldpPHUYQTOIVz8OAKanAHdWgCA4RneIU359F5cd6dj0VrwclnjuEPnM8foIGMzg==</latexit>


<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

<latexit sha1_base64="uxESLMiYAPGWpfODytsf17jL2uM=">AAACEnicjVC7SgNBFL3rM8ZX1NJmMAhWYVcELYM2lgrmAUkIs5O7yZjZBzN3hbDkHyxs/BUbEVsrO//GSbKFJhYeGDiccy537vETJQ257peztLyyurZe2Chubm3v7Jb29usmTrXAmohVrJs+N6hkhDWSpLCZaOShr7DhD68mfuMBtZFxdEejBDsh70cykIKTleptGiDxbqnsVtwp2CLxclKGHP+Ld0uf7V4s0hAjEoob0/LchDoZ1ySFwnGxnRpMuBjyPrYsjXiIppNNTxqzY6v0WBBr+yJiU/XnRMZDY0ahb5Mhp4GZ9ybiX14rpeCik8koSQkjMVsUpIpRzCb9sJ7UKEiNLOFCS/tXJgZcc0G2xaI93Zs/dJHUTyueW/Fuz8rVy7yzAhzCEZyAB+dQhWu4gRoIuIdHeIZX58l5cd6c91l0yclnDuAXnI9vVJ6WoQ==</latexit>
<latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AAAB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmY2ZllplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDDbH7tlJ45pU8H2rhSSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MMBRThxh3Ah3K+UjZhhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl88ckz/wPn8Ao/ePKA==</latexit>

Foundations of Computer Vision Torralba, Isola, Freeman 2024


Concluding remarks

Textbooks on optimization can fill your whole bookshelf.

But the methods account for a large portion of what is used in computer
vision today, especially in systems based on deep learning.

Gradient descent in high-dimensional parameter spaces can be


surprisingly effective.

Foundations of Computer Vision Torralba, Isola, Freeman 2024

You might also like