When Geometry Meets Deep Learning

Patrick R. Nicolas - Original Substack Article

Target audience: Beginner

Estimated reading time: 13' Follow on LinkedIn

Geometric Deep Learning can appear intimidating at first. This article helps ease the learning
curve by (1) organizing the topic around its mathematical foundations and (2) introducing it
through the unifying concept of symmetry.

Newsletter GitHub Articles Blog

What you will learn: Learn the fundamental role of symmetry in models and non-Euclidean
geometry is essential. This article provides an overview of the diverse and higher-order
representations of data that emerge from these principles.

1 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 🎯 OVERVIEW 3
🔁 THE QUEST FOR SYMMETRY 3
Introduction 3
Definitions 3
Symmetry in Images 4
Symmetry in Shapes 5
Symmetry in Mathematics 6
🧮 GEOMETRIC DEEP LEARNING 6
Graph Neural Networks 7
Theory 7
💡 Graph Metrics 8
💡 Common types 9
Applications 9
Graph classification 9
Node classification & Link prediction 10
Mesh Modeling 11
Theory 11
Applications 12
Manifold & Group-Based Learning 12
Theory 13
💡 Differential Geometry 13
💡 Smooth Manifolds 13
💡 Riemannian Geometry 13
💡 Lie Groups & Algebras 16
Applications 17
Topological Deep Learning 18
Theory 19
💡 Point Set Topology 19
💡 Category Theory 19
💡 Topological Domains 19
💡 Laplacians & Graph Lifting 20
💡 Persistent Homology & Cohomology 20
💡 Homotopy 21
💡 Topological Deep Learning 21
Applications 22
Information-Geometric Learning 22
Theory 23
💡Statistical manifolds 23
💡 Fisher-Rao metric 23
💡Non-exponential Distributions 24
💡Divergences 24
Applications 24
📚 ABRIDGED GLOSSARY 24
📘 REFERENCES 25

The original, complete article is available at

A Friendly Primer on Geometric Deep Learning

2 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 🎯 Overview
Geometric Deep Learning (GDL) empowers data scientists to understand the underlying shape
and structure of data. Yet, the topic can feel overwhelming unless approached through the
unifying concept of symmetry.

⚠ This article does not present topics such as topology or differential geometry from a rigorous
mathematical standpoint - no PhD required! Rather, the subsequent sections are designed for
data scientists and engineers navigating the theoretical foundations of geometric learning
without being burdened by mathematical abstractions.

Optional sections with mathematical formulation are specified🤖

as For math-minded readers

🔁 The Quest for Symmetry

Introduction
Symmetry is the organizing principle behind geometric deep learning. It tells you what should
not change (invariance) or how things should change (equivariance) when you transform the
input [ref 1].
The benefits of symmetry in differential geometry or topology are:
• Simplification: Symmetry makes geometry easier to handle by turning intricate geometric
problems into simpler algebraic ones.
• Insight: It reveals how local and global properties of manifolds are connected, providing a
clearer view of their structure.
• Applications: In physics, symmetric spaces are central to theories such as general
relativity and string theory, which use them to model the universe.

Definitions
The terms symmetry, invariance and equivariance can be confusing.
• Symmetry (Webster dictionary): The property of remaining invariant under certain
changes (as of orientation in space, of the sign of the electric charge, of parity, or of the
direction of time flow) —used of physical phenomena and of equations describing them
• Invariance: The output doesn’t change when the input is transformed. Example: a cat
classifier still predicts “cat” for an upside-down photo.
• Equivariance: The output changes in a predictable way under the same transformation.
Example: rotate an image, and the predicted segmentation mask rotates accordingly.
🤖
For math-minded readers
A function f is invariant, given a group G if for any transformation g
𝑔 ∈ 𝐺, 𝑥 ∈ 𝑋 𝑓)𝑔(𝑥), = 𝑓 (𝑥)
A function f is equivariant, given a group G if for any transformation g
𝑔 ∈ 𝐺, 𝑥 ∈ 𝑋 𝑓)𝜙(𝑔),(𝑥) = 𝜌(𝑔))𝑓(𝑥),

3 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 The formulation of invariance and equivariance varies depends on the field they apply such
📌
as Metric, Laplace-Beltrami operator, Lie derivative or Tensor field.

Symmetry in Images
The concept of symmetry and invariance can be easily illustrated in the context of computer
vision. Vision models are often trained to be invariant to common geometric transforms:
• Translation: horizontal/vertical shifts; the object is recognized regardless of image location.

• Rotation: turning the image by some angle

• Scaling: resizing the object.

• Affine transforms: compositions of translation, rotation, scaling, and shear; they preserve
straightness and parallel lines.

• Projective transforms: changes in camera viewpoint; lines remain straight, but parallels
may meet at a vanishing point, which is especially challenging.

4 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 Symmetry in Shapes
Symmetric shapes arise in many applications and scientific domains.

Differential geometry & manifold learning: the gradient and divergence operators on vector fields
often exhibit symmetric patterns.

Graphs: repeating motifs appear in colored, undirected graphs.

Homotopy (algebraic topology): studies continuous deformations of shapes and relates closely
to equivariance.

Graph Neural Networks: exploit permutation invariance of node indices.

Biochemistry: components of Aurora A kinase (a serine/threonine kinase) inhibitors for tumor-

suppressor pathways show notable structural similarities.

5 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 Symmetry in Mathematics
Beyond geometry, symmetric (and sometimes antisymmetric) patterns appear throughout
mathematics. Here are some examples:

Convolution (in the usual linear/abelian setting) is commutative.

The Lie bracket is the algebraic operation that captures how infinitesimal symmetries interact.
The symmetry is abelian if the bracket is null.

The Special Orthogonal Group SO(3) —the rotations of 3-dimension Euclidean space—is a
continuous symmetry (Lie) group acting on space.

The tensor product of vector spaces comes with a universal bilinear map with symmetry property.

The wedge product is antisymmetric and equivariant under linear transformations T. It is

invariant under rotations.

🧮 Geometric Deep Learning

Geometric Deep Learning (GDL) is a branch of machine learning dedicated to extending deep
learning techniques to non-Euclidean data structures. Currently, there is no universally agreed-
upon, well defined scope for Geometric Deep Learning, as its interpretation varies among
authors.

6 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 GDL unlocks the ability to model complex, structured, and relational data, enabling advanced
problem-solving in areas like social networks, molecular science, design optimization, and
transportation systems.

Geometric Deep Learning (GDL) has been introduced by M. Bronstein, J. Bruna, Y. LeCun, A.
Szlam and P. Vandergheynst in 2017 [ref 2]. Readers can explore the topic further through a
tutorial by M. Bronstein [ref 3, 4].

⚠ This article does not delve into the mathematical foundations of Geometric Deep Learning—
such as topology or differential geometry—which were covered in a previous article [ref 5] and
comprehensively detailed in the excellent paper Mathematical Foundations of Geometric Deep
Learning [ref 6].

We arbitrary break down the field into 5 categories

1. Graph Neural Networks
2. Mesh Modeling
3. Manifold & Group-Based Learning
4. Topological Deep Learning
5. Information-Geometric Learning

📌
The layout of this article traces the evolution of geometry in learning systems—from discrete
structures to discretized continuums, onward to continuous and differential, then global, and
finally statistical geometry. Within this continuum, meshes act as the intermediary stage, rooted
in discrete differential geometry and linking Graph Neural Networks to manifold learning.

Graph Neural Networks

A Graph Neural Network (GNN) is an optimizable transformation on all attributes of the graph
(nodes, edges, global context) that preserves graph symmetries (permutation invariances). GNN
takes a graph as input and generate/predict a graph as output [ref 7, 8].

Theory
Data on manifolds can often be represented as a graph, where the manifold’s local structure is
approximated by connections between nearby points. Graph Neural Networks (GNN) and their
variants (like Graph Convolutional Networks (GCNs)) extend neural networks to process data

7 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 on non-Euclidean domains by leveraging the graph structure, which may approximate the
underlying manifold [ref 9, 10, 11].

💡 Inductive vs Transductive Modes

Inductive GNNs learn from existing (training) and new, unseen nodes, links or graphs
(inference). There is no need for these models to store and node/link embeddings [ref 12].

Transductive graph models are trained on the complete, fully defined graph (all nodes and
edges). As a result, they tend to overfit the training set and struggle to generalize to unseen
nodes, edges, or entirely new graphs. Their message passing and aggregation schemes are
typically straightforward.

💡 Graph Metrics
The selection of the topology of a GNN depends on centrality and homophily among other
metrics.

The various centrality measures quantify a node’s (or edge’s) structural importance so you can
prioritize, rank, or act on parts of the network.
🔹 Degree Centrality – Who’s most connected?
🔹 Closeness Centrality – Who’s closest to everyone?
🔹 Betweenness Centrality – Who bridges others?
🔹 Eigenvector Centrality – Who’s connected to the important ones?
🔹 PageRank – Who’s most influential?
🔹 Clustering Coefficient – Who’s in tight-knit groups?
🔹 Graph Density – How connected is the network?
🔹 Connected Components – How many isolated groups exist?

Homophily provides a principled way to assess whether proximity in the graph structure—
through nodes or edges—correlates with similarity in labels or classes [ref 13]. In a nutshell,
Edge homophily answers: “How likely is it that a random edge connects same-label nodes?”
while Node homophily answers: “How homophilic is the neighborhood of a typical node?

🤖
For math-minded readers
Let’s start with the mathematical definition. Let G=(V,E) be a graph with a set of nodes (vertices)
V, a set of edges E, label yv of node v and a neighbor of node v, N(v), there are 3 types of
homophily ratios:
Node homophily ratio: Average fraction of same-label neighbors per node.

Edge homophily ratio: The fraction of edges in a graph which connects nodes that have the
same class label.

8 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 Class insensitive edge homophily ratio: Edge homophily is modified to be insensitive to the
number of classes and size of each class. C number of classes, |Ck| number of nodes of class k
and hk denotes the edge homophily ratio of class k.

💡 Common types
Detailed information is available in a previous article [ref 14].
• Graph Convolutional Networks (GCNs): GCNs generalize the concept of convolution from
grids (e.g., images) to graphs. They aggregate information from a node’s neighbors using
normalized adjacency matrices and apply transformations to learn node embeddings.
• Spectral Graph Neural Networks (SGNN): These networks operate in the spectral domain
using the graph Laplacian. They use eigenvectors of the Laplacian for convolution-like
operations.
• Relational Graph Convolutional Networks (R-GCNs):R-GCNs extend GCNs to handle
heterogeneous graphs with different types of nodes and edges.
• Graph Transformers: They adapt the Transformer architecture to graph-structured data
using attention mechanisms and global context.
• Graph Autoencoders: These are used for unsupervised learning on graphs, aiming to
reconstruct graph structure and node features.
• Diffusion-Based GNNs: These networks use graph diffusion processes to propagate
information. Most research papers train these models in transductive mode although it can
theoretically be run in inductive mode.
• GraphSAGE (Graph Sample and Aggregate) learns node embeddings by sampling and
aggregating features from a fixed-size neighborhood of each node, enabling scalable
learning on large graphs.
• Graph Attention Networks (GATs) use attention mechanisms to learn the importance of
neighboring nodes dynamically. Each edge is assigned a learned weight during
aggregation that doesn’t depend on the specific training graph’s Laplacian.
• Graph Isomorphism Networks (GINs): GINs are designed to be as powerful as the
Weisfeiler-Lehman (WL) graph isomorphism test, distinguishing graph structures more
effectively.
• Graph Pooling Networks: They summarize graph information into a smaller representation,
similar to pooling in CNNs. They can be categorized into Global and hierarchical pooling.
Spectral pooling methods that depend on a graph’s specific eigenbasis are generally
transductive to that graph.
• Hyperbolic Graph Neural Networks: These networks operate in hyperbolic space, which is
well-suited for representing hierarchical or tree-like graph structures).
• Dynamic Graph Neural Networks: These networks are designed to handle temporal graphs,
where nodes and edges evolve over time.

Applications
Graph classification
• Molecules: property prediction, docking, affinity, generative design, retro-synthesis.
• Biology: structure graphs for residue, Protein-Protein interaction function prediction, fold
& complex modeling.

9 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 • Chemistry: force fields, Δ-learning for Discrete Fourier Transform, reaction outcome
prediction.
• Vision: point clouds & meshes, scene classification, segmentation, pose.
• Software Engineering: bug detection, optimization, code similarity.
• Routing: placement & layout optimization.
• Finance: default risk, alpha signals.
• …

Node classification & Link prediction

• Recommenders: user–item graphs; link prediction for Click Through Ratio, cold-start,
ranking.
• Risk Analysis: transaction networks, identity graphs; anomaly detection, entity resolution.
• Cybersecurity: log graphs, call graphs; suspicious edge & node detection.
• Knowledge Graphs: entity/relation reasoning
• Healthcare: patient–event graphs for Electronic Health Records, disease–gene networks
& risk prediction.
• …

⚠
As seen in this section, the distinction between transduction and induction is a bit fuzzy. For
instance, a Graph Isomorphism Network) applies to unseen graph and therefore is inductive.
However, it can be run transductively on a single fixed graph for node classification.

The table below categorizes the most common Graph Neural Networks according to their
training paradigm—transductive, inductive, or a combination of both. The intensity of the green
shading indicates approximately the relative proportion between transductive and inductive
modes.

📌 The principles of graph theory serve as the mathematical backbone of Graph Neural
Networks, guiding their configuration and performance optimization [ref 15, 16, 17, 18 & 19].

10 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 👉 Related Python libraries and accompanying newsletter articles on
Graph Neural Networks can be found in the original Substack publication
A Friendly Primer on Geometric Deep Learning

Mesh Modeling
A mesh in discrete differential geometry is a finite, piecewise-linear approximation of a smooth
geometric object. The most common type are simplicial and combinatorial complexes (vertices,
edges, faces) together with a geometric embedding of its vertices in Rn (most often 3D). The
mesh supplies the discrete scaffold on which smooth notions—metric, curvature, differential
operators—are defined and computed.

Theory
While a Riemannian manifold in differential geometry models data in a continuous domain, a
mesh—rooted in discrete differential geometry [ref 23]—represents data through a structured
assembly of interconnected topological elements.

Point clouds consist of discrete 3D points—typically captured by 3D scanners or depth

cameras—where each point is defined by its spatial coordinates (X, Y, Z) and may carry extra
attributes such as color or intensity. They offer a flexible way to represent the geometry of real-
world objects or scenes without requiring explicit connectivity between points.
Meshes, in contrast, are structured collections of vertices, edges, and faces that together define
the surface of a 3D object. Unlike point clouds, they encode explicit topological relationships
between elements, providing a continuous surface representation. Meshes are widely used in
3D modeling, computer graphics, and additive manufacturing (3D printing).

A mesh is the discrete surface/volume on which we do geometry: it carries both the topology
(connectivity) and the metric (lengths/areas/angles), enabling faithful, stable discretization of
smooth geometric operators.

11 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 A triangle mesh is a manifold, but only if
• Every edge is incident to exactly 2 faces or 1 face
• Every triangle incident to each of its vertices is homeomorphic to a disk or half-disk.

Meshes are related to persistent homology.

The cotangent Laplacian is the standard discrete Laplace–Beltrami operator on a triangle mesh.
It’s the matrix you get when you discretize the continuous Laplacian using piecewise-linear (P1)
finite elements; its edge weights are built from triangle cotangents. The cotangent Laplacian is
derived from the Dirichlet energy.

Applications
• 3D shape classification & retrieval: Mesh CNNs or spectral GNNs process vertices and
faces with DDG-derived operators.
• Surface reconstruction: Deep neural-mesh methods rebuild smooth surfaces from point
clouds.
• Differentiable simulation: Learn physical parameters (e.g., Young’s modulus) using mesh-
based finite element method.
• Physics-informed neural networks on meshes: Uses unstructured meshes instead regular
(Euclidean) grids.
• Surface-based graph learning: Message passing over vertex adjacency with geometric
weights from discrete differential geometry.
• Surface smoothness loss: Regularize geometry by penalizing curvature variation
• 3D reconstruction in computer vision: Mesh prediction networks for deformable surfaces.
• Articulated object modeling: Jointed mesh deformations (e.g., human body models)
• Scene understanding: Meshes integrated with point clouds for semantic segmentation and
navigation.
• …

Manifold & Group-Based Learning

Machine learning models based on manifolds leverage differential geometry [ref 24]. A manifold
is essentially a space that, around every point, looks like Euclidean space, created from a
collection of maps (or charts) called an atlas, which belongs to Euclidean space.

12 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 Theory
Differential (or smooth) manifolds have a tangent space at each point, consisting of vectors.
Riemannian manifolds are a type of differential manifold equipped with a metric to measure
curvature, gradient, and divergence

The primary goal of learning Riemannian geometry is to understand and analyze the properties
of curved spaces that cannot be described adequately using Euclidean geometry alone.

💡 Differential Geometry
Differential geometry is an extensive and intricate area that exceeds what can be covered in a
single article or blog post. There are numerous outstanding publications, including books [ref
25, 26 & 27], this newsletter article [ref 28] and tutorials [ref 29, 30], that provide foundational
knowledge in differential geometry and tensor calculus, catering to both beginners and experts

💡 Smooth Manifolds
A manifold is a topological space that, around any given point, closely resembles Euclidean
space. Specifically, an n-dimensional manifold is a topological space where each point is part of
a neighborhood that is homeomorphic to an open subset of n-dimensional Euclidean space.
Examples of manifolds include one-dimensional circles, two-dimensional planes and spheres,
and the four-dimensional space-time used in general relativity.

Smooth or Differential manifolds are types of manifolds with a local differential structure, allowing
for definitions of vector fields or tensors that create a global differential tangent space.

A Riemannian manifold is a differential manifold that comes with a metric tensor, providing a
way to measure distances and angles [ref 31].

The tangent space at a point on a manifold is the set of tangent vectors at that point, like a line
tangent to a circle or a plane tangent to a surface. Tangent vectors can act as directional
derivatives, where you can apply specific formulas to characterize these derivatives.

Intuitively, the directional derivative measures how a function changes as you move in a specific
direction. The directional derivative
• is large and positive, the function is increasing quickly in that direction.
• is zero, the function doesn’t change in that direction.
• is negative, the function is decreasing in that direction.
💡
Riemannian Geometry
A geodesic is the shortest path between two points on a surface, volume or high dimension
space it lives in. It is a straight line on a plane, a curve on a sphere similar to the trajectory of a
airplane on a long-haul route.

The exponential map translates an initial direction and velocity—represented by a tangent vector
at a given point—into a new point on the manifold reached by following the geodesic that starts
in that direction for a unit amount of time. Conceptually, it provides a way to move from the linear
world (tangent space) to the curved world (manifold) while preserving the notion of “straight
motion.”

13 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 The logarithm map does the inverse: it takes two points on the manifold and expresses the
shortest displacement between them as a tangent vector in the tangent space of one of the
points. In essence, it “flattens” the manifold locally so that the difference between two points can
be represented linearly. This operation enables linear reasoning, differentiation, and optimization
on curved domains—allowing algorithms to work with familiar linear tools in the tangent space
and then project results back onto the manifold via the exponential map.

The exponential and logarithm maps form a conceptual bridge between local linear geometry
and global curved geometry. They are essential in Riemannian optimization, geometric statistics,
and manifold-based machine learning because they enable movement, interpolation, and
computation on non-Euclidean spaces using local linear approximations while respecting the
manifold’s curvature and structure.

🤖
For math-minded readers
Given a Riemannian manifold M with a metric tensor g, the geodesic length L of a continuously
differentiable curve f: [a, b] -> M

An exponential map is a map from a subset of a tangent space of a Riemannian manifold. Given
a tangent vector v at a point p on a manifold, there is a unique geodesic Gv and an exponential
map exp.

Given a differentiable function f, a vector v in Euclidean space Rn and a point x on manifold, the
directional derivative in v direction at x is defined as:

14 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 A Riemannian metric on a smooth manifold M is a choice, at every point of M, of an inner product
on the tangent space that varies smoothly with the point. Formally, it’s a smooth, symmetric,
positive-definite (0,2)-tensor field [ref 32].

Parallel transport moves a vector along a curve on a manifold so that it stays “as unchanged as
possible” according to the manifold’s connection.

The Levi–Civita connection is the canonical way to differentiate vector fields on a Riemannian
manifold that satisfies two conditions - Torsion free & metric compatibility [ref 33].

Riemannian curvature measures how a Riemannian manifold bends—i.e., how its geometry
deviates from flat Euclidean space. Tt encodes how a vector changes after parallel transport
around a tiny loop (holonomy). On a Riemannian manifold with Levi–Civita connection, a vector
field along a smooth curve is parallel if its covariance derivative is null.

Ricci curvature is a way to estimate the full Riemann curvature tensor by averaging sectional
curvatures through each tangent direction.

Ricci flow is a geometric evolution equation that smooths a Riemannian metric by diffusing its
curvature. It can be interpreted as the heat equation for the metric: regions of high curvature
diffuse out, tending to uniformize the geometry [ref 34].

Intrinsic geometry involves studying objects, such as vectors, based on coordinates (or base
vectors) intrinsic to the manifold’s point. For example, analyzing a two-dimensional vector on a
three-dimensional sphere using the sphere’s own coordinates

Extrinsic geometry in differential geometry deals with the properties of a geometric object that
depend on its specific positioning and orientation in a higher-dimensional space.

Here’s what extrinsic geometry involves:

• Embedding: Extrinsic geometry examines how a lower-dimensional object, like a surface or
curve, is situated within a higher-dimensional space.
• Extrinsic coordinates: These are coordinates that describe the position of points on the
geometric object in the context of the larger space.
• Normal vectors: An important concept in extrinsic geometry is the normal vector, which is
perpendicular to the surface at a given point. These vectors help in defining and analyzing
the object’s orientation and curvature relative to the surrounding space.
• Curvature: Extrinsic curvature measures how a surface bends within the larger space.
• Projection and shadow: Extrinsic geometry also looks at how the object projects onto other
surfaces or how its shadow appears in the ambient space.
🤖
For math-minded readers
The Riemann metric g is defined as

15 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 Christoffel symbol for a Levi-Civita Connection equipped with a Riemannian metric g

Riemann curvature tensor using Christoffel symbols

Ricci curvature as contraction of the Riemann curvature tensor

💡 Lie Groups & Algebras

Lie groups play a crucial role in Geometric Deep Learning by modeling symmetries such as
rotation, translation, and scaling. This enables non-linear models to generalize effectively for
tasks like object detection and transformations in generative models [ref 35, 36 & 37].

In differential geometry, a Lie group is a mathematical structure that combines the properties of
both a group and a smooth manifold. It allows for the application of both algebraic and geometric
techniques. As a group, it has an operation (like multiplication) that satisfies 4 axioms:
• Closure
• Associativity
• Identity
• Invertibility

A Lie algebra is a mathematical structure that describes the infinitesimal symmetries of

continuous transformation groups, such as Lie groups. It consists of a vector space equipped
with a special operation called the Lie bracket, which encodes the algebraic structure of small
transformations. Intuitively, a Lie algebra is a tangent space of a Lie group.

16 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 The Special Euclidean Group [ref 38] and Special Orthogonal Group [ref 39] are the most applied
Lie groups.

🤖 For math-minded readers

— Special Orthogonal Groups SO(2), SO(3)

The Special Orthogonal Groups SO(2) (resp. SO(3)) are the Lie groups that represents rotations
in 2-dimensional (resp. 3-dimensional) space. It is widely used in robotics, physics, computer
vision, and geometric deep learning to describe rigid body rotations without scaling or reflection.

— Special Euclidean Group SE(3)

The Special Euclidean group is a subset of the broader affine transformation group. It contains
the translational and orthogonal groups as subgroup. In robotics and physics, an element of
SE(3) is often referred to as a rigid transformation, since it describes a rotation + translation
without scaling or deformation.

Applications
The following highlights the advantages of utilizing differential geometry to tackle the difficulties
encountered by researchers in the creation and validation of generative models.

• Understanding data manifolds: Data in high-dimensional spaces often lie on lower-

dimensional manifolds.
• Improving latent space interpolation: In generative models, navigating the latent space
smoothly is crucial for generating realistic samples. .
• Optimization on manifolds: The optimization processes used in training generative models
can be enhanced by applying differential geometric concepts.
• Geometric regularization: Incorporating geometric priors or constraints based on differential
geometry can help in regularizing the model.
• Advanced sampling techniques: Differential geometry provides sophisticated techniques
for sampling from complex distributions (important for both training and generating new
data points).
• Enhanced model interpretability: By leveraging the geometric structure of the data and
model, differential geometry can offer new insights into how generative models work.
• Physics-Informed Neural Networks: Projecting physics law and boundary conditions such
as set of partial differential equations on a surface manifold improves the optimization of
deep learning models.
• 3D rotations & poses: SO(3), SE(3) manifolds for robotics, molecular dynamics, pose
estimation.
• Shape analysis: manifold of diffeomorphisms or curves for medical imaging and object
alignment.

17 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 • Computer vision: intrinsic geometry of images, geodesic flows between shapes.
• Differential equations: Neural ODEs and continuous normalizing flows on manifolds.
• Graph: Node embedding on manifolds
• Word embeddings: Poincare (hyperbolic) embeddings
• Simulation: Heat-flow using Laplace-Beltrami operator
• Clustering & regression: Information geometry and geodesic distances
• Bain-connectivity, MRI & EEG: Symmetric Positive Definite (SPD) groups
• Low-rank factorization, PCA: Stiefel & Grassmann manifolds
• Robotics: Special orthogonal groups and Special Euclidean groups in 3 dimension
• Taxonomy & knowledge graphs: Hyperbolic manifolds
• Encoding directional data: Spherical manifolds
• …

👉 Related Python libraries and accompanying newsletter articles on Manifold

& Group-based Learning can be found in the original Substack publication
A Friendly Primer on Geometric Deep Learning

Topological Deep Learning

Topological Deep Learning (TDL) is an emerging area of machine learning that extends
traditional deep learning and graph neural networks (GNNs) to operate on higher-order
topological structures beyond simple graphs [ref 44, 45]. While GNNs model pairwise
relationships (nodes and edges), TDL leverages the mathematics of algebraic topology to
capture multi-way interactions (e.g., triangles, tetrahedra) and topological invariants like holes,
cycles, and cavities [ref 46].

18 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 Theory
💡 Point Set Topology
Point-set (general) topology treats the foundational definitions and constructions of topology.
Key concepts include continuity (pre-images of open sets are open), compactness (every open
cover admits a finite sub-cover), and connectedness (no separation into two disjoint nonempty
open subsets) [ref 47].

A topological space is a set equipped with a topology—a collection of open sets (or
neighborhood system) satisfying axioms that capture “closeness” without distances. It may or
may not arise from a metric. Common examples include Euclidean spaces, metric spaces, and
manifolds.

💡 Category Theory
Category theory offers a panoramic view of mathematics: it suppresses detail to reveal structure.
From this vantage, unexpected parallels emerge—why the least common multiple mirrors a
direct sum, or what discrete spaces, free groups, and fraction fields share. This topic is described
in detail in Demystifying the Math of Geometric Deep Learning - Category Theory [ref 48, 49].
💡
Topological Domains
Topological Data Analysis (TDA) is a methodology that applies concepts from algebraic topology
and computational geometry to analyze and extract meaningful patterns from complex datasets.
It provides a geometric and topological perspective to study the shape and structure of data.
TDA seeks to develop rigorous mathematical, statistical, and algorithmic techniques to infer,
analyze, and leverage the intricate topological and geometric structures underlying data, often
represented as point clouds in Euclidean or more general metric spaces [ref 50].

The most common topological domains are

• Simplicial Complexes
• Cellular Complexes
• Hypergraphs
• Combinatorial Complexes

In the simplest terms, a simplicial complex is a graph with faces. It generalizes graphs that
model higher-order relationships among data elements—not just pairwise (edges), but also
triplets, quadruplets, and beyond (0-simplex: node, 1-simplex: edge, 2-simplex: Triangle, 3-
simplex: Tetrahedron, ) [ref 51].

You may wonder where simplicial complex fits in the overall picture of data representation.

19 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 💡 Laplacians & Graph Lifting
The following Laplacian matrices generalize the graph Laplacian to higher-dimensional
simplicial complexes. Their purposes are:
• Generalization of the diffusion & message passing scheme
• Capture higher-order dependencies (Boundaries and Co-boundaries)
• Support convolution operations in Spectral Simplicial Neural Networks
• Encoding of topological features such holes or cycles.

The Upper-Laplacian (a.k.a. Up-Laplacian) is a key operator in simplicial homology and Hodge
theory, defined for simplicial complexes to generalize the notion of graph Laplacians to higher-
order structures. The Up-Laplacian captures the influence of higher-dimensional simplices (e.g.,
triangles) on a given k-simplex. It reflects how k-simplices are stitched together into higher-
dimensional structures.

The Down-Laplacian (a.k.a. Low-Laplacian) is like the Up-Laplacian except that the boundary
operator projects k-simplices to (k-1) faces.

The Hodge Laplacian for simplicial complexes is a generalization of the graph Laplacian to
higher-order structures like triangles, tetrahedra, etc. It plays a central role in topological signal
processing, Topological Deep Learning, and Hodge theory. It is the sum of Up and Down
Laplacian [ref 52].

🤖 For math-minded readers

Node, edge and face representation (or embedding) generated through Hodge Laplacian and
incidence matrices are very sparse.

Full Simplicial lifting assigns signal/features to faces or triangles. Partial Simplicial lifting uses
topological domains in the message passing and aggregation only (e.g., L1 Hodge Laplacian).
This article applies Partial Simplicial Lifting.
💡
Persistent Homology & Cohomology
Persistent homology captures the topological structure of data across different spatial
resolutions. Features that remain stable over many scales are interpreted as genuine properties
of the space, while short-lived ones are viewed as noise or artifacts [ref 53].

20 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 Chain complex is a sequence of abelian groups ⋯- d(n+1) → C(n) — d(n)→ C(n-1) —d(n-1)→
… with boundary maps satisfying d(n−1)∘d(n)=0. Cochain complex: the “opposite-direction”
version ⋯- d(n-1) → C(n) — d(n)→ C(n+1) —d(n+1)→ … with d(n+1)∘d(n)=0; this duality mirrors
covariant vs. contravariant behavior [ref 54].

A Cohomology from a cochain complex gives algebraic invariants of a space. Compared to

homology, Cohomology often carries additional structure (e.g., a graded-commutative ring via
the cup product) and many versions arise by dualizing homology constructions. De Rham
Cohomology measures the “holes” of a smooth manifold using differential forms. Two forms
count as the same Cohomology class if they differ by a derivative (an exact form) [ref 55].

Filtrations of a space across scales give rise to persistent homology, summarizing the birth and
death of features; distances between summaries are stable under small perturbations.
💡
Homotopy
Homotopy theory is a branch of algebraic topology that studies the properties of spaces that are
preserved under continuous deformations, called homotopies. It helps in distinguishing spaces
that are topologically the same from those that are not and offers tools for working with complex
spaces in mathematics and applied sciences [ref 56].

💡
Topological Deep Learning
Topological Deep Learning is widely considered a subfield of Geometric Deep Learning (GDL).
Deep Learning distinguished by the way it defines local neighborhoods.

In geometric models built on continuous manifolds, the neighborhood around a point is typically
represented by a local Euclidean space—often formalized as a tangent space or the Lie algebra
at the identity of the manifold.

21 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 The distinction between Topological Data Analysis (TDA) and Topological Deep learning
📌
(TDL) can be confusing as both stem from algebraic topology. While TDA supports analysis of
the shape of input data (e.g., outliers, clusters,), TDL is used to build learning models (e.g.,
Graph, Node, Simplex classification) that exploits both the input data and the output of TDA.

Applications
• Molecular and protein modeling: Capture of 3D binding pockets, loops)
• Mesh and 3D object analysis: Shape classification, segmentation
• Sensor and communication networks: Detection of coverage holes and cycles
• Neuroscience: Study of brain connectivity and functional cycles).
• Point cloud data: Capture pf geometric and topological invariants).
• Complex social and financial networks: Analysis of group interactions
• …

👉 Related Python libraries and accompanying newsletter articles on

Topological Deep Learning can be found in the original Substack publication
A Friendly Primer on Geometric Deep Learning

Information-Geometric Learning
Information geometry applies differential-geometric tools to probability and statistics. It treats
families of distributions as statistical manifolds—Riemannian manifolds whose points are
probability laws—often equipped with a pair of conjugate affine connections; in many cases
these structures are dually flat.

I treat information geometry as distinct from differential geometry, much as I distinguish the
📌
natural gradient from the standard stochastic gradient. Other authors may differ.

22 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 Theory
💡 Statistical manifolds
Information Geometry is derived from mathematical statistics and information theory [ref 59].
The well-known field of statistics covers discrete probability variables such as Binomial or
Poisson distributions and probability density continuous functions such as Exponential or
Gaussian distribution. Information theory pioneered by Claude Shannon owed its origin to
thermodynamics and concern with the quantization of Entropy as a measure of the disorder (or
chaos) in an irreversible system. The Gamma distribution plays a pivotal role in information
theory.

Information geometry leverages Riemannian manifolds and metric, parallel transport,

connection, tangent spaces, curvature and geodesics to extract geometric properties of various
families of probability density functions [ref 60].

Statistical manifolds are smooth families of probability distributions represented as geometric,

parameterized spaces.
💡 Fisher-Rao metric
The canonical Riemannian metric related to the distribution parameters is known as the Fisher-
Rao metric or Fisher Information Matrix. Alongside of the Fisher-Rao metric, information
geometry equips the manifold with two α-connections, known as dual affine connections: e-
connection and m-connection. Parameters of exponential distributions are affine e-connections
while expectation parameters are affine m-connection. Statistical manifolds can be induced with
a Hessian metric for exponential distribution [ref 61, 62].

🤖
For math-minded readers
A Fisher-Riemann manifold is a differentiable manifold M whose points correspond to probability
distributions p(x∣θ) from a statistical model parameterized by θ∈Θ⊂Rn, and which is equipped
with the Riemannian metric g

The Fisher information metric for a continuous probability density function p is defined (outer
product formulation).

Given that the expectation and differentiator operators can be interchanged. the metric can
also be computed as (Hessian formulation)

23 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 💡 Non-exponential Distributions
Information Geometry applies to Bivariate family of probability distribution such as McKay
Bivariate Gamma and Log Gamma manifold, Freund Bivariate Exponential and Log
Exponential manifold and Bivariate Gaussian manifold.
Information geometry extends spatial stochastic (Poisson) processes

💡 Divergences
Kullback–Leibler (KL) divergence—a.k.a. relative entropy—measures how one probability
distribution departs from a reference/target distribution. It’s not a metric (it’s asymmetric and
fails the triangle inequality).

Jensen–Shannon (JS) divergence—the symmetrized, KL-based measure—quantifies similarity

between two distributions and has a closed form for Gaussian families.

Bregman divergence (or Bregman “distance”) is a difference measure generated by a strictly

convex function. When points represent probability distributions (e.g., model parameters or
datasets), it serves as a statistical divergence. A basic example is squared Euclidean distance.

Applications
• Neural network training: Stability using Natural Gradient.
• Reinforcement learning: Application of Natural Policy Gradient and exponential
distributions.
• Manifold learning: Fisher-Rao geodesics to compute statistical distances.
• Kullback-Leibler divergence: Minimization using information projection on manifolds.
• Computer vision: Embedding distribution and covariance pooling.
• Metric learning: Information geometric distances such as Log-Euclidean or Affine-
Invariance metrics.
• Uncertainty estimation: Fisher-Rao metric and Wasserstein distance.
• Natural/Jeffrey priors: Fisher information geometry.
• Variational Autoencoder: Statistical manifold representation of latent space.
• Regularization: Fisher Information Matrix
• Support Vector Machines: Information-geometric kernels
• …

📚 Abridged Glossary
👉
The complete glossary can be found in the original Substack publication
A Friendly Primer on Geometric Deep Learning

24 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 📘 References
1. A Brief Introduction to Geometric Deep Learning AI for Complex Data - J. McEwen - Medium,
2022
2. Geometric deep learning: going beyond Euclidean data M. Bronstein, J. Bruna, Y. LeCun, A.
Szlam and P. Vandergheynst - 2017
3. Geometric foundations of Deep Learning - M. Bronstein - 2025
4. Geometric Deep Learning Book M. Bronstein, J. Bruna, T. Cohen, P. Veličković - 2021
5. Demystifying the Math of Geometric Deep Learning Hands-on Geometric Deep Learning - 2025
6. Mathematical Foundations of Geometric Deep Learning H. S. de Ocariz Borde, M. Bronstein -
University of Oxford - 2015
7. A Practical Tutorial on Graph Neural Networks I. Ward, J. Joyner, C. Lickfold, Y. Guo, M.
Bennamoun - 2012
8. Introduction to Graph Neural Networks: A Starting Point for Machine Learning Engineers - J. Tanis,
C. Giannella, A. Mariano, The MITRE Corp. - 2024
9. A Comprehensive Introduction to Graph Neural Networks - Datacamp - 2022
10. Graph Neural Networks: A Gentil Introduction - YouTube. A. Persson - 2021
11. Stanford CS: Machine Learning with Graphs - YouTube - CS-224 Stanford University Online - 2022
12. Revisiting Inductive Graph Neural Networks Hands-on Geometric Deep Learning - 2025
13. Neighbors Matter: How Homophily Shapes Graph Neural Networks Hands-on Geometric Deep
Learning - 2025
14. Introduction to Geometric Deep Learning - Graph Neural Networks Hands-on Geometric Deep
Learning - 2025
15. Graph Theory Schaum’s Outlines - V. K. Balakrishnan, 1997
16. Introduction to Graph Theory Chap 1 Definition and Examples R. Wilson - 4th Edition - Addison-
Wesley - 1996
17. An introduction to Graph Theory - Chap 3 Multigraphs D. Grinberg - Drexel University, Korman
Center - 2025
18. Spectral Graph Theory D. Spielman - Yale University - 2011
19. Graph and Matrices - R. B. Bapat - Springer - 2009
20. PyTorch Geometric Documentation [Link]
21. [Link]
22. Taming PyTorch Geometric for Graph Neural Networks - Hands-on Geometric Deep Learning -
2025
23. Discrete Differential Geometry Series K. Crane - YouTube - 2021
24. Differential Geometric Approaches to Machine Learning A. Pouplin - DTU Library -2023
25. Introduction to Smooth Manifolds J. Lee - Springer Science+Business media New York - 2013
26. Differential Geometric Structures - W. Poor - Dover Publications, New York - 1981
27. Differential Geometry - From Elastic Curves to Willmore Surfaces - U, Pinkall, O. Gross - Compact
Textbooks in Mathematics, Birkhauser - 2023
28. Riemannian Manifolds: Foundational Concepts - Hands-on Geometric Deep Learning - 2025
29. Differential Geometry - Tensor Calculus - R. Davie - YouTube - 2024
30. Differential Geometry - Khan Academy - YouTube - 2024
31. Tensor Analysis on Manifolds - Chap 5: Riemannian Manifolds R Bishop, S. Goldberg - Dover
Publications, New York - 1980
32. Tensor Analysis on Manifolds - Chap 3: Vector Analysis on Manifolds R Bishop, S. Goldberg -
Dover Publications, New York - 1980
33. Differential Geometric Structures- Chap 3: Riemannian Vector Bundles W. Poor - Dover
Publications, New York - 1981
34. An Illustrated Introduction to the Ricci Flow G. Khan - 2022
35. Introduction to Lie Groups and Lie algebras Chap 1 & 2 - A Kirillov. Jr - SUNNY at Stony Brook -
2021
36. Lie groups & Lie algebras Series - J. Evans, YouTube - 2020
37. Lie Groups Beyond an Introduction - A. Knapp - 2023

25 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas
 38. SE(3): The Lie Group That Moves the World - Hands-on Geometric Deep Learning - 2025
39. Mastering Special Orthogonal Groups with Practice - Hands-on Geometric Deep Learning - 2025
40. geomstats: a Python Package for Riemannian Geometry in Machine Learning N. Miolane, J.
Mathe, C. Donnat, M. Jorda, X. Pennec
41. Geomstats API GitHub
42. Pymanopt API GitHub
43. Exploring Geometric Learning with Geomstats - Hands-on Geometric Deep Learning - 2025
44. Topological Deep Learning: Going Beyond Graph Data - M. Hajij et all - 2023
45. An introduction to Topological Data Analysis: fundamental and practical aspects for data
scientists - F. Chazal, B. Michel - 2021
46. Algebraic Topology for Data Scientists M. Postol - The MITRE Corporation, 2024
47. Topology - Chap 2: Elements of Point-Set Topology J. Hocking, G. Young - Dover Publications -
1988
48. Demystifying the Math of Geometric Deep Learning - Category Theory - Hands-on Geometric
Deep Learning - 2025
49. Category Theory in Machine Learning - D. Shiebler, B. Gavranovic’, P. Wilson - University of
Oxford, Strathclyde, Southampton - 2021
50. From Nodes to Complexes: A Guide to Topological Deep Learning - Hands-on Geometric Deep
Learning - 2025
51. Exploring Simplicial Complexes for Deep Learning: Concepts to Code - Hands-on Geometric Deep
Learning, 2025
52. Topological Lifting of Graph Neural Networks - Hands-on Geometric Deep Learning - 2025
53. A gentle introduction to persistent homology - C. Bock - 2019
54. Chain Complexes - MIT Libraries - 2021
55. Introduction to Differential Geometry Chap 6 De Rham Cohomology J. Robbin,
56. Topology - Chap 4: The Elements of Homotopy Theory J. Hocking, G. Young - Dover Publications
- 1988
57. TopoX documentation - GitHub
58. GUDHI: Geometric Understanding of Higher Dimensions - INRIA, Fr
59. An Elementary Introduction to Information Geometry F. Nielsen - Sony Computer, 2 Science
Laboratories, 2020
60. The Many Faces of Information Geometry F. Nielsen - Notices of the American Mathematical
Society, 2022
61. What is Fisher Information? Ian Collings - YouTube, 2024
62. Shape your Model with Fisher-Rao Metric - Hands-on Geometric Deep Learning, 2025

Patrick Nicolas is a software and data engineering veteran with 30 years of experience in
architecture, machine learning, and a focus on geometric learning. He writes and consults
on Geometric Deep Learning, drawing on prior roles in both hands-on development and
technical leadership. He is the author of "Scala for Machine Learning", Packt Publishing ISBN
978-1-78712-238-3 and Geometric Learning in Python newsletter and Hands-on Geometric
Deep Learning blog.

26 When Geometry Meets Deep Learning - Hands-on Geometric Deep Learning - Patrick Nicolas