6 Machine Learning in Computational Mechanics

Leon Herrmann
Stefan Kollmannsberger
Chair of Data Engineering in Construction
Bauhaus-Universität Weimar

Deep Learning in Computational Mechanics – an introductory course,

Herrmann et al. 2025

website book
Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 1
Contents
• 5 Advanced Physics-Informed Neural Networks
• 6.1.1 Singular Value Decomposition
• 6.2 Reduced Order Models
• 6.3 Sparse Identification of Non-Linear Dynamical Systems – SINDy
• 6.4 Clustering
• 6.5 Support Vector Machines
• 7 Material Modeling with Neural Networks

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 2
6.1.1 Singular Value Decomposition
Singular value decomposition is a unique matrix factoriziation (𝑨 ∈ ℂ𝑛×𝑚 )
𝑨 = 𝑼𝚺𝑽∗ 𝑉 ∗ is the conjugate transpose of 𝑉
𝑛×𝑚
With 𝑼 ∈ ℂ𝑛×𝑛 , 𝚺 ∈ ℝ+
0 , 𝑽∗ ∈ ℂ𝑚×𝑚

Matrices 𝑼, 𝚺, 𝑽 are obtained through eigendecompositions of 𝑨𝑨∗ and 𝑨∗ 𝑨

• Columns of 𝑼 consist of the eigenvectors of 𝑨𝑨∗ 𝑼, 𝑽 are unitary matrices, i.e., 𝑼∗ 𝑼 = 𝑰, 𝑽∗ 𝑽 = 𝑰
• Columns of 𝑽 consist of the eigenvectors of 𝑨∗ 𝑨 (for real matrices: orthogonal matrices)
• Diagonal entries of 𝚺 are the square roots of the corresponding eigenvalues Non-zero eigenvalues of 𝑨𝑨∗
Signs of the eigenvectors of 𝑼 and 𝑽 must be consistent and can be checked with and 𝑨∗ 𝑨 are the same
𝚺 = 𝑼∗ 𝑨𝑽
If an entry of 𝚺 is negative, the signs of the corresponding eigenvector in 𝑼 or 𝑽 must be flipped.
Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 3
6.1.1 Singular Value Decomposition
Singular value decomposition as dimensionality reduction technique
• Economy singular value decomposition

• Exact reconstruction of 𝑨 with truncated 𝑼, 𝚺, 𝑽 (for 𝑛 ≠ 𝑚)

for 𝑛 > 𝑚 𝑨 = 𝑼 ෩𝚺
෩ 𝑽∗
for 𝑛 < 𝑚 𝑨 = 𝑼𝚺 ෩𝑽෩∗
• Truncated singular value decomposition: truncation of 𝑼, 𝚺, 𝑽 beyond the mismatch in dimensionality

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 4
6.1.1 Singular Value Decomposition
Truncated singular value decomposition: truncation of 𝑼, 𝚺, 𝑽 beyond the mismatch in dimensionality
𝑟×𝑟
• 𝑼 ഥ ∈ ℂ𝑛×𝑟 , 𝚺
ഥ ∈ ℝ+
0
ഥ ∈ ℂ𝑚×𝑟 are truncated up to 𝑟 th singular value
,𝑽

• Reconstruction is now approximative!

𝑨≈𝑼 ഥ𝚺
ഥ𝑽 ഥ∗
• The truncated singular value decomposition enables dimensionality reduction
• Principal components analysis is a related dimensionality reduction technique (see 6.1.2)

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 5
[Link] Image Compression
Image compression as example of dimensionality reduction with singular value decomposition
• Gray-scale image as matrix with pixel entries in 𝑥1 and 𝑥2 direction

Original image (1′ 868 × 1468) Singular values (diagonal entries of Σ)

Truncated singular value decomposition leads to 𝑛𝑟 + 𝑟𝑟 + 𝑚𝑟 values instead of 𝑛𝑚 values

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 6
 Low-dimensional procedure for the characterization
[Link] Image Compression of human faces, Sirovich et al. 1987 Further compression possible if
multiple images are considered
Compression ratios for 𝑟 = 1, 2, 10, 20, 100, 200:
simultaneously, i.e., a common
0.116%, 0.243%, 1.22%, 2.43%, 12.5%, 25.8% compression, e.g., eigenfaces from a
library of faces can reconstruct yet
unseen faces

𝑟=1 𝑟 = 10 𝑟 = 100

𝑟=2 𝑟 = 20 𝑟 = 200
Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 7
Exercises
• E.22 Singular Value Decomposition (P & C)
• First, perform a singular value decomposition on a given matrix. Next implement a general singular value
decomposition and apply it to an image.

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 8
[Link] Identification of a Reduced Basis
Consider the spatio-temporal function
𝑦 = 𝑓 𝑥, 𝑡 = sin 2𝜋𝑥 + 2sin 4𝜋𝑥 𝑥𝑡 2 + 𝑥 2 𝑡 + 5
Measurements obtained at 𝑚 temporal snapshots on a spatial grid with 𝑛 points
(stored in a snapshot matrix 𝑿)
𝑓(𝑥1 , 𝑡1 ) 𝑓 𝑥2 , 𝑡1 … 𝑓(𝑥𝑛 , 𝑡1 )
𝑓(𝑥1 , 𝑡2 ) 𝑓 𝑥2 , 𝑡2 … 𝑓 𝑥𝑛 , 𝑡2
𝑿=
⋮ ⋮ ⋱ ⋮
𝑓(𝑥1 , 𝑡𝑚 ) 𝑓(𝑥2 , 𝑡𝑚 ) … 𝑓(𝑥𝑛 , 𝑡𝑚 )
Singular value decomposition of snapshot matrix
𝑿 = 𝑼𝚺𝑽∗
• Spatial modes can be extracted from columns of 𝑽
• Temporal modes can be extracted from columns of 𝑼

Only the first 3 modeshapes are associated

with non-zero singular values:
→ 𝑿 can be reconstructed exactly with 𝑟 = 3

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 9
[Link] Identification of a Reduced Basis
Modeshapes identified from 𝑿 using singular value decomposition
𝑿=𝑼 ഥ𝚺ഥ𝑽ഥ∗
provide a reduced basis (using time-dependent coefficients 𝒄 𝑡 ∈ ℝ𝑚×𝑟 )
𝒄 𝑡 = 𝑿𝑽 ഥ
𝑿≈𝒄 𝑡 𝑽 ഥ 𝑇 Exploiting the orthonormal property of 𝑽 (𝑽∗ 𝑽 = 𝑰)

Consider a system of equations (task is to solve for 𝒚)

𝑲𝒚 = 𝒇
Prior knowledge of the solution 𝒚 is stored in the snapshot matrix 𝑿
ഥ
Singular value decomposition yields 𝑽, i.e., 𝑽
Projection into a lower dimensional system
ഥ = 𝒇𝑽
𝑲𝒚𝑽 ഥ
ഥ
𝑲𝒄 = 𝒇𝑽
Solve for 𝒄 instead of 𝒚 and recover 𝒚 with
ഥ
𝒚 ≈ 𝒄𝑽

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 10
Exercises
• E.23 Introduction to Reduced Order Models (C)
• Sample a basic spatio-temporal function and extract the essential spatial and temporal modes using a singular
value decomposition

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 11
6.2 Reduced Order Models
Acceleration of a dynamic finite element simulation using a reduced basis
Consider the semi-discrete finite element equations 𝑴 is the mass matrix, 𝑲 is the stiffness
𝑴𝒖ሷ 𝑡 + 𝑲𝒖 𝑡 = 𝑭(𝑡) matrix, 𝑭 is the source vector
Solvable by integration through time, e.g., central difference approximation of 𝒖ሷ
𝒖 𝑡 + Δ𝑡 − 𝒖 𝑡 + Δ𝑡
𝒖ሶ 𝑡 ≈
2Δ𝑡
𝒖 𝑡 + Δ𝑡 − 2𝒖 𝑡 + 𝒖(𝑡 − Δ𝑡)
𝒖ሷ 𝑡 ≈
Δ𝑡 2
Inserted in the finite element equations
𝒖 𝑡 + Δ𝑡 − 2𝒖 𝑡 + 𝒖 𝑡 − Δ𝑡
𝑴 + 𝑲𝒖 𝑡 ≈ 𝑭 𝑡
Δ𝑡 2
Rewritten in an explicit form
1 2 1
𝒖 𝑡 + Δ𝑡 𝑴 ≈ 𝑭 𝑡 − 𝑲 − 𝑴 𝒖 𝑡 − 𝑴𝒖(𝑡 − Δ𝑡)
Δ𝑡 2 Δ𝑡 2 Δ𝑡 2
Compute next time step 𝒖(𝑡 + Δ𝑡) by inverting 𝑴

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 12
6.2.2 Reduced Order Modeling with Finite Elements
From a snapshot matrix
𝑇
𝑿 = 𝒖 𝑡1 , 𝒖 𝑡2 , … , 𝒖 𝑡𝑚 ∈ ℝ𝑛×𝑚
A reduced basis is identified (with the truncated singular value decomposition)
𝝍=𝑽 ഥ ∈ ℝ𝑛×𝑟
Projection of degrees of freedom 𝒖 𝑡 ∈ ℝ𝑛 onto reduced basis 𝒄 𝑡 ∈ ℝ𝑟
𝒖 𝑡 ≈ 𝝍𝒄(𝑡)
Insertion into the semi-discrete finite element equations
𝑴𝝍𝒄ሷ 𝑡 + 𝑲𝝍𝒄 𝑡 = 𝑭(𝑡) 𝑟 degrees of freedom with 𝑛 equations
Pre-multiplication with 𝝍𝑇 to project the system onto the reduced space (to reduce number of equations to 𝑟)
𝝍𝑇 𝑴𝝍𝒄ሷ 𝑡 + 𝝍𝑇 𝑲𝝍𝒄 𝑡 = 𝝍𝑇 𝑭(𝑡)
Solve for 𝒄(𝑡) and recover full solution vector with
𝒖 𝑡 ≈ 𝒄 𝑡 𝝍𝑇

Proper orthogonal decomposition: projection of solution (to differential equation) to lower-dimensional subspace
using orthogonal basis functions

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 13
6.2.3 Cantilever Beam
Simulation of a cantilever beam with two-dimensional finite elements

• Full finite element simulation for 𝑚 = 5000 timesteps with 𝑛 = 410 degrees of freedom → 𝑋

• Singular value decomposition on 𝑿 to obtain reduced basis 𝝍 = 𝑽 ഥ

• Identification of the first four mode shapes of an Euler-Bernoulli beam
𝑚𝑛 𝑥1 = cosh 𝛽𝑛 𝑥1 − cos 𝛽𝑛 𝑥1 − 𝜎𝑛 sinh 𝛽𝑛 𝑥1 − sin 𝛽𝑛 𝑥1

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 14
6.2.3 Cantilever Beam

From the singular values, a truncation level of 𝑟 = 5 is selected (reducing the system from 410 × 410 to 5 × 5)

Full system: 2.87 ⋅ 10−3 s per time step

Reduced system: 1.65 ⋅ 10−4 s per time step

Measurement for snapshot matrix 𝑿 Simulation with reduced order model

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 15
Exercises
• E.24 Reduced Order Models with Finite Elements (C)
• Modify a finite element code to incorporate a reduced order model. After collecting a snapshot matrix and
performing a singular value decomposition, the reduced basis is to be applied within the system of equations.

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 16
6.3 SINDy Discovering governing equations from data: Sparse identification
of nonlinear dynamical systems, Brunton et al. 2015
SINDy: Sparse Identification of Non-Linear Dynamical Systems

The goal of SINDy is the discovery of sparse dynamical system models

described by systems of differential equations of the form
𝑑
𝒙 𝑡 = 𝑓(𝒙 𝑡 )
𝑑𝑡
𝑇
Where 𝒙 𝑡 = 𝑥1 𝑡 , 𝑥2 𝑡 , … , 𝑥𝑚 𝑡

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 17
6.3 SINDy
ሶ
1. Sample temporal snapshots of 𝒙 𝑡 and its first temporal derivative 𝒙(𝑡) in snapshot matrices 𝑿, 𝑿ሶ
−
𝒙 𝑡1 − − 𝒙ሶ 𝑡1 −
−
𝒙 𝑡2 − ሶ 2 ) − If the temporal derivatives 𝒙(𝑡)
− 𝒙(𝑡 ሶ are
𝑿= , 𝑿ሶ =
⋮ ⋮ unavailable, numerical schemes can be used
− 𝒙 𝑡𝑛 − ሶ 𝑛) −
− 𝒙(𝑡
2. Select a library of candidate functions Θ for the sparse regression
2 2
1 𝒙1 (𝑡1 ) 𝒙2 (𝑡1 ) ⋯ 𝒙1 𝑡1 𝒙2 𝑡1 ⋯ 𝒙1 𝑡1 𝒙2 𝑡1 ⋯
2 2
𝚯 = 1 𝒙1 (𝑡2 ) 𝒙2 (𝑡2 ) ⋯ 𝒙1 𝑡2 𝒙2 (𝑡2 ) ⋯ 𝒙1 𝑡2 𝒙2 𝑡2 ⋯
⋮ ⋮ ⋮ ⋱ ⋮ ⋱ ⋮ ⋮ ⋮
2 2
1 𝒙1 (𝑡𝑛 ) 𝒙2 (𝑡𝑛 ) ⋯ 𝒙1 𝑡𝑛 𝒙2 (𝑡𝑛 ) ⋯ 𝒙1 𝑡𝑛 𝒙2 𝑡𝑛 ⋯
3. The sparse regression problem with sparse regression coefficients 𝜩
𝑿ሶ = 𝚯 𝑿 𝚵
4. Compute the sparse regression coefficients 𝚵 with
• Sequential thresholded least squares
• (or Lasso) Least squares with a 𝐿1 penalty to promote sparsity

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 18
6.3 SINDy – Sequential Thresholded Least Squares
The regression is solved iteratively with least squares. In each iteration the coefficients smaller than a threshold 𝑡 are
set to zero. This results in a sparse coefficient matrix

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 19
6.3.1 Learning a Differential Equation
A system of differential equations is given as
𝑥ሶ 1 = −𝑥1 + 2𝑥2
𝑥ሶ 2 = −𝑥1 + 𝑥2
With the initial conditions
𝑥1 0 = 1
𝑥2 0 = 1
And with the solution
𝑥1 𝑡 = sin 𝑡 + cos 𝑡
𝑥2 𝑡 = cos(𝑡)
The goal of SINDy is to find the underlying differential equation with snapshots of the solution 𝑥1 𝑡 , 𝑥2 𝑡

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 20
6.3.1 Learning a Differential Equation
𝑇
𝒙 𝑡 = sin 𝑡 + cos 𝑡 , cos 𝑡
𝜋 3
Snapshots collected at 𝒕 = [0, , 𝜋, , 2𝜋]
2 4𝜋

𝑥1 (𝑡1 ) 𝑥2 (𝑡1 ) 1 1 𝑥ሶ 1 (𝑡1 ) 𝑥ሶ 2 (𝑡1 ) 1 0

𝑥1 (𝑡2 ) 𝑥2 (𝑡2 ) 1 0 𝑥ሶ 1 (𝑡2 ) 𝑥ሶ 2 (𝑡2 ) −1 −1
𝑿 = 𝑥1 (𝑡3 ) 𝑥2 (𝑡3 ) = −1 −1 , 𝑿ሶ = 𝑥ሶ 1 (𝑡3 ) 𝑥ሶ 2 (𝑡3 ) = −1 0
𝑥1 (𝑡4 ) 𝑥2 (𝑡4 ) −1 0 𝑥ሶ 1 (𝑡4 ) 𝑥ሶ 2 (𝑡4 ) 1 1
𝑥1 (𝑡5 ) 𝑥2 (𝑡5 ) 1 1 𝑥ሶ 1 (𝑡5 ) 𝑥ሶ 2 (𝑡5 ) 1 0
Selected candidate functions and the corresponding matrix
1, 𝑥1 , 𝑥2 , 𝑥12 , 𝑥22 , 𝑥1 𝑥2
2 2
1 𝑥1 (𝑡1 ) 𝑥2 𝑡1 𝑥1 𝑡1 𝑥2 𝑡1 𝑥1 𝑡1 𝑥2 (𝑡1 ) 1 1 1 1 1 1
2 2
1 𝑥1 (𝑡2 ) 𝑥2 𝑡2 𝑥1 𝑡2 𝑥2 𝑡2 𝑥1 𝑡2 𝑥2 (𝑡2 ) 1 1 0 1 0 0
𝚯= 2 2 =
1 𝑥1 (𝑡3 ) 𝑥2 𝑡3 𝑥1 𝑡3 𝑥2 𝑡3 𝑥1 𝑡3 𝑥2 (𝑡3 ) 1 −1 −1 1 1 1
1 𝑥1 (𝑡4 ) 𝑥2 𝑡4 𝑥1 𝑡4 2
𝑥2 𝑡4 2
𝑥1 𝑡4 𝑥2 (𝑡4 ) 1 −1 0 1 0 0
1 𝑥1 𝑡5 𝑥2 (𝑡5 ) 𝑥1 𝑡5 2
𝑥2 𝑡5 2
𝑥1 𝑡5 𝑥2 (𝑡5 ) 1 1 1 1 1 1

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 21
6.3.1 Learning a Differential Equation
1 𝑥1 (𝑡1 ) 𝑥2 𝑡1 𝑥1 𝑡1 2 𝑥2 𝑡1 2 𝑥1 𝑡1 𝑥2 (𝑡1 )
1 𝑥1 (𝑡2 ) 𝑥2 𝑡2 𝑥1 𝑡2 2 𝑥2 𝑡2 2 𝑥1 𝑡2 𝑥2 (𝑡2 )
𝚯= 1 𝑥1 (𝑡3 ) 𝑥2 𝑡3 𝑥1 𝑡3 2 𝑥2 𝑡3 2 𝑥1 𝑡3 𝑥2 (𝑡3 )
1 𝑥1 (𝑡4 ) 𝑥2 𝑡4 𝑥1 𝑡4 2 𝑥2 𝑡4 2 𝑥1 𝑡4 𝑥2 (𝑡4 )
1 𝑥1 𝑡5 𝑥2 (𝑡5 ) 𝑥1 𝑡5 2 𝑥2 𝑡5 2 𝑥1 𝑡5 𝑥2 (𝑡5 )
ሶ
Sparse regression problem 𝑿 = 𝚯 𝐗 𝚵 solved with sequential thresholded least squares with 𝑡 = 0.01
Initial least squares regression

−5.98𝑒 − 15 −3.93𝑒 − 15 0 0
−1 −1 −1 −1
2 1 2 1
𝚵 = 𝚯𝑇 𝚯 −1
𝚯𝑇 𝑿ሶ = →
6.56𝑒 − 15 3.84𝑒 − 15 0 0
−2.35𝑒 − 15 −1.06𝑒 − 15 0 0
2.19𝑒 − 15 1.81𝑒 − 15 0 0

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 22
6.3.1 Learning a Differential Equation
Recovered system

Original system
𝑥ሶ 1 = −𝑥1 + 2𝑥2
𝑥ሶ 2 = −𝑥1 + 𝑥2

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 23
6.3.2 Structural Dynamics Model
A two degree of freedom mass-spring-damper model

Governing equations obtained from equilibrium

𝑚1 𝑥ሷ 1 + 𝑘1 𝑥1 + 𝑑1 𝑥ሶ 1 + 𝑘2 𝑥1 − 𝑥2 + 𝑑2 𝑥ሶ 1 − 𝑥ሶ 2 = 0
𝑚2 𝑥ሷ 2 + 𝑘2 𝑥2 − 𝑥1 + 𝑑2 𝑥ሶ 2 − 𝑥ሶ 1 = 0
And initial conditions
𝑥1 0 = 𝑥10 , 𝑥ሶ 1 0 = 𝑥ሶ 10 , 𝑥2 0 = 𝑥20 , 𝑥ሶ 2 0 = 𝑥ሶ 20

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 24
6.3.2 Structural Dynamics Model
Governing equations obtained from equilibrium
𝑚1 𝑥ሷ 1 + 𝑘1 𝑥1 + 𝑑1 𝑥ሶ 1 + 𝑘2 𝑥1 − 𝑥2 + 𝑑2 𝑥ሶ 1 − 𝑥ሶ 2 = 0
𝑚2 𝑥ሷ 2 + 𝑘2 𝑥2 − 𝑥1 + 𝑑2 𝑥ሶ 2 − 𝑥ሶ 1 = 0
Substitutiono to transform the system of second order differential equations into a system of first order differential
equations Any higher order system of differential
𝑦1 = 𝑥1 , 𝑦2 = 𝑥ሶ 1 , 𝑦3 = 𝑥2 , 𝑦4 = 𝑥ሶ 2 equations can be rewritten in terms of a
system of first order differential equations
𝑦ሶ 1 = 𝑦2
−𝑘1 𝑦1 − 𝑑1 𝑦2 − 𝑘2 𝑦1 − 𝑦3 − 𝑑2 (𝑦2 − 𝑦4 )
𝑦ሶ 2 =
𝑚1
𝑦ሶ 3 = 𝑦4
−𝑘2 𝑦3 − 𝑦1 − 𝑑2 (𝑦4 − 𝑦2 )
𝑦ሶ4 =
𝑚2

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 25
6.3.2 Structural Dynamics Model – Linear
Assuming linear springs and dampers
𝑘𝑖 𝑥 = 𝐾𝑖 𝑥
𝑑𝑖 𝑥ሶ = 𝐷𝑖 𝑥ሶ
Example values 𝐾1 = 2, 𝐾2 = 4, 𝐷1 = 2, 𝐷2 = 3, 𝑚1 = 5, 𝑚2 = 2
SINDy estimation with 300 snapshots and the original system

Measurement

Prediction with SINDy

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 26
6.3.2 Structural Dynamics Model – Linear
SINDy is made for nonlinear dynamics
Nonlinear springs and dampers
𝑘1 𝑥 = 𝑥, 𝑘2 𝑥 = 2𝑥 3
𝑑1 𝑥ሶ = 𝑥,ሶ 𝑑2 𝑥ሶ = 1
𝑚1 = 5, 𝑚2 = 2
SINDy estimation with 500 snapshots and the original system

Measurement

Prediction with SINDy

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 27
Exercises
• E.25 Introduction to SINDy (P & C)
• Apply SINDy to a simple system of first order differential equations using pen-and-paper. Next implement SINDy
and the sequential thresholded least sqaures algorithm and recompute the pen-and-paper computations.
• E.26 SINDy for Structural Dynamics (P & C)
• Apply the implementation of SINDy to a two-degree-of-freedom example from structural dynamics.

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 28
6.4 Clustering
Clustering: Discovers similarities between data and creates discrete clusters (unsupervised)

One approach is 𝑘-means clustering

• Subdivision of dataset into 𝑘 clusters
• Each datapoint 𝒙 belongs to cluster with the closest mean 𝝁
• Framed as minimization problem
𝑘
2
arg min ෍ ෍ 𝒙𝑖 − 𝝁𝑗
𝝁𝑗 2
𝑗=1 𝒙𝑖 |𝑦(𝒙𝑖 =𝑦𝑗 )
• Where 𝝁𝑗 is the mean of the 𝑗𝑡ℎ cluster consisting of datapoints associated with the cluster, i.e., 𝑦𝑗 = 𝑦(𝒙𝑖 )
• NP-hard problem, solved by heuristic algorithms, e.g., Lloyds algorithm

1. Computation of distances 𝑑𝑖𝑗 , used to identify closes cluster 𝑦𝑖 by taking the minimum distance min 𝑑𝑖𝑗
𝑗
2. Recomputation of cluster centers 𝝁𝑗 with the mean of the associated data points 𝒙𝑖 ∈ 𝐷𝑗

Repeat until convergence

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 29
6.4 Clustering – Lloyds algorithm

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 30
6.4.1 Cross-Section Clustering
Consider the cross-section of an I-beam with the properties
𝑏ℎ3 𝐵
𝐼1 =
+ 𝐻3 − ℎ3
12 12
𝑏3 ℎ 𝐵3
𝐼2 = + 𝐻−ℎ
12 12
𝐴 = 𝐻𝐵 − ℎ(𝐵 − 𝑏)
Assuming three different cross-sections (with different distributions)

Can we identify the three different cross-sections from the three properties, 𝐼1 , 𝐼2 , 𝐴?

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 31
 6.4.1 Cross-Section Clustering
𝑘-means clustering with Lloyds algorithm

Original cross-sections

𝐼1 /𝐼2 slice 𝐼2 /𝐴 slice 𝐴/𝐼1 slice

Identified cross-sections
with clustering

Cluster centers are

indicated by silver centers
Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 32
6.4.1 Cross-Section Clustering
Evolution of cluster identification in 𝐼1 /𝐼2 -slice (initial guess + three iterations

Iteration 0: 86% Iteration 1: 94%

Iteration 2: 95% Iteration 3: 95%

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 33
6.5 Support Vector Machines
Classification: Prediction of a discrete category via a mapping between input and a (discrete) category

Support vector machines as classification algorithm

• Binary classifier, i.e., (-1 and 1)
• Predecessor of neural networks

• Linear support vector machines construct a hyperplane between two categories

𝒘⋅𝒙+𝑏 =0
• Hyperplane should maximize margin between closest points to the hyperplane

Minimal margin Maximal margin

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 34
6.5 Support Vector Machines
Prediction 𝑦ො is made depending on what side of the plane the datapoint 𝒙 lies
𝑦ො = sign(𝒘 ⋅ 𝒙 + 𝑏)
Optimization is formulated
• 1. Correct classification of data Mislabeling a datapoint increases the loss by one
0 if 𝑦෤𝑖 = sign(𝒘 ⋅ ෥
𝒙𝑖 + 𝑏)
ℒ 𝑦෤𝑖 , 𝑦ො ෥
𝒙𝑖 =ቊ
1 if 𝑦෤𝑖 ≠ sign(𝒘 ⋅ ෥
𝒙𝑖 + 𝑏)
• With the constraint
Ensures that closest data points (support
min ෥
𝒙𝑖 ⋅ 𝒘 = 1
𝑖 vectors) are at a standardized distance
• 2. Maximization of the margin
𝑚
1 2
𝐶 = ෍ ℒ 𝑦෤𝑖 , 𝑦ො ෥
𝒙𝑖 + 𝒘 Achieved by the regularization term
2
𝑖=1
Loss ℒ 𝑦෤𝑖 , 𝑦ො ෥
𝒙𝑖 is not differentiable and therefore exchanged with the differentiable Hinge loss function
ℒ𝐻 𝑦෤𝑖 , 𝑦ො 𝑥෤𝑖 = max(0, 1 − 𝑦෤𝑖 𝑦ො𝑖 )
Optimization via gradient-based optimization determines optimal hyperplane

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 35
6.5 Support Vector Machines – Examples

Linear support vector machine applied Nonlinear support vector machine enabled by
to the cross-section example projection to higher-dimensional space
𝑇
𝑒 𝑥1 , 𝑥2 = 𝑥1 , 𝑥2 , sin 𝜋𝑥1 , sin 2𝜋𝑥1

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 36
Contents
• 5 Advanced Physics-Informed Neural Networks
• 6.1.1 Singular Value Decomposition
• 6.2 Reduced Order Models
• 6.3 Sparse Identification of Non-Linear Dynamical Systems – SINDy
• 6.4 Clustering
• 6.5 Support Vector Machines
• 7 Material Modeling with Neural Networks

Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 37
6 Machine Learning in Computational Mechanics
Leon Herrmann
Stefan Kollmannsberger
Chair of Data Engineering in Construction
Bauhaus-Universität Weimar

Deep Learning in Computational Mechanics – an introductory course,

Herrmann et al. 2025

website book
Leon Herrmann & Stefan Kollmannsberger || Deep Learning in Computational Mechanics || Bauhaus-Universität Weimar 38