Additional Slide

04 – 1
[Link]
Chair of Automotive Technology
TUM School of Engineering and Design
Technical University of Munich

Autonomous Vehicle Perception

Perceive the world around us

Prof. Dr.-Ing. Markus Lienkamp

Felix Fent, [Link].
 Lecture Overview

Lecture 10:15 – 11:45 | Practice 12:00 – 12:45

1 Sensors 29.04.2025
Sensors
2 Calibration 06.05.2025
3 Localization 13.05.2025
Localization &
4 Odometry 27.05.2025
Mapping
5 SLAM 03.06.2025
6 Segmentation 10.06.2025
Object
7 Detection 17.06.2025
Detection
8 Fusion 24.06.2025
Tracking & 9 Tracking 01.07.2025
Prediction 10 Prediction 08.07.2025
11 End-to-End 15.07.2025
Outlook
12 Teleoperated Driving 22.07.2025

04 – 5
Autonomous Vehicle Perception – Odometry Strategies (Leitenstern)
Prof. Dr.-Ing. Markus Lienkamp | Felix Fent, [Link].

1 Introduction

2 Odometry Fundamentals

3 Scan Registration

4 Visual Odometry

5 Summary & Outlook

Autonomous Vehicle Perception – Odometry Strategies (Leitenstern)
Prof. Dr.-Ing. Markus Lienkamp | Felix Fent, [Link].

1 Introduction

2 Odometry Fundamentals

3 Scan Registration

4 Visual Odometry

5 Summary & Outlook

Recap: Localization
Summary

» Fusion for localization poses several challenges (different modalities, frequencies, …)

» Bayesian filters (Kalman Filters, Particle Filter) are methods for effective sensor fusion
» Motion model to describe the physical behavior of the system
» Measurement model to describe the relation between non-observable state and
measurements
» Usage of “conventional sensors” → GNSS, IMU, Wheel speed sensors
» Fusion of sensors in a Bayesian filter leads to huge improvement Outcome of
last week
» GNSS features limitations for level 5 autonomous driving

04 – 10
Recap: Localization
Summary

» Sensors have different modalities and coordinates frames

» How
Fusioncan we leverage
for localization poses information about
several challenges the environment (LiDAR,
» camera,
Bayesian filters (KalmanRaDAR) in our
Filters, Particle sensor
Filter) fusion
are methods formethod?
effective sensor fusion
» Motion model to describe the physical behavior of the system
» Measurement model to describe the relation between state and measurements
» Usage of “conventional” sensors → GNSS, IMU, Wheel speed sensors
» Huge improvement by sensor fusion  Limitations of GNSS

[Link]

04 – 11
Introduction
Odometry Definition

Goal: Estimate the pose of the vehicle

» Available sensors containing information about the change in position of the vehicle:
» GNSS (distance to satellites)
» Wheelspeeds (angular wheel velocity)
» IMU (accelerations + angular rates)
» LiDAR / camera / RaDAR (point clouds / images)

How can we use this information for pose estimation in our

Bayesian Filter?
» Odometry Definition:
» Odometry defines the change in position of a robot between two points in time using
various sensors
[YAN22] 04 – 12
Introduction
GNSS Odometry
𝑦
» GNSS measures distance to satellites
» Computation of global position (lat / lon / alt) Ψ𝑡1
» Direct usage of position in sensor fusion 𝑦𝑡1,𝑣𝑒ℎ𝑖𝑐𝑙𝑒
» Orientation: Ψ𝑡0
» Dual-Antenna GNSS → requires high accuracy
(e.g. D-GNSS, RTK) 𝑦𝑡0,𝑣𝑒ℎ𝑖𝑐𝑙𝑒

𝑥
𝑥𝑡0,𝑣𝑒ℎ𝑖𝑐𝑙𝑒 𝑥𝑡1,𝑣𝑒ℎ𝑖𝑐𝑙𝑒
𝑡0 𝑡1

[YAN22, 04 – 13
SIC16]
Introduction
Wheelspeed Odometry

» Wheelspeed sensor measure the angular velocity of the wheel 𝜔(𝑡)

» Computation of traveled distance from the integration of angular
velocity and dynamic tire radius
𝑡
» 𝑠 = ‫ 𝑡׬‬1 𝜔 𝑡 𝑟(𝑡) dt
0
» → relative change in position
» Limitations:
» Accumulation of integration errors („Drift“)
» Longitudinal slip 𝑥
𝑥𝑡0,𝑣𝑒ℎ𝑖𝑐𝑙𝑒 𝑥𝑡0,𝑣𝑒ℎ𝑖𝑐𝑙𝑒 + 𝑠
𝑡0 𝑡1

Travelled distance 𝑠

[YAN22, 04 – 14
SIC16]
Introduction
IMU Odometry
𝑦 Ψ𝑡0 − ΔΨ
» IMU measures linear accelerations in x/y/z and angular rates
around x/y/z
» Computation of relative change in position by
double integration of linear accelerations: Ψ𝑡0 𝑦𝑡0,𝑣𝑒ℎ𝑖𝑐𝑙𝑒
𝑡 𝑡
» 𝑠 = ‫ 𝑡׬‬1 ‫ 𝑡׬‬1 𝑎 𝑡 𝑑𝑡 𝑑𝑡 + 𝑦𝑠
0 0 𝑦𝑡0,𝑣𝑒ℎ𝑖𝑐𝑙𝑒
» Computation of relative change in orientation by 𝑥𝑡0,𝑣𝑒ℎ𝑖𝑐𝑙𝑒 + 𝑥𝑠
single integration of angular rates
𝑡
» ΔΨ = ‫ 𝑡׬‬1 𝜓ሶ 𝑡 𝑑𝑡 (for z-axis – yaw rotation) 𝑥
0 𝑥𝑡0,𝑣𝑒ℎ𝑖𝑐𝑙𝑒
» Limitations: 𝑡0 𝑡1
» Accumulation of integration errors
» High calibration effort

[YAN22, 04 – 15
SIC16]
Introduction
LiDAR / Camera / RaDAR Odometry
𝑦 Ψ𝑡0 − ΔΨ
» LiDAR / Camera / RaDAR give us information about the
environment
» Objects (especially static ones) may be perceived
from different perspectives over time Ψ𝑡0 𝑦𝑡0,𝑣𝑒ℎ𝑖𝑐𝑙𝑒
» How to compute the relative change in position / rotation + 𝑦𝑠
from this information? 𝑦𝑡0,𝑣𝑒ℎ𝑖𝑐𝑙𝑒
𝑥𝑡0,𝑣𝑒ℎ𝑖𝑐𝑙𝑒 + 𝑥𝑠
𝑥
𝑥𝑡0,𝑣𝑒ℎ𝑖𝑐𝑙𝑒
𝑡0 𝑡1

04 – 16
Additional Slide

Autonomous Vehicle Perception – Odometry Strategies (Leitenstern)

Prof. Dr.-Ing. Markus Lienkamp | Felix Fent, [Link].

1 Introduction

2 Odometry Fundamentals

3 Scan Registration

4 Visual Odometry

5 Summary & Outlook

04 – 18
Odometry Fundamentals
Definition – Landmarks vs. Features

» Landmarks:
» Points in the environment that are recognized in different frames
» Features:
» Clear-cut attribute in the perceived data (from LiDAR / camera /RaDAR) that can be
extracted by algorithms

[THU06] 04 – 19
Odometry Fundamentals
Definition – Landmarks vs. Features

Features

Landmarks

[THU06] 04 – 20
Odometry Fundamentals
Principle

Data Association Estimate Transformation

» Compute transformation
» Identify correspondences in between frames given a set of
different frames correspondences by minimizing
» Usually “unknown” for real-world an objective function
applications » Usage of iterative / non-iterative
methods

04 – 22
Odometry Fundamentals
Principle

Data Association Estimate Transformation

» Compute transformation
» Identify correspondences in between frames given a set of
different frames correspondences by minimizing
» Usually “unknown” for real-world an objective function
applications » Usage of iterative / non-iterative
methods

04 – 23
Odometry Fundamentals
Data Association

How do we know which feature / point corresponds to which

landmark in different frames?

𝑡0 𝑡1

[THU06] 04 – 24
Odometry Fundamentals
Data Association

» Data Association has great effect on the estimation of the transformation

» Known data association:
» Correspondences between different frames are known
» → Closed-form solution of transformation can be computed (e.g. using Singular Value
Decomposition (SVD))
» Unknown data association:
» Correspondences between different frames computed based on a matching strategy
» Generally impossible to compute the optimal transformation

04 – 25
Odometry Fundamentals
Data Association

» In
Data Association
general, the has great
data effect on theisestimation
association unknown of the
andtransformation
has to be
estimated!
» Known data association:
Strategies for data association differ between the different sensors
» Correspondences betweenLiDAR/RaDAR/camera
different frames are known
» → Closed-form solution of transformation can be computed (e.g. using Singular Value
Decomposition (SVD))
»We will firstdata
Unknown learn, how the
association: transformation can be estimated, given a set
of known (estimated)
» Correspondences correspondences
between different frames computed and afterwards
based seestrategy
on a matching the
» application for the different
Generally impossible to computesensors with
the optimal the specific data association
transformation
strategies
04 – 26
Odometry Fundamentals
Principle

Data Association Estimate Transformation

» Compute transformation
» Identify correspondences in between frames given a set of
different frames correspondences by minimizing
» Usually “unknown” for real-world an objective function
applications » Usage of iterative / non-iterative
methods

Iterative Methods
04 – 27
 Rotation Translation
Odometry Fundamentals 𝑁𝑦
Transformation Estimation
1 2
𝐸 𝑅, 𝑡 = ෍ 𝑥𝑖 − 𝑅𝑦𝑖 − 𝑡
{𝑥𝑖 } 𝑁𝑦
𝑖=1

Number of points

{𝑦𝑖 }

» Orange curve/set of points translated + rotated compared to blue one

» Correspondences from curvature

[UME91] 04 – 28
Odometry Fundamentals
Transformation Estimation

» Given:
» 2 corresponding point sets (correspondences from data association):
» 𝑋 = 𝑥1 , … , 𝑥𝑁𝑥
» 𝑌 = {𝑦1 , … , 𝑦𝑁𝑦 }
» Wanted:
» Translation 𝑡 + Rotation 𝑅 minimizing the sum of squared errors:

1 𝑁𝑦 2
» 𝐸 𝑅, 𝑡 =
𝑁𝑦
σ𝑖=1 𝑥𝑖 − 𝑅𝑦𝑖 − 𝑡 Least squares problem
» 𝑥𝑖 and 𝑦𝑖 are corresponding points

[UME91] 04 – 29
Odometry Fundamentals
Singular Value Decomposition

» For known correspondences between the 2 sets of points, the Singular Value
Decomposition (SVD) may be used to determine the transformation:
1
» 𝑋 ′ = 𝑥𝑖 − 𝜇𝑥 = 𝑥𝑖′ , 𝜇𝑥 = σ𝑁𝑥
𝑖=1 𝑥𝑖 (center of mass)
𝑁𝑥

1 𝑁𝑦
» 𝑌 ′ = 𝑦𝑖 − 𝜇𝑦 = 𝑦𝑖′ , 𝜇𝑦 = σ𝑖=1 𝑦𝑖 (center of mass)
𝑁𝑦
» Compute the SVD between the 2 sets of points as follows (3D-case):
𝜎1 0 0
𝑁𝑦 𝑇
» 𝑊 = 𝑈 0 𝜎2 0 𝑉 𝑇 , 𝑊 = σ𝑖=1 𝑦𝑖′ 𝑥𝑖′
0 0 𝜎3
» Reminder: 𝑈, 𝑉 are unitary matrices, 𝜎1 > 𝜎2 > 𝜎3 are the singular values of 𝑊
[UME91] 04 – 30
Odometry Fundamentals
Singular Value Decomposition

» Given the SVD between the 2 sets of points, and 𝑟𝑎𝑛𝑘 𝑊 = 3, the unique, optimal
solution for minimizing the 𝐸 𝑅, 𝑡 is defined by (without proof):
𝑅 = 𝑈𝑉 𝑇
𝑡 = 𝜇𝑦 − 𝑅𝜇𝑥
» SVD solution assumes known correspondences
» More general approaches for least squares problem needed for estimated
correspondences:
» More complex error functions (e.g. considering covariances / uncertainties)
» Often iterative algorithms (e.g. Gauss-Newton minimization, Levenberg-Marquardt)
[UME91] 04 – 31
Autonomous Vehicle Perception – Odometry Strategies (Leitenstern)
Prof. Dr.-Ing. Markus Lienkamp | Felix Fent, [Link].

1 Introduction

2 Odometry Fundamentals

3 Scan Registration

4 Visual Odometry

5 Summary & Outlook

Scan Registration
Problem Definition

04 – 33
[Link]
Scan Registration
Problem Definition

{𝑦𝑖 }
{𝑥𝑖 }
𝑅, 𝑡

1 𝑁𝑦 2
» Least-squares minimization: 𝐸 𝑅, 𝑡 = σ𝑖=1 𝑥𝑖 − 𝑅𝑦𝑖 − 𝑡
𝑁𝑦

» Correspondences generally unknown → need to be estimated / computed

[SIC16] 04 – 34
Scan Registration
Strategies Overview

Scan Registration

Iterative Strategies Feature-based Strategies

» Estimation of temporal » Extraction of features from point
correspondences with given clouds = correspondences:
strategy » Hand-crafted algorithms (e.g.
» Temporal correspondences → sharp edges, planar surfaces)
computed for each iteration » Deep Learning-based
» Estimation of transformation from algorithms
correspondences » Estimation of transformation from
» → initial guess for first iteration sets of features
[SIC16] 04 – 35
Scan Registration
Strategies Summary

» Strategies can be differentiated by the way correspondences are computed

» Feature-based strategies → often used to perform coarse registration, as no initial
guess required
» Iterative strategies → often used to perform fine registration with a given initial guess
» Initial guess from feature-based registration
» Initial guess from motion model (e.g. constant velocity)

04 – 36
Scan Registration
Example: KISS-ICP

[Link] 04 – 37
Scan Registration
Algorithm – ICP

» Proposed by: „Besl, McKay: A Method for Registration of 3D Shapes“ (1992)

» Correspondences from minimal Euclidean distance (point-2-point – Vanilla-ICP)

𝑦𝑖 = 𝑠𝑜𝑢𝑟𝑐𝑒
𝑥𝑖 = 𝑟𝑒𝑓𝑒𝑟𝑒𝑛𝑐𝑒

[BES92] 04 – 38
Scan Registration
Algorithm – ICP

» Proposed by: „Besl, McKay: A Method for Registration of 3D Shapes“ (1992)

» Correspondences from minimal Euclidean distance (point-2-point – Vanilla-ICP)

Initial/current transformation

[BES92] 04 – 39
Scan Registration
Algorithm – ICP

» Proposed by: „Besl, McKay: A Method for Registration of 3D Shapes“ (1992)

» Correspondences from minimal Euclidean distance (point-2-point – Vanilla-ICP)

Final transformation

[BES92] 04 – 40
Scan Registration
Algorithm – ICP

Init tranformation (𝑅, 𝑡) (initial guess)

Apply transformation (𝑅, 𝑡) to target points

Determine corresponding source points (Data Association)

Compute transformation (𝑅, 𝑡) from correspondences (e.g. SVD,

Gauss-Newton)

Compute error 𝐸(𝑅, 𝑡)

𝐼𝑓 𝐸 𝑅, 𝑡 < 𝜀 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 → 𝑠𝑡𝑜𝑝, 𝑟𝑒𝑝𝑒𝑎𝑡 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

[SIC16] 04 – 41
Scan Registration
Algorithm – ICP

04 – 42
Scan Registration
Performance Criteria & Variants

Challenges

Stability (local minima)

Basin of convergence (maximum

initial misalignment)

04 – 43
Scan Registration
Performance Criteria & Variants

Challenges

Stability (local minima)

Inefficient!

Basin of convergence (maximum

initial misalignment)

Speed (correspondence search)

Point to search
correspondences for

04 – 44
Scan Registration
Performance Criteria & Variants

Challenges How to deal with this

correspondence?
Stability (local minima)

Basin of convergence (maximum

initial misalignment)

Speed (correspondence search)

Outlier / noise
Tolerance against noise / outliers

04 – 45
Scan Registration
Performance Criteria & Variants

Challenges Variants

Stability (local minima) Point subsets

Basin of convergence (maximum

Point storage
initial misalignment)

Speed (correspondence search) Correspondence estimation

Weighting of correspondences /
Tolerance against noise / outliers
Rejection of outlier point pairs
04 – 46
Scan Registration
Point subsets

» Naive approach: use all points

» High computation effort
» → sub-sampling, e.g.
» Uniform
» Normal-space
Uniform sampling Normal-space sampling
» Random
» Feature-based

04 – 47
Scan Registration
Point storage

» Store points efficiently to speed up

correspondence search
Inefficient!

Point to search
correspondences for

04 – 48
Scan Registration
Point storage
Voxel grid
» Store points efficiently to speed up
correspondence search
» Voxel hash map
» KD tree

Point to search
correspondences for

04 – 49
Scan Registration
Correspondence Strategies

» Matching strategies, e.g.:

» Closest Euclidean point (point-2-point ICP)
» Closest compatible point (e.g. additional constraints from colors/intensity)
» Point-2-Plane
» Distribution-2-Distribution (Generalized ICP – GICP)

𝑟 𝑟 𝑟 𝑟
𝑟 𝑟
𝑟 𝑟 𝑟 𝑟 𝑟
𝑟

Point-2-Point Point-2-Plane Distribution-2-Distribution

04 – 50
Scan Registration
Rejection of Outlier Pairs

» Key Idea: Weigh correspondences depending the quality of the match

» E.g. Reject correspondences with a point-2-point distance above a given threshold
» Robust Kernels → Correspondence Weighting:
» Huber
» Cauchy
» Geman-McClure
» Welsch
» …
» Adjust parameters of kernel
residual
[CHE20]

04 – 51
Scan Registration
Summary

» Registration of point clouds needed to integrate LiDAR/RaDAR data into sensor fusion
algorithm (e.g. Bayesian filter) (but also relevant for other tasks, e.g. calibration)
» Computes the transformation (rotation + translation) between two sets of points
» Distinction between iterative / non-iterative approaches
» Known data association → Solution can be computed using SVD
» Unknown data association → Usually iterative approaches
» Iterative strategies are State-of-the-Art ➔ Strategies for initial guess computation
» Most popular example: Iterative-Closest-Point (ICP)

04 – 53
Scan Registration
Example: KISS-ICP

» Point-2-Point ICP
» Few parameters, but working on various domains
» Check it out!
[Link]

[VIZ23] 04 – 54
Autonomous Vehicle Perception – Odometry Strategies (Leitenstern)
Prof. Dr.-Ing. Markus Lienkamp | Felix Fent, [Link].

1 Introduction

2 Odometry Fundamentals

3 Scan Registration

4 Visual Odometry

5 Summary & Outlook

04 – 55
Visual Odometry
Problem Definition

… from point sets to images

Features Projected LiDAR points
» Cameras way cheaper than LiDARs!

[Link] 04 – 56
Visual Odometry
Problem Definition

𝑝0 𝑝1

» Estimate transformation (𝑅, 𝑡) between two camera poses given two images mapping the
environment according to:
𝑥 𝑋
𝑦 =𝑃 𝑌
𝑍
1 3D world coordinate
1
2D pixel coordinate
Projection matrix 04 – 57
Visual Odometry
Depth Information in Images

» Cameras project 3D points onto a 2D plane → no depth information

What do we need depth for?

» Projection of pixel in 2D image plane into 3D world
» Projection of 3D point into another image

How to obtain depth information?

» RGB-D cameras → direct depth information (e.g. time-of-flight sensor)
» Stereo / binocular cameras → compute pixel depth from disparity
» Monocular camera → depth unkown and part of the optimization problem

04 – 58
Visual Odometry
Strategies Overview

Visual Odometry

Indirect Strategies Direct Strategies

» Extraction of geometric features » Direct estimation of transformation
from camera images from raw camera frames (pixels)
» Estimation of transformation from » Camera model to determine
sets of features photometric error
» Hand-crafted feature descriptors » → Optimization of photometric
» Deep Learning-based error
approaches
» → Optimization of geometric
error
[TUM23] 04 – 60
Visual Odometry
Example: ORB-SLAM

[Link] 04 – 61
Visual Odometry
Indirect Visual Odometry – Visual Features

» Keypoint: (locally) distinct location in the image

» Feature descriptor: describes local structure around the keypoint

Keypoint +
descriptor (e.g. gradient):
0.01
0.05
𝑓=
⋮
0.03

04 – 62
Visual Odometry
Indirect Visual Odometry – Visual Features

Keypoint = distinct location in image Harris corner detector

» Corner Detectors
» invariant to translation, rotation, illumination
» Corner = 2 roughly orthogonal edges
(edge = sudden change in brightness)
» Corner Detectors: e.g. FAST, Förstner, Harris, Shi-Tomasi
» Difference of Gaussians
» Identify corners + blobs (constant region, but different from surroundings)
» Iterate over scales + blur
[Link] 04 – 63
Visual Odometry
Indirect Visual Odometry – Visual Features

Feature descriptor = describing a keypoint ORB features

» Examples:
» Hand-crafted:
» SIFT (Scale Invariant Feature Transform)
» SURF (Speeded-Up Robust Features)
» BRIEF (Binary Robust Independent Elementary Factors)
» ORB (Oriented FAST and rotated BRIEF)
» Learning-based:
» SuperGlue ([Link]

[Link] 04 – 64
Visual Odometry
Indirect Visual Odometry - Summary

Wide baseline matching → robust to Sparse set of features → discarding

viewpoint changes information
Illumination invariance → robust to Dependent on keypoints
inconsistencies in model/system Needs high resolution camera images
Transition from image data to geometric Feature extraction/matching costly
information

[TUM23] 04 – 66
Visual Odometry
Strategies Overview

Visual Odometry

Indirect Strategies Direct Strategies

» Extraction of geometric features » Direct estimation of transformation
from camera images from raw camera frames (pixels)
» Estimation of transformation from » Camera model to determine
sets of features photometric error
» Hand-crafted feature descriptors » → Optimization of photometric
» Deep Learning-based error
approaches
» → Optimization of geometric
error
[TUM23] 04 – 67
Visual Odometry
Example: Direct Visual Odometry

04 – 68
[Link]
Visual Odometry
Direct Visual Odometry

Simultaneously determine camera motion + correspondences?

» → Brightness Consistency Assumption
» Assumption: Correspondences in successive frames have same brightness
» → minimize brightness difference between two frames
» Photometric error of single pixel: 𝐼1 𝐼2
𝑒 = 𝐼1 𝑝1 − 𝐼2 (𝑝2 )

Image → 2D matrix of intensities

𝑝1 𝑝2
» Extending this to multiple pixels:
𝑁

min 𝐽 𝑇 = ෍ 𝑒𝑖2 Least squares problem

𝑇
𝑖=1
[TUM23] 04 – 69
Visual Odometry
Direct Visual Odometry – Pixel Tracking

Pixel Tracking

Dense Direct Method Semi-Dense Direct Sparse Direct Method

» Computing photometric Method » Compute photometric
error for all existing » Compute photometric error for keypoints
pixels error for pixels with » → no need for
» Computationally significant gradients → descriptors (direct
expensive! significant influence on matching of intensity)
least squares problem » fastest, but only allows
for sparse
reconstruction
[TUM23] 04 – 70
Visual Odometry
Direct Visual Odometry – Pixel Tracking
Dense Direct Method Semi-Dense Direct Sparse Direct Method
» Computing photometric Method » Compute photometric
error for all existing » Compute photometric error for keypoints
pixels error for pixels with » → no need for
» Computationally significant gradients → descriptors (direct
expensive! significant influence on matching of intensity)
least squares problem » fastest, but only allows
for sparse
reconstruction

[TUM23] 04 – 71
Visual Odometry
Direct Visual Odometry - Summary

Usage of all available information → Heavily dependent on illumination

higher robustness to motion blur/weak High-frequency camera required
gradients Precise camera model required
Applicable for lower resolution cameras Generally slower than indirect method
Distortion resistant

[TUM23] 04 – 72
Autonomous Vehicle Perception – Odometry Strategies (Leitenstern)
Prof. Dr.-Ing. Markus Lienkamp | Felix Fent, [Link].

1 Introduction

2 Odometry Fundamentals

3 Scan Registration

4 Visual Odometry

5 Summary & Outlook

04 – 74
Summary
Odometry for Sensor Fusion

» Odometry Definition:
» Odometry defines the change in position of a robot between two points in time using
various sensors
» Odometry Methods:
» IMU odometry (2x Integration)
» Wheelspeed odometry (1x Integration)
Measurement/Observation Model
» Scan Registration (LiDAR/RaDAR)
» Visual Odometry (Camera)
Fusion within a Bayesian Filter (KF, PF)

04 – 75
Summary
Classification of Perception Sensors

IMU Wheelspeed GNSS

» Relative Position » Global Position

» High-frequency » Low frequency

» Error wind-up » Noisy

04 – 76
Summary
Classification of Perception Sensors

IMU Wheelspeed GNSS

RaDAR LiDAR Camera

» Relative Position » Relative Position » Global Position

» High-frequency » Low frequency » Low frequency

» Error wind-up » (smaller) error wind-up » Noisy

04 – 77
Summary
Classification of Perception Sensors

IMU Wheelspeed GNSS

RaDAR LiDAR Camera

» Relative Position » Relative Position » Global Position

» High-frequency » Low frequency » Low frequency

» Error wind-up » (smaller) error wind-up » Noisy

04 – 78
Summary
Classification of Perception Sensors

IMU Wheelspeed GNSS

RaDAR LiDAR Camera

» Relative Position » Relative Position » Global Position

» High-frequency » Low frequency » Low frequency

» Error wind-up » (smaller) error wind-up » Noisy

04 – 79
Summary
Classification of Perception Sensors

IMU Wheelspeed GNSS

RaDAR LiDAR Camera

Only relative measurements with error wind-up!

» Relative Position » Relative Position the drift? » Global Position
How can we eliminate
» High-frequency » Low frequency » Low frequency

» Error wind-up » (smaller) error wind-up » Noisy

04 – 80
Outlook
Loop Closure & SLAM

04 – 81
References

SIC B. Siciliano and O. Khatib, Springer Handbook of Robotics, Second Edition. Springer, 2016, pp. 1–6.
16

CHE N. Chebrolu, T. Läbe, O. Vysotska, J. Behley, and C. Stachniss, “Adaptive Robust Kernels for Non-Linear Least Squares Problems,” IEEE Robotics and Automation Letters, vol. 6,
20 pp. 2240–2247, 2020.

YAN M. Yang et al., “Sensors and Sensor Fusion Methodologies for Indoor Odometry: A Review,” Polymers, vol. 14, no. 10, 2022, doi: 10.3390/polym14102019.
22

THU S. Thrun, W. Burgard, and D. Fox, Probabilistic Robotics. Cambridge, Massachusetts ; London, England: MIT Press, 2006.
06

UME S. Umeyama, “Least-squares estimation of transformation parameters between two point patterns,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 13, no. 4,
91 pp. 376–380, 1991, doi: 10.1109/34.88573.

BES P. J. Besl and N. D. McKay, “A method for registration of 3-D shapes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239–256, 1992, doi:
92 10.1109/34.121791.

VIZ I. Vizzo, T. Guadagnino, B. Mersch, L. Wiesmann, J. Behley, and C. Stachniss, “KISS-ICP: In Defense of Point-to-Point ICP Simple, Accurate, and Robust Registration If Done the
23 Right Way,” IEEE Robotics and Automation Letters, vol. 8, no. 2, pp. 1029–1036, Feb. 2023, doi: 10.1109/lra.2023.3236571.

TUM TUM Computer Vision Group, “Computer Vision II: Multiple View Geometry - Lecture Material,” Summer 2023. Accessed: May 19, 2025. [Online]. Available:
23 [Link]

04 – 82