Module -1

History and Evolution of Autonomous Vehicles, , Levels of driving automation (SAE Levels 0–5), History and evolution of
autonomous vehicles, Levels of driving automation (SAE Levels 0–5), ADAS Levels (1-5), Market overview and real-world
applications. ADAS Vs Fully Autonomous Vehicles (FAV’s), Adaptive Cruise Control (ACC), Lane Keeping Assist (LKA),
Autonomous Emergency Braking (AEB), Blind Spot Detection, Driver Monitoring Systems, Ethical and legal considerations.
History and Evolution of Autonomous
Vehicles

That’s when Leonardo da Vinci designed a small, three-wheeled, self-propelled cart

[Link]
fall-into-levels-of-driving-automation
The taxonomy of an ADAS. PMD: photonic mixer device; IMUs:
inertial measurement units; GPSs: global positioning systems.
Application of ADAS

Significant automotive safety improvements in the past (e.g., shatter-resistant glass, three-point seatbelts, airbags) were
passive safety measures designed to minimize injury during an accident. Today, ADAS systems actively improve safety
with the help of reducing the occurrence of accidents and injury to occupants.
The implementation of cameras in the vehicle involves a new AI function that uses sensor fusion to identify and process
objects. Sensor fusion, like how the human brain process information, combines large amounts of data with the help of
image recognition software, ultrasound sensors, lidar, and radar. This technology can physically respond faster than a
human driver ever could. It can analyze streaming video in real time, recognize what the video shows, and determine
how to react to it.
Adaptive Cruise Control
Adaptive cruise control is particularly helpful on the highway, where drivers can find it difficult to monitor their speed
and other cars over a long period of time. Advanced cruise control can automatically accelerate, slow down, and at
times stop the vehicle, depending on the action’s other objects in the immediate area.
Glare-Free High Beam and Pixel Light
Glare-free high beam and pixel light uses sensors to adjust to darkness and the vehicle’s surroundings without
disturbing oncoming traffic. This new headlight application detects the lights of other vehicles and redirects the
vehicle’s lights away to prevent other road users from being temporarily blinded.
Adaptive Light Control
Adaptive light control adapts the vehicle’s headlights to external lighting conditions. It changes the strength, direction,
and rotation of the headlights depending on the vehicle’s environment and darkness.
Automatic Parking
Automatic parking helps inform drivers of unseen areas so they know when to turn the steering wheel and stop.
Vehicles equipped with rearview cameras have a better view of their surroundings than traditional side mirrors. Some
systems can even complete parking automatically without the driver’s help by combining the input of multiple
sensors.
Autonomous Valet Parking
Autonomous valet parking is a new technology that works via vehicle sensor meshing, 5G network communication, and cloud
services hat manage autonomous vehicles in parking areas. Sensors provide the vehicle with information about where it is, where
it needs to go, and how to get there safely. All this information is methodically evaluated and used to perform drive acceleration,
braking, and steering until the vehicle is safely parked.
Navigation System
Car navigation systems provide on-screen instructions and voice prompts to help drivers follow a route while concentrating on
the road. Some navigation systems can display exact traffic data, and if necessary, plan a new route to avoid traffic jams.
Advanced systems may even offer heads-up displays to reduce driver distraction.
Night Vision
Night vision systems enable drivers to see things that would otherwise be difficult or impossible to see at night. There are two
categories of night vision implementations: Active night vision systems project infrared light, and passive systems rely on the
thermal energy that comes from cars, animals, and other objects.
Automatic Emergency Braking
Automatic emergency braking uses sensors to detect whether the driver is in the process of hitting another vehicle or other
objects on the road. This application can measure the distance of nearby traffic and alert the driver to any danger. Some
emergency braking systems can take preventive safety measures such as tightening seat belts, reducing speed, and engaging
adaptive steering to avoid a collision.
Crosswind Stabilization
This relatively new ADAS feature supports the vehicle in counteracting strong crosswinds. The sensors in this system can
detect strong pressure acting on the vehicle while driving and apply brakes to the wheels affected by crosswind disturbance.
Driver Drowsiness Detection
Driver drowsiness detection warns drivers of sleepiness or other road distractions. There are several ways to determine
whether a driver’s attention is decreasing. In one case, sensors can analyze the movement of the driver’s head and heart
rate to determine whether they indicate drowsiness. Other systems issue driver alerts similar to the warning signals for
lane detection.
Driver Monitoring System
The driver monitoring system is another way of measuring the driver’s attention. The camera sensors can analyze
whether the driver’s eyes are on the road or drifting. Driver monitoring systems can alert drivers with noises, vibrations
in the steering wheel, or flashing lights. In some cases, the car will take the extreme measure of stopping the vehicle
completely.
5G and V2X
This hot new 5G ADAS feature provides communication between the vehicle and other vehicles or pedestrians with
increased reliability and lower latency, generally referred to as V2X. Today, millions of vehicles connect to cellular
networks for real-time navigation. This application will enhance existing methods and the cellular network to improve
situational awareness, control or suggest speed adjustments to account for traffic congestion, and provide real-time
updates to GPS maps. V2X is essential to support over-the-air software updates for the now-extensive range of
software-driven systems in cars, from map updates to bug fixes to security updates and more.
The Google self-driving-car-generated 3-D image of its surroundings using lidar
Vision Based ADAS- Components
Automotive Cameras: Automotive cameras are the eyes of the vision-based advanced driver assistance systems
in a vehicle. While front cameras in vehicles are used to detect lane markings, pedestrian and traffic signs, the
side and rear cameras help in cross-traffic alerts, blind spot detection, and parking assistance. To optimally cover
the front, rear and surround view of a vehicle, automotive companies use monocular and stereo cameras in
vehicles. Where monocular camera (with single camera sensor) is primarily used for object detection, stereo
cameras (with two camera sensors) provide an additional feature to calculate distance-to-object for the system.

Camera Modules: Camera modules are systems comprised of Automotive cameras, image sensors, and lens
modules. Image sensors in the camera module are used to convert images from cameras into electronic signals
for processing. There are two types of image sensors available for ADAS applications, namely CCD (charge-
coupled device) and CMOS (complementary metal-oxide semiconductor) sensors, of which CMOS sensors are
highly preferred and used because of their low power consumption, easy integration, faster frame rate, and low
manufacturing cost.
ADAS Algorithms: Embedded vision systems in ADAS are incomplete without algorithms for specific functions.
Embedded system requires multiple sequential image frame processing and a set of complex and sophisticated
algorithms to analyze the image and reach to a decision for the ADAS function. To achieve real-time performance in the
system, these algorithms require specialized high-performance DSPs (digital signal processors) or GPUs (Graphics
Processing Units).To develop any embedded vision application such as a lane departure warning system or automatic
emergency braking system, a complex set process needs to be followed. At first, the application needs to be analyzed
with a proper algorithm research, and then functional prototyping of the algorithm is done. After that, the code is
optimized for embedded system and finally ported to the target system.
Monocular camera
A monocular camera is a camera that we often see in everyday life. Monocular cameras are the
mainstream camera that can be used as a monitor function.

Usage of Monocular Camera in ADAS

In ADAS, monocular cameras are used in the following scenes.

[Link] camera that compensates for the blind spots behind and sideways.
[Link] of image of parking assistance round view monitor
[Link] recognition for lane keeping
[Link] function to the driver by recognizing traffic signs such as speed limit
[Link] recognition
Stereo Camera
A stereo camera is a camera that simultaneously photographs an object from a plurality of different
directions using two cameras (two-eyed camera) in the same manner as the principle that a person
views an object. It is a camera that can measure information in the depth direction from the position
information of the pixel of the camera.

•A stereo camera can detect objects of any distance- If the calculation algorithm and the object can be
matched, the distance to the object can be measured, and recognition processing can be done by
utilizing the information of the image.
•Stereo camera has high distance accuracy of near object- There is a characteristic that the measurement
accuracy of the neighboring object is higher than the calculation result of the depth information.
•Absolute distance from the camera can be acquired by 3 dimensional measurement -Since distance
measurement (distance measurement) using two cameras is performed, it can measure with absolute
distance, such as a few meters from the camera.
•A detection range can be expanded by combining a plurality of cameras- You can use the multiple
cameras and the distance of the entire surroundings or Image recognition, you can reduce the dead
angle of the moving body.
•Robustness to environmental change such as nighttime, snow, heavy rain- Output stable range finding
results even in rain and snow, night traveling by image processing.
Principal of Camera or vision system
Digital camera
Different Lighting Techniques
Steps In Image processing
Image acquisition
• Video camera
• Infrared camera
• Range camera
• Line-scan camera
• Hyperspectral camera
• Omni-directional camera
• and more …

24
CCD (Charged-Coupled Device) cameras
• Tiny solid state cells convert light energy into electrical
charge.
• The image plane acts as a digital memory that can be read
row by row by a computer.
Image digitization

• Sampling means measuring the value of an image at a finite number

of points.
• Quantization is the representation of the measured value at the
sampled point by an integer.
Image Acquisition
Image segmentation

28
Image digitization (cont’d)
 Black and
White to Gray
scale image
processing
Image Processing Steps
Following article flow describes the steps used to analyze an image in an ADAS system.

1. Image Pre-Processing
The image preprocessing is applied to make the image appropriate for processing. Image preprocessing is performed on the road
image to locate the left-right lane edges by separating the sky region and analyzing road region known as the region of the
interest (ROI) to extract the prominent road features such as lane markers.

2. Object Classification & Recognition

Vision systems rely on object classification, meaning that they must be “trained” to detect what certain objects look like. Those
objects are categorized as “classes”. Classes may include pedestrians, cyclists, passenger cars, trucks, etc. each in its own separate
class. For each captured image, an algorithm tries to match one or more of the “classes” in its library.

Traffic Signal Recognition Example: Traffic sign recognition systems consist of two main components: classification and
recognition. The classification component is strictly focused on the classification of the type of traffic sign after the traffic sign has
been detected. The recognition component, on the other hand, will concentrate on locating the traffic sign in a sequence of
images.

3.
Object Detection Techniques
[Link] by Color- It is a known approach to recognize color-based traffic signs by identifying areas of interest for an
image using a plain threshold or more progressive methods of image segmentation. The resulting regions are immediately
marked as traffic signs or shift the suppositions of traffic signs (i.e. attention areas) into later phases.
[Link]-based Detection- Several approaches have proposed shape features for traffic sign recognition. Traffic sign shapes
are generally circular, triangular, rectangular or polygonal. Hough Transform technique is probably the most common
method of retrieving arbitrary patterns from an image.
[Link] based on Machine Learning- This concentrates on the extracting of hand-crafted features for both the
detection of a traffic sign and the classifying into one of several classes. The extracted features were employed to train
classifiers such as support vector machines (SVMs), Bayesian classifiers, or logistic regression models.
[Link] based on Deep Learning- it is based on a regional proposal and is also referred to as a two-stage approach (R-
CNN) which significantly enhance the average detection accuracy in the challenge of visual object classes (VOC). R-CNN
provides a remarkable improvement over previously published sliding window-based techniques, RCNN uses selective
search which is an unattended algorithm for generating feature extraction by CNN's separately.
. Image Noise
Image noise is random variation of brightness or color information in the images captured. It is degradation in
image signal caused by external sources. Images containing multiplicative noise have the characteristic that the
brighter the area the noisier it. But mostly it is additive. Image noise results in pixels that look very different from
their neighbors.

Sources of Image noise:

•While image being sent electronically from one place to another.

•Sensor heat while clicking an image.
•With varying ISO Factor which varies with the capacity of camera to absorb light.
Types of Image Noises

[Link] Noise
[Link] and Pepper Noise
[Link] Noise
[Link] Noise
Edge Detection Identifies sudden changes (discontinuities) in an image, intuitively, most semantic and shape information
from the image can be encoded in the edges. This also helps in the information/feature extraction and to further recognize
objects that aren’t clear in the image.

Following discontinuities/sudden changes are considered:

[Link] normal discontinuity

[Link] discontinuity
[Link] color discontinuity
[Link] discontinuity
Criteria for an “optimal” edge detector:
•Good detection: the optimal detector must minimize the probability of false positives (detecting spurious edges caused by
noise), as well as that of false negatives (missing real edges)
•Good localization: the edges detected must be as close as possible to the true edge
•Single response: the detector must return one point only for each true edge point; that is, minimize the number of local
maxima around the true edge
Vision Systems Reference: Lane Detection
The basic concept of lane detection is that, from the image of the road and vehicle surroundings, the on-board controller
should understand the limits of the lane and should warn the driver when the vehicle is moving closer to the lanes. In an
autonomous car, lane detection is important to keep the vehicle in the middle of the lane, at all-time, other than while
changing lanes.
A typical lane detection algorithm can be split into simple steps:

[Link] the ROI (Region of Interest) - The images collected by the on-board camera are color images. Each pixel in the image
is made up of R, G, and B three color components, which contains large amount of information. Processing these images
directly makes the algorithm consume a lot of time. A better idea for this problem is to select the region of interest (ROI) from
the original image by concentrating on just the region of the image that interests us.
[Link] Preprocessing - Image preprocessing includes grayscale conversion of color image, gray stretch, median filter to
eliminate the image noise and other interference information.
[Link] the edge information from the image- edge detection on the output of the preprocessing is basically to detect the lines
around the objects in the images. One of the common methods of edge detection is called Canny Edge Detection.
[Link] Transform (or other algorithms)- From the edge detection algorithm, we get a set of points which are considered
part of the lane markings, Hough transform is used to generate a line through these points from the image.
Few Challenges and Solutions
[Link] lane detection: The divider present in the middle of the road should not be detected as a lane. The height and the
other unique properties of the metal dividers should be used by object detection algorithms to differentiate the lanes and
dividers.
[Link] lane transfer: Vision algorithms developed for vehicle localization with the proper datasets can be used to filter out
non-relevant objects under various environmental conditions such as poor visibility and fog weather to help in a proper lane
transfer. ADAS Image processing algorithms can be used for identifying the presence of other vehicles, their speed, and
direction from the image frames data taken from the front, rear, and sides of the vehicle to help the driver in deciding for
proper lane transfer.
[Link] object detection: The objects detected from the rear and side camera image data should be classified properly as
The stop sign detection: (a) a binary stop sign and (b) stop sign classification
using optical character recognition.
Cruise Control
Representative Cruise Control System
• Automotive cruise control is an example of the electronic feedback control system. The components of a closed-
loop control system include the plant or system being controlled and a sensor for measuring the plant variable
being regulated.
• It also includes an electronic control system that receives inputs in the form of the desired value of the regulated
variable and the measured value of that variable from the sensor. The control system generates an error signal
constituting the difference between the desired and actual values of this variable.
• It then generates an output from this error signal that drives an electromechanical actuator. The actuator controls
the input to the plant in such a way that the regulated plant variable is moved toward the desired value.

Consider a simplified traditional cruise control for a vehicle traveling along a straight road (along the x-axis in our
ECEF coordinate system). In the case of a traditional cruise control, the variable being regulated is the vehicle
speed:

where x is the translation of the vehicle in the ECEF (Earth-Centered, Earth-Fixed) frame.

Disclaimer:All the content and figures are used for Educational

Purpose only
Cruise Control

Fig: External forces acting on a vehicle under normal driving condition

Disclaimer:All the content and figures are used for Educational
Purpose only
Cruise Control

Disclaimer:All the content and figures are used for Educational

Purpose only
Cruise Control

Disclaimer:All the content and figures are used for Educational

Purpose only
Cruise Control

Disclaimer:All the content and figures are used for Educational

Purpose only
Cruise Control

Disclaimer:All the content and figures are used for Educational

Purpose only
Cruise Control

Disclaimer:All the content and figures are used for Educational

Purpose only
Cruise Control

Fig:
Disclaimer:All the content and figures are used for Cruise
Educational Control Configuration
Purpose only
Cruise Control

Fig: Block Diagram for Cruise Control

Disclaimer:All the content and figures are used for Educational
Purpose only
Cruise Control

• Taking the Laplace transform and further

solving gives following equation,

Fig: Block Diagram for Cruise Control

Disclaimer:All the content and figures are used for Educational
Purpose only
Cruise Control

Representative Cruise Control System

1. Digital Cruise Control

Fig: Digital Speed Control Block Diagram

Disclaimer:All the content and figures are used for Educational
Purpose only
Cruise Control

Representative Cruise Control System

1. Digital Cruise Control
The transfer function is given as
The z-transfer function for the combination of ZOH and plant (G(z)) is
given by

Disclaimer:All the content and figures are used for Educational

Purpose only
Cruise Control

Disclaimer:All the content and figures are used for Educational

Purpose only
Cruise Control

Disclaimer:All the content and figures are used for Educational

Purpose only
Cruise Control

Disclaimer:All the content and figures are used for Educational

Purpose only
Cruise Control

Fig: Speed Sensor Configuration

Disclaimer:All the content and figures are used for Educational
Purpose only
Cruise Control

Representative Cruise Control System

2. Hardware Issues

• In a representative vehicle speed measurement system, the vehicle speed information is mechanically coupled
to the speed sensor by a flexible cable coming from the driveshaft, which rotates at an angular speed
proportional to vehicle speed. A speed sensor driven by this cable generates a pulsed electrical signal that is
processed by the computer to obtain a digital measurement of speed.
• A speed sensor can be implemented magnetically or optically.
Considering an optical sensor for the current system, a flexible cable
drives a slotted disk that rotates between a light source and a light
detector.

• The placement of the source, disk, and detector is such that the slotted
disk interrupts or passes the light from source to detector, depending on
whether a slot is in the line of sight from source to detector. The light
detector produces an output voltage whenever a pulse of light from the
light source passes through a slot to the detector.
Fig: Speed Sensor Output
Disclaimer:All the content and figures are used for Educational
Purpose only
Cruise Control

Representative Cruise Control System

2. Hardware Issues

• The number of pulses generated per second is proportional to the number of slots in the disk and the vehicle speed

Fig: Speed Sensor Output

Disclaimer:All the content and figures are used for Educational
Purpose only
Cruise Control

The number P(n) is proportional

to vehicle speed V at speed
sample n. The electrical signal in
the binary counter is in a digital
format that is suitable for reading
by the cruise control computer Fig: Digital Speed Measurement System
Disclaimer:All the content and figures are used for Educational
Purpose only
Adaptive Cruise Control
Light Detection and Ranging (LiDAR) technology powers the
“eyes” of autonomous vehicles. LiDAR principle is pretty easy
to understand: the system measures how long it takes for
laser pulses to bounce off objects and return to the sensor.
This simple concept creates detailed 3D maps with
exceptional precision – about 0.1 degrees of spatial
resolution.

LiDAR systems integrate 4 main components whether they

are mounted on automotive vehicles, aircraft or UAV:
•Laser transmitter: Sends out infrared light pulses at specific
wavelengths (850nm, 905nm, or 1550nm)
•Photodetector receiver: Catches the reflected laser points
•Scanner: Groups laser points into “point clouds” to build 3D
models
•Processing unit: Turns spatial data into precise position
measurement
The LiDAR scanning process follows clear steps. First, a laser beam fires and starts a
timing clock. The beam travels through optics, hits objects, bounces back through receiver
optics, reaches photodetectors that convert light to electrical signals, and finally becomes
digital data in point cloud form.

Two main methods dominate automotive LiDAR systems today. Direct Time-of-Flight
(dToF) calculates distance by measuring light travel time. Frequency Modulated
Continuous Wave (FMCW) looks at frequency differences between sent and received laser
waves. FMCW adds another useful feature – it can measure velocity using the Doppler
effect.

LiDAR stands out for autonomous vehicles because it “sees” everything around the car
with excellent depth perception. While cameras need light to work, LiDAR functions just
as well in complete darkness and tough weather. The technology can spot different
objects with high accuracy – from cars and people down to small items like football
helmets.

Safety requirements demand that automotive LiDAR must see 200–300 meters ahead.
This range lets vehicles traveling over 60 mph react safely to obstacles. Eye safety rules
also matter – IEC 60825-1 standards limit laser power, which affects what wavelengths
manufacturers can use.
Understanding Laser Generation in Automotive LiDAR

LiDAR starts with a laser source inside the unit. This source creates short light pulses lasting just nanoseconds to
microseconds. Before reaching the environment, these pulses pass through beam controllers and optical systems.

Two main methods generate these laser pulses:

•Pulsed Time-of-Flight (ToF): Sends out separate light pulses at specific times
•Frequency Modulated Continuous Wave (FMCW): Creates a steady beam with changing frequency patterns

LiDAR systems use three types of laser sources, each with unique benefits:
[Link]-Emitting Lasers (EEL): Popular in 905nm systems, offering good value but less stable spectrally
[Link]-Cavity Surface-Emitting Lasers (VCSEL): Better temperature stability (0.06 nm·K⁻¹ vs 0.25 nm·K⁻¹ for
EELs)
[Link] Lasers: Used in 1550nm systems, highest beam quality but costs more
shows the basic principle of lidar, with light shining out in
certain patterns and information extracted based on the
reflections gathered at the receiving end. Pulse power, round-
trip time, phase shift and pulse width are common parameters
used to extract information from light signals
Lidar, radar, ultrasonic sensors and cameras have their own niche sets of benefits and disadvantages. Highly or
fully autonomous vehicles typically use multiple sensor technologies to create an accurate long- and short-range
map of a vehicle’s surroundings under a range of weather and lighting conditions. In addition to the technologies
complementing each other, it is also important to have sufficient overlap in order to increase redundancy and
improve safety. Sensor fusion is the concept of using multiple sensor technologies to generate an accurate and
reliable map of the environment around a vehicle.

Ultrasonic waves suffer from strong attenuation in air beyond a few meters; therefore, ultrasonic sensors are
primarily used for short-range object detection.

Cameras are a cost-efficient and easily available sensor; however, they require significant processing to extract
useful information and depend strongly on ambient light conditions. Cameras are unique in that they are the only
technology that can “see color.” Cars that have the lane keep assist feature use cameras to achieve this feat.
• Mechanical lidar uses high-grade optics and a rotating assembly to create a wide (typically 360 degree)
FOV. The mechanical aspect provides a high signal-to-noise ratio (SNR) over a wide FOV, but results in a
bulky implementation (although size has also been shrinking).

• Solid-state lidar has no spinning mechanical components and a reduced FOV; thus, it costs less. Using
multiple channels at the front, rear and sides of a vehicle and fusing their data creates an FOV that rivals
mechanical lidar
Microelectromechanical systems (MEMS) lidar. A MEMS lidar system uses tiny mirrors whose tilt angle varies when
applying a stimulus such as a voltage. In effect, the MEMS substitutes mechanical scanning hardware with an
electromechanical equivalent. The receiver light collection aperture that determines the receive SNR is typically
quite small (a few millimeters) for MEMS. To move the laser beam in multiple dimensions requires cascading
multiple mirrors. This alignment process is not trivial, and once installed, it is susceptible to shocks and vibrations
typically encountered in moving vehicles. Another potential pitfall with a MEMS- based system is that automotive
specifications start at –40°C, which can be challenging for a MEMS device.

• Flash lidar. Flash lidar operation is very similar to that of a standard digital camera using an optical f lash. In flash
lidar, a single large-area laser pulse illuminates the environment in front of it, while a focal plane array of photo
detectors placed in proximity to the laser captures the back-scattered light. The detector captures the image
distance, location and reflected intensity. Since this method captures the entire scene in a single image compared to
the mechanical laser scanning method, the data capture rate is much faster. In addition, since the entire image is
captured in a single flash, this method is more immune to vibration effects that could distort the image. A downside
to this method is the presence of retroreflectors in the real-world environment. Retroreflectors reflect most of the
light and back scatter very little, in effect blinding the entire sensor and rendering it useless. Another disadvantage
to this method is the very high peak laser power needed to illuminate the entire scene and see far enough. To
comply with eye safety requirements, flash lidar is primarily used in short- to medium-range detection systems.
Optical phase array (OPA). The OPA principle is similar to phased-array radar. In an OPA system, an optical phase
modulator controls the speed of light passing through the lens. Controlling the speed of light enables control of
the optical wave-front shape, as shown in Figure 2. The top beam is not delayed, while the middle and bottom
beams are delayed by increasing amounts. This phenomenon effectively “steers” the laser beam to point in
different directions. Similar methods can also steer the back-scattered light toward the sensor, thus eliminating
mechanical moving parts.
Frequency-modulated continuous wave (FMCW) lidar. While the methods listed so far are based on Sensor
fusion Vehicle interface ETH/CAN An Introduction to Automotive Lidar Ethernet Load Heating Heater/ the
ToF principle using narrow light pulses, FMCW lidar uses a coherent method, producing brief chirps of
frequency- modulated laser light. Measuring the phase and frequency of the return chirp enables the system
to use the Doppler principle to measure both distance and velocity. The computational load and optics are
simpler with the FMCW method, although the chirp generation adds complexity. The laser power required
for FMCW systems is considerably lower than what pulsed ToF systems require, making FMCW suitable for
very long-range sensing applications. They also perform well in adverse weather conditions such as fog, rain
and snow
The entire functional lidar module
subsystem, including the signal chain,
power, interface, clocking and monitoring
or diagnostics subsystem. The main
subsystems of the lidar signal chain
comprise a transmitting system (Tx), a
receiving system (Rx) and a custom digital-
processing system to extract point-cloud
information. TI offers device options for
the function blocks shown in real.
What is an IMU?
• Inertial Measurement Unit (IMU) measures acceleration and rotation
• Provides 6-axis motion data (3 linear + 3 rotational)
• Operates independently using physics principles
• Key enabler for vehicle stability and autonomous systems
IMU Components

Accelerometers
Measure linear acceleration in X, Y, Z axes

Gyroscopes
Measure angular velocity and rotation rates

Tilt Sensors
Measure angular position relative to gravity
ADAS Applications (Part 1)
• Electronic Stability Control (ESC) - prevents skidding and loss of control
• Anti-Lock Braking System (ABS) - optimizes braking force distribution
• Traction Control - maintains grip during acceleration
• Roll Detection - identifies vehicle rollover risk
ADAS Applications (Part 2)
• Slip Angle Estimation - determines vehicle dynamics state
• Autonomous Parking - assists in automated maneuvering
• Sensor Fusion - combines data with lidar, camera, radar
• Vision Systems - enhances camera-based ADAS modules
Vehicle Dynamics Measurement
IMU measures complete 3D motion capturing all vehicle movements.

Translational Motion
Forward/backward, lateral, vertical

Rotational Motion
Roll, pitch, yaw movements
Sensor Fusion in ADAS
IMU works as central integrator blending data from multiple sensors.

Camera
Visual detection

Lidar
Distance mapping

Radar
Velocity detection

IMU
Central Integration
Critical Safety Features
• Real-time vehicle dynamics detection
• Operates independently of external conditions
• Functions during GPS/sensor failures
• ASIL D functional safety certification support
Modern IMU Technology (2024-2026)
Next-generation 6-axis IMUs enhance ADAS capabilities.

• Compact size with high temperature stability (up to 125°C)

• Eliminates need for multiple 3-axis sensors
• ASIL B/D certification ready systems
Challenges & Solutions

Challenge
Drift accumulation in continuous operation

Solution
Vehicle dynamics model feedback correction
Future: Autonomous Driving (Level 5)
IMU is critical for achieving full vehicle autonomy.

• Ensures data continuity during sensor failures

• Enables safe operation in adverse weather
• Accelerates Level 5 fully autonomous driving
IMU Classifications by Axes:

IMUs are categorized by the number of sensing axes, with automotive ADAS favoring higher integration for comprehensive
motion tracking.
•6-axis IMUs: Combine 3-axis accelerometers (linear acceleration in x, y, z) and 3-axis gyroscopes (angular rates for yaw,
pitch, roll). Examples include TDK InvenSense IAM-20685HP (ASIL B for positioning/vision) and IAM-20689 (ASIL D for
ESC/RSC).
•9-axis IMUs: Add a 3-axis magnetometer for heading reference, though less common in high-safety ADAS due to
magnetic interference vulnerabilities.

Redundancy Types for Safety:

Functional safety (ISO 26262, up to ASIL D) drives redundancy in ADAS IMUs.

•Single IMU: Used in lower-level ADAS (e.g., ACC, lane assist) for basic motion data; cost-effective but lacks fault
tolerance.
•Dual IMU: Two independent units for cross-checking, common in mid-level automation.
•Triple-redundant IMU: Three synchronized IMUs (as in Bosch/Aptiv systems) compare outputs to detect faults via majority
voting, essential for Level 3+ autonomy and fail-operational control.

Technology Basis:

All automotive ADAS IMUs rely on MEMS (Micro-Electro-Mechanical Systems) for compact size, low power, and
ruggedness (-40°C to 125°C).
•Accelerometers: Capacitive or piezoresistive proof-mass sensing.
•Gyroscopes: Vibrating MEMS with Coriolis force detection.
IMU Type Axes ASIL Level Key ADAS Use Example Device
Standard 6-axis 6 B Positioning, ACC IAM-20685HP
Safety 6-axis 6 D ESC, DbW IAM-20689
Triple- Bosch ADAS
6x3 D Full redundancy
redundant IMU
Sensor fusion in automotive and ADAS systems combines data from multiple sensors like cameras, radar, LiDAR, ultrasonic,
IMU, and GNSS to create a robust, accurate perception of the vehicle's environment and motion, overcoming individual
sensor limitations such as weather sensitivity or drift.
Fusion Levels
Sensor fusion occurs at different stages of data processing, each with trade-offs in computational load and accuracy.
•Raw/Low-level fusion: Merges unprocessed sensor signals (e.g., aligning radar range with camera pixels via time-sync and
coordinate transforms) for maximum detail retention.
•Mid-level fusion: Combines detected objects/tracks (e.g., radar velocity with camera classification) using association
algorithms like nearest neighbor or JPDA.
•High-level fusion: Integrates semantic outputs (e.g., lane maps from camera/HD-maps with free-space from LiDAR) for
decision-making like path planning.

Core Algorithms
Kalman filters and extensions dominate real-time fusion in ADAS for state estimation (position, velocity, orientation).
•Linear Kalman Filter (KF): For Gaussian noise, linear models; basic vehicle motion prediction.​
•Extended Kalman Filter (EKF): Handles non-linear dynamics, common for IMU/GNSS fusion in dead-reckoning.
•Unscented/Particle Filters: For highly non-linear/multi-modal data, used in complex urban scenarios.​

Challenges and Solutions

•Data sync/latency: Sensors operate at varying rates (e.g., camera 30Hz, radar 10Hz); solved via timestamp alignment and
buffering.
•Data association: Matching detections across sensors (e.g., same pedestrian); uses gating, probabilistic methods.​
•Safety/validation: ISO 26262 requires redundant fusion paths; real-world data tuning and simulation with sensor models
ensure 75-85% accuracy
Key Fusion Techniques in ADAS

Technique Sensors Involved ADAS Application Benefit

Improves detection
Camera + Radar + AEB, ACC target
Object Data Fusion range/velocity
LiDAR tracking
accuracy
Robust lane
Lane Fusion Camera + HD-Map Lane keeping detection in poor
visibility
Dead-reckoning in
Highway pilot,
Localization Fusion IMU + GNSS + LiDAR GNSS-denied areas
parking
(tunnels)
Radar + Ultrasonic + Reliable drivable
Free Space Fusion Automated parking
Camera area estimation
IMU + Wheel Precise vehicle state
Ego-Motion Fusion Stability control
Encoders + Vision amid sensor noise
Inertial measurement unit (IMU) sensors may not attract the same amount of attention as other sensors,
such as lidar, radar and cameras, but in many ways IMUs are the critical safety sensor component required
for the successful operation of the Level 4 and Level 5 autonomous vehicles that will be appearing on roads
within the next decade.
Role of Kalman Filter in Automotive Applications
The Kalman filter is a mathematical algorithm that estimates the state of a dynamic system from
a series of noisy measurements. In automotive applications, it plays a crucial role in improving
the accuracy and reliability of sensor data, which is essential for various advanced driver-
assistance systems (ADAS) and autonomous vehicles.
Key Applications in Automotive Systems
[Link] Position and Navigation (GPS/INS Integration):
1. Problem: GPS signals can be noisy or temporarily unavailable (e.g., in tunnels or urban
canyons). Inertial Navigation Systems (INS) drift over time.
2. Kalman Filter Solution: Combines GPS and INS data to provide a more accurate and
stable estimate of the vehicle's position and velocity.
3. Example:
1. When a car enters a tunnel and loses GPS signal, the Kalman filter uses the last
known GPS position and INS data (from accelerometers and gyroscopes) to estimate
the current position until GPS is available again.
[Link] Fusion for ADAS (Advanced Driver-Assistance Systems):
1. Problem: Different sensors (radar, lidar, cameras) provide measurements with different
noise characteristics and update rates.
2. Kalman Filter Solution: Fuses data from multiple sensors to track objects (like other
vehicles or pedestrians) more accurately.
3. Example:
1. In adaptive cruise control, the Kalman filter combines radar and camera data to
estimate the distance and speed of the vehicle ahead, even if one sensor is
temporarily blocked or noisy.
[Link] State Estimation (Speed, Yaw Rate, etc.):
1. Problem: Direct measurements of vehicle states (like speed, acceleration, yaw rate) are
noisy due to road conditions and sensor limitations.
2. Kalman Filter Solution: Filters out noise to provide smooth and reliable estimates for
vehicle control systems (e.g., Electronic Stability Control).
3. Example:
1. The Kalman filter processes wheel speed sensor data and accelerometer readings
to estimate the true speed of the vehicle, which is used for traction control.