P1.

1-417
Intelligence means the ability to understand, know and learn. This ability was given to the
human been and enabled him to build and control his world. This becomes the key tasks
in robot perception. Intelligence given to a machine is defined as artificial intelligence (AI).
AI allows a robot to perform its actions even in situations that were not precisely defined
in advance, that is, in changing environmental conditions. For example, if the robot in our
lab had sensors in the gripper, every time the part fell on, it would know it had to go back
to the parts feeder to take another part. Adapting a robot to work in a changing
environment is done by integrating sensors and a processor in the robot system with
various related algorithms.
At the recent time, the development of sensors focuses in making perception to be similar
the five senses that human endowed with. A lot of research and industrial developments
have been invested to reach these capabilities such as touch and recognizing
voice(hearing) and vision that have been developed so far.
In addition to these senses, sensors have been developed that are not similar to human
senses, such as: infrared light sensors to detect for example heat source, proximity
sensors to detect nearby objects and acoustic sensors to detect space and motion. More
notable is the inner ability to feel and feel the forces and processes that take place within
his body through internal sensors.

In general, in robotics we actually talk about perception as a concatenation of modules

within a general perceptual pipeline. Therefore, a perception in general would consist of
first sampling process, then we might be using some signal processing to process the
raw signal values. Then we feed that into a feature extraction component via some
inference model that may or may not be similar to the models we looked at before and
finally we would use that to output interpret a scene or the state of the robot or the world.
So, this all together is our perceptual pipeline, and the length or the sophistication of this
pipeline tells us whether we're using a proximal architecture, whereby we be thinking of
a very short or very simple pipeline versus a more distal architecture where we're looking
at a long pipeline with a lot of sophisticated modules. Therefore, now by adding all these
layers of modeling and perception and representation we can make these architecture
more distal and more sophisticated and ultimately also more powerful.
P1.q2(203 w)

Now the key question that we ask which is relevant to any type of sensor, is how we
actually relate a sensor measurement to a perceptive feature? One way is to use
calibration. As an example calibrating an infrared proximity. This process of calibration
will produces a sensor model which is the formality that relate the output of the sensor
to a perceptive feature.
To demonstrate this relation, the single chip approach and the three separate chips
approach where used in a basic light-measuring process in color camera. Those two
approaches used for “measuring the total number of photons that strike each pixel in
the integration period”(chp4). The three-chip color camera splitting the incoming light
into three complete (lower intensity) copies where each chip measures light intensity
for one color, then the camera must combine the chips’ outputs to create a joint color
image.

Therefor the goal of ant calibration procedure is to determine the mapping between
the sensor reading and the desired feature and its relays on three main steps: the first
step uses a ground-truth telemetry system to set up sensor, the second step then
consist of measuring and tabulating these measured values, the third step is actually
fitting a curve to create a more compact representation of the relationship between the
sensor output and the perceptive feature.

In order to create these compact representations there is a different ways to achieve it.
One way is to create a curve or fitting a curve to the collected sensor measurements.
And this curve then ideally correspond for example in a case of infrared sensor to the
signal who intensity decreases with the square of the distance. This is because some
thing that we know from first principles in how infrared proximity is suppose to behave.
And we can expect in the plot that the sensor actually not return the same value for
each distance! This is because in any robotic system there will be uncertainty that
arises due to unwanted function or chip production of the sensors but also uncertainty
of an modeled or under modeled in the environment. And we can see this limitation in
the three-chip and the single-chip color cameras where both “suffer from the fact that
photodiodes are much more sensitive to the near-infrared end of the spectrum. This
means that the overall system detects blue light much more poorly than red and
green”(chp5).
Even repeated measurement will not be the same since a noise could be happened on
the environments. And one of the key questions that calibration in sensor model deal
with is exactly how to deal with the uncertainty? Dealing with the uncertainty is crucial to
effectively and efficiently solving the perception problem in robotics. Therefore
representing uncertainty is one of the future main challenge.

P1.3
Representing uncertainty requires having a methodology by which we can represent
sensor uncertainty. Since sensors are imperfect devises, we know there will be a
random errors, therefore we need a representation for random errors that are
associated to a given sensor and its systematic errors, and the main challenge in the
future remain is how to represent uncertainty in a way that exploit and use it in our
perceptions algorithms and systems. so the key thing here is representation. There are
several way we can create representations for sensors-uncertainty. Since we want to
infer features from measurements, we need to know mush we can trust this inference.
Therefore we need to know what the confidence is in interpreting what the sensor is
telling us, and the key question that we should deal with is what is the sensor errors
distribution look like? And the process we would follow is doing measurements as
describe in previous paragraph, and then create a model. There are three different
options for creating sensor-uncertainty model: The first option consist of storing the
original measurements with the true/false measurements. The second option to store a
histogram of measurements, and the third option is to compute a compact
representation of the error distribution.

The main problem with option one is clearly will be a lot of data that make some
difficulties to update new values to be measured and how to add that data to our
table/data etc. which is not a good way in modeling error distributions. The second
option is more practical from the first option, but the problem with it is that the histogram
is a desecrate collection of data. And finally the problem with the third option is although
it is definitely the practical way of doing things, but models tend to be more general.
This is because we tend to choose a template model with hyper-parameters with it and
then try to fit, but still this model might not be the right model since for example if we
choose a Gaussian one-mode, but perhaps the underling distribution isa Gaussian two-
mode, or it has a heavy tail, etc. these are things that will effect accuracy and the
efficiency and the applicability of the model we are fitting to the underling data.
P2.1(cabridge) 700

In robotics we're interested in certain guarantees such as the real time guarantees that
vision algorithms could deliver, and of course they have to be able to operate very
quickly because we're talking about dynamic decision making when we use vision and
robotics.

The perception is task driven when we use vision. So we're not using vision in general
to just classify anything arbitrarily, we usually use vision for very specific purposes. So
for example, we want to detect pedestrians and other cars, but not necessarily
interested in having classifiers that can detect any possible object that exists In the
world.

We're also using vision in motion, and very commonly the cameras are placed on
objects that are moving, and hence we're using data streams rather than images so
we're really interested in streams which produces completely different types of data
sources and databases that we would have to process in order to make sense
of. Common applications in robotics are, for example, object tracking, so we have a
robot that is trying to hone in to a target or because it wants to protect the target or
because it wants to follow that target for some reason. We might be using vision for
information gathering, so we could think of environmental monitoring purposes or a
wildlife monitoring and we would want to do those things in autonomous manner.

The odometers sensors measure how far a robot has moved, or also speedometers
use ideas as such to determine how far they're traveling or at which speeds. with
odometers we can also use the shaft encoders to measure the direction of motion. And
visual odometry is actually also an application that is perhaps more of a means to an
end visual dormitory what it allows us to do is actually it allows us to estimate positions
of mobile of autonomous mobile vehicles. So how visual dormitory actually works or
otherwise known as visual inertial odometry is by looking at image frames as they come
in our stream, we can actually infer by how much the vehicle for example has moved
along with a degrees of freedom by comparing neighboring frames. Therefore, this idea
is very powerful and can allow us to use the idea of a dormitory not using it dormitories,
but instead using visual input. And so these are just a couple of ideas and applications
that are very popular within the field of robotics through leveraging vision as our
perception component.

The LIDAR are based on a simple measurement principle of Time-of-Flight(TOF) of a

pulse of light. When developing a perception model for the LIDAR ,it will may have
multiple components to deal with uncertainty and will depend on the environment. For
example modeling unexpected dynamic objects, random measurements and failures.
We might have a standard Gaussian to model noise around the current range, and
uniformed distribution to model random measurements. Then when we fit the
parameters with the given underling data then we will get an ultimate probability density
function.

Sensor fusion approached by combining and synchronizing group of sensors to get

different features from different sensors. According to(ref), deep learning provide the
ability of the sensor fusion to overcome with different multi-sensor fusion challenges in
the perceptual conditions with a number of strategies: early fusion, late fusion, mid-
level fusion, and sequential fusion.

Odometry and sensor fusion are crucial for mobile robot. In the context of ground robots
will use sensing to localize or navigate. The perception-action loop, start by infer
information about the state of the worlds. And they do this in a such way that the robot
can use output to make decisions about the actions that the robot would next take. In
ground robot the perception mostly will be related to localization and navigation.
Therefore the decisions will be related to motion and navigation.

P2.2(500)
Simultaneous localization mapping (SLAM) plays a vital part in the main technique used
to perform perception on the state estimation problems when using laser range scanners
and camera as a main popular sensors used to turn their information into a geometric
model while they moved through the environment by the robot to build a map of the
environment.

Actually, by using SLAM, we aim to build a model of what the environment looks like. This
model can be a dense 3D model or the positions of distinct points in the environment.
Features that have been extracted from sensors data will be used by SLAM for estimating
the location of those features or landmarks in the environment. Importantly, this can
surface our model so we can again use that model in order to make estimation.

So there are three problems which actually come here together: localization for
estimating the pose, mapping for building a map and SLAM for simultaneous
localization and mapping which is the problem of addressing those two problems at the
same point in time. That is a kind of challenging, because we typically need to have a
map in order to estimate our pose. On the other side we need to have a good pose
estimate in order to build a map. So, we have a dependency between those two tasks
when trying to solve both of them at the same time. While the robot moved through the
environment, this could add more challenge since not everything is perfect and this
leads to an uncertainty. Therefore, the real map and the map estimated by the robot are
not identical.
So there are variant various variants of this simultaneous localization mapping
algorithms. The full slam algorithms trying to approach the problem in which all the
sensor data is active initially and then we have all the sensor data at hand and want to
build the best possible map of the environment. Another alternative and notable if we
think about robotics applications, drones, self driving cars is that we process algorithms
in real time or near real time and want to turn the new observation that we get directly
into a model. This is something that we call online slam that is also very related to the
bundle adjustment problem and this is basically the SLAM problem for cameras using
only camera observations.
P2.3-no cambrage(the video) (ML,NN)

In order to enable applications to truly interact with the real world, SLAM algorithms must
provide a number of advanced capabilities. These include capturing accurate
photorealistic 3D models while handling changing scenes with varying lighting conditions
and occlusions. Real-world SLAM algorithms must also be able to handle colorless
scenes and scenes with flat surfaces.

Existing SLAM technologies cannot meet these challenges, unless it can handle a
dynamic Slam which addresses various challenges with the following techniques that
are [--ML NN--]:
-taking full advantage of multiple sensor inputs to support sensor fusion with RGB
cameras, 3D depth, sensors and inertial motion units as well as stereoscopic cameras.
- tracking technique known as sparse tracking. Sparse tracking uses color information to
identify key features, and this reduces the need for dense 3D depth data. And the
processing requirements are also greatly reduced by applying machine learning
techniques with the use of colored dots indicate key feature areas identified using these
color information. It is also possible to augment the scene by inserting a colored cube
frame and use ML algorithms to exact location where they were originally inserted. The
Dynamics SLAM also uses a technique called dense tracking, which is automatically
applied when needed to improve sparse tracking accuracy and to capture accurate
photorealistic 3D models.

- Also, stereo cameras can be incorporated for outdoor applications. One of the
challenges in scanning a complete 360 degree scene is how to correct a phenomenon
known as drifting. As the camera pans across the scene, it introduces slight errors in the
estimated path taken. The accumulation of these errors results in misaligned loop
ends. [---]provides advanced loop closure capability to fix these misalignments.

And what we want to estimate is the map of the environment as well as the path
including positions and orientation information parts. Therefore, this is a probabilistic
approach therefore we must provide a probability distribution to be close as possible to
the true pose of the platform, but we also have a challenge of uncertainty. And this
typically relates to the uncertainty of the observations and the uncertainty of our robot
motion and both will impact the uncertainty of the overall map that we actually have.

P2.4 -in dept….

If we look to the question how is this SLAM problem typically solved?

If we look to the 3 traditional paradigms:
1) the Kalman filter: using this as an estimator to estimate where the platform is in the
environment and where our landmarks are.

2) the particle filter based representations which have the advantage that they get rid of
the Gaussian assumption about the world and we can estimate multimodal beliefs those
particle filter applications.
3) the graph based approaches which typically refer to Least squares formulations of
the simultaneous localization mapping problem.

The graph based representations because today most of the systems for tackling this
SLAM problem are using graph based representations.

Also, when taking into account multiple different sensors when being able to revise
previous data station decisions, for example, the idea of using this graph based
representation, in least squares approach is very attractive and is today the most
frequently used representation and therefore it is good to stick with graph based
paradigm for the simultaneous localization and mapping problem.

Since everything boils down to observations and controls which are used in order to
estimate where I am and what the world looks like.
So the information about how good my observations are and the information on how
well I can describe my motion has a key impact on the robot state estimation system.

Based on the graphical model representation, we could see the position of the platform
at time T, decision of the platform at time T - 1 and the control command which guides
the platform from T - 1 to T, we can use them to describe and build a motion
model. Then the probability distribution of an observation given the map of the
environment given the position of the platform is something that we refer to as the
observation model. And these are the two models, the observation model and the
motion model which have a key
Impact on our robot perception system, and sensor fusion.