The Robotics Primer

Chapter 1: What is a Robot?

Defining Robotics

● The definition of a robot has evolved, as research has made strides and technology has
advanced.
○ The word “robot” resulted from combining the Czech words rabota, meaning
“obligatory work”, and robotnik, meaning “serf.”
○ Robotics is much more than obligatory labour.
● The idea of a robot/machine that can help people has no real origin.
○ Many engineers in the past thought of it in some form.
○ This form changed over time, as science and technology advanced.
○ Many previously unachievable dreams of robots have become a reality or entered the
realm of possibility.
● With the advancement of science and technology, the notion of a robot has become more
sophisticated.
● The notions of robots started to include thought, reasoning, problem-solving, and even
emotions and consciousness.
○ Robots are starting to look more and more like biological creatures.
● A robot is an autonomous system which exists in the physical world, can sense its
environment, and can act on it to achieve some goals.
○ An autonomous robot acts based on its own decisions, and is not controlled by a
human.
■ Machines that are externally controlled by humans are teleoperated.
■ True robots act autonomously.
○ Existing in the physical world is a fundamental property of robots.
■ It makes robotics a real challenge.
■ Robots that exist on computers are simulations.
○ Sensing the environment means the robot has sensors to get information from the
world.
■ A simulated robot can be given information or knowledge.
■ A true robot can sense its world only through its sensors.
○ Taking actions to respond to sensory inputs and achieve what is desired is a
necessary part of being a robot.
■ A machine that does not act is not a robot.
○ A true robot has one or more goals to act so as to achieve those goals.
■ Goals can be very simple or quite complex.
● Robotics is the study of robots, which means it is the study of their autonomous and
purposeful sensing and acting in the physical world.
The Robotics Primer

Chapter 2: Where Do Robots Come From?

A Brief but Gripping History of Robotics

2.1: Control Theory

● Control theory is the mathematical study of the properties of automated control systems
ranging from steam engines to airplanes, and much in between.
○ It is one of the foundations of engineering and studies a vast variety of mechanical
systems that are a part of everyday life.
○ Its formalisms help us understand the fundamental concepts which govern all
mechanical systems.

2.2: Cybernetics
● Cybernetics was the study and comparison of communication and control processes in
biological and artificial systems.
○ It combined theories and principles from neuroscience and biology with those from
engineering.
○ It had the goal of finding common properties and principles in animals and machines.
● In cybernetics, the idea was that machines would use a similar steersman to produce
sophisticated behaviour similar to nature.
● A key concept of cybernetics focuses on the coupling, combining, and interaction between
the mechanism or organism and its environment.
○ The interaction is necessarily complex and is difficult to describe formally.
○ This is an important component of robotics.

2.2.1: Grey Walter’s Tortoise

● Machines with properties similar to those of biological systems are biomimetic.
○ This means they imitate biological systems in some way.
● 1940s: William Grey Walter builds a variety of cleverly designed machines
(“turtles”/”tortoises”).
The Robotics Primer

● Walter gave the turtles Latin names to describe their behaviours.

○ “Machina Speculatrix” means “machine that thinks” or “speculates.”
○ “Machine Docilis” means “machine that can be tamed/taught.”
● Machine Speculatrix consisted of…
○ One photocell - a sensor that detects light levels.
○ One bump sensor - a sensor that detects contacts with objects.
○ One rechargeable battery.
○ Three motors, one for each wheel.
○ Three wheels.
○ One analogue electronic circuit with two vacuum tubes, serving as the “brain” and
connecting the two sensors with the wheels.
■ An analogue signal is continuous in time and amplitude.
■ An analogue electronic circuit processes analogue signals.
● Walter endowed Machine Speculatrix with the following behaviours/capabilities…
○ Find the light.
○ Head toward the light.
○ Back away from bright light.
○ Turn and push to avoid obstacles.
○ Recharge battery.
● This robot and others constructed by Walter used reactive control.
○ Reactive control is a means of controlling robots using a collection of prioritised
“reflexes.”
○ These rules resulted in animal-like behaviour.
● The turtles manifested a variety of complex patterns of behaviours that were not
programmed in - emergent behaviour.
○ They were excellent examples of how animal-like realistic behaviour could be
achieved with very simple artificial mechanisms.
● Cybernetics effectively combined “thinking,” “acting,” and the interaction with the
environment.
○ These aspects would split between the fields of artificial intelligence and robotics.
The Robotics Primer

○ Artificial intelligence took over thinking.

○ Robotics concerned itself with action in the physical world.
● Walter’s turtles represent some of the earliest examples of artificial life.
○ Artificial life/alife mostly focuses on computational, not physical systems that exist
within a computer, not in the real world.

2.2.2 Braitenberg’s Vehicles

● 1984: Valentino Braitenberg publishes Vehicles.
○ It describes a series of ideas - thought experiments/gedanken experiments - that
shows how one could design simple robots and get them to produce behaviours that
appear very animal-like and life-like.
● Braitenberg’s vehicles started with a single motor and single light sensor and gradually
progressed to more motors and sensors, and more interesting connections between them, all
using analog electronic circuits.
○ The sensors were directly connected to the motors so that the sensory input could
drive the motor output.
● A connection between the sensors and motors in which the stronger the sensory input, the
stronger the motor output, is called an excitatory connection because the input excites the
output.
● A connection in which the stronger the sensory input, the weaker the motor output, is called
an inhibitory connection, because the input inhibits the output.
● Braitenberg’s book describes how many simple mechanisms can be employed to store
information, build a memory, and even achieve robot learning.

2.3: Artificial Intelligence

● 1956: For machines to be intelligent, they will have to do some heavy-duty thinking and to do
that, they will need to use the following…
○ Internal models of the world
○ Search through possible solutions.
○ Planning and reasoning to solve problems.
○ Symbolic representation of information.
○ Hierarchical system organization.
○ Sequential program execution.
● 1980s: Robotics enters a phase of very rapid development.
○ Several new robotics directions emerged, shook things up, and eventually got
organized into the types of robot control used today.
■ Reactive control.
■ Hybrid control.
■ Behaviour-based control.
● AI was (and is) very different from cybernetics in its goals and approaches.
○ Robotics needs to combine both to create robust, fast, and intelligent machines.
The Robotics Primer

● Modern robotics emerged from historic developments and interactions among a handful of
key research areas…
○ Control theory.
○ Cybernetics.
○ Artificial intelligence.
● These fields have had an important and permanent impact on robotics, and at least two of
them continue as major areas of research in their own right.
○ Control theory thrives as a field of study applied to various machines.
■ Control-theoretic ideas are at the core of low-level robot control.
○ Cybernetics no longer exists under that name, but research into biologically inspired
biomimetic methods for robot control is still present.
○ Artificial intelligence is now a very large and diverse field of research.

Chapter 3: What’s in a Robot?

Robot Components

● A robot’s main components are…

○ A physical body, so it can exist and do work in the physical world.
○ Sensors so that it can sense/perceive its environment.
○ Effectors and actuators, so it can take action.
○ A controller, so it can be autonomous.

3.1: Embodiment
● Embodiment refers to having a physical body.
● Embodiment is necessary for a robot, but…
The Robotics Primer

○ We are all in the same boat.

■ An embodied robot must obey the same physical laws that all physical
creatures obey.
■ It cannot be in more than one place at a time, and can’t change shape and
size arbitrarily.
■ It must use effectors on its body to actively move itself around.
■ It needs some source of energy to sense, think, and move.
■ It must take time to speed up and slow down when moving.
■ Its movement affects the environment in various ways.
■ And so on…
○ You are in my space.
■ All physical robots have to worry about running into or hitting other things in
the environment.
○ I can only do so much.
■ Every robot body has its limitations.
○ All in good time.
■ The body influences things that have to do with time.
■ The body determines how fast the robot moves and reacts to the
environment.

3.2: Sensing
● Sensors are physical devices that enable a robot to perceive its physical environment in order
to get information about itself and its surroundings.
○ Sensing and perception refer to receiving information about the world through
sensors.
● Robots have a niche, consisting of their environment and their task.
○ The better they fit that niche, the longer they survive.
● Sensing allows the robot to know its state.
○ State is a general notion from physics that is borrowed by robotics and refers to the
description of a system.
○ A robot’s state is a description of itself at any point in time.
○ The more detailed description, the larger the state is, because it takes more bits or
symbols to write it down.
● To a robot, a state may be visible (observable), partially hidden (partially observable), or
hidden (unobservable).
● A state may be discrete or continuous.
○ This refers to the type and amount of information used to describe the system.
● State space consists of all possible states a system can be in.
○ The term space refers to all possible values or variations of something.
● It is useful for a robot to distinguish two types of states relative to itself.
○ External state refers to the state of the world as the robot can perceive it.
○ Internal state refers to the state of the robot as the robot can perceive it.
■ The internal state can be used to remember information about the world.
The Robotics Primer

■ This is called a representation or an internal model.

● All of the robot’s sensors create the space of all possible sensory readings, which is called the
robot’s sensor space/perceptual space.
○ The robot’s sensor space grows pretty quickly as you add more sensors or more
complex sensors.

3.3: Action
● Effectors enable a robot to act, and to do physical things.
○ Effectors use underlying mechanisms called actuators, which do the actual work for
the robot.
● Robotic effectors and actuators are used for two main activities.
1. Locomotion: Moving around, going places.
2. Manipulation: Handling objects.

3.4: Brains and Brawn

● Robots need a brain to function properly.
○ Protect the brain to avoid problems!
● Power is a major problem in practical robotics, and issues include…
○ Providing enough power for a robot without loading it down with heavy batteries.
○ Effectively keeping the electronics isolated from the sensors and effectors.
○ Preventing loss of performance as the power levels drop because batteries wear
down or there is a sudden high power demand.
○ Autonomously replenishing power by the robot itself instead of by people.
○ And so on…
● Brains and brawns are directly connected and related and should be designed and treated
together.

3.5: Autonomy
● Controllers provide the hardware and/or software that makes the robot autonomous by
using the sensor inputs and any other information to decide what to do and then control the
effectors to execute that action.
○ They play the role of the brain and the nervous system.
○ Typically, there is more than one controller so that various parts of the robot can be
processed at the same time.
● Autonomy is the ability to make one’s own decisions and act on them.
○ For robots, autonomy means that decisions are made and executed by the robot
itself, not by human operators.
○ Autonomy can be complete or partial.
■ Completely autonomous robots make their own decisions and act on them.
The Robotics Primer

■ Teleoperated robots are partially or completely controlled by a human and

are more like complex puppets.

Evolutionary Computation
● Reseasearchers in artificial intelligence have developed algorithms based on evolution theory.
○ The class of these algorithms are called evolutionary algorithms and consists among
other others of genetic algorithms, evolutionary strategies, and genetic
programming.
● Evolutionary algorithms are optimisation algorithms inspired by Darwin’s evolution theory
(natural selection/survival fo the fittest).
○ They can find very good solutions in very large spaces.
○ The principle is that all solutions are evaluated after which the best solutions are
allowed to reproduce most offspring (children).
○ If the the parents form good solutions, they are likely to possess good building blocks
of genetic material that may be useful for creating new individuals.
● Genetic algorithms usually take two parent individuals and recombine their genetic material
to produce a child that inherits genetic material from both parents.
○ If a child performs well on an evaluation test, it will be selected for reproduction.
■ The genetic material can be propagated to new generations.

1.1: Solving Optimisation Problems

● A lot of research in computer science/artificial intelligence has been devoted to solving
optimisation problems.
● Exhaustive search arpproaches cannot be used for solving combinatorial solutions in a short
time.
○ Heurisitc search algorithms have to be used which can find good solutions in a good
time, although not always the most optimal ones.
● Genetic algorithms differer from other algorithms in a way that they keep a population of
solutions and use recombination operators to form new solutions.

1.1.1: Formal description of an optimisation problem

● Optimisation problems consist of two components.
1. The representation space denotes all possible solutions.
2. The evaluation/fitness function compares different solutions to each other.
a. The goal is to find the solution smax which has the maximimal fitness.

b. There can be multiple different solutions with the same maximal fitness
value.
● The goal is to search through the representation space for a solution which has the maximal
possible fitness value given the fitness function f(.).
The Robotics Primer

● The representation space can consist of a huge number of possible solutions or may be
continuous.
○ The optimal solution may be very hard to find…
● In practice algorithms, they are compared by the best found solutions within the same
amount of computational time.
○ Among these algorithms, there could be a human which tries to come up with a
solution.
○ If the fitness function gets more complicated and the representation space becomes
bigger, the advantage of computers in their ability to try out millions of solutions
within a short period of time outcompetes the ability of any human in finding a good
solution.

1.1.2: Finding a solution

● Heuristic search algorithms usually start with one or more random solutions which are then
evaluated.

● Another possibility is simulated annealing.

○ Simulated annealing always accepts improving solutions, but can also select a new
solution with lower fitness than the current one, albeit with a probability less than 1.

■ T is the temperature which allows the algorithm to explore (T >>> 0) or to

only accepting improving solutions (T = 0).
● Temperature is cooled down/annealed from a high temperature to
zero.

1.2: Genetic Algorithms

● Genetetic algorithms use a population of individuals to search for solutions.
○ Search is done in a distributed way.
○ Individuals are enabled to exchange genetic material.
● Making the search using a population allows for parallel computation!
The Robotics Primer

● The real advantage lies in the possibility of individuals to exchange genetic material by using
recombination operators and by the use of selective pressure on the population.
○ The best inidivudals are most likely to reproduce and continue the search for novel
solutions.

● There is a state of every individual.

● Since the population consists of N individual, the population also has a state.
○ After each iteration of the algorithm/generation, the population state makes a
transition to a new state.
○ After a long time, the population contains the optimal solution.
● We always store the best solution found so far in some place.

1.2.1: Steps for making a genetic algorithm

● Construct a representation.
● Initialise a population.
● Design a mutation operator and a recombination operator.
● Determine how parents are selected for reproduction.
● Decide how individuals are used to construct a new population.
● Decide when the algorithm has to stop.

1.2.2: Constructing a representation

● There are often many possible representations, so we have to examine the problem to
choose one.

Binary representations and finite discrete sets

● The most often used representation in genetic algorithms uses binary values, encoding a
chromosome using a bitstring of N bits.
The Robotics Primer

● If we have a binary representation for the genotype, we can use to construct different
representations for phenotypes.
○ Search using the genetic operators take place in the genotype space.
○ The phenotype is an intermediary representation which is easier to evaluate by the
fitness function.
● The mapping from genotype to phenotype can be an identity mapping.
○ They can be exactly the same.

Representing real numbers

● It is more a natural way to encode real numbers immediately in the genotype and to search in
the space of real numbers.
○ This can lead to more precise solutions.
○ The encoding is much smaller, but this comes at the cost of creating a continuous
search space.
● For real numbered representations, we have to use a fitness function which maps a solution
to a real number.

Representing ordering problems

● For particular problems, there are natural constraints which the representation should obey.
● A natural representation for an ordering problem is to use a list of numbers where each
number represents a value.

1.2.3: Initialisation
● Before running a genetic algorithm, there must be an initial population.
● Initialisation is usually done using a pseudo-random generator.
○ This depends on the representation.
The Robotics Primer

● For…
○ Binary strings, each single bit on each location in the string of each individual receives
50% probability to become a 0 and 50% probability to become a 1.
○ Real numbers, if the space is bounded by the lower and higher limits, generate a
uniform number between the boundaries.
○ Ordered lists, we generate numbers randomly and eliminate ones that been used
before during the initisilisation of an individual coding a tour.
● With possessing a-priori o possible good solutions, although the starting population may have
higher average fitness, there are some disadvantages…
○ It is more likely that genetic diversity in the initial population is decreased, which can
make the population converge much faster to a population of equal individuals.
○ Due to the initial bias which is introduced in this way, it is more difficult for the
algorithm to search through the whole state space, possibly making it almost
impossible to find a global optimum which is distant from the individuals in the initial
population.

1.2.4: Evaluating an individual

● In some cases, evaluating an individual can be very fast or slow.
● Populations cannot be very large nad the number of generations should be limited.
● For particular problems, there are hard/soft constraints which a solution has to obey…
● To deal with these constraints…
○ Use a penalty term which punishes illegal solutions.
■ In some cases where there are many constraints, a large proportion of a
population may consist of illegal solutions.
■ Even if these are immediately eliminated, the search is less efficient.
○ Use specific evolutionary operators which make sure that all individuals from legal
solutions.

1.2.5: Mutation operators

● Mutation is used to pertuabe (slightly change) an individual so that a new individual is
created, but still resembles the previous one.
○ This is an important operator which allows us to explore the representation space.
○ It is useful to create more diversity and to escape from a converged population which
would be unable to explore different solutions anymore.
● It is important that…
○ At least one mutation operator should make it possible to search through the whole
space of solutions.
○ The size of the mutation operator should be controllable.
○ Mutation should create valid (legal) individuals.

Mutation for binary representations

● Mutation on a bitstring is performed by changing a bit to its opposite.
The Robotics Primer

Mutation for real numbers

● If a representation of a real numbers, we need a different mutation operator.

Mutation for ordered representations

● We should try to make sure the resulting individual respects the constraint of the problem.
○ We can do this by using a swap of two values on two different loci.

1.2.6: Recombination operators

● With recombination operators, it becomes possible to combine useful genetic material from
multiple parents.
○ The offspring will posses the best building blocks from both parents.
○ Good individuals will be kept in the population and worse ones will die.
● With recombination operation, it is important that…
○ The child must inherit particular genetic material from both parents.
■ If it only inherits genetic material from one of the parents, it is basically a
mutation operator.
○ The recombination operator must be designed together with the representation of
an individual and the fitness function so that recombination is not often a
catastrophe (generating bad individuals).
○ The recombination operator should generate legal individuals.
The Robotics Primer

Recombination for binary strings

● There are different crossover operators.

Recombination for real numbered representations

● We can use the same methods for binary strings.
● Another option is to average the numbers on the same location.

Recombination for ordered representations

● Select a part of the first parent and copy that to the child.
● Use information from the second parent about the order of values which is not yet copied to
the child.

1.2.7: Selection strategies

● An important topic with designing GAs is selecting which parents are allowed to create
children.
The Robotics Primer

○ Good individuals must have a larger probability for generating offspring than worse
individuals.
● The selection strategy determines how individuals of a population are chosen for generating
offspring.
● Using bad individuals may create more diverse populations, so do not necessarily restrict
them from creating offspring.

Fitness proportional selection

● Parents which are allowed to reproduce themselves are assigned a probability for
reproduction that is based on their fitness.

○ pi indicates probability.
○ i indicates an individual.
○ fi indicates fitness of the ith individual.
● There are some disadvantages…
○ There is a danger of premature convergence, since good individuals with a much
larger fitness value than other individuals can quickly take over the whole population.
○ There is little selection pressure if the fitness values all lie close to each other.
○ If we add some constant to all fitness values, the resulting probabilities will become
different, so that similar fitness functions lead to completely different results.

Tournament selection
● k Individuals are selected from the population without replacement.
● The individual of this group of k individuals is used for creating offspring.
○ k is the tournament size, and is usually set to 2 or 3.

Rank-based selection
● All individuals receive a rank where higher ranks are assigned to better individuals.
● The rank is used to select a parent.
The Robotics Primer

○ pi indicates probability.
○ i indicates an individual.
○ ri indicates the rank of the ith individual.

Truncated selection
● The best M < N individuals are selected and used for generating offspring with equal
probability.
○ The problem is that it does make distinctions between the best and the Mth best
individual.

1.2.8: Replacement strategy

● In a generational genetic algorithm, one usually kills the old population and replaces it by a
completely new population.
● In a steady-state genetic algorithm, at each time one new individual is created which
replaces one individual of the old population (usually the worst one).

1.2.9: Recombination versus mutation

● The recombination operator causes new individuals to depend on the whole population.
○ Its utility relies on the schemata-theorem.
■ If the crossover operator does not destroy good building blocks too often,
they can be quickly mixed and stay in the population.
● An individual consisting of two good building blocks (schemata) is
likely to have a higher fitness value and is more likely to propagate
genetic material.
○ The crossover operators exploits previously found genetic material and leads to faster
convergence.
○ With less diverse populations, the effect of crossover diminishes.
● The mutation operator allows a population to escape from a single local minimum.
○ Allows values of locations which have been lost to be reinserted again.

Genetic algorithms and evolutionary strategies

● The choice to use crossover or not depends on…
○ Is the fitness function separable in additive components?
■ In case of separable fitness functions, the use of recombination can lead to
much faster search times for optimal solutions.
○ Are there building blocks?
■ If there are no real building blocks, then crossover does not make sense.
○ Is there a semantically meaningful recombination operator?
■ If recombination is meaningful it should be used.
The Robotics Primer

1.3: Genetic Programming

● The main element of Genetic Programming (GP) is the use of functional/program trees which
are used to map inputs to outputs.
○ By evolving functional program trees, those programs which work best for the task at
hand will remain in the population and reproduce.
● A program tree may consist of a large number of functions and random constants.
○ The functions have a fixed number of inputs.
● To make a program tree legal, functions which require n arguments (n-ary functions) should
have n branches to child-nodes where each child-node is filled in by another function or
variable.
○ The leaf nodes of the tree are input-variables or random constants.

● GP is flexible for its use of functions and and primitive building blocks.
● Like in GAs, one has to devise mutation and crossover operators for program trees.

1.3.1: Mutation in GP
● The mutation operator can adjust a node in the tree.

● For point-mutations, one only allows mutating a terminal to a different terminal and a
function to a different function of the same arity.

1.3.2: Recombination in GP
● The recombination operator works on program trees.
● Particular subtrees are cut from the main program trees for both parent individuals.
The Robotics Primer

● These subtress are exchanged.

1.3.3: Probabilistic incremental program evolution

● One could use generative prototypes which generate individuals according to some
probability distribution.
● Population Based Incremental Learning (PBIL) encodes a chromosome for generating
bitstrings.
○ The chromosome consists of probabilities for generating 1 on a specific location and
0 for minus that probability.
○ Individuals can be generated and evaluated.
○ The prototype chromosome can be adjusted towards the best individual so that it will
generate solutions around the best individuals with higher probability.
● This idea was pursued by Rafal Salustowicz for transforming populations of program trees in a
representation using a probabilistic program tree (PPT).
○ The idea is known as Probabilistic Incremental Program Evolution (PIPE) and it uses
probabilities to generate functions in a particular node.
○ The PPT which is used for generating program trees consists of a single large tree
consisting of probabilities of functions in each node.

● The PPT is used to generate an individual as follows…

○ Start at the root node and select a function according to the probabilities.
○ Go to the subtrees of the PPT to generate the necessary arguments for the previously
generated functions.
○ Repeat until the program is finished.
● PIPE repeats the following steps.
○ Generate N individuals with the prototype tree.
The Robotics Primer

○ Evaluate these N individuals.

○ Select the best individual and increase the probabilities of the functions and
terminals used by this best individual.
○ Mutate the probabilities of the PPT a little bit.

1.4: Memetic Algorithms

● Memetic algorithms are inspired by memes (pieces of mental ideas) which reproduce
(propagate) themselves through a population of meme carriers.
○ Each meme uses the host (individual) to propagate itself further through the
population.
○ It competes with different memes for the limited resources.
● Each transmitted meme can be changed according to receiving more information from the
environment.
○ This makes it possible to locally optimise each different meme before it is transmitted
to other individuals.
○ How can we optimise a meme or individual?
● Memetic algorithms do not have the problem of fine-tuning a good solution to make an
optimal one.
● Memetic algorithms have the advantage that all indivudals in a population are in local
maxima.

1.5: Discussion

● A problem with evolutionary algorithms is that sometimes the population converges

prematurely to a suboptimal local minimum.
○ Aot of research effort has come up with methods for keeping diversity during the
evolution.
● Many individuals are evaluated and then never used anymore, which seems a waste of
computer power.
● The learning progress can be quite slow for some problems and if many individuals have the
same fitness value there is not much selective pressure.
● The fitness function should be designed in a way to provide maximal informative information.

Artificial Life
● Artificial Life is a broad interdisciplinary field where research runs from biology, chemistry,
and physics to computer science and engineering.
● One of the most ambitious goals is to study the principles of life itself.
○ To study these properties there are two roads.
1. To study carbon life forms and their development.
2. To examine life forms and their properties using a computer.
● Kufe emerges from building blocks which cannot be called alive on their own.
The Robotics Primer

○ The interaction between the elements makes the whole system appear to be alive.
● The study of artificial life is holistic, which means we look at the whole system without being
able to make clear separations in smaller modules.
● Artificial life constructs models and simulates them to study living entities or other complex
systems in computer systems.
● One goal of artificial life is to understand the source and functionality of life.
○ This can involve making computer programs which simulate organisms using some
encoding.
● There are many possible definitions of “alive.”

1.1: Genetic Algorithms and Artificial Life

● In combination with artificial life, genetic algorithms (GA) allow us to study a wide variety of
topics…
○ Robots that interact with an environment to solve some tasks.
○ Competitive evolutionary models.
○ Models of social systems.
○ Economical models.
○ Population genetics models where one examines which groups of genes remain in
the population.
○ The study of the interaction between learning and evolution.

1.1.1: Interaction between evolution and learning

● In evolutionary theory, sociology, and psychology, they consider the difference between
nature and nurture.
○ Nature is what a newborn organism possesses at its birth.
○ Nurture is the knowledge, skills, and behaviours which an organism develops through
its adaption and learning process while interacting with an environment.
● The nature/nurture dilemma is to determine whether something was born inside an
organism or whether it developed due to the interaction with the environment.
○ It is better to say that nature gives a bias towards some behaviour or the other, and
nurture causes some behaviours to be expressed.
● In artificial life simulations, machine learning algorithms can be used to learn from the
interaction with the world.
○ They can be used together with GAs in an environment consisting of many entities
that interact and evolve.
● When studying the interaction between evolution and learning…
○ Evolution is very slow and takes place over generations of individuals.
○ Learning is very fast and takes place within an individual (agent).
● The combination of the two leads to two possible effects.
1. Baldwin Effect: The individual learns during its interaction with the environment.
The Robotics Primer

a. This learning may increase the fitness of the individual so that individuals
who are good at learning may receive higher fitness values than slow-
learning individuals.
b. Individuals who are good at learning may reproduce with a higher probability
leading to offspring who are potentially also good at learning.
c. Although the skill of learning is propagated to offspring, learned knowledge is
not immediately propagated to the offspring.
2. Lamarckian Learning: The individual learns during their life, and when it gets
offspring, it propagates its learned knowledge to their children, who then do not
have to learn this knowledge anymore.
a. Biologically not very realistic.
b. In computer programs, it’s very feasible.

1.2: Cellular Automata

● Cellular automata (CA) are decentralised spatial systems with a large number of simple,
identical components which are locally connected.
○ They are suited for visualising processes.
○ Despite consisting of simple components and some simple rules, they can show very
complex behaviours.
● CA are used in several fields, including…
○ Fluid dynamics.
○ Galaxy formation.
○ Earthquakes.
○ Biological pattern formation.
○ Forest fires.
○ Traffic models.
○ Emergent cooperative and collective behaviour.

1.2.1: Formal description of CA

● A CA consists of two components.
1. Cellular space: This consists of a lattice of N identical cells.
a. Usually, all cells have the same local connectivity to other cells.
2. Transition rule: This gives an update for a cell to its next state as a function of its
neighbourhood.
a. Usually, all cells are synchronously updated.
b. This rule is often implemented as a lookup table.
The Robotics Primer

1.2.2: Example CA

● This CA has periodic boundary conditions.

○ The periodic boundary conditions mean that the most left state has the most right
state as its left neighbour, and vice versa.

1.2.3: Dynamics of the CA

● Iterating over time and showing the dynamics in space-time dimensions, a simple CA can
show complex behaviour.
● A CA with more complex transition rules and a larger number of possible states can show
much more complex behaviour.

1.2.4: Processes in CA
● When we have bounded spaces, we can divide a process resulting in a pattern into three
different classes: Stable, periodic, and chaotic.
● The set of possible complete states of a CA is not only bounded but finite.
● The three possible resulting patterns of a CA are…
1. A stable pattern/point: After entering the stable state, the process remains in the
same state and the change stops.
2. A cyclic pattern: The CA traverses through a repeating pattern of some periodic
length.
a. If there are multiple sub-patterns each with their own periodic length, the
complete pattern will be periodic but with a larger length.
3. A chaotic behaviour: The CA always goes to new, unseen patterns.
a. Since the CA is deterministic, chaotic behaviour is possible.
b. After finite time, there will always be a state which has been seen before
after which the process repeats the same cycle of configurations.
c. Real chaotic behaviour in a CA is not possible; it is only a periodic cycle of a
very large length.
● An initial configuration may lead to a sequence of patterns which are all different, after which
it may enter a stable state or a periodic cycle
○ The time until the CA enters a stable state or periodic cycle is called the transient
period.
The Robotics Primer

● The dynamics of CA can be influenced by the transition rules.

1.2.5: Examples of Cyclic Processes

● Transition rule: If 2 neighbours are active, then the cell is activated (1/black), otherwise the
cell is not activated (0/white).
● This results in a repeating pattern with periodic length 2.

1.2.6: Elimination of basis patterns

● When one evolves a CA, there are often some regularities involved and other parts which are
completely unpredictable.
○ Some researchers have tried to use methods for eliminating the basis of the
evolutionary transitions in a CA.
○ This basis can consist of walls, singularities, and so on, and can be then eliminated
from the process.
● The importance of eliminating the basic patterns is to get more inside possible chaotic or
turbulent processes.
The Robotics Primer

1.2.7: Research in CA
● CA are universal machines.
○ Universal machines can compute any computable function and are just as powerful
as Turing Machines.
○ Any algorithm which can be implemented on the usual sequential computer can in
principle also be implemented in a CA.

Conway’s game of life

● Conway’s game of life balances the tendencies of cells dying too fast and too many cells
being born at a time, making it hard to tell whether a pattern will die out completely, form a
stable population, or grow forever.
● Conways’ game of life uses a 2-dimensional lattice with 8 neighbours for each cell.
● The transition rules are…
○ If a cell is not active (dead/black/1) and it has exactly 3 living neighbours, then the
cell will become active (rule of birth).
○ If a cell is active and it has 2 or 3 neighbours which are active, then the cell stays
active (rule of survival).
○ In all other cases, the cell becomes not active (rule of death due to overcrowding or
loneliness).
● One of the interesting things about the game of life is that it has universal computing power.
○ They rely on particular patterns known as gliders.
○ Gliders are living entities which cross the 2D lattice and can pass information so that
it becomes possible to make logical AND and NOT gates.

● Glider guns can fire gliders and remain stable, making it possible to propagate information at
some rate.
○ By using multiple glider guns that shoot gliders, we can make interactions between
different patterns which are propagated in the cellular space.
● Making a CA using the game of life rules to compute arbitrary functions is very complicated
because it requires very careful development of the initial configuration consisting of glider
guns and other patterns.

Development of cellular automata

● One goal of artificial life is to make artificial systems which can be called “alive.”
The Robotics Primer

○ Reproduction seems necessary.

● 1966: CA which was able to reproduce itself.
● 1972: CA which could recognize context-sensitive languages.
● 1994: GAs evolve the transition rules of CA, using the majority problem as a testbed.
○ In the majority problem, a bitstring is given of some size and each bit of the string
can be on or off.
■ The system should tell whether the majority of bits are on or off.

1.3: Ecological Models

● Simulation models often make use of CA due to their insightfulness and easy implementation
while still providing interesting and complex behaviours.
● Ecological models can be used to study social phenomena, immunology and epidemics,
population dynamics of different species, and so on…
● An artificial ecosystem consists of several individuals (agents) which…
○ Occupy a position in the environment.
○ Interact with the environment and with other agents.
○ Possess some internal state such as amount of energy or money.
● By examining the evolutionary process in an ecosystem, it is possible to research the creation
and continuity of processes such as…
○ Cooperation.
○ Competition.
○ Imitation.
○ Parasitic behaviour: An individual profits from another individual whereas the other
individual is harmed by this.
○ Communities: If a large group of individuals are put together they might form
communities for the benefit of all.
■ Individuals can cooperate and specialise in their tasks.

1.3.1: Strategic Bugs

● 1992: Strategic bugs, a model of an ecosystem, use individuals who try to find food and
reproduce.
● The model consists of…
○ An environment modelled as a 2-dimensional lattice.
○ A cell in the environment can be occupied by food or by a bug or is empty.
○ Food will grow automatically in the environment.
■ Food is added in a cell with some probability if there is no food or bug there.
○ Bugs survive by finding food.
○ Bugs use energy to move and die if they do not have any energy anymore.
○ Bugs can clone themselves or reproduce with another bug if they have sufficient
energy.
● The behaviour of a bug evolves from the interaction of the policy of the bug and the
environment.
The Robotics Primer

○ The bug’s policy uses a lookup table to map environment inputs to actions.

1.4: Artificial Market Models

● Financial markets such as stock markets are difficult to predict.
○ It is a chaotic process emerging from a large number of investors and unforeseen
circumstances.
● Under what conditions will predictions about the dynamics of financial markets be possible?
○ To study this question, we have to look at the efficient market hypothesis (EMH),
which states that in an information-efficient market, all price fluctuations are
unpredictable if all necessary investment information is taken into account by the
investors.

1.4.1: Are real markets predictable?

● When we examine the efficient market hypotheses, it requires rational and completely
informed investors.
○ These assumptions are not realistic.
○ Investors are not completely rational and sometimes hard to predict.
○ Information is difficult to interpret, technologies and companies change, and there
are costs associated with transactions and information gathering.

1.4.2: Models of financial theories

● We can only show general approaches that have been intended to come up with models to
predict price fluctuations.
○ Psychological models: The model tries to analyse the risk-taking behaviour of
investors and examines how human attitudes to the market influence stock prices.
○ Learning models: Data about the stock prices of the past is used to train a model to
predict its development in the future.
○ Agent models: Investors are modelled as agents which use particular strategies.
■ By letting the modelled agents interact the complex dynamic of stock
markets can be simulated.
○ Evolutionary algorithms for developing strategies: The evolution of strategies of
investors is mimicked.
■ Competitive strategies could be used to create other strategies.
■ Finally, a strategy which was observed to gain the most money in the past
could be used to trade in the future.

1.5: Artificial Art and Fractals

● Iterating a simple function can create very complex, artistic patterns.
● A fractal is a pattern which is self-similar to different scales
The Robotics Primer

○ If we look at a zoomed-out picture of some details of the fractal we can recognize

features which were also shown in the bigger pattern.

● Computer artists like to use fractals since although the equations are simple, as long as they
are non-linear, they can produce a large variety of complex patterns, and zooming in the
pictures creates many other patterns.

1.6: Conclusion
● Artificial life is useful for simulating many biological, physical, sociological, and economic
processes.
○ One goal of artificial life is to understand the principles underlying living entities and
the emergence of life forms.
○ Artificial life can be combined with genetic algorithms for optimising individual
behaviours by adapting them to the (changing) environment.
● If multiple individuals adapt themselves and also adapt to the environment, the resulting
dynamics can be very complex and unpredictable.
● Even with simple entities such as those used in cellular automata, complex behaviour can
result from the interaction between simple components.
● Cellular automata are very useful for modelling and visualizing spatial processes.
● Cellular automata are just as powerful as Turing machines

Chapter 4: Arms, Legs, Wheels, Tracks, and

What Really Drives Them
Effectors and Actuators

● An effector is any device on a robot that has an effect (impact/influence) on the

environment.
○ The robot’s controller sends commands to the robot’s effectors to produce the
desired effect on the environment, based on the robot’s task.
○ Effectors must have some mechanism that allows them to do their work.
○ An actuator is the mechanism that enables the effector to execute an action or
movement.
The Robotics Primer

4.1: Active vs. Passive Actuation

● The action of actuators and effectors requires some form of energy to provide power.
○ Some clever designs use passive actuation, utilising potential energy in the
mechanics of the effector and its interaction with the environment instead of active
power consumption.

4.2: Types of Actuators

● There are many different ways of actuating a robotic effector…
○ Electric motors: The most affordable and simplest to use, based on electric current.
○ Hydraulics: These are based on fluid pressure; as the pressure changes, the actuator
moves.
■ They are powerful and precise, but are also large, potentially dangerous,
must be well-packaged, and must be kept from leaking.
○ Pneumatics: These are based on air pressure; as the pressure changes, the actuator
moves.
■ They are typically large, very powerful, potentially dangerous, and must also
be kept from leaking.
○ Photo reactive material: Some materials perform physical work in response to the
amount of light around them; they are called photo-reactive.
○ Chemically reactive materials: These materials react to certain chemicals.
○ Thermally reactive materials: These materials react to changes in temperature.
○ Piezoelectric materials: These materials - crystals - create electric charges when
pushed/pressed.
○ And more!

4.3: Motors
● Motors are the most common actuators in robotics.
○ They are well suited for actuating wheels, since they provide rotational movement,
thereby enabling wheels to turn.
○ Wheels are a popular effector!

4.3.1: Direct-Current (DC) Motors

● Direct current (DC) motors are simple, inexpensive, easy to use, and easy to find.
○ They convert electrical energy into mechanical energy.
○ They use magnets, loops of wire, and current to generate magnetic fields whose
interaction turns the motor shaft.
The Robotics Primer

● To make a motor run, you need to provide it with electrical power in the right voltage range.
○ If the voltage is low, the motor will still run, but will have less power.
○ If the voltage is high, the power of the motor is increased, but wear and tear will
make it break down sooner.
○ When the motor is provided with nice constant voltage in the right range, it draws
current in the amount proportional to the work it is doing.
■ Work is the produce of force and distance.
● The more current the motor uses, the more torque (rotational force) is produced at the
motor shaft.
○ The amount of power a motor can generate is proportional to its torque.
○ The amount of power is proportional to the rotational velocity of the shaft.
● When the motor is spinning freely, with nothing attached to its shaft, then its rotational
velocity is the highest but the torque is zero.
○ The output power is zero.
● When the motor is stalled, its torque is the maximum it can produce but the rotational
velocity is zero, so the output power is again zero.

4.3.2: Gearing
● Combining different gears is used to change the force and torque output of motors.
● The force generated at the edge of a gear is the ratio of the torque to the radius of the gear.
○ We can manipulate the amount of force and torque that gets generated.
The Robotics Primer

● If the output gear is larger than the input gear…

○ The torque increases.
○ The speed decreases.
● If the output gear is smaller than the input gear…
○ The torque decreases.
○ The speed increases.
● Gears are combined by “the skin of” their teeth.
○ Gear teeth require special design so that they mesh properly.
○ Any looseness between meshing gears, called backlash, makes the gear mechanism
move sloppily back and forth between the teeth, without turning the whole gear.
■ Backlash adds error to the positioning of the gear mechanism, which is bad
for the robot because it won’t know exactly where it is positioned.
■ Reducing backlash requires tight meshing between the gear teeth, but that,
in turn, increases friction between the gears, which wastes energy and
decreases the efficiency of the mechanism.
● Gears can be organised in a series or “ganged” in order to multiply their effect.
○ The method of multiplying reduction is the underlying mechanism that makes DC
motors useful and ubiquitous (commonplace), because they can be designed for
various speeds and torques, depending on the robot’s tasks.

4.3.3: Servo Motors

● DC motors are great at rotating continuously in one direction.
● It is often useful for a robot to move a motor and some effector attached to it to a particular
position.
● Motors that can turn their shaft to a specific position are called servo motors or servos for
short, because they can servo to a particular position.
The Robotics Primer

● Servo motors are made from DC motors by adding the following components…
○ Some gear reduction.
○ A position sensor for the motor shaft, to track how much the motor is turning, and in
what direction.
○ An electronic circuit that controls the motor, in order to tell it how much to turn and
in what direction.
● Servo motor operation is all about getting the shaft of the motor in the desired position.
○ The amount of the turn within the zero to 180 degree range is specified by an
electronic signal.
■ The signal is produced as a series of pulses.
■ When the pulse arrives, the shaft of the motor turns.
■ When no pulse arrives, the shaft of the motor stops.
■ The up-and-down pattern of the pulses produces a wave-like pattern, called
the waveform.
● The amount the motor shaft turns when the pulse arrives is determined by the duration of
the pulse.
○ The longer the pulse, the larger the turn angle.
■ This is called pulse-width modulation because the width (duration) of the
pulse modulates the signal.
● Most robot actuators use position control in which the motor is driven so as to track the
desired position at all times.
○ The result is an accurate stiff actuator.
● One alternative is to use torque control, in which the motor is driven so as to track the
desired torque at all times, regardless of its specific shaft position.
○ The result is an actuator that is less stiff.

4.4: Degrees of Freedom

● A degree of freedom (DOF) is any of the minimum number of coordinates require to

completely specify the motion of a mechanical system.
○ It can be thought of as a way in which the system (robot) can move.
○ How many DOF a robot has is important in determining how it can impact its world
and how well it can accomplish its task.
The Robotics Primer

● In general, a free body in 3D space has a total of six DOF.

○ Three of those are called translational DOF, as they allow the body t translate,
meaning move without turning.
○ The other three are called rotational DOF, as they allow the body to rotate (turn).
■ They are called roll, pitch, and yaw.
● An effector can have any number of DOF, starting with one.
● If a robot has an actuator for every DOF, then all of the DOF are controllable.
● The DOF that are not controllable are called uncontrollable DOF.
○ This creates problems for the controller in that they make movement more
complicated.
● The ratio of controllable DOF to total DOF on a robot tells us quite a bit about how easy it is
to control its movement.
○ CDOF = TDOF: The robot is holonomic.
○ CDOF < TDOF: The robot is nonholonomic.
○ CDOF > TDOF: The robot is redundant.
● In general, effectors are used for two basic purposes:
1. Locomotion: To move the robot around.
2. Manipulation: To move other objects around.

Chapter 5: Move It!

Locomotion

● Locomotion refers to the way a body moves from place to place.

● Moving around presents all kinds of challenges.
● Legged locomotion is a more difficult robotic problem compared with wheel location,
because…
1. The comparatively larger number of DOF.
2. The challenge of stability: It is harder to stay stable on legs than it is on wheels.
The Robotics Primer

5.1: Stability
● Most robots need to be stable, meaning they do not wobble, lean, and fall over easily, to get
their job done.
● There are two kinds of stability: Static and dynamic.
○ A statically stable robot can stand still without falling over; it can be static and stable.
● The centre of gravity (COG) of anybody needs to be above the area that is covered by the
ground points (legs or wheels).
○ When it is, the body is balanced and stays in the upright position.
○ When it is not, the body is dragged down, falling over.
○ The area covered by the ground points is called the polygon of support.
■ You can draw a projection of the body onto the surface and trace the shape
or contour of the support points, which will result in a polygon.

● If the robot can walk while staying balanced at all times, we call this statistically stable
walking.
○ This is very safe, but also very slow and energy inefficient.
○ As a robot designer, you have to think about how important continued static stability
is to your robot and if it is worth the time and effort it entails.
● In dynamic stability, the body must actively balance or move to remain stable; it is called
dynamically stable.
● Balancing one-legged robots is formally called the inverse pendulum problem because it is
the same as trying to balance a pendulum on one’s finger and is equally hard.

5.2: Moving and Gaits

● A gait is the particular way a robot moves, including the order in which it lifts and lowers its
legs and places its feet on the ground.
● Desirable robot gaits have the following properties…
○ Stability: The robot does not fall over.
○ Speed: The robot can move quickly.
The Robotics Primer

○ Energy efficiency: The robot does not use a great deal of energy to move.
○ Robustness: The gait can recover from some types of failures.
○ Simplicity: The controller for generating the gait is not unwieldy.
● Not all of the above requirements can be achieved with all the robots.
○ Safety requirements compromise energy conservation, robustness requirements
compromise simplicity, and so on.

● Six legs allow for multiple walking gaits, both statically and dynamically stable.
○ The tripod gait is a statically stable gait in which three legs stay on the ground,
forming a tripod, while the other three lift and move.
○ If the sets of three legs are alternating, the gait is called the alternating tripod gait
and produces efficient walking that can be found in various bugs.
● If a robot has more than six legs, the alternating tripod gait can still be used, in the form of
the ripple gait, because it ripples down the length of the body.

5.3: Wheels and Steering

● Wheels are more efficient than legs.
○ They are the locomotion effector of choice in robotics.
● Wheeled robots are built to be statically stable.
○ They are not necessarily holonomic.
● Having multiple wheels means there are multiple ways in which those wheels can be
controlled, either together or independently.
○ The ability to drive wheels separately and independently, through the use of separate
motors, is called a differential drive.
○ Being able to steer wheels independently is called differential steering.
The Robotics Primer

5.4: Staying on the Path vs. Getting There

● In robot locomotion, we may be concerned with…
○ Getting the robot to a particular location.
○ Having the robot follow a particular path (trajectory).
● Following an arbitrary given path/trajectory is harder than having to get to a particular
destination by using any path.
○ Some paths are impossible to follow for some robots because of their holonomic
constraints.
○ Other paths can be followed, but only if the robot is allowed to stop, change
directions (in place/otherwise), and then go again.
● Trajectory planning/motion planning is a computationally complex process which involves
searching through all possible trajectories and evaluating them, to find one that will satisfy
the requirements.
○ Depending on the task, it may be necessary to find the very best optimal trajectory.
○ Since robots are not just points, their geometry and steering mechanism must be
taken into account.
● Depending on the task, practical robots may not be so concerned with following specific
trajectories as with just getting to the goal location.
○ The ability to get to the goal is quite a different problem from planning a particular
path and is called navigation.

Chapter 11: The Building Blocks of Control

Control Architecture

● The job of the controller is to provide the brains for the robot so it can be autonomous and
achieve goals.
○ Feedback control is a good way to write controllers for making a robot perform a
single behaviour.
■ This behaviour requires little thinking.
○ Most robots have more complex, non-simple tasks.
■ Having to do multiple things at once and deciding what to do at any point in
time is not simple.
● Putting together controllers that will get the robot to produce the desired overall behaviour is
not simple… but is what robot control is really all about.

11.1: Who Needs Control Architecture?

● A robot control architecture provides guiding principles and constraints for organising a
robot’s control system (brain).
○ Helps the designer to program the robot in a way that will produce the desired
overall output behaviour.
The Robotics Primer

○ There is a set of building blocks/tools at your disposal to make the job of robot
control design easier.
○ Uses specific styles, tools, constraints, and rules.
● The robot’s “brain” can be…
○ Implemented with a conventional program running on a microprocessor.
○ Embedded in hardware.
○ Both!
● The robot’s controller does not need to be a single program on a single processor.
○ There is a lot going on in most robots.
○ Sensors, actuators, and decisions need to interact in an effective way to get the robot
to do its job.
○ There is typically no good reason to have all the elements controlled from a single
centralized program.
● Hardware is good for fast and specialized uses, and software is good for flexible, more general
programs.
○ Complicated robot brains typically involve computer programs of some type or
another running in real-time.
○ The brain should be physically on the robot.
■ Biological prejudice.
■ Quick and reliable communication.
○ If wireless radio communication is reliable enough, some or all of the processing
could reside off the robot.
● Brains use their programs to solve problems that stand in the way of achieving their goals and
getting their jobs done.
○ The process of solving a problem using a finite step-by-step procedure is called an
algorithm.
■ Structure on which computer programs are based.

11.2: Languages for Programming Robots

● There is no “best” language for programming robots.
○ There is a variety of languages dependent on…
■ What they are trying to get the robot to do.
■ What they are used to.
■ What hardware comes with the robot.
○ Robot controllers can be implemented in various languages.
● Any worthy programming language is called “Turing universal.”
○ It can be used to write any program.
● To be Turing universal, a programming language has to have the following capabilities…
○ Sequencing.
○ Conditional branching.
○ Iteration.
● With those capabilities, any language can compute everything that is computable.
● All programming languages are just tools.
The Robotics Primer

○ You can use them to program various things.

○ You can use any language to write any program (in theory).
● Some languages are better for programming robots than others.

11.3: And the Architectures are…

● Effective robot programs fall into one of the known types of control architecture.
1. Deliberative control.
2. Reactive control.
3. Hybrid control.
4. Behaviour-based control.
● All new architectures and all robot programs fit into one of the categories.
● It is impossible to tell what control architecture a robot is using, based on observation.
○ Multiple architectures can get the job done, especially for simple robots.
● For deciding on architecture, there are many things to be considered.
○ Is there a lot of sensor noise?
○ Does the environment change or stay static?
○ Can the robot sense all the information it needs? If not, how much can it sense?
○ How quickly can the robot sense?
○ How quickly can the robot act?
○ Is there a lot of actuator noise?
○ Does the robot need to remember the past in order to get the job done?
○ Does the robot need to think into the future and predict in order to get the job done?
○ Does the robot need to improve its behaviour over time and be able to learn new
things?
● Control architecture differs fundamentally in the ways they treat…
The Robotics Primer

○ Time: How fast do things happen? Do all components of the controller run at the
same speed?
○ Modularity: What are the components of the control system? What can talk to what?
○ Representation: What does the robot know and keep in its brain?

11.3.1: Time
● Time scale (time) refers to how quickly the robot has to respond to the environment
compared with how quickly it can sense and think.
○ A key aspect of control.
○ A major influence on the choice of what architecture should be used.
● The four basic architecture types differ significantly in how they treat time.
○ Deliberative control looks into the future.
■ It works on a long-time scale.
○ Reactive control responds to the immediate, real-time demands of the environment
without looking into the past or future.
■ It works on a short-time scale.
○ Hybrid control combines the long-time scale of deliberative control and the short-
time scale of reactive control, with some cleverness in between.
○ Behaviour-based control works to bring the time scales together.

11.3.2: Modularity
● Modularity refers to the way the control system is broken into pieces or components
(modules) and how those modules interact with each other to produce the robot’s overall
behaviour.
● The four basic architecture types differ significantly in how they treat modularity.
○ Deliberative control: The control system consists of multiple modules, and the
modules do their work in sequence, with the output of one providing the input for
the next.
■ Things happen one at a time, not at the same time.
○ Reactive control: Things happen at the same time, not one at a time.
■ Multiple modules are all active in parallel and can send messages to each
other in various ways.
○ Hybrid control: There are three main modules to the system that all work in parallel,
at the same time, but also talk to each other..
1. Deliberative part.
2. Reactive part.
3. Part in-between.
○ Behaviour-based control: There are more than three main modules that also work in
parallel and talk to each other but in a different way than in hybrid systems.

11.3.3: Representation
● Representation is tough to summarize…
The Robotics Primer

Chapter 13: Think Hard, Act Later

Deliberative Control

● Deliberation refers to thinking hard.

○ It is defined as “thoughtfulness in decision and action.”
● In AI, deliberative systems were (and sometimes still are) used to solve problems where
thinking hard is exactly the right thing to do.
○ Taking time to consider the possible outcomes of several actions is both affordable
and necessary.
○ 1960s-70s: AI researchers theorise that the human brain works this way.

13.1: What is Planning?

● Planning is the process of looking ahead at the outcomes of possible actions, and searching
for the sequence of actions that will reach the desired goal.
● Search is an inherent part of planning.
○ It involves looking through the available representation “in search of” the goal state.
○ Sometimes, searching for the complete representation is necessary.
○ Other times, only a partial search is enough to reach the first found solution.
The Robotics Primer

● The process of planning involves the robot trying different turns at each junction, until a path
leads it to the battery.
● Usually the shortest path is considered the best, since the robot uses the least time and
battery power to reach it.
● In some other cases, other criteria may be used…
○ Which path is the safest?
○ Which path is the least crowded?
● The processing ofi mproving a solution to a problem by finding a better one is called
optimisation.
○ Optimisation criteria are various values or properties of a given problem that can be
optimised.
○ Usually, some optimisation criteria conflict, so deciding what and how to potimise is
not simple.
○ Optimising search looks for multiple solutions, in some cases all possible paths.
● In order to use search and plan a solution to a particular problem, it is necessary to represent
the world as a set of states.
● Then, search is performed to find a path that can take the robot from the current state to the
goal state.
○ If the robot wants to find the very best, optimal path, it has to search for all possible
paths and pick the one that is optimal based on the selected optimised criteria.

13.2: Costs of Planning

● As the number of possible states become large, planning becomes slow.
○ The longer it takes to plan, the longer it takes to solve the problem.
● A robot must be able to avoid immediate danger.
○ If path planning takes too long, the robot either…
■ Has to stop and wait for planning to finish before moving on.
The Robotics Primer

■ Risk collisions or running into blocked paths if it forges ahead without a

finished plan.
● Whenever a large state space is involved, planning is difficult.
● One opopular approach to speed things up is to use hierarchies of states, where first only a
small number of “large,” “coarse,” or “abstract” states is considered.
○ After that, more refined and detailed states are used in the parts of the state space
where it really matters.

● There is still a limit to what can be done in real time, the time in which a physical robot
moves around in a dynamic environment.
● Deliberative, planner-based architectures involve three steps that need to be performed in
sequence.
1. Sensing (S)
2. Planning (P)
3. Acting (A): Executing the plan.
● Deliberative architectures - SPA (sense-plan-act) architectures - have serious drawbacks for
robotics.

Drawback 1: Time scale

● It can a very long time to search in large state spaces.
● The combined inputs from sensors in themselves constitute a large state space.
○ When combined with internal models or representations, the result is a state space
that is large and slow to search.
● If the planning process is slow compared with the robot’s moving speed, it has to stop and
wait for the plan to be finished, in order to be safe.
● To make progress, it is best to plan as rarely as possible and move as much as possible in
between.
○ This encourages open loop control, which is bad for dynamic environments.
● If planning is fast, then executing need not be open loop, since replanning can be done at
each step.
The Robotics Primer

○ Unfortunately, this is typically impossible for real-world problems and robots.

Drawback 2: Time scale

● It may take a great deal of space to represent and manipulate the robot’s state space
representation.
○ The representation must contain all information needed for planning and
optimisation.
● Space is not as much of a problem as time, but all memory is finite, and some algorithms can
run out of it.

Drawback 3: Information
● The planner assumes that the representation of the state space is accurate and up-to-date.
○ If the representation is not accurate and updated, the resulting plan is useless.
● The representation used by the planner must be updated and checked as often as necessary
to keep it sufficiently accurate for the task.
● The more information, the better!

Drawback 4: Use of Plans

● Any accurate plan is useful only if…
○ The environment does not change during the execution of the plan in a way that
affects the plan
○ The robot knows what state of the world and of the plan it is in at all times
○ The robot’s effectors are accurate enough to execute each step of the plan in order to
make the next step possible.
● 1980s-Present: Purely deliberative architectures are no longer used for the majority of
physical robots.
○ It is mostly rendered impractical.
● The SPA approach has not been abandoned in robotics - it has been expanded.
● The following improvements have been made…
○ Search/planning is slow, so save/cache important and/or urgent decisions.
○ Open loop plan execution is bad, so use closed loop feedback, and be ready to
respond or replan when the plan fails.

Chapter 14: Don’t Think, React!

Reactive Control

● Reactive control is one of the most commonly used methods for robot control.
○ It is based on a tight connection between the robot’s sensors and effectors.
● Purely reactive systems do not use any internal representations of the environment and do
not look ahead at the possible outcomes of their actions.
○ They operate on a short time scale and react to the current sensory information.
The Robotics Primer

● Reactive systems use a direct mapping between sensors and effectors, and minimally state
information (if any).
○ Consists of collections of rules that couple specific situations to specific actions.
● Think of reactive rules as being similar to reflexes, innate responses that do not involve any
thinking.
● Reactive systems consist of a set of situations (stimuli/conditions) and a set of actions
(responses/actions/behaviours).
○ The situations may be based on sensory inputs or on internal state.
○ Reactive rules can be much more complex, involving arbitrary combinations of
external inputs and internal states.

● The best way to keep a reactive system simple and straightforward is to have each unique
situation (state) that can be detected by the robot’s sensors trigger only one unique action of
the robot.
○ The conditions are said to be mutually exclusive, meaning that they exclude one
another.
■ Only one can be true at a time.
● It is often too difficult to split up all possible situations (world states) in this way.
○ Doing so may require unnecessary encoding.
● To ensure mutually exclusive conditions, the controller must encode rules for all possible
sensory input combinations.
○ As the sensors grow in complexity and number, the combinatorial space of all
possible sensory inputs quickly becomes unwieldy - intractable.
● To do a complete reactive system, the entire state space of the robot should be uniquely
coupled or mapped to appropriate actions.
○ This results in the complete control space for the robot.
● The design of the reactive system is coming up with this complete set of rules.
The Robotics Primer

○ This is done at “design-time,” not at “run-time” - when the robot is active.

○ It takes a lot of thinking by the designer, but no thinking by the robot.
● Complete mappings between the entire state space and all possible responses are not used in
manually-designed reactive systems.
○ The programmer/designer identifies the important situations and writes the rules for
those.
○ The rest are covered with default responses.

14.1: Action Selection

● Action selection is the processing of deciding amount multiple possible actions or

behaviours.
○ It may select only one output action (arbitration) or may combine the actions to
produce a result (fusion).
■ Command arbitration is the process of selecting one action or behaviour
from multiple candidates.
■ Command fusion is the process of combining multiple candidate actions or
behaviours into a single output action/behaviour for the robot.
● Action selection is a major problem in robotics, beyond reactive systems.
○ There is a great deal of work on different methods for command arbitration and
fusion.
● Reactive systems must be able to support parallelism, the ability to monitor and execute
multiple rules at once.
○ This means that the underlying programming languages must have the ability to
multitask - to execute several processes/rules/commands in parallel.
○ The ability to multitask is critical in reactive systems.
The Robotics Primer

○ If a system cannot monitor its sensors in parallel and instead checks them in
sequence, it may miss an event or onset of an event, and fail to react in time.

14.2. Subsumption Architecture

● The basic idea behind Subsumption Architecture is to build systems incrementally - from the
simple parts to the more complex - all the while using the already existing components as
much as possible in the new stuff being added.
● Subsumption systems consist of a collection of modules or layers, each of which achieves a
task.
○ All of the task-achieving layers work at the same time, instead of in sequence.
○ This means the rules for each of them are ready to be executed at any time,
whenever the right situation presents itself.
● The modules or layers are designed and added to the robot incrementally.
○ Higher layers can temporarily disable one or more layers below them.
1. The inputs of a layer/module may be suppressed.; this way the module
● The module receives no sensory inputs.
● It computes no reactions.
● It sends no outputs to effectors or other modules.
2. The outputs of a layer/module may be inhibited.
● The module receives sensory inputs and performs its computations.
● It cannot control any effectors or other modules.

● The name “Subsumption Architecture” comes from the idea that higher layers can assume
the existence of the lower ones and the goals they are achieving
○ The higher layers can use the lower ones to help them achieve their own goals, either
by…
■ Using them while they are running.
■ Inhibiting them selectively.
● There are several benefits to the subsumption style of organizing reactive systems.
1. By designing and debugging the system incrementally, we avoid getting bogged down
in the complexity of the overall task of the robot.
The Robotics Primer

2. If any higher-level layers or modules of a subsumption robot fail, the lower-level ones
will still continue to function unaffected.
● The design of subsumption controllers is called bottom-up, because it progresses from the
simpler to the more complex, as layers are added incrementally.

● Building incrementally helps the design and debugging process, and using layers is useful for
modularizing the robot controller.
○ If everything is lumped together, it is hard to design, hard to debug, and hard to
change and improve later.
● The effectiveness of modularity also depends on not having all modules connected to all
others, as that defeats the purpose of dividing them up.
○ In Subsumption Architecture, the goal is to have very few connections between
different layers.
○ The only intended connections are those used for inhibition and suppression.
○ Inside the layers, there are plenty of connections, as multiple rules are put together
to produce a task-achieving behaviour.
● By keeping rules for separate tasks apart, the system becomes more manageable to design
and maintain.
● In Subsumption Architecture, we use strongly coupled connections within layers, and loosely
coupled connections between layers.
○ Dependent on the specifics of the robot, the environment, and the task.

14.3: Herbert, or How to Sequence Behaviours Through

The World
● You can have one layer or module activate another in a sequence, but that is not the way to
go.
The Robotics Primer

○ Coupling between reactive rules or subsumption layers need to be through the

through the environment.
● The guiding principles of Subsumption Architecture are…
○ Systems are built from the bottom up.
○ Components are task-achieving actions/behaviors (not functional modules).
○ Components can be executed in parallel (multitasking).
○ Components are organized in layers.
○ Lowest layers handle the most basic tasks.
○ Newly added components and layers exploit the existing ones.
■ Each component provides and does not disrupt a tight coupling between
sensing and action.
○ There is no use of internal models - "the world is its own best model."

Chapter 15: Think and Act Separately, in

Parallel
Hybrid Control

● Hybrid control involves the combination of reactive and deliberative control within a single
robot control system.
○ Fundamentally different controllers, time scales, and representations must be made
to work together effectively.
● In order to achieve the best of both worlds, a hybrid system typically consists of three
components (layers/modules).
1. A reactive layer.
2. A planner.
3. A layer that links the above two together.
● Hybrid architectures are often called three-layer architectures and hybrid systems, three-
layer systems.

● The middle layer has a hard job because it has to…

The Robotics Primer

○ Compensate for the limitations of both the planner and the reactive system.
○ Reconcile their different time scales.
○ Deal with their different representations.
○ Reconcile any contradictory commands they may send to the robot.
● The main challenge of hybrid control is achieving the right compromise between the
deliberative and reactive parts of the system.
● The methods for handling the above situations and various others are usually implemented in
the “magic middle” layer of a hybrid system.

15.1: Dealing with Changes in the World/Map/Task

● When the reactive system discovers it cannot do its job, it can inform the deliberative layer
about this new development.
○ The deliberative layer can use this information to update its representation of the
world so that it can now, and in the future, generate more accurate and useful plans.
● It is necessary to update the internal model when things change, but also because we already
know that updating internal models and generating plans takes time and computation, so it
cannot be afforded continually.
○ The input from the reactive layer provides an indication of a very good time for such
an update.

15.2: Planning and Replanning

● Whenever the reactive layer discovers that it cannot proceed, this can be used as a signal to
the deliberative layer to do some thinking, in order to generate a new plan.
○ This is called dynamic replanning.
● Not all information flows from the bottom up - from the reactive to the deliberative layer.
The Robotics Primer

○ The deliberative layer provides the path to the goal and gives the robot the directions
to follow, turns to take, distances to go, etc.
○ If the planner is computing while the robot is moving, it may send a message to the
reactive navigation layer to stop, turn around, and head in a different direction
because a better way has been discovered on the map.
● A complete plan - when finished - has the best answer the deliberator can generate.
○ Sometimes, there is not enough time to wait for that full and optimal answer.
○ It is often best to get the robot going in the generally right direction and in the
meantime keep generating a more accurate and detailed plan, and updating the
navigation layer as planned.
● The timing of reactive navigation and deliberative planning is obviously not synched up.

15.3: Avoiding Replanning

● A useful idea researchers in planning had quite a while ago was to remember/save/store
plans so that they would not have to be generated again in the future.
○ This idea is really popular for situations that happen often and need a fast decision.
● This basic idea of storing and reusing mini-plans for repeated situations has been used in
"contingency tables" - lookup tables that tell the robot what to do and pull out a little plan as
a response.
○ This same idea has appeared in the form of fancy terms such as "intermediate-level
actions" and "macro operators."
○ In all cases, it is basically the same thing: plans that are computed and stored for fast
lookup in the future.
● In some cases, the deliberator’s plan is "the law" for the system, while in others the reactive
system merely considers it as advice that can be ignored.
● In the most effective systems, the interaction between thinking and acting is coupled, so that
each can inform and interrupt the other.
○ In order to know who should be in charge and when, it is necessary to consider the
different modes of the system and specify who gets its say.

15.4: On-line and Off-line Planning

● The next good idea is to preplan for all the situations that might come up, and store those
plans ahead of time.
○ This off-line planning takes place while the robot is being developed and does not
have much to worry about, as compared with on-line planning of the kind that a
busy robot has to worry about while it is trying to get its job and its goal achieved.
● A universal plan is a set of all possible plans for all initial states and all goals within the state
space of a particular system.
● If for each situation a robot has a preexisting optimal plan it only has to look up, then it can
always react optimally, and so have both reactive and deliberative capabilities without
deliberating at all.
○ That robot is reactive since the planning is all done offline and not at run-time.
The Robotics Primer

● Another good feature of such precompiled plans is that information can be put into the
system in a clean, principled way.
○ Such information about the robot, the task, and the environment is called domain
knowledge.
○ It is compiled into a reactive controller, so the information does not have to be
reasoned about/planned online, in real-time but instead becomes a set of real-time
reactive rules that can be looked up.
● Situated automata are not real, physical machines but formal ones whose inputs are
connected to abstract sensors, and whose outputs are connected to abstract effetors.
○ To be situated means to exist in a complex world and to interact with it.
○ Automata are computing machines with particular mathematical properties.
● Unfortunately, this is too good to be true for real-world robots…
○ The state space is too large for most realistic problems, so either generating or
storing a universal plan is simply not possible.
○ The world must not change; if it does, new plans need to be generated for the
changed environment.
○ The goals must not change; this is the same as with reactive systems, where if the
goals change, at least some of the rules need to change as well.
● There are drawbacks to hybrid systems…
○ The middle layer is hard to design and implement, and it tends to be very special-
purpose, crafted for the specific robot and task, so it has to be reinvented for almost
every new robot and task.
○ The best of both worlds can end up being the worst of both worlds; if mismanaged, a
hybrid system can degenerate into having the planner slow down the reactive
system, and the reactive system ignores the planner entirely, minimizing the
effectiveness of both.
○ An effective hybrid system is not easy to design or debug, but that is true for any
robot system.

Chapter 16: Think the Way You Act

Behaviour-based Control

● Behaviour-based systems were inspired by several main challenges…

○ Reactive systems are too inflexible and incapable of representation, adaptation, or
learning.
○ Deliberative systems are too slow and cumbersome.
○ Hybrid systems require complex means of interaction among the components.
○ Biology seems to have evolved complexity from simple and consistent components.
The Robotics Primer

● Behaviour-based control (BBC) involves the use of “behaviours” as modules for control.
○ BBC controllers are implemented as collections of behaviours.
○ The first property of BBC to remember is that it is all behaviours.
● One of the strengths of BBC comes from the different ways in which people have encoded
and implemented behaviours, which are also sometimes called behaviour-achieving modules.
● There are some rules of thumb about behaviours and constraints on how to design them and
what to avoid in implementing them…
○ Behaviors achieve and/or maintain particular goals.
○ Behaviors are time-extended, not instantaneous.
■ They take some time to achieve and/or maintain their goals.
○ Behaviours can take inputs from sensors and also from other behaviours and can
send outputs to effectors and to other behaviours.
■ We can create networks of behaviours that "talk to" each other.
○ Behaviors are more complex than actions.
■ While a reactive system may use simple actions like stop and turn-right, a
BBC uses time-extended behaviours like the ones we saw above.
● Behaviours can be designed at a variety of levels of detail or description.
○ This is called their level of abstraction because to abstract is to take details away and
make things less specific.
○ Behaviours can take different amounts of time and computation.
○ They are quite flexible, a key advantage of BBC.
● The power and flexibility of BBC also come from the organisation of those behaviours, from
the way they are put together into a control system.
○ Behaviours are typically executed in parallel/concurrently, much as in reactive
systems, in order to enable the controller to respond immediately when needed.
○ Networks of behaviours are used to store state and to construct world
models/representations.
■ When assembled into distributed representations, behaviours can be used to
store history and to look ahead into the future.
○ Behaviors are designed so that they operate on compatible time scales.
The Robotics Primer

■ This means it is not good for BBC design to have some very fast behaviours
and some very slow ones.
● BBC has the following key properties…
1. The ability to react in real time.
2. The ability to use representations to generate efficient behaviour.
3. The ability to use a uniform structure and representation throughout the system.
● In some BBC systems, the internal behaviour structure exactly matches the externally
manifested behaviours.
● The most interesting observable robot behaviours are a result not just of the control
program, but also of the interaction of the internal behaviors among themselves and with the
environment the robot is in.
○ Simple reactive rules can interact to produce interesting observable behaviour of the
robot.
● BBC systems are typically designed so the effects of the behaviours interact in the
environment rather than internally through the system, in order to take advantage of
interaction dynamics.
○ Those dynamics refer to patterns and history of interaction and change.
○ The idea that rules or behaviours can interact to produce more complex outputs is
called emergent behaviour.
● When designing a behaviour-based system, the designer begins by listing the desirable
observable, externally manifested behaviours.
● Next, the designer figures out how best to program those behaviours with internal
behaviours.
○ Those internal behaviours may or may not directly achieve the observable goals.
○ To make this process easier, various compilers and programming languages have
been developed.
● In the case of BBC, the approach to modularity is that of using a collection of behaviours, and
having those behaviours be relatively similar in terms of execution time.
○ This means that having one behaviour which contains a centralized world model and
performs reasoning on it would not fit the behaviour-based philosophy, and would
thus not make a good controller.
● BBC mandates that behaviours be incrementally added to the system and that they be
executed concurrently, in parallel, rather than sequentially, one at a time.
○ Behaviours are activated in response to external and/or internal conditions, sensory
inputs, and internal states or messages from other behaviours.
○ Dynamics of interaction arise both within the system itself and within the
environment.
● As long as the controller is structured using behaviours, it creates a BBC system, with all the
benefits that come with the approach, even without the use of representation.
● Behaviours can be used in clever ways to program robots.
○ By having behaviours interact with each other within the robot, they can be used to
store representation.
○ They can serve as a basis for learning and prediction.
○ This means that BBC systems can achieve the same things that hybrid systems can,
but in a different way.
The Robotics Primer

16.1: Distributed Representation

● The philosophy of BBC systems mandates that the information used as an internal
representation is not centralized or centrally manipulated.
● The key challenge in using representation in BBC is how that representation can be effectively
distributed over the behaviour structure.
○ The representation must be able to act on a timescale that is close to the real-time
components of the system.
○ The representation needs to use the same underlying behaviour structure as the rest
of the system.

16.2: An Example: Distributed Mapping

● BBC and hybrid control are equally powerful and expressive.
● There is no fixed answer to which one to choose
○ The decision is often based on the designer’s personal preferences and past
experience and expertise.
○ There are advantages to one approach or the other.
● For single-robot control, hybrid systems are more popular.
● For controlling groups and teams of robots, BBC is often preferred.

A Short Review of Symbol Grounding in

Robotic and Intelligent Systems
1: Introduction
● The main dream of artificial intelligence has been to create autonomous and intelligent
systems that can reason and act in the real world.
○ An essential ingredient is establishing and maintaining a connection between what
the system reasons about and what it can sense in the real world.
■ This is an aspect of the Symbol Grounding Problem.
● The Symbol Grounding Problem (SGP) is the problem of how to ground the meanings of
symbol tokens in anything different than other (meaningless) symbols.
○ Its practical application has been studied in robotics and intelligent systems,
emphasising the problem of grounding symbols acquired by physically embedded
sensors.
● There are two subtopics which relate to the current approaches to SGP in robotics and
intelligent systems.
1. “Physical Symbol Grounding” is the grounding of symbols to real-world objects by a
physical agent interacting in the real world.
The Robotics Primer

a. Physical Symbol Grounding requires constructing a consistent relation

between percepts that may vary under different conditions and often have a
high dimensionality.
2. “Social Symbol Grounding” refers to the collective negotiations for the selection of
shared symbols (words) and their grounding meanings in populations of agents.

2: Physical Symbol Grounding

● One of the basic challenges examined in literature is to ground symbols to perceptual
representations (sensor data).
○ Symbols denote categorical concepts such as colour, shape, and spatial features.
○ The sensor data comes from vision sensors, but other modalities have been used.
● Symbol Grounding is a bi-directional process (double-grounding).
○ Going bottom up the agent acquires a hierarchical composition of human behaviour.
○ Going top down the agent gets intentionally laden interpretations of those
structures.

2.1: Perceptual Anchoring

● A special case of Symbol Grounding is the connection of sensor data coming from physical
objects to higher-level symbolic information that refers to those objects.
○ The process of creating and maintaining these connections is called anchoring.
● This can be used to enable a grounded and shared model of the world that is suitable for
dialogue understanding.
○ Realistic human-robot interactions are considered that deal with complex, partially
unknown human environments.
● The anchoring process constructs and maintains a set of object linkage structures
representing the best possible hypotheses at any time.
○ Each hypothesis can be incrementally generalized or narrowed down as new sensor
data arrives.
○ Symbols can be associated with an object at any level of classification, allowing
symbolic reasoning on different levels of abstraction.

3: Grounding Words in Action

● One method is a hierarchical organisation of concepts used for the acquisition of abstract
words.
○ Higher-order concepts are grounded using basic concepts and actions that are
directly grounded in sensorimotor experiences.
● Another method is where representational complexity is managed using a symbolic feature
representation generated via policies, affordances, and goals.
○ The agent learns new policies, affordances, and goals and adds them to the
dictionary.
○ After each addition, the best common substructure is extracted.
The Robotics Primer

● Another (another) method is a probabilistic approach used to learn word meanings from
large examples and use those meanings to find good groundings in the external world.

4: Social Symbol Grounding

● The social symbol grounding is considered the next step after the connections between the
sensor data and symbols for individual agents are achieved.
○ How can these connections be shared among many agents?
● An unknown word can - theoretically - refer to an infinite number of objects.
● The primary motivation for an agent to construct a symbol-meaning mapping is to solve tasks
more quickly and accurately.
● One can consider the social symbol grounding problem as the initial step that facilitates the
acquisition of language and meaning without which no such relations can be learned.

5: Grounding Symbols in the Semantic Web

● The symbol grounding problem in the semantic web is a problem.
○ An inappropriate correspondence between symbol and referent can result in logically
valid but meaningless inferences.

6: Conclusions
● Environments for Symbol Grounding require a flexible handling of knowledge and a
connection of symbolic and sensory information to be able to successfully operate.
● Systems where humans have an active role are becoming more common.
○ Symbol Grounding is essential to ensure meaningful natural language
communication.
● The use of the web as a source of information about objects and their properties is providing
new opportunities to access a very large and updated storage of symbolic and visual data.
○ Connecting the information on the web to real data is the most important challenge
for this field.

Evolving grounded communication for

robots
● The computational and robotic synthesis of language evolution is emerging as a new exciting
field of research.
○ The objective is to come up with precise operational models of how communities of
agents can arrive at shared grounded communication systems.
● Artificial Life (AL) developed in the early 1990s as a field investigating principles of living
systems by building artificial systems that model some key aspects.
The Robotics Primer

○ These models provide a causal insight into how phenomena arise.

○ The notion of a complex adaptive system has emerged as a key concept.
■ Many independent elements dynamically coordinate their activity through
self-organisation.
● Genetic evolution has made it possible to solve a large set of engineering problems by using
the search power implicit in selection processes.
● In robotics, the AL approach has had an important impact and is responsible for the
developments in autonomous robots in the last decade.
○ The AL approach insists on a bottom-up emergence of complexity and the role of
collective dynamics in behaviour.

Language as a complex adaptive system

● The first goal is scientific: To make precise models of how certain key properties of language-
like communication systems might in principle originate, and how such communication
systems continue to evolve and remain adapted to the needs of the users.
○ Language is viewed as a living system, that self-organises and evolves through the
collective dynamics of agents engaged in situated verbal interactions.
■ The insights and apparatus for studying complex systems and AL are relevant
to the study of language.
● Evolutionary linguistics was born.
○ The focus is to understand the origins of language and meaning.
○ The hypothesis is that language is a complex adaptive system.
○ The methodology is to construct artificial systems as a way to develop and test
theories.
● The relationship between evolutionary linguistics and linguistics is indirect.
○ Linguistics is an empirical science which studies human natural languages as they
exist.
○ Evolutionary linguistics focuses on the evolution of grounded communication
systems in general.
● Artificial communication systems generated in experiments can be similar to human
languages, but do not and are not identical.
● The relationship between computational linguistics and AI is indirect.
○ The goal of evolutionary linguistics is to understand the generic forces and
mechanisms that give capabilities.
■ It is to understand how natural language-like lexicons or grammars arise in
the first place and by what mechanisms they can continue evolving.

Relevance to robotic applications

● This research has a practical goal: To forge a new technology for communication between
humans and robots, or among robots.
● There is a consensus that pre-programmed communication is inadequate for new generations
of robots and software agents.
The Robotics Primer

○ The environment is the real world and open-ended.

■ It is not possible to foresee all things that can happen.
■ It is not possible to foresee all task situations that can be encountered.
■ It is not possible to foresee what communication is needed.
○ Humans negotiate shared conventions as part of dialogues.
■ Robots need the ability to cope with new meanings, variations in speech,
words or shifts in the meaning of existing words, grammatical constructions,
and interaction patterns.
○ Some applications require robots to self-develop.
■ Robots should keep discovering new behaviours and new modes of
interaction, including new ways to communicate.

Evolutionary language games

Game theory
● Game theory plays a profound role in many AL discussions.
○ It has been adopted as a framework for studying the origins and evolution of
communication systems in populations of agents.
○ It is useful for performing large-scale computational and robotic experiments and
also for developing mathematical theories.
● A language game model consists of a population of agents.
○ Each agent is embodied.
■ It has a cognitive apparatus and learning mechanisms relevant to the aspects
of language and meaning one wants to study.
○ The agents interact with the environment and each other through a sensorimotor
system.
■ The environment is the real world itself
○ Agents have or must develop scripts for playing language games
■ They take turns playing the role of speaker and hearer to build up
competence both for interpreting and for producing language utterances.
● There is no access to each other’s mental states - all interaction goes
through the world.
○ There are mechanisms for handling breakdowns in the game…
■ Introducing or learning a new sound.
■ Learning the meaning of a word never heard before.
■ Extending by analogy a new word for a meaning that was never expressed
before.
■ Introduce or acquire a new grammatical construct.
■ And so on…
The Robotics Primer

The Talking Heads experiment

● The objective of this experiment was to show how a population would be able to generate
and self-organise a shared lexicon as well as the perceptually grounded categorisations of the
world expressed by this lexicon, without human intervention or prior specification.

● One figure is chosen randomly by the “speaker” as a topic of the game.

● The “hearer” has to guess the intended topic based on words supplied by the speaker.
● The robots use pan-tilt cameras for visual sensing and for pointing.
● At the start of the game, the speaker moves the camera towards a specific area of the
whiteboard, indicating what the hearer should pay attention to.
● After interpreting the words, the hearer points the camera to the guessed object.
○ When it points to the wrong topic or signals failure in understanding, the speaker
points to the topic it indicated.
● Two processes are required to play these lexical language games.
1. The speaking agent must conceptualise the context in a way that they find a
category/set of categories which distinguishes the topic from other objects.
a. If the agent has no adequate distinction, it should create a new more refined
distinction.
2. The speaker must be able to express the selected categories through words and the
hearer must be able to parse them.
a. The speaker chooses words for maximum communicative success.
b. Agents keep score for each form - meaning pair in their lexicons and choose
the one with the maximum success in the past.
i. If a word-meaning pair fails, the score decreases.
ii. If there is a success, the score increases and the competing word
forms decrease.
iii. If there is no word for a category, it is randomly generated, and the
hearer learns the word by guessing the meaning from context.
The Robotics Primer

● This lateral-inhibition dynamic progressively results in the self-organisation of a lexicon in a

group
● At first, there is a struggle in which different words compete until the population settles on a
single dominant word.
○ The more agents prefer a particular word, the more they use this word, and the more
success the word has.
○ By tightly coupling processes that create/learn categorical distinctions and processes
that lexicalise them, the conceptual repertoires of the agents become coordinated
without having been programmed in, without central control or telepathy.

Horizontal and vertical transmission

● The Talking Heads experiment is an example of a language game model where linguistic
structures arise in the interaction between agents and propagate horizontally in the
population.
○ The structures that emerge depend on the properties of the communication medium,
the environments in which the agents operate, the embodiment of the agents, their
cognitive capacities, and the types of interactions they have.
● In vertical transmission, there is a population of agents which inherits or learns linguistic
behaviour from a previous generation.
○ In the transmission process, novelty (including increased complexity) arises.

The future of evolving communications research

● Evolving communication systems for new generations of robots bring up a wide range of
open issues.
● There are some major research challenges…
1. Pragmatic feedback is crucial for bootstrapping grounded communication.
a. Smooth integration remains an enormous challenge.
b. We need to understand better the operation of these various capabilities and
the nature of prelinguistic, implicit communication.
2. The experiments mentioned assume that agents can play language games, but how
do the games themselves emerge?
a. We need to understand better the dynamical systems properties of coupled
systems and see how they can be embedded in agents.
3. Research in cognitive semantics has yielded intriguing descriptions of the kinds of
conceptualisations humans employ for language.
a. It is mysterious how such conceptualisations can arise and how they can be
embodied in artificial agents.
4. Grammaticalisation processes play a dominant role in the emergence of new
grammatical constructions in natural languages.
a. We are lacking precise models of cognitive mechanisms and collective that
are involved.
The Robotics Primer

b. No convincing simulations of grammaticalisation phenomena have been

shown yet.
c. Grammaticalisation is strongly grounded in many aspects of human culture
and embodiment.
i. These aspects are difficult to incorporate in artificial systems.
5. We need to understand how populations of agents can self-organise a shared
repertoire of discrete building blocks grounded in a continuous physical medium and
how combinatorial systems can arise from them.
6. The question how an embodied agent could invent and learn the linkage between
complex conceptualisations and complex syntactic structures is still a large open
problem.
● Natural languages are extraordinarily complex communication and representation systems.
○ They cannot be built by hand!

Chapter 7: What’s Going On?

Sensors

● A robot typically has two types of sensors based on the source of information it is sensing.
1. Proprioceptive sensors perceive elements of the robot’s internal state.
a. Proprioception is the process of sensing the state of one’s own body.
2. Exteroceptive sensors perceive elements of the state of the external world around
the robot.
a. Exteroception is the process of sensing the world around the robot (including
sensing the robot itself).
● Proprioceptive and exteroceptive sensors constitute the perceptual system of a robot.

● Sensors are physical devices that measure physical quantities.

The Robotics Primer

○ The same physical property may be measurable with more than one type of sensor.
○ Acquiring information from multiple sensors can provide improved accuracy.
Physical Property Sensing Technology

Contact Bump, switch

Distance Ultrasound, radar, infrared

Light level Photocells, cameras

Sound level Microphones

Strain Strain gauges

Rotation Encoders and potentiometers

Acceleration Accelerometers and gyroscopes

Magnetism Compasses

Smell Chemical sensors

Temperature Thermal, infrared

Inclination Inclinometers, gyroscopes

Pressure Pressure gauges

Altitude Altimeters

● Sensor noise and errors are inherent in physical measurement and cannot be avoided,
contributing to uncertainty.
○ The robot’s inability to be certain about the state of itself and its environment, to
take optimal actions at all times.
○ Uncertainty comes from a variety of sources, including…
■ Sensor noise and errors.
■ Sensor limitations.
■ Effector and actuator noise and errors.
■ Hidden and partially observable state.
■ Lack of prior knowledge about the environment, or a dynamic and changing
environment.
● Robots must survive and perform in a messy, noisy, challenging real world.
○ Sensors are the windows into the world.
The Robotics Primer

● The more information a sensor provides, the more processing is needed.

○ Consequently, it takes no brains at all to use a switch, but it takes a great deal of
brains to process inputs from vision sensors.
● There are two ways in which sensory information can be treated.
1. Given that sensory reading, what should I do?
a. This refers to the action in the world.
2. Given that sensory reading, what was the world like when the reading was taken?
a. This refers to the reconstruction of the world.
● The problem of going from the output of a sensor to an intelligent response is sometimes
called the signal-to-symbol problem.
○ By using symbols, we can make information abstract and sensor-specific.
○ Getting from a sensory output (of any sensor) to such an abstract symbolic form
(encoding) of information to make intelligent decisions is a complex process).
○ This is one of the most fundamental and enduring challenges in robotics.
● Since sensors provide signals and not symbolic descriptions of the world, they must be
processed to extract the information the robots need.
○ This is called sensor preprocessing because it comes before anything else can be
done in terms of using the data to make decisions and/or take action.

7.1: Levels of Processing

● A robot needs a variety of factors to process sensory inputs.

○ Analog of digital processing capabilities.
○ Wires to connect everything.
○ Support electronics to go with the computer.
○ Batteries to provide power for the whole thing.
The Robotics Primer

● Perception requires…
○ Sensors (Power and electronics).
○ Computation (More power and electronics).
○ Connectors (To connect it all).
● It is generally not a good idea to separate what the robot senses, how it senses it, how it
processes it, and how it uses it.
○ Doing that results in a large, bulky, and ineffective robot.
● It is best to think about the what, why, and how of sensing as a single complete design,
consisting of the following components…
○ The task the robot has to perform.
○ The sensors best suited for the task.
○ The mechanical design is most suited to enable the robot to get the sensory
information necessary to perform the task.
● Robotics researchers have figured out these important requirements of effective perception,
and have been exploring various methods.
○ Action-oriented perception (“active sensing”) refers to the robot using the
knowledge about the task to look for particular stimuli in the environment and
respond accordingly.
○ Expectation-based perception uses knowledge about the robot’s environment to
help guide and constrain how sensor data can be interpreted.
○ Task-driven attention refers to the robot moving the sense in the direction where
information is most needed or available.
○ Perceptual classes divided the world into perceptual categories for getting the job
done.
■ The robot can consider manageable categories it knows how to handle.
● As a robot designer, you will not get the chance to make up new sensors, but the chance to
design interesting ways to use the sensors you have at your disposal.
● Combining multiple sensors to get better information about the world is called sensor fusion.
○ This is not a simple process and can result in more noise and inaccuracy - more
uncertainty about the world.
○ Some clever processing has to be done to minimize the error and maximize accuracy,
as well as put the different types of information together in an intelligent and useful
way.
● A great deal of our impressive brain is involved in processing sensory information.
○ It is not surprising that this is a challenging and important problem in robotics!

Chapter 8: Switch on the Light

Simple Sensors

8.1: Passive vs. Active Sensors

● Passive sensors measure a physical property from the environment.
○ Consists of a detector which perceives (detects) the property to be measured.
○ Can be simple or complex.
The Robotics Primer

● Active sensors provide a signal/stimulus (requiring extra energy) and use the interaction of
that signal with the environment as the property they measure.
○ Consists of an emitter which produces (emits) the signal, and a detector.
○ They are not necessarily complex.
● Whether a sensor is complex is determined by the amount of processing its data requires,
while whether a sensor is active is determined by its mechanism.

8.2: Switches

● Switches are the simplest sensors.

○ They provide useful information at the electronics (circuit) level since they are based
on the principle of an open vs. closed circuit.
○ If a circuit is open, no current can flow through the circuit.
○ If a circuit is closed, current can flow through the circuit.
○ By measuring the amount of current flowing through the circuit, we can tell if the
switch is open or closed.
■ Switches measure the change in current resulting from a closed circuit,
resulting from physical contact of an object with the switch.
● The measure of current is needed to use the switch as a sensor.
● This principle is applied in a wide variety of ways to create switches, and in turn switches are
used in a variety of clever ways for sensing, such as…
○ Contact sensors detect when the sensor has contacted another object.
○ Limit sensors detect when a mechanism has moved to the end of its range.
○ Shaft encoder sensors detect how many times a motor shaft turns by having a switch
click (open/close) every time the shaft turns.
● Switches can be attached to a great variety of places and components on a robot.
The Robotics Primer

8.3: Light Sensors

● Light sensors measure the amount of light impacting a photocell.
○ Photocells are sensitive to light.
○ This sensitivity is reflected in the resistance in the circuit they are attached to.
○ The resistance of a photocell is low when it is illuminated, sensing a bright light.
○ The resistance of a photocell is high when it is not illuminated, sensing the dark.

● Light sensors are simple, but they can detect a wide range of wavelengths, much broader
than the human eye can see.
● Light sensors can be used as passive or active sensors in a variety of ways, and can measure
the following properties…
○ Light intensity: How light/dark it is.
○ Differential intensity: Difference between photocells.
○ Break in continuity: “Break beam,” change/drop in intensity.

8.3.1: Polarized Light

● “Normal” light emanating from a light source consists of light waves that travel in all
directions relative to the horizon.
○ If we put a polarizing filter in front of the light source, only the light waves in the
direction of the filter will pass through it and travel away.
○ This direction is called the “characteristic plane” of the filter.
○ Polarized light is light whose waves travel only in a particular direction, along a
particular plane.
● By playing around with photocells and filters and their arrangement, you can use polarized
light to make specialized sensors that cleverly manipulate what and how much light is
detected.
○ These are active sensors since they consist of a photocell, light sources, and filters for
polarizing the lights.
The Robotics Primer

8.3.2: Reflective Optosensors

● Reflective optosensors operate on the principle of reflected light.
○ They are active sensors consisting of an emitter and detector.
■ The emitter is usually made with a light-emitting diode (LED).
■ The detector is a photodiode/phototransistor.
○ They do not use the same technology as resistive photocells.
■ Photodiodes and phototransistors are much faster and preferred in robotics.
● There are two basic ways reflective optosensors can be arranged, based on the relative
positions of the emitter and detector…
1. Reflectance sensors: The emitter and the detector are side by side, separated by a
barrier.
a. The presence of an object is detected when the light reflects from it and back
into the detector.
2. Break beam sensors: The emitter and detector face one another.
a. The presence of an object is detected if the beam of light between the
emitter and the detector is interrupted or broken.

8.3.3: Reflectance Sensors

● By measuring light reflectance/reflectivity, you can…
○ Detect the presence of an object.
○ Detect the distance to an object.
○ Detect some surface features.
○ Decode a bar code.
○ Track the rotations of a wheel.
● Light reflectivity is affected by colour, texture, and other properties of the surface it hits.
The Robotics Primer

● Even though we have useful sensors, we cannot have complete and completely accurate
information.
● A light sensor has to operate in the presence of the light existing in the environment, which is
called ambient light.
○ The reflectance sensor must ignore ambient light to be sensitive only to its own
emitter’s reflected light.
○ The ambient light level is measured by taking a sensor reading with the emitter off.
○ To measure only the emitter’s reflected light, the detector takes two/more readings
of the sensor level, one with the emitter on and one with it off.
■ The difference produces the amount of the emitter’s light reflected to the
sensor.
● Calibration is the process of adjusting a mechanism to maximize its performance (accuracy,
range, etc.)
○ Sensors require calibration, some initially and some continuously, to operate
effectively.
■ This can be performed by the designer, the user, or the sensor mechanism
itself.
● Ambient light levels change during the day.
○ If the environment can change, the sensor has to be calibrated repeatedly to stay
accurate and useful.

8.3.4: Infrared Light

● Visible light is light in the frequency band of the electromagnetic spectrum that human eyes
can perceive.
○ Infrared (IR) light has a wavelength different from visible light and is not in the visible
spectrum.
● IR sensors are light sensors which function in the infrared part of the frequency spectrum.
○ They are used in the same ways that visible light sensors are: As reflectance sensors
or as break-beams.
● IR is very useful in robotics because it can easily modulated and made less prone to
interference.
○ IR can also be used for communication, which is how IR modems work.

8.3.5: Modulation and Demodulation of Light

● Light is modulated by rapidly turning the emitter on and off, pulsing it.
○ The resulting pulse signal is then detected by a demodulator, a mechanism that is
tuned to a particular frequency of the modulation, so it can be decoded.
○ The detector has to sense several “on” flashes in a row for the demodulator to
determine its frequency and decode it.
● Strobe light is a type of modulated visible light.
● Modulated IR is commonly used since IR is not in the visible spectrum.
The Robotics Primer

8.3.6: Break Beam Sensors

● Any pair of compatible emitter-detector devices can be used to produce break-beam sensors.

8.3.7: Shaft Encoders

● Shaft encoders measure the angular rotation of a shaft or an axle.
○ They provide position and/or velocity information about the shaft they are attached
to.
● To detect a turn or a part of a turn, we have to mark the thing that is turning.
○ This is usually done by attaching a round notched disk to the shaft.

● If there is only one notch in the disk, then every time the notch passes between the emitter
and detector, the disk has completed one full rotation.
○ This allows for measuring with only a low level of precision.
○ If any noise or error is present, one or more turns might be missed, and the encoder
will be inaccurate.
The Robotics Primer

● To make the encoder more accurate as well as more precise, many notches are cut into the
disk.
● An alternative to cutting notches in the disk is to paint the disk with wedges of alternating,
contrasting colours (black for absorption and white for reflection).
○ This becomes a reflectance sensor.
● The detector of the shaft encoding sensor will output a wave function of the sensed light
intensity of the emitter.
○ The output is processed, using signal processing, with hardware or a simple
processor, to calculate the position and speed by counting the peaks of the waves.
● We can use encoders in at least two ways to measure the speed of a robot…
1. Encode and measure the speed of a driven wheel.
2. Encode and measure the speed of a passive wheel (caster) that is dragged by the
robot.
● We can combine the position and velocity information the encoder provides to have the
robot do more sophisticated things, such as move in a straight line or turn at by exact angle.
○ Shaft encoders can provide feedback to correct some of the errors, but having some
errors remain is unavoidable.
● The mechanism for detecting and measuring the direction of rotation is called quadrature
shaft encoding.
○ Instead of using only one sensor, use two sensors.
○ The two encoders are aligned so that their two inputs coming from the detectors are
90 degrees out of phase.
■ By comparing the outputs of the two encoders at each time step with the
output of the previous time step, we can tell if there is a direction change.
○ Since they are out of phase, only one encoder can change its state.
■ This determines the direction the shaft is rotating.
■ Whenever a shaft is moving in one direction, a counter is incremented in that
encoder, and when it turns in the opposite direction, the counter is
decremented, keeping track of the overall position of the mechanism.

8.4: Resistive Position Sensors

● Bend sensors are much less robust than light sensors, although the two use the same
underlying principle of responding to resistance.

8.4.1: Potentiometers
● Potentiometers (“pots”) are commonly used for the manual tuning of analogue devices.
○ Nowadays, most devices are digital.
● Potentiometers are resistive sensors.
○ Turning the knob or pushing a slider effectively alters the resistance of the sensor.
○ The basic design of potentiometers involves a tab that slides along a slot with fixed
ends.
The Robotics Primer

○ As the tab is moved, the resistance between it and each of the ends of the slot is
altered.
■ The resistance between the two ends remains fixed.
● In robotics, potentiometers are used to tune the sensitivity of sliding and rotating
mechanisms, as well as to adjust the properties of other sensors.

Chapter 9: Sonars, Lasers, and Cameras

Complex Sensors

9.1: Ultrasonic or Sonar Sensing

● Ultrasound means “beyond sound” and refers to a range of frequencies of sound beyond
human hearing.
○ It is also called sonar - so(ound) na(vigation and) r(anging).
● The process of finding your (or robot’s) location based on sonar is called echolocation.
○ Sound bounces off objects and forms echoes that are used to find one’s place in the
environment.
○ The principle comes from nature (ex. bats).
● Artificial ultrasound sensors/sonars are based on the time-of-flight principle.
○ They measure the time it takes something (sound) to travel (“fly”).
● Sonars are active sensors consisting of an emitter and detector.
○ The emitter produces a chirp/ping of ultrasound frequency.
○ The sound travels from the source and reflects from a barrier.
○ If there is no barrier, the sound does not return.
■ The sound wave weakens (attenuates) with distance, breaking down.
○ If a sound comes back, the amount of time it takes for it to return can be used to
calculate the distance between the emitter and the barrier that the sound
encountered.
■ A timer is started when the chirp is emitted and is stopped when the
reflected sound returns.
■ The resulting time is then multiplied by the speed of sound and divided by
two.
● It is a one-way distance!
● Sound travels 1.12 feet per millisecond (1 foot per 0.89 milliseconds)
at room temperature.
● The physical sensor is a round transducer that emits the chirp/ping and receives the sound
(echo) that comes back.
○ Transducers are devices that transform one form of energy into another.
● The hardware of ultrasound sensors involves relatively high power.
○ A significant current is needed for emitting each ping.
○ The amount of current required is much larger than what computer processors use.
The Robotics Primer

9.1.1: Sonar Before and Beyond Robotics

● Simply sensing the distances between the emitter and the environment may be sufficient for
many robotics applications, but in medical sonar imaging, much more complex
postprocessing is involved to create a composite image of the body part.
○ The image is updated in real-time!
● Since sound travels well through water, while vision is almost useless in an underwater
environment, sonar is the favoured sensor for underwater navigation.
● As in-depth sounders, these sonars send an intense beam of sound into the ocean/body of
water and wait for it to return to see how deep the water is or how far the nearest
object/surface is.
● The principle of time-of-flight underlies all uses of sonar as a ranging and imaging device.
○ Multiple sensor units are employed for increased coverage and accuracy.
○ Most robots using sonars are equipped with several covering a cross-section of the
robot’s body.

9.1.2: Specular Reflection

● Sonar sensing is based on the emitted sound wave reflecting from surfaces and returning to
the receiver.
● The direction of reflection depends on several factors, including the properties of the surface
(smoothness), and the incident angle of the sound beam and surface (sharpness).
● A major disadvantage of ultrasound sensing is its susceptibility to specular reflection.
○ Specular reflection is the reflection from the outer surface of the object.
○ The sound wave that travels from the emitter bounces off multiple surfaces in the
environment before returning to the detector.
■ The smoother the surface is, the more likely the sound is to bounce off.
■ The rougher the surface is, the more likely the reflections will return to the
emitter.
● In the worst case of specular reflection, the sound bounces around the environment and
does not return to the detector, fooling the sensor into detecting no object/barrier at all, or
one that is far away instead of nearby.
○ To combat this is by altering the environment so that surfaces are less reflective.
○ Another solution is to use phased arrays of sensors to gain more accuracy.
■ This is effective in some environments.
■ The environment may have discontinuities - sudden and large changes in its
features.
○ Another solution is action-oriented perception.

9.2: Laser Sensing

● Lasers emit highly amplified and coherent radiation at one or more frequencies.
○ The radiation may be in the visible spectrum or not, depending on the application.
● Laser range sensors can be used through the time-of-flight principle.
The Robotics Primer

● When lasers are used for measuring short distances, the light travels so fast that it comes
back more quickly than it can be measured!
○ Phase-shift measurements are used to compute the distance, instead of time-of-
flight.
● Lasers involve higher-power electronics, which means they are larger and more expensive.
○ They are also much more accurate!
● Laser light is projected in a beam rather than a cone.
○ The spot is small (3mm in diameter).
○ Lasers can take more measurements than sonar can, providing data with a higher
resolution.
■ Resolution refers to the process of separating or breaking something into its
constituent parts.
■ High resolution refers to many parts, which refers to more information.

9.3: Visual Sensing

● Cameras are the closest thing to natural eyes that we have available among synthetic sensors.
● The research field that deals with vision in machines, including robots, is called machine
vision.
○ Robots have particular perceptual needs related to their tasks and environments.
● Machine vision and robotics are two separate fields of research with some overlap of
interests, problems, and uses.

9.3.1: Cameras
● Cameras are biomimetic, meaning they imitate biology.
● There are several components for a vision system.
○ Light, scattered from objects in the environment (scene), goes through an opening
(iris) and hits the image plane.
○ The image plane corresponds to the retina of the biological eye, which is attached to
numerous light-sensitive (photosensitive) elements called rods and cones.
○ These are attached to nerves that perform early vision, the first stages of visual
image processing.
○ This information is passed on to other parts of the brain to perform high-level
processing, everything else done with the visual input.
● In machine vision, the computer needs to make sense out of the information on the image
plane.
○ A simple camera and tiny pinhole requires some computation to determine the
projection of objects from the environment onto the image plane.
○ If a lens is involved, more light can get in at the cost of focus.
■ Only objects at a particular range of distances from the lens will be in focus.
■ This range of distances is called a depth of field.
●
● The projection of the scene on the image plane is called the image.
The Robotics Primer

○ The image plane is usually divided into equal parts called pixels.
○ The brightness of each pixel in the image is proportional to the amount of light
reflected into the camera by the part of the object or surface that projects to that
pixel, called the surface pixel.
● All influences that affect the brightness of the patch can be lumped into two kinds of
reflections.
1. Specular Reflection: Off the surface.
2. Diffuse Reflection: Light that penetrates into the object, is absorbed, and then comes
back out.
● Cameras observe the world continually.
○ Processing any time series information over time is complicated.
○ For machine vision, each individual snapshot in time is called a frame.

9.3.2: Edge Detection

● Edge detection is finding all the edges in the image.
● In machine vision, an edge is a curve in the image plane across where there is significant
change in pixel brightness.
○ A simple approach is through differentiating the image and looking for areas where
the magnitude of the derivative is large.
■ This does find edges but also finds shadows and noise, making it hard to
distinguish.
● Noise produces sudden and spurious intensity changes that do not have any meaningful
structure.
○ These peaks can be taken away through smoothing.
● To perform smoothing, we apply convolution, which finds and eliminates the isolated peaks.
○ Convolution applies a filter to the image, convolving the image.
○ The process of finding real edges involves convolving the image with many filters with
different orientations.
● Once edges are determined, the next step is to find objects among the edges.
○ Segmentation is the process of dividing or organising the image into parts that
correspond to continuous objects.

9.3.3: Model-based Vision

● Model-based vision is part of the philosophy about how the brain may recognize familiar
objets, or how we might get robots to do it effectively.
○ This uses models of objects, and some prior information or knowledge about those
objects, represented and stored in a way that can be used for comparison and
recognition.
● Models can be stored in a variety of forms.
○ It is not enough just to compare what the robot sees with the stored model.
○ The robot may be looking at the object from any angle and any distance.
■ The model needs to be scaled and rotated in different orientations, taking in
consideration of all combinations.
The Robotics Primer

9.3.4: Motion Vision

● Visual systems are often attached to things that move.
● The movement of the body and camera on it makes vision processing challenging.
○ Motion vision is a set of machine vision approachs that use motion to facilitate visual
processing.
■ This is by subtracting two consecutive images from one another, getting the
“movement” between the two, while the objects stay the same.
○ A mobile robot using vision has to make clever movement decisions, and has to
subtract its own movement from the visual image to see what it can see.

9.3.5: Stereo Vision

● The main advantage of having two eyes is the ability to see in stereo.
○ Stereo vision/binocular stereopsis is the ability to tuse the combined points of view
from the two eyes or camera to reconstruct 3D solid objects and perceive depth.
● With stereo vision, we get two images which can be subtracted from one another, as long as
we know how the two cameras or eyes are positioned relative to each other.

9.3.6: Texture, Shading, Contours

● Light is reflected different with a variety of textured objects.
○ Surface patches that have uniform texture have consistent/identifical brightness in an
image, so we can assume they come from the same object.
○ By extracting and combining patches with uniform and consistent texture, we can get
a hint about what parts of the image may belong to the same object in the scene.
● Shading, contours, and object shape can be used to simplify vision.

9.3.7: Biological Vision

● We use model-based vision to recognise objects and people we know.
○ Without it, we find it hard to recognise entirely unexpected objects or novel ones, or
to orient ourselves.
● We use motion vision in a variety of ways in order to better understand the world around us,
as well as be able to move around while looking and not having ti all result in a big blur.
○ The latter is done through vestibular ocular reflex (VOR), in which your eyes stay
fixed even though your head is moving, in order to stabilise the image.
● We are “hard-wired” to be sensitive to movement at the periphery of our field of view.

9.3.8: Vision for Robots

● Robot vision has more stringent requirements than some other applications of machine
vision, and only slightly less demanding requirements than biological vision.
○ Robot vision needs to inform the robot about important things!
The Robotics Primer

● Since vision processing can be a very complex problem, responding quickly to the demands of
the real world based on vision information is very difficult.
● There are multiple approaches to simplifying this vision problem…
1. Use colour; look for sepcfiically and uniquely coloured objects, and recognise them
that way.
2. Use the combination of coloru and movement; this is called blob tracking and is
popular in mobile robotics.
a. By marking important objects with salient (noticable/attention-getting) or
recognisable colours, and using movement to track them, robots can
effectively get their work doen without having to actually recognise objects.
3. Use a small image plane.
4. Combine other, simpler and faster sensors with vision.
5. Use knowledge about the environment.
● Complex sensors imply complex processing, so they should be used selectively, for task where
they are required or truly useful.