Understanding Artificial

Intelligence
Of Minds and Machines
Course Guidebook
Patrick Grim, PhD
© 2026 The Teaching Company

This book is in copyright. All rights reserved.

Without limiting the rights under copyright reserved above, no part

of this publication may be reproduced, stored in or introduced into
a retrieval system, or transmitted, in any form, or by any means
(electronic, mechanical, photocopying, recording, or otherwise),
without the prior written permission of The Teaching Company.

4840 Westfields Boulevard, Suite 400

Chantilly, VA 20151‑2299
USA
1-800-832-2412
[Link]
Patrick Grim, PhD
Patrick Grim is Distinguished Teaching Professor of Philosophy Emeritus
at Stony Brook University and Philosopher in Residence with the Center for
the Study of Complex Systems at the University of Michigan. He earned
his BPhil from the University of St. Andrews, Scotland, and his PhD from
Boston University. He has published extensively in scholarly journals across
a range of disciplines and has received numerous awards for his teaching.
His books include The Incomplete Universe, The Philosophical Computer, and
Theory of Categories.

i
Table of Contents
About Patrick Grim, PhD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
1. The Very Human History of AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2. Intelligence: Real and Artificial . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
3. The Foundations of Artificial Intelligence . . . . . . . . . . . . . . . . . 16
4. How AI Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5. Minds, Machines, and the Limits of AI . . . . . . . . . . . . . . . . . . . . 31
6. AI Comes of Age . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
7. AI and the Wisdom of Crowds . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
8. Evolution and Artificial Intelligence . . . . . . . . . . . . . . . . . . . . . . . 52
9. Ethics and AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
10. Promises and Perils of AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
11. The Coming Singularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
12. Predicting Tomorrow’s AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

ii
1
The Very Human
History of AI

A rtificial intelligence (AI) has long been a theme in science fiction. In

movies of that genre, AI is portrayed in different ways—an indication
that society is not clear on what it can expect from the technology. But AI is
no longer a concept in the vision of a distant future; wherever information
matters, AI is already at work. Today’s real world is saturated with AI, and
it is important to understand it: how it got here, how it works, its positive
potential, and its possible dangers. In this lecture, you’ll explore AI’s role in
daily life and discover some of the people who made that possible.

1
 1. The Very Human History of AI

AI in Society
Most people are familiar with platforms such as Google Maps, widely used
to find the most efficient route to wherever they want to go. Google Maps
suggests different routes depending on traffic conditions and the time of
day; monitors construction delays along the chosen route; and even displays
nearby gas and electric charging stations, rest areas, and restaurants. It directs
hundreds of thousands of travelers on their individual journeys nationwide
simultaneously. It wraps GPS—a satellite-based positioning system—in a web
of AI and combines a record of past traffic in a particular area with current
data from users on the road to generate maps, individualize routes, and
predict arrival times.

The transportation sector relies heavily on AI. Today’s ride-sharing services

don’t use human dispatchers; instead, an AI network coordinates riders
and drivers. Even airline routing is determined by AI, down to the details
of individual passengers and their baggage. Reliance on AI in air travel is
particularly evident when computer systems are impacted in some way, as
was the case when a single defect in a software update from a cybersecurity
firm caused a global Microsoft outage. Without AI, air travel came to an
abrupt halt.

AI is also the backbone of the financial industry. It monitors bank deposits

and withdrawals, transfers, loans, and mortgages and makes a large
percentage of stock market trade decisions. The success of hedge funds
was only possible with AI algorithms administered with computer speed.
In addition, bank and credit card companies use AI to generate purchase
verification alerts. The system relies on a form of neural network that
has learned the pattern of the account holder’s buying behavior and flags
any unusual variation. AI is also particularly dominant in the world of
cryptocurrency—an economy that consists of virtually nothing but bytes and
essentially exists only in cloud computing.

In the workplace, robots of increasing AI sophistication dominate the

factory floor in automobile manufacturing, the aeronautical industry, and
the appliance sector. From architecture to advertising, designing involves
computers with AI assistance. AI is even the new doctor’s assistant. Medical
records are stored and accessed electronically, and warnings regarding

2
 1. The Very Human History of AI

potential drug interactions show up instantly. With the tools of machine

learning, detection of some cancers on the basis of X-rays and MRIs
has become as good as or better than human diagnosis. From weather
forecasting and law enforcement to the design and supervision of the power
grid, AI is involved. It has become an integral part of the communication
systems, making it possible for streaming services to provide personalized
recommendations and for advertisers to target specific user groups. Society is
immersed in AI.

Lady Ada Lovelace and

the Analytical Engine
To trace the history of AI, it is necessary to go back to the 19th century. In
1835, the famous poet George Gordon Byron—Lord Byron—and his wife,
Annabella, had a daughter, Ada. Byron,
a notorious libertine, deserted his
wife and daughter within months
of Ada’s birth. Annabella
was deeply worried that her
daughter would take after her
father, so she ensured that
Ada received an education
that emphasized science
and mathematics.
At 17, like other
young women of
her class, Ada
was presented Ada Lovelace
at court. She
made the rounds
of social soirees,
and a few years
later, she got married.
Ada became Lady
Ada Lovelace.

3
 1. The Very Human History of AI

The turning point in Lovelace’s intellectual life came at one of the social
gatherings—a party hosted by Charles Babbage, a brilliant and socially
connected inventor. Babbage demonstrated parts of a machine he had
designed, called the Difference Engine. Powered by steam, it was designed
to automatically calculate logarithms. Lovelace shared Babbage’s interest in
mathematical calculation by way of machines, and this led to years of research
collaboration. Babbage designed two versions of the Difference Engine,
but only part of one was completed in his lifetime. In 2002, the Science
Museum in London built a complete version according to his plans. With
its printing mechanism, the Difference Engine No. 2 uses 8000 machined
parts and weighs more than 5 tons. And it works. It has been said that “if
the technology of the 19th century had been equal to Babbage’s genius,
a computer would have been built in 1822.”

Babbage also sketched plans for a more ambitious project: an Analytical

Engine. This machine was conceived as a numerical calculator, but it was far
more general in mathematical application than the Difference Engine. After
a member of the audience at one of
Babbage’s rare lectures published
a paper on the Analytical Engine
in French, Lovelace translated
the paper into English and
amplified it. Her version was
four times as detailed and
four times as long, one of
the best early outlines
for computing
technology known
to exist. It shows
the vision of
something
more than
a mere calculator,
as Lovelace
recognized that all
symbol manipulation
could be handled by
Charles Babbage’s Analytical Engine such a machine.

4
 1. The Very Human History of AI

The Man Who Broke the Code

The second key character in the development of AI is Alan Turing, who
was born approximately 100 years after Lovelace. He is known for three
major accomplishments—one in the 1930s, one in the 1940s, and one in
the 1950s—all directly related to the development of AI. Although Turing
was trained as a mathematician, his work in the 1930s wasn’t on any specific
mathematical problem. He was interested in a general philosophical question
regarding all mathematical questions: Can mathematical truth always be
calculated step by step, or are there mathematical truths beyond the reach
of step-by-step calculation? Turing developed an abstract idea of what any
calculation—any computation—required. His purely abstract idea, a mere
thought experiment, became the inspiration for the machines that were built
later. All computers can be thought of as versions of Turing machines.

In the 1940s, Turing became a major figure in breaking the phenomenally

complex code the Nazis used for message transmission. The Nazis used
a device known as the Enigma machine, which was an automatic coder and
decoder. A message typed into a machine at one end was encoded, and that
coded message—fed into another Enigma machine at the other end—was
then automatically decoded. The Nazis had good reason to think their codes
for the Enigma machine were unbreakable: There were billions of ways of
setting up the code, and they changed the code every day. However, Turing
masterminded the British strategy for breaking the code.

Part of the code-breaking effort depended on exploiting predictabilities in

messages. Many German transmitters started their radio reports the same way
every day: “Today is Tuesday, and the weather over Berlin is …” The messages
also contained predictable phrases, such as “Heil Hitler.” The code-breaking
effort was further aided by a particular weakness in the Enigma machine:
A letter could encode any letter except itself. But the main tool in cracking
the code was another machine. Turing led a team that designed and built
a machine called the Bombe, a computer that could sort likely possibilities at
immense speed. Because of Turing and his machine, the British cracked the
codes and defeated the Enigma machine. Later in the war, the Nazis made
the machine even more complicated, but the Allies were way ahead of them.
For almost the entire war, the Allies could decode almost all intercepted
Nazi messages.

5
 1. The Very Human History of AI

In the 1950s, Turing

introduced the Turing test
in “Computing Machinery
and Intelligence,” an
article that was published
in a major philosophy
journal. The first sentence
states, “I propose to
consider the question
‘Can machines think?’”
It’s a clear landmark in
the history of AI. Sadly,
in the Cold War that
emerged in the 1950s,
certain population
groups were considered
high security risks in
sensitive government
research. Turing, a known
homosexual, lost his
security clearance and
was blocked from further
Bletchley Park bombe
work in computer research
and development.

From Neural Network

Concept to AI Student
The third important actor is Walter Pitts, the early developer of the neural
networks that power all of today’s AI. In 1935, in Detroit, Pitts read
Bertrand Russell and Alfred North Whitehead’s Principia Mathematica, an
important book on logic and mathematics. Pitts thought the book contained
several mistakes, and he wrote to Russell, who then invited him to come
to Cambridge as one of his doctoral students. Pitts had to decline the
offer; he was only 12 years old and in the seventh grade at the time. Later,

6
 1. The Very Human History of AI

having dropped out of school, Pitts ended up homeless in Chicago. Without

registering, he attended college classes at the University of Chicago, including
advanced classes by the logician Rudolph Carnap. Again, he pointed out
mistakes, this time in Carnap’s most recent book, and he was right.

Pitts was eventually introduced to the neurologist Warren McCulloch, who

gave him a home. Pitts and McCulloch started a collaboration of profound
importance for the future of AI. They wrote a paper together, “A Logical
Calculus of Ideas Immanent in Nervous Activity.” It introduced the concept
of computational neural networks. Pitts and McCulloch realized that
a simple model of the neurons in the brain can produce the effect of logical
connectives: the concepts of and, or, not, and if then that are fundamental to
all logic.

The history of AI is an account of not only astounding inventions but also

human tragedy. Lady Lovelace died of uterine cancer at the age of 36. Turing
died in 1954, at the age of 41, by biting into an apple dipped in cyanide. Pitts
passed away at the age of 46 of cirrhosis of the liver and complications of
alcoholism. In his last years, he shrank away from publicity, refused university
affiliations, and burned years of his unpublished research.

Today, as an educational experiment, a computer AI system named Ann is

enrolled as a student at Ferris State University, taking classes alongside her
human classmates—artificial alongside human intelligence. People such as
Lady Ada Lovelace, Alan Turing, and Walter Pitts have made that possible.

Reading
Hodges, Andrew. Alan Turing: The Enigma. Princeton University Press,
2014. (Also available as an audiobook.)
Padua, Sydney. The Thrilling Adventures of Lovelace and Babbage: The
(Mostly) True Story of the First Computer. Pantheon Books, 2015. (A
well-informed comic novel.)

Questions
1 Which image comes to mind when you think of the future of AI:
R2-D2 or the Terminator?

7
 1. The Very Human History of AI

2 What is your favorite AI application? What AI application do you

hate the most?

3 Suppose you could invite Lady Ada Lovelace, Charles Babbage,

Alan Turing, and Walter Pitts to dinner. What do you think the
conversation would be like?

Other Media
Tyldum, Morten, dir. The Imitation Game. Black Bear Pictures and Bristol
Automotive, 2014. (Starring Benedict Cumberbatch as Alan Turing.)

8
2
Intelligence: Real
and Artificial

W hat exactly is intelligence? How should this basic term be defined?

Across philosophy, the various branches of psychology, the
neurosciences, cognitive science, and the information sciences, it is widely
accepted that no agreed-upon definition of intelligence exists, or at least no
agreed-upon definition sharp enough to provide a precise way of measuring
intelligence. In this lecture, you will explore some of the deep questions
behind this lack of agreement and some of the important disagreements
and disputes regarding the very concept of intelligence—both real and
artificial.

9
 2. Intelligence: Real and Artificial

What Is Intelligence?
When it comes to understanding what intelligence is, there are various views.
One extreme, from a classic experimental psychologist, is that “intelligence
is what is measured by intelligence tests.” The opposite extreme, from a more
contemporary psychologist, claims that “there seem to be almost as many
definitions of intelligence as there were experts asked to define it.” And
according to a key actor in the history of AI, “the very concept of intelligence
is like a stage magician’s trick. Like the concept of the unexplored regions of
Africa, it disappears as soon as we discover it.”

In 50 recently reviewed definitions of intelligence from psychologists and

psychological organizations, however, some terms are repeated. Learning is the
most repeated term, as many experts agree that the ability to learn is crucial to
intelligence. Reasoning and understanding are also repeated often. Other terms
and phrases that frequently appear in the definitions are abstract thought,
problem solving, knowledge, imagination, and planning. Various phrases make
the claim that intelligence demands
flexible interaction with the
environment.

Looking for a positive

definition of intelligence is
one approach, but it can
also be instructive to
consider intelligence
in terms of what it’s
not. In medieval
philosophy, that
approach was
called the
via negativa.
The behavior
of the Sphex wasp
exemplifies the
concept of via negativa.
The great black wasp with When it comes time to
paralyzed insect lay its eggs, the female

10
 2. Intelligence: Real and Artificial

Sphex wasp digs a burrow. She then finds a cricket, katydid, or cicada and
stings it to paralyze but not kill it. She carries it into the burrow, lays her eggs
beside it, closes the burrow, and flies away, knowing that when her offspring
awake, a meal is waiting for them.

The female Sphex wasp, or digger wasp, has a genetically hardwired routine.
She brings the paralyzed cricket to the burrow, leaves it on the threshold,
goes inside to make sure all is well, then drags the cricket in. If the paralyzed
cricket is moved just a few inches away from the threshold while she is
inspecting the burrow, she does not drag it straight inside. Instead, she comes
out, drags the cricket to the threshold, and again goes into the burrow to
check things out. The wasp will continue this behavior until the cricket is still
on the threshold when she has inspected the burrow. This rigid behavioral
routine may have been evolutionarily advantageous, but it also serves as an
example of what intelligence is not.

The Role of Flexibility

The via negativa example indicates a crucial element of intelligence: Genuine
intelligence is environmentally flexible in ways the digger wasp is not. The
right thing to do in a situation may not be the intelligent thing to do if the
situation changes. Intelligence is an environmentally perceptive, reflective,
and flexible behavioral capacity. True intelligence is imaginative, creative,
reflective, and responsive in ways the digger wasp’s behavior is not.

Researchers pursuing AI generally include in their definitions of intelligence

that a core component is the adaptive ability to meet goals in a complicated
or shifting environment. They almost unanimously indicate that what they
aim for is “computational devices with the flexibility to pursue goals by
adapting their responses to a shifting environment”—more like humans than
digger wasps. Still, there is a great deal of animal behavior that seems far
more intelligent.

Systematic studies of problem-solving in primates go as far back as 1913, when

Wolfgang Köhler documented chimpanzees stacking wooden crates to retrieve
out-of-reach bananas. They also retrieved bananas placed outside their cages
by using sticks. In both cases, Kohler claimed, problem-solving involved

11
 2. Intelligence: Real and Artificial

an aha moment—intelligently
recognizing how to solve the
problem at hand. Primates
have been observed not only
using tools but also making
tools for new purposes,
both in captivity and in
the wild.

The problem-
solving skills of
octopuses are
numerous.
In the wild,
octopuses have
been known to
get fresh but dead
crabs by stealing them
out of lobster traps or
climbing aboard fishing
boats to hide in containers
of dead or dying crabs. In
Octopus controlled experiments, they
have shown the ability to figure
out how to open complicated
acrylic boxes or unscrew jar lids to
get the food inside. The complexity of the animal’s behavior is not the only
factor that demonstrates intelligence. Other components are the planning
involved in problem-solving and the ability to adapt behavior to meet goals
despite environmental change and complexity—the latter is an element that
appears often in the definitions of intelligence offered by AI researchers.

In the attempt to create AI, the question is how to achieve that flexibility.
Human beings are sensitive to patterns not only in the environment but also
in their own reactions to their environment. In humans, behavioral flexibility
works by way of reflection: the ability not only to react to a situation but
also to think about how to react and to critically examine those thoughts. In
many cases, this is achieved on a conscious level. And if flexible intelligence

12
 2. Intelligence: Real and Artificial

demands conscious reflection—or at least the capacity for such—two

different possibilities exist regarding people’s conceptions of themselves
and machines.

If flexible intelligence demands reflective consciousness and machines can

never be conscious, humans and much of the animal kingdom possess
a unique cognitive ability. However, if flexible intelligence demands
consciousness and it is possible to produce intelligent flexibility in machines,
it will need to be recognized that machines are conscious.

Multiple Intelligences
People’s mental lives involve more than intelligence alone, and many questions
abound about how intelligence relates to other aspects of the human mind.
How does intelligence relate to intuition? How does it relate to emotion?
What is the relationship between intelligence and creativity? Some argue that
many forms of intelligence exist. When it comes to the relationship between
intelligence and intuition, a widely accepted view is that humans have two
different but interrelated forms of intelligence.

The Nobel Prize–winning psychologist Daniel Kahneman distinguishes

between System 1 and System 2 intelligence, called thinking fast and thinking
slow, respectively. During waking hours, System 1 is always “on.” It comprises
immediate impressions, flash judgments, and feelings—things that involve
operating quickly and intuitively. System 2 is only triggered when needed,
when something is too complex for System 1 to handle. Standard intelligence
tests tend to measure only System 2. Based on Kahneman’s view, those tests
leave out a major part of intelligent processing: System 1, which is the system
that humans predominantly rely on.

The most controversial theory of multiple intelligences is associated with the

work of Howard Gardner, who proposed several distinct types of intelligence:
musical intelligence, linguistic intelligence, logic-mathematical intelligence,
visual-spatial intelligence, bodily kinesthetic intelligence, interpersonal
intelligence, and intrapersonal intelligence. Later in his career, Gardner
suggested a further intelligence: naturalistic intelligence, including a ready
ability to classify plants and animals.

13
 2. Intelligence: Real and Artificial

Gardner and his followers argue that intelligence is a multifaceted capacity.

The alternative view posits that the term intelligence should only refer to one
thing: the g factor, also called general intelligence, general mental ability, IQ,
or simply intelligence. There is an undisputed fact at the core of the theory,
which is evident when people take a series of tests, such as on vocabulary,
recognizing similarities, reading comprehension, picture completion, block
design, and arithmetic. People who do well on one of the tests also tend to
do well on the others; the results are correlated. This has been referred to as
“arguably the most replicated result in all psychology.” However, the existence
of a statistical g factor doesn’t refute theories of multiple intelligences.

Artificial AI versus Real AI

AI researchers recognize that computers are good at specialized abilities. For
example, the world’s best chess-playing computer can’t play a decent game of
checkers or even variations of chess in which some of the rules are changed.
Some researchers have said that what has been developed so far is artificial AI.
They’re looking for real AI, the kind that transcends the limits of specialized
skills. The goal is to develop artificial general intelligence—a machine with
the ability to learn in all areas; the ability to think in all areas; the ability to
satisfy internal goals by adapting to a changing environment, whatever that
environment is like; and reflexive cognitive flexibility across all topics.

In “Computing Machinery and Intelligence,” Turing wondered if humans

could build machines that could participate in communication exchanges
in such a way that it would be impossible to know whether the replies came
from a person or a machine. He called it the imitation game, now known as
the Turing test. Turing predicted that by the year 2000, a computer would be
able to play the imitation game so well that the “average interrogator will not
have more than 70 percent chance of making the right identification after five
minutes of questioning.”

He was wrong about the timing, but computer programs keep getting better.
There are claims that the Turing test has been passed by a chatbot posing
as a 13-year-old Ukrainian boy in one case and by an advanced version of
ChatGPT in another. In both cases, human-like spelling and grammatical

14
 2. Intelligence: Real and Artificial

errors helped fool the judges. But the Turing test assesses whether a computer
can simulate intelligence—which is still artificial AI. Researchers are aiming
for real AI.

It is important to note that a research symbiosis exists between computer

science and cognitive science. The exploration of AI is double-sided.
A solid understanding of human intelligence could be carried over to
implementations in computer science. Conversely, if a genuinely intelligent
machine could be developed, it could provide insight into the workings of
human intelligence. At this stage, the research uses input and interaction
involving both sides. Research in psychology and cognitive science is therefore
relevant to the ongoing attempt to build intelligent machines.

Reading
Kahneman, Daniel. Thinking, Fast and Slow. Farrar, Straus and
Giroux, 2011.
Van Pelt, Shelby. Remarkably Bright Creatures: A Novel. Ecco, an imprint of
HarperCollins Publishers, 2022.

Questions
1 Do you think intelligence requires consciousness? Does it require
free will?

2 Do you think intelligence is one thing or many?

3 How would you define intelligence? How would you test for it?

Other Media
Ehrlich, Pippa, and James Reed, dirs. My Octopus Teacher. Off
the Fence, The Sea Change Project, A Netflix Original
Documentary, 2022.
Scott, Ridley, dir. Blade Runner. Warner Brothers, 1982.

15
3
The
Foundations of
Artificial Intelligence

A t its core, today’s AI uses the ancient development of logic. At the

lowest levels of hardware and software, computers use the simplest
concepts of formal logic: and, or, not, and if then. The basic idea of logic is
a radically simplified representation of steps in thinking. The key to all logic
is a particular kind of simplification: The simple syntax of all and some or
and, not, and if then. The development of this formal logic traces back to
the 4th century BCE. Many figures in the history of philosophy pondered
theoretical issues of mind and machine long before those same questions
were phrased in terms of digital computers. In this lecture, you’ll explore the
early thinking and events that led to the development of today’s AI.

16
 3. The Foundations of Artificial Intelligence

The Development of
Logical Computation
In the 17th century, the German
polymath Gottfried Wilhelm
Leibniz envisioned a language
built to represent all
fundamental human
ideas, together with
a logical mechanism
for rational inference
in that language.
He believed that
such a tool
would be both
a mechanism
for intellectual
invention and
a way to settle all
intellectual debate.
“When there are
disputes among persons,”
he said, “we can simply
say ‘Let us calculate,’ and Gottfried Wilhelm Leibniz
without further ado, see who
is right.”

Alternatively, René Descartes

emphasized the contextual flexibility
of general human intelligence. “[R]eason is a universal instrument which can
serve for all contingencies,” he stated. “[I]t is morally impossible that there
should be sufficient diversity in any machine to allow it to act in all the events
of life in the same way as our reason causes us to act.”

The development of logic in the 19th and 20th centuries went far beyond
this early thinking, feeding directly into the development of computers. One
of the early 20th century landmarks was Russell and Whitehead’s Principia

17
 3. The Foundations of Artificial Intelligence

Mathematica, in which the authors tried to prove that all mathematics

reduced to logic. However, Turing’s work in the 1930s was the real turning
point in logical computation.

At the time, the notion of effective procedures or algorithms but had not
yet progressed beyond a vague idea. Turing was the first to produce a formal
model for what the concepts of effective procedures or algorithms might
mean. His basic idea was to think of step-by-step information processing
as something that a machine could do, and he even described the necessary
components for such a machine. He proposed that a Turing machine
could clearly handle logic, and if Russell and Whitehead were right, it
could therefore handle not only all mathematics but any step-by-step
symbolic transformation.

Turing’s vision of how an information-processing machine would have to

operate was extremely simple. In many cases, an idea’s simplicity gives it the
power of generalization. Any effective procedure or algorithm, on any kind of
symbols, would be performed by a simple Turing machine. Turing’s abstract
model even included the idea of a general-purpose computer that could be
programmed to perform any effective procedure or algorithm.

In the 1930s, no concrete machines could match Turing’s abstract idea.

That changed in the 1950s with the impetus of World War II and through
the work of John von Neumann and others. Turing’s abstract model of
information-processing machines paved the way for the actual machine.
Digital computing machines were built that were similar to today’s computers.
AI in the contemporary sense began to progress.

The Dartmouth Conference

AI is one of the few research categories that can be traced to a particular
individual at a particular time. In 1955, 28-year-old John McCarthy took
a position in the mathematics department at Dartmouth College. He wanted
to host a 2-month, 10-person summer project to explore the idea of thinking
computers. In applying for grant funding to run the conference, he said later,
“I had to call it something, so I called it ‘artificial intelligence.’” The meeting

18
 3. The Foundations of Artificial Intelligence

was to be based on “the conjecture that every aspect of learning or any other
feature of intelligence can in principle be so precisely described that a machine
can be made to simulate it.”

The conference, which took place in Dartmouth in 1956, is often cited

as the birthplace of AI. It assembled many people who became important
figures in the development of AI, including Claude Shannon, the father of
information theory; Arthur Samuel, who programmed the first computer to
play checkers; and Marvin Minsky, who founded and directed the Artificial
Intelligence Laboratory at the Massachusetts Institute of Technology. Some of
the participants were primarily interested in trying to understand the nature
of human cognition and intelligence—the cognitive science project. They
considered the attempt at AI a useful means toward that end. Others were
mainly interested in building smart machines, no matter how they operated—
the computer science project.

Those whose main goal was developing smart machines thought of

intelligence as a matter of information processing. Turing had made that
possible. A Turing machine could handle a program—a step-by-step
procedure. All that was needed was a program that captured intelligence.
This strand is now known as symbolic AI or GOFAI—good old-fashioned
AI. GOFAI’s initial accomplishments were astounding. Allen Newell and
Herbert A. Simon developed a working program that was able to prove 38 of
the first 52 theorems in Principia Mathematica. Minsky produced a program
that could construct geometrical proofs. The group was optimistic about
further developments.

For the conference group more interested in cognitive science, the inspiration
wasn’t Turing’s universal information-processing machine but the human
brain—the concept of neural networks. This group considered that the road
to understanding intelligence, and perhaps even to producing it, was not by
way of symbolic programming but by attempting to build something that
functions like the human brain. A major step in that direction is seen in the
work of the psychologist Frank Rosenblatt, who built on the work of Pitts and
McCulloch. Rosenblatt developed computer models of neurons that were able
to learn. He called them perceptrons.

19
 3. The Foundations of Artificial Intelligence

Rosenblatt’s
Perceptrons
The neurons in the brain
are cells that, on one
end, have dendrites
coming in from
other neurons—
sometimes
thousands of
dendrites.
Those are their
input lines,
which function
electrochemically
to bring signals from
other neurons to the
cell body. When input
from the dendrites reaches
a certain level in the cell
body, the neuron fires, again
Mark I Perceptron
electrochemically, sending
a pulse down an axon at the other
end of the cell. The process involves
multiple inputs from the dendrites
combining to produce neuron firing, sending an output pulse down the axon.

Rosenblatt’s perceptrons are mathematical routines in a computer, and they

function differently from the neurons in the human brain. A perceptron
takes numbers as inputs, rather than electrochemical signals, and it generates
a number as output in place of firing down an axon. Each input number
is multiplied by a unique weight on its way to a central calculation, where
the weighted inputs are added together. If the resulting sum is greater than
a given numerical threshold, the perceptron fires with an output number.
Thus, perceptrons function like neurons in which each electrochemical

20
 3. The Foundations of Artificial Intelligence

step is replaced with a simple mathematical calculation. All contemporary

AI relies on neural networks, and perceptrons were the first and simplest
neural networks.

For example, if the weather forecast for tomorrow is that it will rain and
that the temperature will be more than 80°, the forecast is only accurate if it
rains and the temperature is more than 80°—if both parts are true. That is
a logical “and.” A perceptron can give the logical equivalent of an “and” if it
is programmed to only output a 1 (for “true”) in the case that its two inputs
are both 1s. That’s a perceptron that works like a logical “and.” Similarly,
a perceptron can calculate the logical concept of “or” if it is programmed to
output a 1 (for “true”) if either input is a 1.

Rosenblatt’s perceptrons could not only be built to perform this logic but
could learn to perform this logic. An off-the-shelf perceptron can be trained
to work like an “and.” With a different training routine, it will learn to
function like an “or.” One term that comes up repeatedly in psychologists’
definitions of intelligence is learning. Perceptrons were the first computer
structure that could learn. Training them is a matter of adjusting the weights
until the right answer is achieved. This is called supervised learning. With lots
of examples, nudging the weights each time, perceptrons will learn the desired
logical concept.

The AI Winter
Both GOFAI and Rosenblatt’s perceptrons were the subject of much hype.
McCarthy aimed to build “a fully intelligent machine in a decade.” Simon
predicted that “machines will be capable, within twenty years, of doing any
work that a man can do.” Minsky stated that “within a generation … the
problem of creating ‘artificial intelligence’ will be substantially solved.” The
New Yorker claimed that the perceptron was capable of original thought,
a serious rival to the human brain. Rosenblatt himself spoke of building
brains that would be conscious of their own existence.

The two approaches from the Dartmouth conference were in direct and
intense competition for grant funding to support them. Rosenblatt’s approach
lost when Minsky and Seymour Papert, in their book Perceptrons, showed

21
 3. The Foundations of Artificial Intelligence

that there were simple logical concepts that perceptrons couldn’t learn,
such as “exclusive or,” the logical concept of this or that but not both. Sadly,
Rosenblatt died a few years later, on his 43rd birthday, in a boating accident
in Chesapeake Bay.

Despite its early successes, GOFAI crashed and burned on tasks that are easy
for people. Eventually, it became involved in expert systems, such as medical
diagnosis and treatment suggestions. It proved to be excruciatingly time-
consuming and exorbitantly expensive. Moreover, many areas of expertise
require pattern recognition, which was precisely the kind of task that GOFAI
wasn’t good at.

This pattern of events has often been repeated in the history of AI: initial
enthusiasm and overhyping, a massive influx of commercial and research
funds, and then a crash when results turn out not to be as advertised. By the
mid-1970s, both lines of research that began with such promise at the time of
the Dartmouth conference had fizzled in disappointment—this is known as
the AI winter.

Reading
Čejková, Jitka, ed. R.U.R and the Vision of Artificial Life. MIT Press, 2024.
(A new translation of Karel Čapek’s 1920 play Rossum’s Universal Robots
(R.U.R.), which introduced the term robot, with essays by contemporary
authors.)
McCorduck, Pamela. Machines Who Think: A Personal Inquiry into the
History and Prospects of Artificial Intelligence. 2nd ed. A K Peters/CRC
Press, 2004.
Pickover, Clifford A. Artificial Intelligence: An Illustrated History.
Sterling, 2019.

Questions
1 Do you think artificial intelligence would have developed the same
way without commercial funding? Without World War II?

22
 3. The Foundations of Artificial Intelligence

2 What are some of the things that seem easy for computers but are
hard for you? What are some of the things that seem easy for you but
hard for computers?

3 Is it possible that GOFAI and neural nets just represent different

kinds of intelligence?

Other Media
Lang Fritz, dir. Metropolis. Universum Film A. G., 1927. (A futuristic
vision of humanoid AI from the distant past, made in 1927 but with a
story set in 2026.)

23
4
How AI Works

B y the 1970s, the two approaches from the Dartmouth conference

seemed to have reached a dead end. Research and development
money dried up, and commercial interest evaporated. Research applications
and commercial proposals avoided the term AI altogether. But eventually,
the AI winter was followed by an astounding development, growth, and
flowering of AI in what is known as the AI spring—the final triumph of neural
nets. In this lecture, you will discover how the main components of today’s
AI work and explore some of its important limitations.

24
 4. How AI Works

Perceptron and Neural

Network Training
The training of a perceptron involves asking it a question pertaining to
a training set and, if it gives the wrong answer, adjusting the weights until
the correct answer is given. Next, the procedure is repeated with a different
training set. Using that procedure with more and more training examples, the
perceptron will get the answer right more frequently; it is, in fact, learning.
Once it is trained, a perceptron can generalize beyond its training set, giving
the right answer even in cases it has never seen before.

The selling point of perceptrons was that they could learn patterns. However,
what Minsky and Papert proved was that there were certain patterns that
perceptrons couldn’t learn, no matter how thoroughly they were trained.
The structure of perceptrons was too simple for them to learn the logical
concept of “exclusive or,” as they consisted of just an input layer and an output
layer. Another problem was that firing worked in terms of a sharp “yes” or
“no” threshold.
Even at that time, it was known that larger structures incorporating
perceptrons could recognize a logical concept such as “p or q and not both”
and could recognize all possible patterns, but they needed more than two
layers. A single perceptron is analogous to a single neuron—multiple dendrite
inputs, leading to a single axon output. What was needed was a neural net:
a multilayered neural network, with the first layer feeding outputs to points
on a second layer, where they become inputs for a third layer, and so on.
Inputs feed to multiple middle spots, which then serve as inputs to further
spots, eventually feeding down to the output.

The training technique used for perceptrons initially only worked for two
layers. The promise of neural nets was that they could learn, but no one knew
how to train those more complicated nets or how to make multilayered neural
nets learn. Eventually, however, the AI spring was brought on by three factors:
big data, computational power, and a way to train multilayered neural nets.

25
 4. How AI Works

The AI Spring
Perceptrons use a threshold firing function. If the weighted inputs add up to
anything over the threshold, regardless of the amount, a perceptron answers
with a definite “yes”; if not, it answers with a definite “no.” But the sudden
thresholds made it impossible to pass training from one layer to another in
multilayered neural nets. This problem was solved by replacing the abrupt
thresholds with more gradual ones. Gradual changes in added input result in
gradual changes in output; thus, this adjustment replaced the all-or-nothing
threshold activations with smooth activation functions.

The ability to train multilayered neural nets was one factor in initiating
the AI spring. The basic perceptron training pattern remains the same,
although the weights may need to be adjusted on all the inputs on all
the layers. If the network gives the wrong answer, each input weight is
adjusted proportionately, working backward through the net. This is called
backpropagation of errors—progressively assigning blame for a wrong answer,
nudging weights backwards through the net.
A neural network with hundreds or thousands of layers and millions of
weights requires significant computer power. In 1975, Gordon E. Moore,
one of the founders of the computer chip manufacturer Intel, predicted that
computer power would double about every 2 years—an exponential rate of
increase—measured in terms of the number of transistors on an integrated
circuit. This is referred to as Moore’s law, and much of the history of
computer technology has followed that pattern closely. The development of
massive computer power was the second factor in producing the AI spring.

Multilayered neural nets need a large amount of data for training. The deep
learning of deep neural nets requires big data, particularly for complex tasks
such as general image recognition and natural language processing. That
data can be found on the internet, to which people have uploaded many
billions of pictures. In addition to images, there are large amounts of political,
sociological, medical, and scientific data. The availability of that massive
body of potential training data was the third crucial factor in initiating the
AI spring.

26
 4. How AI Works

Multilayered neural nets remain the core technology, but variations continue
to be made in how neural nets are trained and used. In addition to supervised
learning, which involves training on labeled data, unsupervised learning
can be used. With unsupervised learning, the neural net is given the task of
finding patterns in a data set but is not told what patterns to find; it must
find patterns on its own. Unsupervised learning has become widely used
in AI. Some of the patterns that neural nets find in big data are patterns of
use. Unsupervised neural net learning therefore plays a significant role in
controlling fraud and is at the core of recommender systems.

Neural Net
Limitations
While neural nets are
modeled after real neurons,
they have little in
common with them.
Artificial neurons are
simple electronic
devices. Real
neurons are
complex cells
that function
electrochemically.
The chemical
aspect of connection
at synapses—
drawing from a sea of
neurotransmitters—is
far more complex than the
simple digital processing of Brain neuron
network
artificial neural networks. All
artificial neurons are basically the
same. Real neurons are said to be

27
 4. How AI Works

the most diverse kinds of cells in the human body. There are thousands of
different types of neurons, using different neurotransmitters, receiving and
sending messages in different ways.

Given a particular input, artificial neurons just fire once. The output of
real neurons involves particular frequencies of repeated firing. On the input
side, neurons are sensitive to not only incoming signals but also their pattern
or frequency. This frequency coding is crucial to how neurons function in
linkage with other neurons in the brain.

And once a neural network is set up, its architecture—the structure of

links between different levels—is essentially frozen. That is not true of real
neurons. New neurons are added, particularly when the brain is young but
throughout a lifetime of learning as well. Neurons can even migrate to where
they are needed.

The brain is a massive net of neurons. Those neurons operate simultaneously

and in parallel. Patterns of neurons are synchronized across the brain.
Although AI increasingly uses parallel processing for reasons of efficiency,
there is no analogy to the widespread simultaneous synchronization
characteristic of the brain.

Artificial neural nets don’t come close to the complex functioning of the
brain, but they are much faster. Computer chips are already running at tens
of millions of times the speed of human neurons and are getting faster all the
time. In one major respect, however, artificial neural nets are like the brain:
Very often, it is not known exactly how they work. That is a severe limitation,
with both technical and ethical consequences.

A Lack of Transparency
Among the participants at the Dartmouth conference, two different goals and
two different approaches were evident. Some of the participants wanted to use
AI to understand human intelligence. They ended up leaning toward neural
nets as models of the brain. Other participants just wanted smart machines.
They leaned toward GOFAI programming. In the end, it wasn’t GOFAI

28
 4. How AI Works

but neural nets that led to today’s smart machines. Yet, neural nets have not
provided the desired understanding of intelligence or brains because why
neural nets do what they do is often not fully understood.

Humans have set up the basic architecture of the network—often itself a

matter of guesswork—and have trained it on a massive body of data. The
network may end up doing precisely what it is meant to do, but how it does
that is not clear. This problem is a lack of transparency; the operation of
neural nets is opaque. The more powerful the networks, the more mysterious
their operation. A look inside them doesn’t show how they work, and even
their designers can’t be expected to know. That alone raises ethical issues
of responsibility.

The lack of transparency also points to immediate technological limitations.

There are concerns that neural networks might not have learned what they
were supposed to have learned. The lack of transparency can be a real problem
when AI is used for applications in industry, medicine, finance, the military,
and criminal sentencing, which is increasingly the case.

It also turns out that neural networks can be fooled. In one study, researchers
were able to trick an image identification network into giving the wrong
answer by manipulating a few pixels in a photograph just slightly—so
slightly that a person could see no difference. With that small change, the
researchers could get the network to identify a school bus as an ostrich. In
another study, researchers were able to use a similar pixel trick to fool a face-
recognition neural network. These are known as adversarial examples and
a warning: Because of the lack of transparency, neural networks may come
with not only inadvertent limitations and blind spots but also loopholes open
to manipulation.

Reading
Bringsjord, Selmer, and Naveen Sundar Govindarajulu. “Artificial
Intelligence.” Stanford Encyclopedia of Philosophy Archive (Fall
2024). [Link]
intelligence/.

29
 4. How AI Works

Mitchell, Melanie. Artificial Intelligence: A Guide for Thinking Humans.

Farrar, Straus and Giroux, 2019. See esp. chap. 6, “A Closer Look at
Machines That Learn.”

Questions
1 Are there ways that you learn that are like neural nets? Are there
ways that you learn that are entirely different?

2 If you were developing the next stage of artificial intelligence, what

aspects of brain function do you think might offer important clues?

3 What are some of the ethical problems raised by the lack of

transparency in neural nets?

30
5
Minds, Machines,
and the Limits of AI

P owerful as they are, the neural networks at the core of today’s AI

have crucial limitations. The mechanized training of neural nets on
big data may give the impression that the designing of neural nets is itself
mechanized, but that is not the case. The design of information-processing
machines still requires human minds. People decide how many inputs
will turn out to be the right number and which data characteristics will
be important. People determine how many levels multilevel neural nets
should have to accomplish the desired goal without wasting computational
resources. In this lecture, you’ll dive deeper into the technological
limitations, questions of minds and machines, and challenges leveled
against the idea of AI.

31
 5. Minds, Machines, and the Limits of AI

The Dark Art of Design

Neural net design for a particular purpose is regarded as one of the dark arts
of computer science. It calls for experience, judgment, and what has been
referred to as an almost cabalistic tacit knowledge—an intuitive grasp of both
the problem to be solved and how neural nets might handle it. It also involves
a lot of guesswork.

One of the problems designers face is called the curse of dimensionality.

Whether a neural net captures the desired reality depends on how many
characteristics are important in that reality and how many of those
characteristics the neural net captures, which cannot be known in advance. It
might seem safe to allow for more factors rather than fewer, but each added
factor adds a dimension, increasing the neural network’s complexity. Not only
do complexity and dimensionality increase, but they do so exponentially.

Problems also arise in the training of neural nets. Neural nets can generalize
from their training data. For example, with proper training, a neural net
can recognize cats in pictures to which it has never been exposed before.
A neural net can be trained to recognize particular faces, suspicious financial
transactions, or malignancies in medical images by applying the lessons it
learned from its initial sample data to new and expanded data.

A common problem is that a neural net can train too well on its initial
sample. In the example of cat images, it is trained to output “cat” for all and
only those images in the training set that are labeled as cats. It may learn to
do a perfect job of that but then recognize only or almost only the cat images
in the initial training set. That is called overfitting; whatever the neural net
has learned is too closely tied to its training sample, preventing the ability
to generalize.

Can Machines Have a Mind?

Network structure or architecture problems, the curse of dimensionality,
overfitting, and opacity are real issues in AI design and interpretation, but
they pale in comparison with deeper challenges that have been leveled against

32
 5. Minds, Machines, and the Limits of AI

the very idea of AI, not only in its current form but in any form. Prominent
minds, historical and contemporary,
can be found on opposite sides of
the issue.

The history of AI was guided by

a vision that perhaps genuine
intelligence could be
formally or mechanically
produced. Alan Turing
offered a conceptual
blueprint for the
construction and
programming
of the real
computers
that were to
come. The actual
construction of
computing machines
took place during and
shortly after World War
II. A key figure in that
development was John von
Alan Turing’s Automatic
Neumann, whose vision went Computing Engine
beyond computing to full-blown
AI. He thought machines could
match the mind, without cognitive
limits. In von Neumann’s words, “You
insist that there is something a machine cannot do. If you tell me precisely
what it is a machine cannot do, then I can always make a machine which will
do just that.”

That vision was represented in the proposal for the Dartmouth conference
of 1956, credited with the first use of the term AI. The conjecture of
the conference was “that every aspect of learning or any other feature of
intelligence can in principle be so precisely described that a machine can be
made to simulate it.”

33
 5. Minds, Machines, and the Limits of AI

The questions remain: Are those optimistic visions actually possible? If given
a precise enough description of what minds do, could a machine do anything
and everything that a mind can? Could a machine have a mind? Could
a machine be a mind?

Weak and Strong AI

A distinction is often drawn between strong and weak AI. Weak AI already
exists, as it refers to the ability to perform at the human level on specific tasks.
Computer programs can match and even exceed humans at playing video
games, checkers, chess, and the ancient game of go. The current machines can
match human performance in many forms of pattern recognition, in language
translation, and in many other so-called mental tasks. What makes those
weak AI is that a given machine or computer program is strictly constrained
to the specific tasks for which it was trained.

Weak AI has a transfer problem. None of the programs designed for

a particular game can transfer their mastery of that game to anything even
slightly different. In chess, rooks can move up, down, or sideways any number
of squares. If the rules are changed so that rooks can move only two squares
at a time, people can flexibly transfer their knowledge of chess to compete in
that slightly different environment, but chess-playing programs don’t have
that adaptive flexibility. Weak AI is therefore limited to the special purpose
for which it has been trained.

A widespread goal is general AI—as broad, flexible, and adaptive as human

intelligence. Human intelligence doesn’t need to be trained from scratch
for each confined and specific task. It easily transfers ability in one area to
a related area. General AI would represent a core integration of cognitive and
learning abilities with the flexibility of human intelligence. But despite the
development of many specific weak AI abilities, the goal of general AI seems
as far away as ever.

Computer science researchers equate general AI with strong AI, but many
philosophers consider that strong AI would require genuine conceptual
understanding. In that sense, the goal would be machines that have all
the mental powers humans possess, including deliberate decision-making,

34
 5. Minds, Machines, and the Limits of AI

curiosity, creativity, feelings of pain and contentment, and phenomenal

consciousness—the entire realm of subjective experience. Strong AI wouldn’t
demand machines that might be mistaken for minds but machines that have
minds in the full sense. Both the challenges to the goal of strong AI and the
responses to those challenges revolve around those deeper human capabilities.

Conflicting Visions
Lady Lovelace had a prescient vision of what a computer could be, but she
emphasized that it could not be creative: “[T]he Analytical Engine has no
pretensions to originate anything. It can do whatever we know how to order it
to perform.” Turing, in “Computing Machinery and Intelligence,” considers
several objections to his claims regarding thinking machines. He questions
whether human intelligence is itself creative or simply combines ideas and
inputs from somewhere else. The defenders of strong AI can claim that the
problem is a lack of knowledge about creativity, making it impossible to know
what to ask a machine to exhibit.

A better psychological understanding of what creativity is would improve the

ability to determine whether a machine could be genuinely creative and, thus,
provide a better answer to the question of whether a strong AI that demands
genuine creativity is or isn’t possible. One of the strongest philosophical
arguments that strong AI in the full philosophical sense is not possible is
a thought experiment by the philosopher John Searle, who suggests that real
understanding cannot be programmed.

Searle’s argument also works as a critique of the Turing test. It seems to

show that such a test is irrelevant with regard to the questions of human
capabilities at issue in strong AI. A machine might simulate understanding
without having real understanding. According to Searle, the argument shows
that whatever the human brain does that generates understanding—and
consciousness in general—it is not merely running a program in the way that
a computer does.

The philosopher Daniel Dennett takes the opposite position: “I think it’s
absolutely possible that we can have conscious robots, conscious AI.” But there
are philosophical subtleties. Dennett argues that consciousness is actually

35
 5. Minds, Machines, and the Limits of AI

a matter of simultaneous information processing, synchronized across many

different areas of the brain. That parallel processing is something he thinks
a machine could be capable of as well. However, although he believes it’s
possible to develop conscious robots, he thinks it isn’t necessarily a good idea:
“[I]f we really succeeded, then they would be precisely as autonomous as we
are. And we are very dangerous.”

David Chalmers describes the difficulty in imagining a conscious machine

as “the hard problem of consciousness.” The seat of consciousness is clearly
the human brain, but it represents all subjective experiences—what it is
subjectively like to be human. The brain itself is approximately 3 pounds
of wet gray matter located in the skull, and it is difficult to grasp how it is
possible for that matter to produce the wealth of phenomenal experience.

This once again emphasizes the challenge of computer science meeting

cognitive science in issues of AI. To determine whether a machine can be
conscious, it is necessary to really understand what consciousness is, how
subjective consciousness happens in a physical brain, and how human
beings are conscious. Many goals of AI are premised on the idea that there
is one thing to aim for: intelligence. Perhaps the kind of intelligence that
machines will be great at is Daniel Kahneman’s System 2—formal systems,
logic, mathematics, and complex games. That type of AI, even general AI
of that sort, may well be within reach. But System 1 intelligence—quick
and intuitive judgments, with conscious feeling—may be beyond anything
machines can ever have.

Reading
Bison, Terry. “They’re Made Out of Meat.” [Link]
theyre-made-out-of-meat-2/. (A very short story.)
Churchland, Paul M., and Patricia Smith Churchland. “Could a Machine
Think?” Scientific American 262, no. 1 (1990): 32–39. https://
[Link]/wp-content/uploads/2020/05/1990-Could-A-
[Link]. (A critique of Searle’s Chinese Room.)
Hofstadter, Douglas, and Daniel Dennett. The Mind’s I: Fantasies and
Reflections on Self and Soul. Basic Books, 2000.

36
 5. Minds, Machines, and the Limits of AI

Searle, John. The Mystery of Consciousness. The New York Review of

Books, 1997.

Questions
1 Do you think a machine could surprise you? Would that be enough
to show that it is creative?

2 What is your take on the “hard problem of consciousness”? How

can three objective pounds of wet matter produce the subjective
experience of consciousness?

Other Media
Garland, Alex, dir. Ex Machina. University Pictures and A24, 2015.
(Testing a humanoid robot for consciousness forms part of the plot.)
Morris, Errol, dir. Fast, Cheap, and Out of Control. Sony Pictures Classics,
1997. (This is documentary of interviews with a lion tamer, a topiary
gardener, a naked mole rat specialist, and the roboticist Rodney Brooks.
Only Brooks is directly relevant to this lecture, but the film as a whole
is as peculiarly entertaining as it is simply peculiar.)

37
6
AI Comes of Age

M ultilayered neural nets paved the way for the AI spring, and they
remain the core engine of AI. But the developments of the 21st
century have allowed those neural nets to be used in even more powerful
ways, with success in a wide range of applications. AI has really come into
its own, contributing to and sometimes dominating humans’ lives in radically
new ways. In this lecture, you’ll take closer look at the technology’s state
of the art and explore two of its new forms—reinforcement learning and
ChatGPT—as well as what the current AI can do and how it does it.

38
 6. AI Comes of Age

AI Is Good at Games
Games have repeatedly been
targeted for the development
of AI because they have clear
and simple rules—the type
that should be possible to
program—and a clear
criterion of success,
namely, winning. A
milestone in chess
programming came
just before the
start of the 21st
Man playing chess
century, when
IBM’s Deep Blue
computer beat
the reigning world
chess champion. Deep
Blue was not a true
neural net. It relied on
sheer number-crunching
and the fact that it could
store massive data, operate at
lightning speed, and follow the
branching possibilities of future
board positions much faster and
farther than people can.

Another milestone in the application of AI to games occurred in 2011, when

IBM’s Watson beat two human champions in the game show Jeopardy!
In addition to its access to a massive database, Watson had a neural net
component that was trained to handle the show’s peculiar format, using
archived examples of Jeopardy! answers and questions to parse clues and
prioritize possible responses.

39
 6. AI Comes of Age

Neural nets’ deep involvement with games came with reinforcement learning.
Most of what humans learn isn’t the result of supervised training but comes
through interacting with an environment that rewards some actions and
penalizes others—positive reinforcement and negative reinforcement. The key
to AI’s world-level gaming was reinforcement learning in an environment in
which the positive reinforcements were game scores and wins.

The research group DeepMind made many breakthroughs in complex games.

They started with simple games, such as the video game Breakout. The idea of
Breakout is to move a paddle to repeatedly bounce a ball overhead, knocking
out a line of blocks. The more blocks are knocked out, the more points are
gained. The team built into the neural net that it would favor moves that
would maximize points. Training involved the net learning series of moves
with high payoff, but this did not require supervision; the game itself supplied
the payoff in the form of points, reinforcing successful patterns of play.
DeepMind’s network not only learned to play Breakout but even discovered
clever strategies in the process.

Reinforcement and Self-Learning

After their success with simple video games, DeepMind researchers moved
on to more complex games. Reinforcement became more complicated. In the
game of chess, there are no reinforcing points for individual moves. The only
reward comes from winning at the end—a difficult target for training. Thus,
individual moves need to have values assigned to them. With the winning
move as the endpoint and working backward, values can be assigned to
possible board configurations, with associated values for the moves that can
lead to them. This makes it possible to give reinforcement rewards for specific
moves that increase the chance of an eventual win.

The idea of assigning values to all possible moves in all possible configurations
is beyond the reach of even the fastest machines. However, neural nets can
play games of chess and learn some of the important values from the course of
play. By playing millions of games, they can learn which patterns of play have
a high probability of leading to a win. They can even learn winning strategies
through self-play—playing millions of games against themselves.

40
 6. AI Comes of Age

Reinforcement training led to the

conquest of an even more complex
game. Go is played on a 19-by-
19 board of squares, which
starts blank. Players take
turns putting their tokens
on the board—one player
using white tokens, the
other using black.
The goal is to
capture territories
by surrounding
either empty
spaces or the
other player’s
tokens with one’s
own. DeepMind
called the network
used for this game
AlphaGo. It was trained on
data from 150,000 human
games of go, then left to learn Go board
by reinforcement learning in
self-play. DeepMind’s goal was
to have AlphaGo beat the world’s
best player of go, and in March of
2016, a five-game match between AlphaGo and the Korean Lee Sedol took
place. AlphaGo made aggressive, nonstandard moves and won four of the five
games, beating the best human go player.

DeepMind decided to try the idea again, this time without using human
games in the training. The neural net would learn the rules, then learn
through reinforcement learning by playing itself. That variation was called
AlphaZero, and it routinely beat AlphaGo. The neural net performed better
without human help. The next step in the Alpha series was AlphaFold, the
development of which was awarded a Nobel Prize in chemistry. AlphaFold
opens up significant prospects for pharmacology and medicine. Further
applications of reinforcement learning are bound to follow.

41
 6. AI Comes of Age

The Generative Pre-

Trained Transformer
The second major 21st-century accomplishment in AI is the mastery
of natural language and conversational communication through the
development of ChatGPT and related systems. These systems package
information with impressive completeness and confidence. The programs
are essentially next-word prediction programs. They start with a line of text
and predict the next word. That gives them a slightly longer text, which
they use to predict a next word, and so forth. The result appears to be
a sensible response.

GPT stands for Generative Pre-trained Transformer. The generative aspect

is its ability to generate new output. It generates an answer to a question or
some other kind of text based on the prompt. The core of GPT is a massive
neural net of hundreds of layers and trillions of weights. Each time that neural
net is applied, a calculation happens at each of those weights. It is trained
on sequences of words taken from the vast database of available material—
millions of digitized books and hundreds of billions of human-written web
pages. In training, the net is given a sequence of words from that database and
is asked to guess the next word. If it’s wrong, weights are nudged toward the
right answer. The trained network can generalize; it can guess a next word
when given a series of words it has never seen before.

That neural net at the core of GPT is used each time the system is used, but
it’s not trained each time. It is pre-trained, once and for all, and used over and
over again. The result of the training can be conceptualized as a language
map that lays out the 50,000 most common words in terms of their
relationships, with words that belong in the same category grouped together.
Groups that are similar are close to each other, while those that are different
are farther apart. It is crucial for GPT that its map represents relationships
between words that occur together and words that tend to follow other series
of words. That allows it to predict the next word in a series. The power of
GPT lies in the fact that it can generalize well beyond its training sample.

42
 6. AI Comes of Age

The transformer idea is perhaps the most innovative aspect. The transformer
is a two-step form of processing, with the second step using the word-map
neural network. The first step is a process called attention, which refers to
attention to context. It was initially introduced for purposes of translation,
for which attention to context is crucial. The attention step of a transformer is
sometimes called self-attention. It codes each word in a line of text in terms of
the other words that are important in understanding it.

In the second step of the transformer, the words and their attention codes
are fed through the massive neural net. In GPT, a line of text is processed
through many transformer levels of this kind, attention-coding the words at
each level and then feeding them through the central neural net. Each level
further refines the result, and it is only at the last step of refinement that
a next word is generated. The entire process is then repeated to generate the
next word. The stack of two-step transformers generates text in nanoseconds.

There are almost always multiple candidates for that next word, assigned
different probabilities in the semantic space of the database. If the computer
were to always pick the most probable next word, the resulting text would
sound flat and mechanical. The solution, standardly built into GPT systems,
is some randomness, occasionally picking next words that are not the most
probable. That makes the output less predictable, more human, and more
interesting. It also explains why the system gives slightly different answers
when it is asked the same question twice.

It is important to be aware that GPT gives the appearance of being well

informed. The fact that tacking on the next most probable word in a series
results in a grammatical sentence is no guarantee that it will be true. There
are many cases in which GPT has given laughably wrong responses, but not
all the misinformation will be humorous or even harmless, and developers
warn users to always check the generated outcome. Much of the current work
in AI is an attempt to add reasoning models trained through reinforcement
learning to complement GPT and compensate for some of its shortcomings.

43
 6. AI Comes of Age

Reading
Twain, Mark. The Jumping Frog: In English, Then in French, and
Then Clawed Back into a Civilized Language Once More by
Patient, Unremunerated Toil. Leopold Classic Library, 2015. Originally
published 1865. [Link]
(A humorous example of the perils of word-by-word translation.)
Wolfram, Stephen. What Is ChatGPT doing … and Why Does It Work?
Wolfram Media, 2023.

Questions
1 Do you think there is some innate structure that humans bring
to the task of learning language? Or do we learn language “from
scratch,” by mere exposure?

2 Try to work the way GPT does. Add a plausible word to fill in
the blank:

3 It was a dark and _____.

4 Now add another word. And another.

5 Most people learn chess “forward,” starting with opening moves.

The chess legend Bobby Fischer taught chess “backward,” starting
with simple end-game positions: “What single move would give you
checkmate?” Which do you think is a better approach? How about a
combination?

Other Media
Kohs, Greg, dir. AlphaGo. Documentary. Moxie Pictures, 2017. (Available
on YouTube.)
Kohs, Greg, dir. The Thinking Game. Documentary. Cityspeak Films and
Reel as Dirt, 2025. (A documentary on Demis Hassabis and DeepMind
from video games to AlphaGo and AlphaFold. Available on a number of
platforms.)

44
7
AI and the Wisdom
of Crowds

I n Discourse on Method and Meditations, Descartes describes that he shut

himself away in a hot little room “at full liberty to discourse with myself
about my own thoughts.” His goal was to acquire knowledge—knowledge
with certainty—but humans seldom acquire knowledge purely as individuals.
What people know of language, history, science—even what is known
about knowledge—is a product of social interaction with a social body of
knowledge. Yet, philosophy has largely ignored the essential role of social
knowledge—the role of social epistemology. To understand the information
processing that AI, it is necessary to understand the social mechanisms
involved. In this lecture, you’ll explore social information processing and
group intelligence and discover how they relate to AI.

45
 7. AI and the Wisdom of Crowds

Collective Wisdom
The internet is composed of billions of web pages. Contemporary
information-processing AI, including generative AI such as ChatGPT, is
built from those web pages. Any information supplied by such programs
is information scraped from the
internet and is therefore inherently
social in origin. Any intelligence
evident in such programs is
inevitably derivative—in
a sense, it is the people’s
collective intelligence,
repackaged and fed back
to them. Collective
information
processing can
be extremely
powerful, and
AI therefore
comes with
both a promise
of potential gains
and a threat of
serious risks.

Sir Francis Galton,

a British scientist and
founder of the field of
Francis Galton eugenics, had two obsessions:
accurate scientific measurement
of mental abilities and selective
breeding. In 1906, he attended a
livestock show where people were invited to guess the weight of an ox that was
on display. Eight hundred people entered the contest—all kinds of people,
many considered by Galton to be intellectually inferior. In letters to the
journal Nature, Galton wrote, with aristocratic snobbishness, “[T]he average

46
 7. AI and the Wisdom of Crowds

competitor was probably as well fitted for making a just estimate of the …
weight of the ox as an average voter is of judging the merits of most political
issues …”

When the contest was over, Galton laid the recorded guesses out in order,
from lowest to highest weight. He then found the median guess, which was
1207 pounds. The actual weight of the ox was 1198 pounds—just 9 pounds
less. The average of all the guesses was 1197 pounds, just 1 pound off the
actual weight—an error of less than one-tenth of 1%. This is an example of
the wisdom of crowds.

Similarly, in the classic version of the game show Who Wants to Be a

Millionaire, the studio audience that at times would be called upon to help
a contestant would be composed of all kinds of people, each knowledgeable
in some but ignorant in other areas. Yet, their majority vote turned out to
be right 91% of the time. By comparison, the phone-a-friend option, which
allowed a contestant to call a knowledgeable friend, resulted in the correct
answer just 65% of the time. The statistical explanation for the wisdom of
crowds is that people who don’t know the correct answer will make random
guesses, and the incorrect guesses will cancel each other out.

The Wisdom of Diversity

Another way that collective wisdom—such as the wisdom of the internet—
can be powerful is through the phenomenon of diversity. Lu Hong and Scott
Page recently explored the idea that “diversity can trump ability.” They
programmed a computer model in which an assortment of artificial agents
tried to solve a problem. The model was programmed to represent a variety
of abilities. The researchers put together two teams of agents. One team was
composed of the very best agents; the other team’s members were picked at
random. Under the conditions laid out in the Hong–Page model, the random
team consistently performed slightly better.

Hong and Page explained that diversity can trump ability because the
members of a team consisting of those with the best scores individually will
think much alike, exploring a problem in much the same way and leaving

47
 7. AI and the Wisdom of Crowds

many possible approaches unexplored. A randomly selected team will have the
strength of diversity. It will embody many more perspectives and will explore
more of the possible approaches, often finding a better answer.

The wisdom of crowds is on full interactive computerized display in what are

known as prediction markets. A prediction market creates contracts that will
give a payoff if a certain event occurs. How much someone is willing to pay
for a given contract is a measure of how probable they think an event is, and
with lots of people buying and selling, the contract’s price on the open market
is a measure of how probable buyers and sellers think the event is—a wisdom-
of-the-crowd estimate.

Prediction markets have been highly successful. The Iowa Electronic Market,
founded in 1988, has a near-perfect
record in predicting the results of
Marquis de Condorcet
major elections, far better than
traditional polling. TradeSports
has a similarly outstanding
record in predicting sports
scores. Large firms,
including Microsoft
and Hewlett Packard,
now use prediction
markets to forecast
both sales
and product
success.

There is a classic
probability result
in political science
that also supports
the wisdom of crowds.
Condorcet’s jury theorem
was developed by the
Marquis de Condorcet at the
time of the French Revolution.
The theorem holds that the

48
 7. AI and the Wisdom of Crowds

probability of a majority vote getting the answer to a question right is better

than the probability of any of these voters getting the answer right on their
own. As long as each voter has a higher probability of getting the answer right
than wrong, no matter how small the difference, with enough people voting
independently, it becomes almost certain that the majority decision will be
the correct one. Condorcet’s theorem is positive justification for democratic
decision-making.

The Dangers of Collective Ignorance

Today’s AI is built on and can exploit this collective wisdom. ChatGPT
and similar programs are trained on the length and breadth of the internet;
therefore, their answers reflect the internet wisdom of crowds. However,
this does have a negative side, a phenomenon from the dark side of
social epistemology.

Condorcet’s theorem holds in the negative direction just as strongly as it does

in the positive direction. Thus, if each independent voter’s probability of
getting the answer right is less than 50%, it becomes almost certain that the
majority decision of enough individuals will be wrong. Majority decisions
amplify individual knowledge, but they also amplify individual ignorance.
And people don’t form their opinions in a vacuum; they are influenced by
others. Especially in today’s internet- and AI-saturated environment, voters
are not likely to form their opinions independently.

The problem of an independence assumption in the Condorcet result

shows up strongly in another phenomenon from the dark side of
social epistemology—the phenomenon of misinformation cascades.
A misinformation cascade occurs when the first few people act on imperfect
information and happen to make the wrong decision. Even if everyone after
that would have made the right decision if they had acted on their own,
they end up making the wrong decision because they pay attention to what
everyone else has decided. The result is that everyone ends up making the
wrong decision just because the first few people did. Misinformation cascades
are quite common. False rumors and propaganda that go viral, runs on banks,
financial bubbles, and market crashes all characteristically take the form of
misinformation cascades.
49
 7. AI and the Wisdom of Crowds

Opinion polarization is a further phenomenon on the dark side of social

epistemology. Views on issues have become progressively polarized. The
theory behind misinformation cascades offers a partial explanation for
this phenomenon, and it involves an element of trust. People use the first
information they receive as a mark against the source of contrary information
they receive later. In other words, the earlier message poisons their trust in
the later messenger. Typically, some of the conclusions people draw from
information pertain to the reliability of the sources from which they get that
information. And once people have decided which information to believe,
they will trust any information that supports their initial inclination on one
side or the other.

If the first information different groups of people receive oppose each other,
eventual opinion polarization of radical dimensions seems inevitable. The
current informational environment of AI searches and the internet is bound
to make that polarization dynamic worse, simply because people are able
to find information sources that will support virtually any opinion they
happen to start out with. They seek out informational silos and end up in
opinion echo chambers. New information is interpreted in ways that vindicate
existing opinions.

To understand the informational environment in which society is immersed,

it is crucial to apply the tools of social epistemology. Descartes shut himself
away to think things through for himself because he had come to doubt the
solidity of a whole range of what he had been taught—“the multitude of errors
that I had accepted as true in my earliest years.” He decided to set time aside
for independent thinking. Lack of independent thinking fuels many of the
negative phenomena. Somewhat paradoxically, people must be independent
thinkers to really benefit from social intelligence.

Reading
Page, Scott E. The Difference: How the Power of Diversity Creates Better
Groups, Firms, Schools and Societies. Princeton University Press, 2007.
Surowiecki, James. The Wisdom of Crowds. Anchor Books, 2005.

50
 7. AI and the Wisdom of Crowds

Questions
1 This lecture included cases of the wisdom of crowds. Has there been
something in your own experience that fits that picture?

2 This lecture also included negative cases of information cascades.

Has there been something in your experience that fits that picture?

3 Genuinely independent thinking is hard. What are your major

sources for national news? Can you tell how much you turn to those
sources because they reinforce views you already hold?

Other Media
McCabe, Daniel. “The Wisdom of Crowds.” PBS NOVA, April 24, 2018.
[Link]
(A PBS NOVA replication of “Galton’s Ox” with a crowd guessing the
number of jelly beans in a jar.)

51
8
Evolution and
Artificial Intelligence

B oth biology and philosophy changed with Charles Darwin, who

profoundly altered people’s view of the world and their place in it.
Before Darwin, the apparent biological order and perfection of nature were
generally taken as evidence of intelligent design and divine creation. That
assumption changed because Darwinian evolution offered an alternative
explanation. Without disproving the creation theory, Darwin offered an
alternative explanation for the astounding and intricate ways in which
biological organisms are adapted to their environments. His explanation fit
the data without requiring the assumption of a divine creator. In this lecture,
you’ll explore evolution and intelligence and the use of evolution in AI.

52
 8. Evolution and Artificial Intelligence

Artificial Selection
Darwin started by discussing not
natural selection but artificial
selection. In the world of
domesticated animals,
selective breeding has long
been practiced. Selective
breeding means matching
a specimen with the
desired traits with
another that has
complimentary
traits. For
example,
successful
racehorses are the
result of selective
breeding from
successful racehorses,
a process that is clearly
reflected in thoroughbred
pedigrees. New highly
productive corn varieties Charles Darwin
are produced by selecting the
most productive corn plants and
hybridizing them.

Artificial selection involves starting with a population, picking those

individuals that have the desired features, and using those for further
breeding. All food plants and domestic animals are the products of hundreds
and thousands of years of selective breeding.

The algorithm that drives Darwinian evolution works in precisely the same
way—by selective reproduction. The only difference is that the natural
environment does the selecting. Reduced to its essentials, the program of
Darwinian evolution requires just two things: a population with a variety of
inheritable traits and selective reproduction on the basis of those traits.

53
 8. Evolution and Artificial Intelligence

Evolution is an optimization program: It optimizes populations of organisms

for survival in their environment. That same environment can also include
other evolving groups of organisms. The result is the complexity of co-
evolving species and evolving ecosystems. The biological order and perfection
of nature self-assembled through millions of years of natural selection.
Natural selection is often explained as survival of the fittest, in which case
“the fittest” needs to be interpreted as those best adapted to survive. The result
of the evolutionary algorithm is the survival of those kinds best adapted
to survive.

Genetic Algorithms
John Holland was the central figure in the development of genetic algorithms
and evolutionary computing. He believed that the basic principles of
biological evolution could be used in designing computer programs.
Indeed, genetic algorithms have become a key component of the tool kit
of contemporary AI. Wherever optimization is the goal—from designing
airplane wings to optimizing the structure of neural networks—genetic
algorithms are in play.

Biological evolution produces better-adapted combinations of inheritable

traits through mutation—random changes in the DNA—and by sexual
reproduction. In genetic algorithms, analogues of both of those techniques are
used. Genetic algorithms try to solve each problem as a form of AI, and the
program works just like Darwinian evolution.

For example, the well-known board game Clue, or Cluedo, with the added
complexity of having to guess the date of the crime, may start with a
population of 20 guesses. These are random combinations, with each guess
comprising five components: person, weapon, place, month, and day. The
guesses can be scored in terms of how many of those components in the
guess were right. The score won’t indicate which ones were right, just how
many. Some guesses will score poorly, while others may score higher. At the
top of the initial 20 guesses might be two possible solutions, with a score of
two points each. One way to explore the possibility of a better guess is to

54
 8. Evolution and Artificial Intelligence

use mutation: randomly changing one of the categories. The new guess can
replace one of the guesses that had a score of zero. If the mutation performs
better, further mutations can be explored.

The other technique used in genetic algorithms is more powerful. It’s

a form of recombination borrowed from sexual reproduction. Chromosomes
are essentially strands of DNA. Segments of DNA called alleles dictate
an individual’s physical traits, such as eye color. In the process of sexual
reproduction, the chromosomes of the parents line up and trade segments
of DNA. That same process can be used on the two best guesses in the
Clue example, resulting in new guesses that combine segments of the earlier
two high scorers. If any of those combinations produce a higher score, the
process can be repeated on those guesses. This procedure, when repeated,
will eventually obtain guesses with increasingly higher scores, evolving a
solution to the crime. It mimics the evolutionary processes of mutation and
sexual reproduction.

Genetic algorithms are a general tool for optimization, now widely used in
AI. They are used in constructing facial composites from eyewitness reports
in law enforcement, in the design of electronic circuits, in robotics, in
designing software, in code-breaking, and in antiterrorism systems. Genetic
algorithms are used not only for airline scheduling but also for designing the
aircraft themselves.

Evolution as Inspiration
Evolution, in both natural and artificial forms, can be seen as a form of
exploration. Theorists sometimes think of all the possibilities as forming
a possibility space. The search for optimization is an exploration of that
space, moving from point to point to find better options. It always involves
a trade-off between two options—what Holland called exploration and
exploitation—and aiming for the right balance between further exploration
and using what has already been found.

Biological evolution has provided the inspiration for evolutionary computing.

Biological evolution and AI have moved in parallel. On one side, it’s the
story of how the human eye functions. On the other side, it’s the story of

55
 8. Evolution and Artificial Intelligence

how AI “sees.” The human eye is

not only a complex organ but is
formed of parts—lens, pupil,
Human eye anatomy retina, optic nerve—that can
make no functional sense
independently. As such, it
was long considered to
be proof of creation.

It is now known
that vision
evolved in
different
forms along
many different
branches of the
tree of life, gradually
making more complex
visual representations
possible. In the human
eye, light activates cells
in the retina, but visual
processing doesn’t operate on
the retinal image as a whole—at
least not at first. It works on input
from localized portions of the retina.
The input from each small section forms the receptive field for independent
neurons on a first layer of processing in the visual cortex, located at the back
of the brain.

The neurons on that first layer take input from small sections of the retina
and respond to edges between light and dark in their specific receptive field.
Some neurons in that layer respond to vertical edges in their receptive fields,
some respond to horizontal edges, and some respond to other angles in
between. All respond just to localized patterns in their receptive fields, and all
seem to respond just to edge orientation.

56
 8. Evolution and Artificial Intelligence

Those neurons then feed into a second layer, which look for the form
the edges produce. Layer by layer, visual processing combines data from
increasingly larger areas, putting together a composite recognition of shapes,
then objects; faces, then facial expressions. Visual processing therefore involves
three key aspects: localized receptive fields, lots of layers, and specialized
functions that are fairly uniform on specific layers.

Convolutional Neural Networks

Visual processing has been a long-term goal for AI, with a history of setbacks
and disappointments. A breakthrough came when researchers started using
a particular kind of neural network: the convolutional neural network,
or ConvNet. In ConvNets, localized patches of the visual input feed to
designated nodes on the first layer below. Neighboring groups of those first-
layer nodes then connect to their own dedicated node on the second layer,
assigned just to that group. Processing proceeds layer by layer, analyzing larger
visual sections as it goes.

The other distinguishing feature of ConvNets is that in the training process,

all the nodes on a given layer are nudged to act alike, performing the same
step in visual processing. The ConvNet analyzes visual input in progressive
steps, through progressive layers, thus using the same pattern found in human
visual processing.

ConvNets have swept the field because they are efficient and fast. The
bite-sized calculations can work in parallel, all at the same time, without
forcing computers to wait for one calculation to go to the next. ConvNets
are trained with the same basic techniques used to train any neural network,
and everything that happens on any of the layers is learned from data.
Surprisingly, a ConvNet trained for visual processing will learn, on its own, to
detect edges in the first processing layer. Not only does the ConvNet see, but
it has learned to see in much the way that humans see.

Another reason ConvNets are fast is that they can run on a piece of computer
hardware that was initially developed for generating images fast enough for
video games. That piece of hardware is the graphics processing unit (GPU),

57
 8. Evolution and Artificial Intelligence

and it also works on the basis of layers of simple calculations on parts of an

image or array. Today, ConvNets are put to wide use in many areas of AI, and
GPUs form a major component in massive AI data centers.

Reading
Dennett, Daniel. Darwin’s Dangerous Idea. Simon & Schuster, 1995.
Holland, John H. “Genetic Algorithms.” Scientific American 267, no. 1
(1992): 66–73.

Questions
1 In what ways is evolution really like an algorithm or a computer
program? In what ways is it different?

2 How do you play Clue? In what ways is your strategy like a genetic
algorithm? In what ways is it entirely different?

3 A deep question: Could the mechanisms of evolution themselves

evolve?

Other Media
Bongard, Josh, Victor Zykov, and Hod Lipson. “The Resilient Machines
Project.” YouTube video. 2 min, 10 sec. [Link]
watch?v=x579QKA6fkY. (Josh Bongard and his collaborators used
genetic algorithms to program a “star” robot. First, it uses a genetic
algorithm with input from sensors and random motion to figure out
the shape of its own body. Then it figures out how to move toward the
end of the platform. The genetic algorithm found a successful way of
moving that we wouldn’t have thought of.)

58
9
Ethics and AI

I saac Asimov is best remembered for his science fiction. His stories
featured visions of AI, and he often emphasized questions of ethics
in relation to the concept. He envisioned a policy in which the first thing
programmed in a robot was a code of ethics: Asimov’s Three Laws of
Robotics. First, a robot may not injure a human being or, through inaction,
allow a human being to come to harm. Second, a robot must obey orders it
is given by human beings—except where such orders would conflict with
the first law. Third, a robot must protect its own existence—as long as such
protection does not conflict with either of the first two laws. Yet, Asimov
didn’t think those laws were sufficient. In his fiction, he anticipated several
of the ethical difficulties with today’s AI that you’ll explore in this lecture.

59
 9. Ethics and AI

Unintended Bias
In the real world, some of the problems in the use and application of
AI are problems involving bad actors—those who deliberately use the
technology in an unethical manner. For example, AI offers a powerful tool
for propaganda and misinformation. At first glance, what is at issue is the
unethical application of AI by some of its users. The problems that need
to be considered are the ease with which potentially dangerous tools are
made available to potential bad actors and how difficult they are to detect.
However, some of the potential dangers and ethical difficulties of AI involve
not deliberate wrongdoing but all the things that can go wrong even when the
intention is to do the right thing.

For example, in the justice system,

a parole board may consider
statistics—among other
factors—when deciding if an
individual can be eligible
for early release. Those
statistics will reveal how
many individuals in
the same situation
did or did not
return to crime.
COMPAS,
which
stands for
Correctional
Offender
Management
Profiling for
Alternative Sanctions,
is one of several programs
designed to help in that
kind of estimation. It is a
form of statistical AI that takes
into consideration variables such

60
 9. Ethics and AI

as age, age at first arrest, criminal history, chemical dependency, and family
support. AI statistical analysis is increasingly used for decisions regarding
pretrial detention, and in some cases, judges have relied more on AI than on
agreements reached by the defense and prosecution.

In many jurisdictions, the use of COMPAS or a similar AI system is now

required in parole decisions. By checking its predictions against the data, it
is possible to determine how accurately COMPAS predicts recidivism. Did
those who were predicted to return to crime actually do so, and did those who
were predicted to become law-abiding citizens stay out of trouble? Validations
studies showed that COMPAS had an accuracy rate of 61% with regard to
recidivism. The studies also indicated that algorithmic techniques were as
good as and sometimes better than other forms of prediction, such as the
predictions human parole boards used to make.

Still, an accuracy rate of 61% meant that COMPAS was wrong 39% of the
time. In some cases, it gave false positives, predicting that someone would
return to crime when in fact they didn’t. In other cases, it gave false negatives,
predicting that someone would not return to crime when they did. The really
bad news was that those wrong decisions turned out to be skewed in terms
of race. The false positives disproportionately involved the Black population.
Black people were far more likely to be classified by COMPAS as high risk for
recidivism—and wrongly so—than White people. Individuals were also far
more likely to be wrongly classified as low risk if they were White. In reality,
among those classified as low risk, White people were more likely to return to
crime. No one intended COMPAS to be racially discriminatory, and attempts
continue to try to make it and similar systems better.

Bias in Source Data

Machine learning programs built on big data are generally statistical
programs, searching for correlations between different factors. For example,
they can discover what the risks are of a second heart attack for people who
have survived a first or what the correlation is between obesity and diabetes.
Knowing those correlations can be useful in medicine, and they are applicable
in individual treatment programs. Correlations are also potentially useful in
dealing with social issues such as crime.
61
 9. Ethics and AI

But some correlations should be handled with greater care. For example,
statistical data is used to identify areas in cities in which crime rates are higher
and to assign more intensive policing in those areas. It often turns out that
these are predominantly minority areas. This creates a tricky situation: The
intention may be good—to cut down on crime—but the ethical concern is
that the use of AI may end up reinforcing selective enforcement on the basis
of race.

This same kind of concern will exist in the use of AI in deciding who should
be hired, who should be admitted to college, or who should get a mortgage
and who should not. Race may not be one of the factors built into the system,
but unintentionally, there may be factors—or combinations of factors—that
serve as proxies for race. Therefore, in designing ethical AI, awareness of
factors that may serve as unintended but unethical proxies is crucial. There
are many ways in which AI can produce unintentional bias. Massive neural
networks are trained to match patterns in big data, and they can amplify
patterns of bias that already exist in the training data.

Large language models are trained on massive samples of text vacuumed up

from millions of sources. Those text samples may have built-in biases that
show up when the software is used. There are many examples of AI systems
having built-in gender bias, such as a Google image search for pilot or brain
surgeon coming up with only pictures of males and a search for nurses
delivering only images of females. There is no deliberate intent to produce
gender-discriminatory responses; the data on which programs are trained
already exhibits bias. AI didn’t create this gender bias; it mined associations
that were already in the data—already in society.

To correct such bias, the obvious solution is to modify the training data.
Other solutions have been tried, but some of those turned out to be as bad as
the problem they were trying to solve. For a certain period, Google artificially
programmed a bias toward showing minorities in its image-generating
software, but that resulted in different mistakes. For example, a request for an
image of a Nazi soldier returned a picture of a Black person in an SS uniform.

62
 9. Ethics and AI

Wider Consequences
Bias is not the only ethical concern with AI. The purpose of a search engine
is to recommend some sites rather than others. For example, recommender
systems work by telling consumers that “people who bought this also bought
this” or recommending “other films you might like.” In many cases, such
purchasing alternatives and recommendations can be helpful. But what works
for a consumer search may not work for everything.

The information people need regarding political choice may be very different.
In that context, the algorithms used in search engines and social media might
do something far less helpful, perhaps even socially harmful. If political
searches lead those with different
views toward increasingly extreme
versions of those views, political
polarization is inevitable. If
people are led to sites that
simply reinforce their
current views, they will be
caught in echo chambers
that deny them the
full information
they need. Many
researchers have
placed at least
part of the
blame for the
polarization
in America on
the polarization
dynamics built into
the algorithms in search
engines and social media.

Ethical issues in the use

of AI are not problems for
a distant future; they are here
now. Society is already immersed

63
 9. Ethics and AI

in AI. People’s personal data is

inevitably part of the big data on
which AI operates. That raises
obvious privacy and security
concerns. A large percentage
of Americans have been
victims of identity theft,
and that percentage is
growing.

The possibility
of pinpoint
geographical
tracking,
a complete
data profile on
everyone, and
widespread facial
recognition capabilities
raises the specter of
a ubiquitous surveillance
state—a frightening Big
Brother trope that is all too
familiar in the dystopias of
science fiction. And at this point,
there are more ethical questions than
clear answers. One of these questions is whether AI should be regulated, and
if so, how.

The Search for Solutions

There are some immediate proposals for regulating AI that seem both
simple and sensible. It could be made a matter of law that AI-generated
photos, video, and audio must carry a watermark labeling them as such. In
journalism and writing generally, it should perhaps become common practice
for publishers and media outlets to demand that pieces that rely on AI-

64
 9. Ethics and AI

generated text are labeled. Legislation to that effect is one course of action,
but regulations alone will not protect society from bad actors determined to
circumvent labeling and watermarks.

Another possible target for regulation is the issue of authorship. The neural
net training used in large language models has scraped text from all available
sources, for example, years of articles from The New York Times and copyright
novels by Stephen King. Existing laws against impersonation of businesses or
government agencies could perhaps be extended to applications of AI.

Questions regarding ethics of design are best considered in terms of ethical

responsibility, but the transparency issue creates complications. Neural
networks are generally opaque in operation, and even their developers can’t
say why they do what they do. They operate as a web of numerically weighted
links trained on massive amounts of data, and they are self-taught. They don’t
operate in terms of justification. It is thus difficult to assign responsibility
for errors. In his article “Can We Survive Technology?,” John Von Neumann
stated that “useful and harmful techniques lie … so close together that it
is never possible to separate the lions from the lambs.” One seems forced to
reach the uncomfortable conclusion that certain forms of AI should perhaps
be banned outright.

There is a general issue of trying to make sure that the goals that are built into
AI systems are aligned with society’s goals. Part of the problem is not knowing
what goals a system really is pursuing, but another part of the problem is
trying to figure out what society’s values really are and what its goals should
be—an area in which, it appears, much remains to be learned.

Reading
Asimov, Isaac. I, Robot. Basic Books, 1991.
Bradbury, Ray. “I Sing the Body Electric.” In I Sing the Body Electric! and
Other Stories. Perennial, 2001. (A short story that first appeared as a
screenplay for The Twilight Zone, season 3, episode 35. It aired May 18,
1962.)
Coeckelbergh, Mark. AI Ethics. MIT Press, 2021.

65
 9. Ethics and AI

Questions
1 The runaway trolley is headed for five workers on the right-hand
track. Would you flick the switch to save them, killing one worker
on the left?

2 What if a child is on the left? What if it is your child?

3 You’re watching from a footbridge as the runaway trolley heads for

five workers. You can save them by pushing the very large man on
your right onto the tracks. Should you?

4 Do you think a robot could be a moral agent in its own right? Could
it have rights?

Other Media
Proyas, Alex, dir. I, Robot. 20th Century Studios, 2004. (The movie is only
peripherally related to Asimov’s original, but it raises some of the same
ethical questions he did.)

66
10
Promises and
Perils of AI

T he development of AI seems to carry promises of advances in the

areas of medicine, education, communications, and energy efficiency.
However, it also seems to arrive with significant threats to privacy, security,
employment, and social order in general. The promise of AI may extend to
a cure for cancer, while the threat of AI may extend to killer robots. In this
lecture, you’ll discover some of AI’s promises and perils, including those in
human–computer interaction that are often overlooked, and their potential
impact on individuals and society.

67
 10. Promises and Perils of AI

Voice Cloning
Alexis Bogan was a normal vocal high schooler until a life-threatening tumor
was discovered near the back of her brain. The operation to remove it was
successful, but her voice was gone—she could no longer speak. But before
her surgery, Alexis had recorded a cooking demonstration for a high school
project. Fifteen seconds of that recording was used to clone her voice. Using
a phone app and AI, she can key in whatever she wants to say. She has her
voice back.

Casey Harrell could no longer speak as a result of Lou Gehrig’s disease. But
surgeons at University of California, Davis, were able to implant electrodes
in his brain that detected the impulses that should but were no longer able
to control movement of his mouth and tongue. They fed those impulses to
a voice synthesized from old recordings Casey had made, and he can now
speak again.

Voice cloning offers hope for millions of victims of stroke, throat cancer,
and a range of neurodegenerative diseases. However, the technology has
a darker side. The same techniques that recovered Casey Harrel’s and Alexis
Bogan’s voices from old recordings can also be used to make people appear
to say something they never actually said. Joe Rogan’s voice has been used
to promote a scam, and voice cloning has been used to update an older
scam in which a relative supposedly makes a plea for emergency money. By
using available online clips, those pleas may now be made in a loved one’s
own voice.

Thus, voice cloning can be both a good thing and a bad thing. In that
respect, it’s typical of AI technologies, offering both promise and peril. Like
other aspects of AI, it offers new prospects for human–computer interaction,
including some that border on the macabre—such as apps that allow
mourners to continue conversations with the dead—with new potential
psychological advantages and disadvantages that are yet unknown.

68
 10. Promises and Perils of AI

Advancing AI Use
The challenge for science is no longer information gathering but information
management—making sense of the data. In this context, AI data mining
offers the ability to find unseen connections in what has been called
undiscovered public knowledge. As early as the 1980s, the information
science pioneer Don Swanson applied data processing techniques to medical
literature. He looked for the hidden knowledge of undiscovered medical
connections. Essentially, if one batch of literature connects a topic A with a
topic B, and another, far removed batch of literature connects topic B with
topic C, then there may be an overlooked connection between A and C, even
if no literature links those two. Swanson proved that AI can be used to make
those hidden connections.

Today’s tools are far more

sophisticated than Swanson’s,
making it possible to look for
connections both within and
across enormous bodies of
data. Causal patterns do
not necessarily take the
simple form of one
cause having one
effect. They can
involve subtle
and complex
combinations
of factors—or
the absence
of factors—that
only AI can detect,
leading to hypotheses
to be investigated using
traditional methods.

Through the use of AI, many US government’s genome

promising new drugs are being sequencing center
developed. Human genome

69
 10. Promises and Perils of AI

sequencing has the potential to lead to genetic medicine, which is the ability
to actively edit and engineer snippets of genetic code. AI is increasingly used
in everyday medical practice as well. The “expert systems” available for a
doctor’s consultation are AI-generated. They offer the prospect of information
that not only takes the most recent findings into account but also allows for
targeted and personalized application.

In other areas, the promise of AI is just as profound. The energy grid has long
been a vulnerable patchwork resulting from a piecemeal history. Progressive
redesign can create greater robustness and security and an estimated 40%
increase in efficiency in energy delivery. The design of next-generation
batteries is in the hands of AI. In engineering in general, from the design of
bridges to international satellite and communication systems, AI is now the
tool of choice.

Even forms of automated science are already in use. In a lab in Cambridge,

Massachusetts, AI software is linked directly to physical experiments in the
lab. Image processing from a network of telescopes is in place to offer early
warning of asteroids and comets that might cross Earth’s path. On Mars,
AI software was uploaded to the Curiosity rover, allowing the vehicle to
autonomously pick rocks of particular interest for analysis. The next step
may be to send AI robots to explore planets circling distant suns, collect data,
formulate hypotheses, and guide further testing. And among all the promises,
humankind may learn more about itself. That has long been the goal of the
philosophy of science.

Potential Risks
The potential perils of AI should not be underestimated. Many are already
evident, such as image cloning and visual deep fakes. The potential use of
deep fakes in political misinformation is clear: One could make an opponent
appear to say on camera precisely what one wants them to say. The mere
existence of AI can pose the threat of political misinformation even when it
is not actually used, as even legitimate videos may be labeled falsely as AI-
generated. Bad actors have new and powerful tools to generate propaganda
and spread misinformation.

70
 10. Promises and Perils of AI

Some of the problems that have emerged are consequences of the technology
itself. Algorithms such as ChatGPT and their visual counterparts are
notorious for what is called hallucinating: giving authoritative responses
that are simply false. At the core, the programs predict the next verbal or
visual component based on a background database scraped from the Web.
This method of predicting the next word or completing a picture based on
cues cannot guarantee truth or accurate representation. Overreliance on the
authority of GPT results can have serious consequences.

In addition, good training data may be a diminishing resource. Publishers,

authors, and web designers increasingly guard against unauthorized use of
their work. It has been estimated that 5% of the kinds of sources used in
training in the past and 25% of the highest-quality sources are now restricted.
And when AI is trained on its own
content, its accuracy can deteriorate
significantly. As AI-generated text
and images themselves become
part of the data on which
future generations of AI are
trained, misinformation
and misrepresentation
can be recycled
and amplified.
Repeatedly training
AI on AI may
prove a threat to
AI itself.

Increased use
of AI in the
workplace has the
potential to cause
social and economic
upheaval. The hope is that
AI will allow an increase
in productivity; the threat is
that many jobs will disappear.
In the areas of crime prevention

71
 10. Promises and Perils of AI

and national security, intelligence agencies’ access to AI surveillance tools is

cause for concern. In the management of energy, communication, financial,
and transportation systems, centralized control means that a centralized
misfunction or attack can bring any of those systems to its knees, making
them targets of future cyber warfare.

And in the military, remote-controlled attack drones may soon be joined

by autonomous weapons, with AI systems deciding on appropriate targets
on their own. The promise is that AI systems may be better at targeting
combatants and reducing collateral damage for civilians; the potential danger
is that robots may not or will not make that distinction. Bioterrorism also
needs to be on the list of potential perils. Jason Matheny, the president and
CEO of RAND, has concluded that with currently available AI, it may
be possible to develop a virus that can kill millions of people for less than
$100,000.

The Human Connection

The changes in human–computer interaction as a result of AI are often
overlooked. For example, battlefield drones can be operated by a soldier
sitting in front of a screen thousands of miles away from the war zone. Several
studies indicate that disconnecting a person from the immediacy of the
killing can make committing atrocities all too easy. An army chaplain and
ethics instructor put it this way: “As soldiers are removed from the horrors of
war and see the enemy not as humans but as blips on a screen, there is a very
real danger of losing the deterrent that such horrors provide.”

A different threat in human interaction with AI lies in treating computer

technology as if it is far more human than it is. Human social psychology
is geared to interacting with humans, and typically, such interaction
assumes truthfulness. When applications are designed to mimic humans
but are technologies that can handle misinformation as fluently as
legitimate information, the impact can have disastrous consequences.
Anthropomorphism also impacts human emotion. Some have proposed
chatbots and AI characters as emotional aids in dealing with what has been
called an epidemic of loneliness. Character AI offers routines in which to

72
 10. Promises and Perils of AI

create virtual boyfriends and girlfriends, and evidence exists showing that
vulnerable individuals may abandon human contact for obsessive involvement
with an AI avatar.

By exploring AI, it may be possible to explore humankind, using AI as a tool

toward understanding intelligence and cognition. The result, however, may
be disappointing. The point is illustrated in fears expressed by Douglas
Hofstadter, a cognitive and computer scientist who won a Pulitzer Prize for
his book Gödel, Escher, Bach. He found that a computer-generated piece of
music could appear to carry all the humanity and emotion of a real composer,
perhaps even more than the composer themselves. Hofstadter’s fear is that
what AI may teach humankind is that everything humans take pride in—
intelligence, human emotions, consciousness—may turn out to be far too
simple. Perhaps the lesson will be that these traits are no more complicated
than an algorithmic bag of tricks.

Reading
Harari, Yuval Noah. Nexus: A Brief History of Information Networks
from the Stone Age to AI. Random House, 2024. (The second half in
particular is recommended.)
Lee, Kai-Fu Chen Quifan. AI 2041: Ten Visions for Our Future.
Currency, 2021.
Narayanan, Arvind, and Sayash Kapoor. AI Snake Oil: What Artificial
Intelligence Can Do, What It Can’t, and How to Tell the Difference.
Princeton University Press, 2024. (The authors’ main target is
predictive AI, as in hiring and sentencing, but with thought-provoking
warnings regarding misuse of machine learning in scientific research.)

Questions
1 What do you most hope for from AI?

2 What is your greatest AI fear?

3 What was it about AI that terrorized Douglas Hofstadter? Do you

think his fears are justified?

73
 10. Promises and Perils of AI

Other Media
Block, Hans, and Moritz Riesewieck, dirs. Eternal You. Documentary.
Gebrüder Beetz Filmproduktion, 2024. (A documentary tracking
startups creating digital avatars of dead loved ones. Available on several
platforms.)
Jonze, Spike, dir. Her. Warner Brothers, 2013. (Scarlett Johansson, the
voice of Samantha in the movie, charged OpenAI with cloning her
voice for a ChatGPT chatbot.)
Schrader, Maria, dir.. I’m Your Man. Bleecker Street, 2021. (Life and love
with a humanoid robot.)

74
11
The Coming
Singularity

T he common theme in science fiction is that of an AI that aims for world

domination in ways that entail human extinction. But the theme isn’t
limited to science fiction. Various experts and AI researchers have issued
warnings that there is real danger of creating an AI that leaves humans far
behind. The idea is that the future holds the prospect of a technological
singularity—known simply as the Singularity—which is when machines
will become far more intelligent than human beings and become people’s
masters. In this lecture, you will explore this vision of a coming Singularity as
well as some basic questions and criticisms of that vision.

75
 11. The Coming Singularity

The Threat of Superintelligence

The holy grail of current AI
research is general AI—systems
that exhibit learning and
intelligence across all fields of
human interest. If or when
that is achieved, artificial
systems will match
human intelligence.
The question is: Will
the next step be
superintelligent
AI that
surpasses
human
intelligence?
That’s where
the vision of the
Singularity comes
into play.

Human intelligence can

design AI systems, but
those systems can design
High-tech data centers
even more intelligent artificial
systems. Once AI matches human
intelligence, it could design systems
well beyond that level, and those systems could design still more intelligent
systems. The idea is that once the process has begun, AI will be able to
self-improve beyond human reach. There will be a tipping point at which
self-improving AI will spiral into superintelligence beyond human control and
even beyond human understanding.

Given the speed of computer systems—far greater than the speed of

human brains—the explosion of self-improving AI might happen suddenly,
unexpectedly, and quickly. Some speculate that it could happen in a matter
of hours or days rather than decades. The concept is alarming. What will

76
 11. The Coming Singularity

happen if Homo sapiens is no longer the most intelligent species on Earth?

Are superintelligent machines the next step in evolution, bound to replace
humankind? Does AI’s inevitably unlimited self-development spell the end
of humankind?

A number of AI experts take the prospect seriously and have issued warnings.
According to Geoffrey Hinton, a recent Nobel Prize recipient:

There’s a serious danger that we’ll get things

smarter than us fairly soon and that these things
might get bad motives and take control. … I
think it’s quite conceivable that humanity is just a
passing phase in the evolution of intelligence.

And in the words of physicist Stephen Hawking: “Success in creating AI

would be the biggest event in human history. Unfortunately, it might also be
the last.”

Kurzweil’s Predictions
The concept of the Singularity isn’t new; it can be found in both fiction
and nonfiction from the middle 1800s. But alarms started to sound in the
later part of the 20th century. More recently, the foremost proponent of the
Singularity theory has undoubtedly been Ray Kurzweil, author of both The
Singularity Is Near and The Singularity Is Nearer.

Kurzweil developed a system to convert the written word to speech, a clear

benefit for the visually impaired. His invention caught the attention of
Stevie Wonder and led to several meetings with the famous musician, which
resulted in Kurzweil founding a company that produced some of the best
music synthesizers ever made. Since then, Kurzweil has worked in natural
language processing, produced educational software, and developed a pocket-
sized optical reader for the visually impaired—an updated version of his
earlier invention.

Some of Kurzweil’s ideas are quite outlandish. He has arranged for his body
to be cryonically frozen at death, in the hope that future developments
in medicine will allow him to be brought back to life. He foresees
77
 11. The Coming Singularity

a nanotechnology of miniature robots that will allow most human ailments

to be cured from the inside. He predicts that it will be possible to enlarge
the human brain to include virtual neurons in the cloud. Human life—
at least in the form of computer uploads of the human mind—will be
extended indefinitely.

Kurzweil predicts an exponential increase in the power of AI that will be

so great “as to appear essentially vertical.” That is the Singularity, at which
point machines will have an intelligence “trillions and trillions” times that
of humans. Kurzweil expects the Singularity by the year 2045, but rather
than being alarmed by this, he writes positively of a future event “utterly
transformative for humanity.” This optimistic view is not his alone.

The Singularity Theory

Visions of the Singularity weave several ideas together. At the core is the
idea that AI will be capable of recursive self-improvement. It is not difficult
to measure the success of AI software against metrics such as standard
intelligence tests, and GPT systems can already write and improve software
programming. Taken together, these facts suggest that an AI system should
be able to create a more intelligent AI system or even rewrite its own software
to achieve a higher level of intelligence. The Singularity theory is based on
the idea that such a process will continue without limits, each system creating
an even more intelligent AI. If this is true, there will be an exponential
intelligence explosion in AI: the Singularity.

The phenomenon of exponential growth in some aspects of knowledge is

relatively well documented, and an exponential pattern is also evident in
the development of computer hardware. Kurzweil has traced exponential
acceleration of innovation through agriculture; writing; the experimental
method; printing; the Industrial Revolution; and the invention of the
telephone, the radio, and the computer. He extends Moore’s law to all of life,
drawing an exponential graph through the first multicellular organisms, the
Cambrian explosion, reptiles, mammals, primates, Homo sapiens, and the
development of today’s technology. He calls it the law of accelerating returns.

78
 11. The Coming Singularity

His graphs are meant to support

the claim that the Singularity is
approaching at an exponential
rate, and the dominance of
Homo sapiens will turn out
to be but a passing phase
in evolutionary history.
The evolutionary
component of the
Singularity theory
is one of the
oldest. In 1863,
four years after
Darwin’s Origin
of Species was
published, Samuel
Butler extended the
ideas of evolution to
humanity’s creations.
In his essay “Darwin
among the Machines,” he
wrote: “There is no security
… against the ultimate Samuel Butler
development of mechanical
consciousness. [I]t appears to us
that we are ourselves creating our own
successors.”

A final component of the Singularity vision is the observation that the goals
of the superintelligent machines may not coincide with society’s goals. Its
goals may not even include continued human existence. This relates to what
is known as the alignment problem: How can human designers align AI with
human values rather than values it might adopt on its own? Furthermore, it is
important to recognize that human values differ from person to person, and
cultural values do not always align. Therefore, it is not clear what society’s
values and goals should be or whose values should be embodied in AI. Even
if agreement can be reached, the challenge of unambiguously controlling the
appropriate implementation of the goals in a machine remains.

79
 11. The Coming Singularity

Critique of the Singularity Vision

The plausibility of the Singularity becoming reality is not accepted by
all. A survey of researchers in machine intelligence asked whether “the
intelligence explosion argument is broadly correct.” Among the respondents,
21% said that the odds were “about even,” 29% said that it was either “likely”
or “quite likely,” and 50% answered that it was “unlikely” or “quite unlikely.”

Critics have noted several problems with Kurzweil’s various charts. Kurzweil’s
inspiration is Moore’s law, whose predictions held for more than 50 years.
But the growth of computer power seems to have slowed. Another critique of
Kurzweil’s exponential growth charts is that other curves fit the points but
don’t follow a nearly vertical line to a Singularity.

A common form of growth starts with an early burst and a period of rise
followed by a gradual slowing down to a plateau. This is called logistic
growth. It follows an S-shaped curve and is typically seen in the spread
of disease, in the spread of information, in the adoption of technological
innovations, and in economics. Early on in a growth pattern, it is impossible
to tell whether the eventual outcome will be exponential growth or
logistic growth.

There are reasons to believe that an eventual plateau in the growth of AI is

inevitable. Exponential growth would demand an exponential increase in
computer speed, and there are probably physical limits to how fast computers
can run. Genuinely exponential growth would demand exponential resources
of energy, an area with built-in limitations. These factors indicate that AI
can’t explode without limits.

A fundamental critique of the Singularity vision involves the central concept

of intelligence. If the efforts to date are any indication, AI has captured only
certain aspects of intelligence—those that show up in formal games and in
patterns and predictable content that can be read off massive linguistic data.
But these are different than the forms of intelligence required to deal with
a complex and changing real world.

Paul G. Allen, who co-founded Microsoft with Bill Gates, argues that
advances in the design of AI will require expanding the understanding of
natural intelligence. This alone will prevent exponential growth. Kurzweil’s

80
 11. The Coming Singularity

projections demand an exponential curve in scientific discovery regarding the

brain, but scientific progress rarely follows that pattern. Far more common
are fits and starts of altered hypotheses and abandoned paradigms. If the past
is any guide, each necessary breakthrough in the attempt to understand how
the brain functions will become increasingly more difficult. Therefore, each
breakthrough required in the attempt to artificially model and expand that
intelligence will become increasingly harder. The process will slow.

Yet, the importance of preventing an out-of-control intelligence explosion,

however unlikely, has gained wide recognition. Within research and
development groups, government, and the law, there are movements to try to
ensure friendly or beneficial AI. Still, it must be admitted that, at present, no
one has a solid grasp on precisely how that can be achieved.

Reading
Bostrom, Nick. Superintelligence: Paths, Dangers, Strategies. Oxford
University Press, 2014.
Kurzweil, Ray. The Singularity Is Near. Viking Penguin, 2006.
Shanahan, Murray. The Technological Singularity. MIT Press Essential
Knowledge Series. MIT Press, 2015.

Questions
1 How would you answer this survey question: How likely is it that
the intelligence explosion argument is broadly correct?

{ quite likely

{ likely

{ odds are about even

{ unlikely

{ quite unlikely

2 If there is a Singularity in our future, what is the best way to prepare

for it?

81
 11. The Coming Singularity

Other Media
Cameron, James, dir. The Terminator. Orion Pictures, 1984.
Wachowski, Lana, and Lilly Wachowski, dirs.. The Matrix. Warner
Brothers, 1999. (With multiple sequels.)

82
12
Predicting
Tomorrow’s AI

A ttempts to gaze into the future of AI don’t only focus on doomsday

scenarios. It is important to consider realistic projections of AI
benefits just around the corner, as well as the risks that might accompany
those benefits. Even in short-term predictions, the shortcomings in human
knowledge must be recognized. If the past is any indication, even those who
should know aren’t very good at predicting the future. For example, in 1927,
Harry Warner of Warner Brothers predicted that sound movies would just be
a passing fad, while Thomas J. Watson Sr., president of IBM, is reputed to
have said, “I think there is a world market for maybe five computers.” In this
lecture, you’ll explore more attempts to peer into the future of AI.

83
 12. Predicting Tomorrow’s AI

The Role of Social Forces

Prediction failures haven’t been limited to single individuals. Entire disciplines
have repeatedly missed the mark. Political scientists failed to predict the
breakup of the Soviet Union, economists have repeatedly failed to predict
major downturns, and a common joke is that the stock market has predicted
nine of the last four recessions. And despite past predictions, no one makes
their commute in a flying car or lives in a mile-high skyscraper, and not one
doomed underwater city has been found.

The first way predictions go wrong is by simply extrapolating current

trends in a straight line. Housing prices have been going up for decades;
therefore, they are expected to continue to increase, and they do … until the
housing bubble bursts. Liberal democracies have been on the rise over the
last 100 years, and the trend is projected to continue … until a totalitarian
backlash occurs. A second way predictions go off the rail is by ignoring
complexity, particularly social complexity. Often, those predictions are based
on the feasibility of producing a particular technology, ignoring the social and
economic dynamics that will determine whether that technology is likely to
be adopted.

To be plausible, projections need to take into consideration not only the

research and development of the devices themselves but also the economic and
social dynamics of the culture in which they would have to be adopted. Social
forces can slow or prevent the adoption of an envisaged technology, or they
can accelerate it. For example, Henry Ford found a way to produce affordable
automobiles in large numbers, but he didn’t build better roads for them. As
people bought automobiles, public pressure mounted to build better roads.
The better the roads, the more incentive to buy an automobile. Social forces
accelerated the adoption of the car.

No one predicted a future with personal computers (PCs). Watson’s prediction

was based on the room-filling computers that IBM produced at the time.
Small computers were made possible by advancements in the development
of microprocessors and integrated chips, but the crucial factor in the
adoption of PCs was that virtually everyone could buy them, complete with
primitive, cheap software. A mutually reinforcing loop was established. As

84
 12. Predicting Tomorrow’s AI

the number of PCs increased, so did

the incentive to develop better
software. As the quality of
software increased, so did the
incentive to buy a PC.

Predictions will also fail

if they don’t take into
account the economic
competition in
a capitalist
marketplace.
Different
systems attempt
to ride the
social dynamics
of self-reinforcing
economic loops with
new offerings, auxiliary
apps, and add-ons. The
economic war between Macs
and PCs, the legacies of Steve
Jobs and Bill Gates, has raged
for more than 40 years, and there
seems to be no end in sight. The
development of AI will be shaped and
guided in an environment of economic forces, as computation and capitalism
cannot be expected to operate on separate tracks.

A range of classic models of economic growth have been built on the work of
Nobel Prize winner Robert Solow. These abstract mathematical models try to
trace how an economy will grow based on some simple assumptions regarding
how much it invests in its machines each year, how productive those machines
are, the costs of their maintenance, and rates of popular consumption. Solow’s
models show an initially optimistically rising curve of economic growth, but
eventually, that growth levels off and stalls, indicating limits to economic

85
 12. Predicting Tomorrow’s AI

growth. However, the models do not include innovation, and economists have
therefore reached the conclusion that a society’s economic growth can only
continue to rise with the introduction of new ideas and technologies.

Plausible Projections
People will increasingly be
surrounded by what is called the
Futuristic transport vehicles Internet of Things. At present,
cars can be made to signal
when another vehicle is
too close; homes can be
equipped with smart
switches that are
triggered remotely;
and voice-activated
systems can
bring up
specific pieces
of music and
information
on request. The
prediction is that the
range of devices with
sensors and software
processing—as well as
linked networks of devices—
will proliferate to meet
demand.

AI has already given an astounding

range of new tools and is bound to
offer more. Some might believe that this will result in massive unemployment,
but the more likely scenario is that professionals who adopt the new tools
will replace those who don’t. New tools from AI are already extending the
frontiers of science and can plausibly be predicted to continue doing so. For

86
 12. Predicting Tomorrow’s AI

example, it may become possible to interpret whale songs or to recover and

read priceless works of antiquity that are too damaged to examine using
traditional methods.

Another easy prediction is that human beings will be able to become

cyborgs—essentially, bionic humans. AI techniques have been developed
that allow some paralyzed individuals to move their legs and arms again.
A chip implanted in the brain can interpret brain signals, processed through
a computer and connected to muscle stimulation that produces the intended
movement. Elon Musk’s Neuralink is predicted to use similar techniques to
link brain activity directly with the operation of external devices.

Human–machine interaction is bound to change as machines change,

particularly as machines become more human-like. In Japan, robotic
assistance has already been used in health care for the elderly for decades.
AI can monitor medications, detect health needs, and signal emergencies.
But it is predictable that AI will also be used for social interaction—or as
a replacement for social interaction.

And as humans tend to anthropomorphize their machines, AI designed

to be socially responsive—whether in ameliorating dementia, addressing
major psychological conditions, relieving loneliness, or making one feel
more accepted or loved—will be built on that tendency. It remains to be
seen whether human-imitative AI of this type proves to be good or evil,
filling in for desperate human needs or replacing genuine human contact
with something far more superficial or even dangerous because it is far less
real. Like other forms of AI, anthropomorphic AI may come with both
positive and negative aspects, bringing both psychological benefits and
psychological dangers.

It’s an easy prediction that the future will be filled with AI-generated
spam, known as slop. Unwanted advertising and attempts at persuasive
political messaging will predictably become more targeted to individuals’
characteristics and lifestyle. Slop is not likely to become any less invasive,
time-consuming, manipulative, or annoying. Other AI developments will
be greeted by hype, followed by a corrective disappointment, leveling off to
a point at which it will be possible to learn to use and appreciate those new
tools within their limits and for what they really are.

87
 12. Predicting Tomorrow’s AI

AI technology itself will be relied upon to help address some of the

shortcomings of AI technology. The companies behind generative AI are
already attempting to use the technology itself to combat the hallucinogenic
misinformation that generative AI can produce. They aim to develop AI deep
fake detectors that will better detect AI deep fakes. This signals a predictable
dynamic: an arms race between new technologies and attempts to hack or
exploit those technologies. AI software will be introduced on one side, with
AI attempts to find its vulnerabilities or to use it maliciously on the other side.
AI patches will be introduced to plug the original vulnerabilities or to block
misuses, leading to further AI attempts to find loopholes or workarounds.

Uncertainty and Unknowns

While it is not possible to predict a radical conceptual innovation, several
exciting technologies are now in the experimental stages of development.
Primary among these is quantum computing. Practical quantum computers
don’t exist yet, but some predict that they will be the next big thing.
Quantum mechanics is the physics of the very small, at the scale of subatomic
particles such as electrons. The idea of quantum computing is to use some
of the strangeness of that world of the very small to do some strange but
powerful computing. In addition, quantum computers would be significantly
faster than the current computers.

This additional speed could be important in the encryption and decryption

of secret codes. Historically, the common denominator among encryption
systems has been that the receiver needs to be sent the key to the code. That
communication of the key itself is a clear vulnerability. In the 1970s, a new
form of encryption was invented in which the transmission key—the code
needed for a computer to encrypt and send a message—no longer has to be
kept secret. For decoding, however, a computer needs a special key.

Prior to this new system, all secret codes were symmetrical systems—
they used the same key for encoding and decoding. The new system is
asymmetrical. A public key allows anyone to encode a message and send
it, but only a system’s private key can decode it. The actual encryption and
decryption are achieved using numbers in a mathematical formula, and the
crucial asymmetry of the system is based on the fact that factoring is hard,
88
 12. Predicting Tomorrow’s AI

even for computers. The larger the number, the harder it gets. One of the
problems for which a quantum computer might provide exponential speedup
is factoring large numbers. It would then be able to crack many of the existing
security encryption systems—a serious prospect.

A famous quote by former Secretary of Defense Donald Rumsfeld can be

paraphrased as follows. There are known knowns: things we know we know.
There are known unknowns: things we know we don’t know. But there are
also unknown unknowns: things we don’t know we don’t know. When it
comes to trying to peer into the future of AI, it is impossible to predict the
unknown unknowns.

Reading
Silver, Nate. The Signal and the Noise: Why So Many Predictions Fail—but
Some Don’t. Penguin Press, 2012.
Tegmart, Max. Life 3.0: Being Human in the Age of Artificial Intelligence.
Knopf, 2017.

Questions
1 Pick a future prediction that you heard about long ago. Did it come
true? If not, why do you think the prediction failed?

2 What are your predictions for AI over the next 5 years? Over the
next 10 years? By the end of the 21st century?

89
Image Credits
3: NYPL; 4: Science Museum London/flickr/CC BY-SA 2.0; 6: Antoine
Taveneaux/Wikimedia Commons/CC BY-SA 3.0; 10: Dennis
Fernkes/Wikimedia Commons/Public Domain; 12: Getty Images;
17: Rijksmuseum; 20: United States Navy; 27: Getty Images;
33: Antoine Taveneaux/Wikimedia Commons/CC BY-SA 3.0;
39: Getty Images; 41: Getty Images; 46: National Library of Medicain;
48: Rijksmuseum, Amsterdam; 53: Portrait photograph of Charles
Darwin. Wellcome Collection; 56: Getty Images; 60: Getty Images;
63: Getty Images; 64: Getty Images; 69: Retrofootage/Pond5;
71: Digital Storm/Shutterstock; 76: Getty Images; 79: Wikimedia
Common/Public Domain; 85: Getty Images; 86: Getty Images

90