THE MYTH OF THE COMPUTER

John R. Searle
The following essay was first published in The New York others. Indeed it has become something of a scandal of
Review of Books (29 April 1982) as a review of a book
twentieth-century intellectual life that we lack a science of
edited by Douglas R. Hofstadter and Daniel C. Dennett
(The Mind’s I: Fantasies and Reflections on Self and Soul). the human mind and human behavior, that the methods of
This essay has since been widely reprinted as concise the natural sciences have produced such meager results
introduction to Searle’s arguments against the program of when applied to human beings.
artificial intelligence. Dennett responded to Searle’s essay
a few months later, and this response, along with Searle’s The latest candidate or family of candidates to fill the
reply, are reprinted immediately after this essay. gap is called cognitive science, a collection of related
Searle’s “Chinese room argument” was discussed investigations into the human mind involving psychology,
earlier in The Behavioral and Brain Sciences, Vol. 3 philosophy, linguistics, anthropology, and artificial intel-
(Cambridge University Press, 1980), along with twenty-
seven responses and Searle’s reply to the responses. ligence. Cognitive science is really the name of a family of
research projects and not a theory, but many of its practitio-
Our ordinary ways of talking about ourselves and other ners think that the heart of cognitive science is a theory of
people, of justifying our behavior and explaining that of the mind based on artificial intelligence (AI). According to
others, express a certain conception of human life that is so this theory minds just are computer programs of certain
close to us, so much a part of common sense that we can kinds. The main ideological aim of Hofstadter and Den-
hardly see it. It is a conception according to which each nett’s book is to advance this theory.
person has (or perhaps is) a mind; the contents of the mind [...]
— beliefs, fears, hopes, motives, desires, etc. — cause and [Dennett and Hofstadter’s] theory, which is fairly widely
therefore explain our actions; and the continuity of our held in cognitive science, can be summarized in three prop-
minds is the source of our individuality and identity as ositions.
persons. 1. Mind as Program. What we call minds are simply
In the past couple of centuries we have also become very complex digital computer programs. Mental states are
convinced that this common-sense psychology is grounded simply computer states and mental processes are computa-
in the brain, that these mental states and events are some- tional processes. Any system whatever that had the right
how, we are not quite sure how, going on in the neurophysi- program, with the right input and output, would have to
ological processes of the brain. So this leaves us with two have mental states and processes in the same literal sense
levels at which we can describe and explain human beings: that you and I do, because that is all there is to mental states
a level of common-sense psychology, which seems to work and processes, that is all that you and I have. The programs
well enough in practice but which is not scientific; and a in question are “self-updating” or “self-designing” “systems
level of neurophysiology, which is certainly scientific but of representations.”
which even the most advanced specialists know very little 2. The Irrelevance of the Neurophysiology of the Brain.
about. In the study of the mind actual biological facts about actual
But couldn’t there be a third possibility, a science of human and animal brains are irrelevant because the mind is
human beings that was not introspective common-sense an “abstract sort of thing” and human brains just happen to
psychology but was not neurophysiology either? This has be among the indefinitely large number of kinds of comput-
been the great dream of the human sciences in the twentieth ers that can have minds. Our minds happen to be embodied
century, but so far all of the efforts have been, in varying in our brains, but there is no essential connection between
degrees, failures. The most spectacular failure was the mind and the brain. Any other computer with the right
behaviorism, but in my intellectual lifetime I have lived program would also have a mind.
through exaggerated hopes placed on and disappointed by Theses 1 and 2 are summarized in the introduction where
games theory, cybernetics, information theory, generative the authors speak of “the emerging view of the mind as
grammar, structuralism, and Freudian psychology, among software or program — as an abstract sort of thing whose
John Searle, “The Myth of the Computer” 2

identity is independent of any particular physical embodi- as we know these neuron firings are a very large part of the
ment.” cause of thirst. Now obviously there is more to be said, for
3. The Turing Test as the Criterion of the Mental. The example about the relations of the hypothalamic responses
conclusive proof of the presence of mental states and to the rest of the brain, about other things going on in the
capacities is the ability of a system to pass the Turing test, hypothalamus, and about the possible distinctions between
the test devised by Alan Turing and described in his article the feeling of thirst and the urge to drink. Let us suppose
in this book. If a system can convince a competent expert we have filled out the story with the rest of the biochemical
that it has mental states then it really has those mental states. causal account of thirst.
If, for example, a machine could “converse” with a native Now the theses of the mind as program and the ir-
Chinese speaker in such a way as to convince the speaker relevance of the brain would tell us that what matters about
that it understood Chinese then it would literally understand this story is not the specific biochemical properties of the
Chinese. angiotensin or the hypothalamus but only the formal com-
The three theses are neatly lumped together when one of puter programs that the whole sequence instantiates. Well,
the editors writes, “Minds exist in brains and may come to let’s try that out as a hypothesis and see how it works. A
exist in programmed machines. If and when such machines computer can simulate the formal properties of the sequence
come about, their causal powers will derive not from the of chemical and electrical phenomena in the production of
substances they are made of, but from their design and the thirst just as much as it can simulate the formal properties of
programs that run in them. And the way we will know they anything else — we can simulate thirst just as we can simu-
have those causal powers is by talking to them and listening late hurricanes, rainstorms, five-alarms fires, internal com-
carefully to what they have to say.” bustion engines, photosynthesis, lactation, or the flow of
We might call this collection of theses “strong artificial currency in a depressed economy. But no one in his right
intelligence” (strong AI).1 These theses are certainly not mind thinks that a computer simulation of a five-alarm fire
obviously true and they are seldom explicitly stated and will burn down the neighborhood, or that a computer simu-
defended. lation of an internal combustion engine will power a car or
Let us inquire first into how plausible it is to suppose that computer simulations of lactation and photosynthesis
that specific biochemical powers of the brain are really will produce milk and sugar. To my amazement, however, I
irrelevant to the mind. It is an amazing fact, by the way, have found that a large number of people suppose that com-
that in twenty-seven pieces about the mind the editors have puter simulations of mental phenomena, whether at the level
not seen fit to include any whose primary aim is to tell us of brain processes or not, literally produce mental phenome-
how the brain actually works, and this omission obviously na.
derives from their conviction that since “mind is an abstract Again, let’s try it out. Let’s program our favorite PDP-
sort of thing” the specific neurophysiology of the brain is 10 computer with the formal program that simulates thirst.
incidental. This idea derives part of its appeal from the We can even program it to print out at the end “Boy, am I
editors’ keeping their discussion at a very abstract general thirsty!” or “Won’t someone please give me a drink?” etc.
level about “consciousness” and “mind” and “soul,” but if Now would anyone suppose that we thereby have even the
you consider specific mental states and processes — being slightest reason to suppose that the computer is literally
thirsty, wanting to go to the bathroom, worrying about your thirsty? Or that any simulation of any other mental phen-
income tax, trying to solve math puzzles, feeling depressed, omena, such as understanding stories, feeling depressed, or
recalling the French word for “butterfly” — then it seems at worrying about itemized deductions, must therefore produce
least a little odd to think that the brain is so irrelevant. the real thing? The answer, alas, is that a large number of
Take thirst, where we actually know a little bit about people are committed to an ideology that requires them to
how it works. Kidney secretions of renin synthesize a believe just that. So let us carry the story a step further.
substance called angiotensin. This substance goes into the The PDP-10 is powered by electricity and perhaps its
hypothalamus and triggers a series of neuron firings. As far electrical properties can reproduce some of the actual causal
powers of the electrochemical features of the brain in pro-
1 “Strong” to distinguish the position from “weak” or ducing mental states. We certainly couldn’t rule out that
“cautious” AI, which holds that the computer is simply a eventuality a priori. But remember: the thesis of strong AI
very useful tool in the study of the mind, not that the is that the mind is “independent of any particular embodi-
appropriately programmed computer literally has a mind.
John Searle, “The Myth of the Computer” 3

ment” because the mind is just a program and the program manipulation” or, to use the term favored by Hofstadter and
can be run on a computer made of anything whatever Dennett, the whole system is a “self-updating representa-
provided it is stable enough and complex enough to carry tional system”; but these terms are at least a bit misleading
the program. The actual physical computer could be an ant since as far as the computer is concerned the symbols don’t
colony (one of their examples), a collection of beer cans, symbolize anything or represent anything. They are just
streams of toilet paper with small stones placed on the formal counters.
squares, men sitting on high stools with green eye shades — The computer attaches no meaning, interpretation, or
anything you like. content to the formal symbols; and qua computer it couldn’t,
So let us imagine our thirst-simulating program running because if we tried to give the computer an interpretation of
on a computer made entirely of old beer cans, millions (or its symbols we could only give it more uninterpreted sym-
billions) of old beer cans that are rigged up to levers and bols. The interpretation of the symbols is entirely up to the
powered by windmills. We can imagine that the program programmers and users of the computers. For example, on
simulates the neuron firings at the synapses by having beer my pocket calculator if I print “3 x 3 = ,” the calculator will
cans bang into each other, thus achieving a strict correspon- print “9” but it has no idea that “3” means 3 or that “9”
dence between neuron firings and beer-can bangings. And means 9 or that anything means anything. We might put
at the end of the sequence a beer can pops up on which is this point by saying that the computer has a syntax but no
written “I am thirsty.” Now, to repeat the question, does semantics. The computer manipulates formal symbols but
anyone suppose that this Rube Goldberg apparatus is attaches no meaning to them, and this simple observation
literally thirsty in the sense in which you and I are? will enable us to refute the thesis of mind as program.
Notice that the thesis of Hofstadter and Dennett is not Suppose that we write a computer program to simulate
that for all we know the collection of beer cans might be the understanding of Chinese so that, for example, if the
thirsty but rather that if it has the right program with the computer is asked questions in Chinese the program enables
right input and output it must be thirsty (or understand it to give answers in Chinese; if asked to summarize stories
Proust or worry about its income tax or have any other in Chinese it can give such summaries; if asked questions
mental state) because that is all the mind is, a certain kind of about the stories it has been given it will answer such
computer program, and any computer made of anything at questions.
all running the right program would have to have the Now suppose that I, who understand no Chinese at all
appropriate mental states. and can’t even distinguish Chinese symbols from some
I believe that everything we have learned about human other kinds of symbols, am locked in a room with a number
and animal biology suggests that what we call “mental” of cardboard boxes full of Chinese symbols. Suppose that I
phenomena are as much a part of our biological natural am given a book of rules in English that instruct me how to
history as any other biological phenomena, as much a part match these Chinese symbols with each other. The rules say
of biology as digestion, lactation, or the secretion of bile. such things as that the “squiggle-squiggle” sign is to be
Much of the implausibility of the strong AI thesis derives followed by the “squoggle-squoggle” sign. Suppose that
from its resolute opposition to biology; the mind is not a people outside the room pass in more Chinese symbols and
concrete biological phenomenon but “an abstract sort of that following the instructions in the book I pass Chinese
thing.” symbols back to them. Suppose that unknown to me the
Still, in calling attention to the implausibility of sup- people who pass me the symbols call them “questions,” and
posing that the specific casual powers of brains are irrele- the book of instructions that I work from they call “the pro-
vant to minds I have not yet fully exposed the preposter- gram”; the symbols I give back to them they call “answers
ousness of the strong AI position, held by Hofstadter and to the questions” and me they call “the computer.” Suppose
Dennett, so let us press on and examine a bit more closely that after a while the programmers get so good at writing the
the thesis of mind as program. programs and I get so good at manipulating the symbols that
Digital computer programs by definition consist of sets my answers are indistinguishable from those of native Chi-
of purely formal operations on formally specified symbols. nese speakers. I can pass the Turing test for understanding
The ideal computer does such things as print a 0 on the tape, Chinese. But all the same I still don’t understand a word of
move one square to the left, erase a 1, move back to the Chinese and neither does any other digital computer because
right, etc. It is common to describe this as “symbol all the computer has is what I have: a formal program that
John Searle, “The Myth of the Computer” 4

attaches no meaning, interpretation, or content to any of the me obviously yes. But to say that is in no way to endorse
symbols. the systems reply, since in this case we are dealing with the
What this simple argument shows is that no formal pro- specific causal powers of the human brain, whereas the
gram by itself is sufficient for understanding, because it systems reply claims that a system made of any substance at
would always be possible in principle for an agent to go all could have mental states.
through the steps in the program and still not have the rel- The details of how the brain works are immensely com-
evant understanding. And what works for Chinese would plicated and largely unknown, but some of the general
also work for other mental phenomena. I could, for exam- principles of the relations between brain functioning and
ple, go through the steps of the thirst-simulating program computer programs can be stated quite simply. First, we
without feeling thirsty. The argument also, en passant, know that brain processes cause mental phenomena. Mental
refutes the Turing test because it shows that a system, states are caused by and realized in the structure of the
namely me, could pass the Turing test without having the brain. From this it follows that any system that produced
appropriate mental states. mental states would have to have powers equivalent to those
[...] of the brain. Such a system might use a different chemistry,
The rest of what they have to say is mostly a repetition of but whatever its chemistry it would have to be able to cause
points made by other authors and already answered by me. what the brain causes. We know from the Chinese room
Specifically, they endorse the “systems reply” to the argument that digital computer programs by themselves are
Chinese room argument, according to which the man in the never sufficient to produce mental states. Now since brains
room does not understand Chinese, but the system of which do produce minds, and since programs by themselves can’t
he is a part — including the instruction book, the Chinese produce minds, it follows that the way the brain does it can’t
symbols, etc. — really does understand Chinese. Adherents be by simply instantiating a computer program. (Every-
of this view believe, to my constant amazement, that though thing, by the way, instantiates some program or other, and
the man fails to understand, the room does understand brains are no exception. So in that trivial sense brains, like
Chinese. The obvious objection to this is that the system everything else, are digital computers.) And it also follows
has no way of attaching meaning to the uninterpreted that if you wanted to build a machine to produce mental
Chinese symbols, any more than the man did in the first states, a thinking machine, you couldn’t do it solely in virtue
place. The system, like the man, has a syntax but no of the fact that your machine ran a certain kind of computer
semantics. And you can see this by simply imagining that program. The thinking machine couldn’t work solely in
the man internalizes the whole system. Suppose he has a virtue of being a digital computer but would have to
super memory and a super intelligence so that he memorizes duplicate the specific causal powers of the brain. [...]
the instruction book and does all the calculations in his
head. To get rid of the room, we can even suppose he works DANIEL DENNETT’S REPLY TO SEARLE
outdoors. Now since the man doesn’t understand Chinese, Daniel Dennett, a professor of philosophy at Tufts Universi-
and since there’s nothing in the system that is not in the ty and an important scholar in the field of cognitive science,
replied to Searle in the June 24, 1982 edition of The New
man, there is no way the system could understand Chinese. York Review of Books.
As near as I can tell Hofstadter and Dennett’s only reply to
this is to observe that no normal human being could perform To the Editors:
such a feat of memory. This is of course quite true, but also In The Mind’s I, Douglas Hofstadter and I reprint (cor-
quite irrelevant to the point, which, to repeat, is that from rectly) John Searle’s much-discussed article, “Minds,
syntax alone you don’t get semantics. Brains, and Programs,” and follow it with a “Reflection”
For reasons that seem to me utterly confused they think that is meant to refute his position, as he notes in his review
that my reply to one of the thought experiments actually [NYR, April 29]. [...]
commits me to accepting the systems reply. Suppose that We claim that [Searle] has frankly misunderstood the
the neuronal connections of a Chinese-speaking woman are systems reply, and that his remark about “bits of paper”
broken, but suppose that a tiny, lightning-fast demon in her betrays this — and has “blinded him to the realities of the
head makes all the connections in just the right order. situation.” Sometimes it even seems as if he deliberately
Would she then understand Chinese? Assuming that the misrepresents the systems reply, as when he says in his
powers of her brain are fully restored the answer seems to review: “Adherents of this view believe, to my constant
John Searle, “The Myth of the Computer” 5

amazement, that though the man fails to understand, the Searle stresses that a computer program, being “purely
room understands Chinese.” Searle’s amazement stops just formal,” has no causal powers of its own. True, but of
short of inspiring any doubt in his mind about the fidelity of course when a program is physically realized in some
his interpretation, but perhaps this is to be explained by a hardware, and attached by “transducers and effectors” to
certain exegetical carelessness rather than willful caricature. relevant portions of the rest of the world, that physically
What is the heart of the systems reply? It is a distinction realized program can have lots of causal powers: such a
of levels that is not at all mysterious, or new, though program can control an oil refinery, make out payroll checks
Searle’s diminutive “bits of paper” acts to minimize (or or — terrible to say — guide nuclear missiles to their
obfuscate) the point. “The conjunction of a person and bits targets. Let’s call such causal powers control powers. Such
of paper” doesn’t sound like a very different system from a powers are not simulated but real; the computer doesn’t
person alone, does it? How about “the conjunction of a simulate controlling the refinery; it really does control the
person and the Library of Congress with its attendant staff”? refinery. (The distinction between simulating and duplicat-
Does that sound like a supersystem that just might have ing is not as unproblematic as Searle supposes, but we will
some interesting powers or properties lacked by any of its give him the distinction here for the sake of argument.)
proper parts or subsystems? [...] Now Searle has admitted (in conversation on several
Searle, in a letter to me (which he has kindly permitted occasions) that in his view a computer program, physically
me to quote), says: realized on a silicon chip (or for that matter a beer-can con-
In any case you and Hofstadter still miss the point. No traption suitably sped up and hooked up) could in principle
matter how big the program, the conjunction of man duplicate — not merely simulate — the control powers of
and bits of paper is no different from man alone. All the human brain. That is, such a computer program (some-
of the bits of paper in the world add nothing to the neu-
rophysiological powers of the man’s brain. The whole how realized) could control a human body in all its activi-
point of reminding the reader that these are just “bits of ties. Would such a body have a mind? We on the outside
paper” is that they are not in any way an addition to the would find its behavior indistinguishable from that of a
specific neurophysiological powers of the man’s brain.
normal human being, but whether or not it really had a mind
Here Searle manifestly misunderstands the systems would depend, Searle insists, on whether the hardware real-
reply. No one claims the supersystem gives the subsystem ization of the control program shared with the missing brain
by itself special new powers or properties. Rather, we (and not only all its control powers (granted ex hypothesi) but
many others) claim that the supersystem itself — the whole also some other “causal powers” entirely undetectable by
supersystem — has these powers. Searle’s persistent deaf others in behavior, including the behaviors of introspective
ear to this point puzzles me, particularly since it is really speech, emotional reaction, and so forth.
just a “category mistake” claim of the sort that was all the What powers could these be? Where would the physical
rage during Searle’s graduate student days at Oxford. In his effects of these neurophysiological powers show up? Searle
reply to my earlier commentary on his paper (in Behavioral answers that they would show up in the individual subject’s
and Brain Sciences) he objects to my rather Oxonian claim consciousness of his own intentionality. But would these be
that I understand English — my brain doesn’t — with the physical effects? If so, they must be detectable (in princi-
retort: “I find his claim as implausible as insisting, ‘I digest ple) by outsiders. Would they register on the instruments of
pizza; my stomach and my digestive tract don’t.’” How neuroscientists (if not “behaviorists”)? Searle does not say,
important a single word can be! The verb “digest” is nicely but since he insists that the effects are introspectible (only?)
chosen, for note how radically the image shifts if we switch it is tempting to conclude that the effects are presumed to be
to “eat” or “enjoy.” Does Searle find it quite all right to say non-physical, and that Searle is some sort of dualist. He
that his stomach eats pizza? Can his mouth eat pizza? adamantly denies it; he insists the causal powers he is
Which proper part of him could be said to enjoy the pizza? discussing are physical, so they must have physical, publicly
Levels do make a difference. Anyone who hunts for a observable effects. Where, if not in the subject’s behavior?
pizza-enjoying subsystem in a human being is on a fool’s Just in the brain? What would these effects do?
errand, and anyone who denies that a supersystem under- These are mysterious causal powers indeed, despite their
stands Chinese on the grounds that none of its subsystems scientific-sounding name. We frankly disbelieve in them —
do is making the same error moving in the other direction. which is the extent of our “behaviorism.” Surely we all
[...] agree that anything that has all the relevant causal powers of
John Searle, “The Myth of the Computer” 6

food — it saves one from starving, sustains growth and shrink from imputing such a silly view to him, and await his
repair, tastes good, etc. — is food. And anything that has all further clarification of his position.
the causal powers of oxygen is oxygen. We think that you Searle paints us as taken in by the “mythology” of com-
could in principle give a body an artificial brain by giving it puters. We see ourselves as demythologizers, and Searle as
something that duplicated all the brain’s control powers. the victim of several superannuated myths, but perhaps we
And any creature so equipped would “have a mind” in the have misinterpreted his view.
only sense that makes any sense: it would have a well-
functioning (prosthetic) brain. Now perhaps we are wrong; Daniel C. Dennett
Tufts University
perhaps there are some other causal powers that matter.
Medford, Massachusetts
Searle thinks so; he thinks organic brains “produce inten-
tionality.” It sometimes seems as if he thinks intentionality
is some marvelous fluid secreted by the brain — but we