From: AAAI-94 Proceedings. Copyright © 1994, AAAI ([Link]). All rights reserved.

CHATTERBOTS, TINYMUDS,and the Turing Test

Entering the Loebner Prize Competition
Michael L. Mauldin
Carnegie Mellon University Center for Machine Translation
5000 Forbes Avenue
Pittsburgh, PA 15213-3890
fuzzy @[Link]

Abstract AI focused on the program’s lack of an internal world

model that influenced and tracked the conversation.
The Turing Test was proposed by Alan Turing in 1950; he PARRY simulates paranoid behavior by tracking its own
called it the Imitation Game. In 1991 Hu h Loebner started the internal emotional state on a few different dimensions.
Loebner prize competition, offering a f 100,000 prize to the
author of the first computer program to pass an unrestricted
Colby subjected PARRY to blind tests with doctors ques-
Turing test. Annual competitions are held each year with tioning both the program and three human patients diag-
smaller prizes for the best program on a restricted Turing test. nosed as paranoid. Reviews of the transcripts by both
This paper describes the development of one such Turing psychiatrists and computer scientists showed that neither
System, including the technical design of the program and its group did better than chance in distinguishing the computer
performance on the first three Loebner Prize competitions. We from human patients.
also discuss the program’s four year development effort, which Often overlooked is Colby’s comparison of PARRY’s
has depended heavily on constant interaction with people on and human dialogs with RANDOM-PARRY. He showed that
the Internet via Tinymuds (multiuser network communication
servers). Finally, we discuss the design of the Loebner com-
merely choosing responses at random did not model the
petition itself, and address its usefulness in furthering the human patients’ responses as well as standard PARRY.
development of Artificial Intelligence. Shieber argues that PARRY fooled its judges because
paranoid behavior makes inappropriate responses or non
sequiturs appropriate. But there is still a certain logic to
Introduction them that PARRY simulates effectively. It is simpler to
In 1950, Alan Turing proposed the Imitation Game as a simulate paranoid behavior, perhaps, but it is no&trivial.
replacement for the question, “Can machines think?” He In our view, PARRY is an advance over ELIZA because
predicted that by the year 2000 technological progress PARRY has a personality. The Rogerian therapist strives to
would produce computing machines with a capacity of log eliminate all traces of his or her own personality, and
bits, and that with such machinery, a computer program ELIZA therefore succeeds without one.
would be able to fool the average questioner for 5 minutes
about 70% of the time (Turing, 1950). TINYMUD
In 1991, Dr. Hugh Loebner, the National Science
In August 1989, Jim Aspnes opened TINYMUD, an elegant
Foundation, and the Sloan Foundation started the Loebner reimplementation of Richard Bartle’s multiuser dungeon
Prize Competition: an annual contest between computer
(MUD). See (Rheingold, 1991) for more details. Key
programs to identify the most “human’~ programs, and features of TINYMUD include:
eventually to award $100,000 to the program that first
passes an unrestricted Turing test (Epstein, 1992). This 0 multiplayer conversation,
competition has been criticized as a parlor game, reward- 0 textual “scenery” simulating physical spaces,
ing tricks rather than furthering the field of Artificial Intel- 0 user extensibility.
ligence (Shieber, 1992). This last feature, the ability of players to create their own
In this paper, we discuss our own entry in the Loebner subareas within the world model, was a key feature that
competition, including a description of our own tricks, and made TINYMUD very popular.
describe how techniques and methods from AI are used to TINYMUD provided a world filled with people who
go beyond tricks. One of our goals is to encourage more
communicate by typing. This seemed to us to be a ripe
participation by the AI community in the Loebner Com-
opportunity for work on the Turing test, because it
petition. provided a large pool of potential judges and interviewees.
In TINYMUD, computer controlled players are called
History “hots,” short for robots. Many simple robots were
Fifteen years after Turing proposed the imitation game, created, and even ELIZA was connected to one stationary
Weizenbaum’s ELIZA program demonstrated that “a robot (if a player went alone into a certain cave, he could
simple computer program” could successfully play the im- chat with ELIZA).
itation game by resorting to a few “tricks,” the most im- We created a computer controlled player, a “Chatter
portant being to answer questions with questions (Weizen- Bot,” that can converse with other players, explore the
baum, 1976). world, discover new paths through the various rooms,
ELIZA sparked the interest of many researchers, but answer players’ questions about navigation (providing
perhaps the most interesting result was Colby’s work on shortest-path information on request), and answer ques-
PARRY (Colby, 1975). Criticism of ELIZA as a model for tions about other players, rooms and objects. It can even

16 The Arts
join in a multi-player card game of “Hearts” It has won J: Being liberal or coneenmtive implies
many rounds, “shooting the moon ” on several occasions. that we can attach these labels and
have some kind of self image and free
The conversational abilities were originally im-
will. Do you agree that we have free
plemented as simple IF-THEN-ELSE rules, based on pat- will?
tern matching with variable assignment. Most patterns C: YOU have free will only if you believe
have multiple outputs that are presented in a random, non- you have free will1 Can you converse
repeating sequence to handle repeated questions. with an arrdvaark? Pleaee continue...
A primary goal of this effort was to build a conver-
sational agent that would answer questions, instead of ig- Weintraub’s program won the first competition using
noring them, and that would be able to maintain a sequence the topic of “Whimsical Conversation.” Even though the
of appropriate responses, instead of relying on topics he has chosen for the two following years are more
non-sequiturs. We included a lot of humor among the serious, the program retains a comedic flavor, and humor is
responses, and succeeded in making an agent more inter- the program’s main strength.
esting than ELIZA; Wired magazine described our program Bad Marriage. The second place program was written by
as “...a hockey-loving ex-librarian with an attitude.” Peter and Ken Colby of Malibu Artifactual Intelligence
(Wired, 1993). Works, based on a conversational program they sell.
The CHATTERBOT succeeds in the TINYMUD world be- J: my husband is inumtent and im a
cause it is an unsuspecting Turing test, meaning that the nymphomaniac can you help me dot?
players assume everyone else playing is a person, and will C: what makes your husband so helplese?
give the CHATR%BOT the benefit of the doubt until it J: he has environmental poisoning
makes a major gaffe. C: He has lost interest in life.

The Loebner Prize Although Ken Colby was PARRY’S author, this program
is not a variant of PARRY. Instead is is derived from a
By mid-1991 the framework of the robot was mature, and program that serves as a conversational companion for
work was concentrated on improving the conversational depressed patients. It is rule-based, with the left-hand side
component. When the first Loebner competition was an- matching concepts expressed in the user input string, and
nounced that same year, we submitted logs from the the right-hand side containing multiple options for substr-
robot’s sessions on TINYMUD. The program qualified for ings, which are instantiated randomly to provide lexical
entry, and placed third of six programs in the first competi- variation.
tion, ranked above a human by one of the 10 judges.
The final round of competition was smaller in 1992 and
1993, down to three computer programs from six, and the
same three programs have finished first, second, and third A CWA'TTERBOTis a robot TINYMUD player whose main
all three years. In 1992, we chose hockey as a domain for service is chattering. When few players are logged into the
discourse, and the program finished dead last, partly be- game, some people find talking to a robot better than talk-
cause of a lack of hockey fans among the judges (leading
to more difficult questions). The conversational model
was expanded to include a tree-shaped network of
input/output patterns, much like the opening book of a
chess program, but the mismatch of expectation between
the program and the judges was so great that very few
conversations followed the expected lines of questioning.
For 1993, we chose a more universal topics, “Pets,”
on the premise that everyone has had a pet at some time in
their life. The tree-based model was scrapped in favor of
an activation network that was less rigid and allowed the
program to make more associations between the judge’s
input and its own output. Although the program still
finished third of three, it was no longer dead last: two of Figure 1: CHA~RBOT Configured for TINYMUD
eight judges ranked it more human than the second place
program, and a third judge ranked it above the first place Architecture
program.
The CHATTERBOT is implemented as a C program with
Two other computer programs qualified for the final several different modules for dealing with the various func-
round; the first place entry discussed “Liberal or Conser- tions required to automate a player in the TINYMUD world:
vative,” and the second place program tried to help people
discuss “Bad Marriage.” Q)communications, handles the TCP/IP connections.
e protocol interpreter, decodes game related messages.
Liberal or Conservative. The winning entry was written
by Joseph Weintraub of Thinking Softwure, and was based 8 world model, tracks the various rooms and objects,
on his PC Therapist program. modeling the world as a directed graph, and providing
shortest path searches as needed.
J: What would you say is a liberal position
on drugs?
e player memory, tracks the other players and records up
C: Do you have liberal or conservative views to 2000 bytes of their most recent utterances.
about the legalization of pot? Oh, all e exploration module, directs an open-ended exploration
this typing is getting to meI Please of the world when the robot is not conversing.
continue.. . e conversation module, provides the “chatter.”

Believable Agents 17
 Figure 1 shows the CHATTERBOT configured for play
on a TINYMUD. Records of other players’ appearances,
rooms within the MUD, and other players’ utterances are
kept in long term (world) memory. The CHA-ITERBOTalso
has a contest mode, in which it simulates human typing
using a Markov model. Because the program does not
expect to talk with the same judge again, no long term
world model is kept in contest mode.
The conversation module is implemented as a
prioritized layer of mini-experts, each an ordered collection
of input patterns coupled with a set of multiple possible
responses.
0 command patterns are the highest priority. These
represent direct commands from the robot’s owner, and
include hand-shaking challenges, “What’s the code re 2: Portion of conversational network
word?, ” to prevent other players from spoofing com-
mands to quit the game. <havepet>
0 hi priority responses include common queries that the a:l.O
keyword patterns handle well, “How do I get from the p:l *
Town Square to the Library Desk? *’ ~:DO you have any pets?
0 activation network includes the bulk of the topic +:<havepet-1s <havepet-2* <haveget-3s . ..
oriented responses; weights on the nodes of the network
*havepet-1s
encode state information about what the user and a:O.l
program have said. p:l NE0
e lo priority responses include a series of patterns for r:Why not?
common sense things the robot should know about itself, +:<havepet-l-1, thaveget-l-2>
“Where do you live?” “What’s 2 times 23?” “What -:<havepet-9, &d-46,
color is your hair?, ” that have been collected over 4 <have-t-l-1,
years of interaction on TINYMUD. a:0.02
0 sorry responses are the typical last ditch responses that p:2 *apartment*
are used when no input pattern matches. As a debugging p:3 *allerg*
aid, any input that generates a “Go on,” “So?” or “I’ll r:You could still have a flab tank, or
remember that” response is logged in a separate file. maybe a terrarium with a turtle or two.
-:<havepet-9>
Activation-based Responses
Figure 3: Sample conversational nodes
The bulk of the topic-oriented responses are encoded in an
activation network, partially shown in Figure 2. Details of
the starting node and two subnodes are shown in Figure 3; Given a sufficiently large network of conversational
each node has 5 attributes: nodes (our program ran with 224 nodes, plus 529 fixed
responses), the conversation problem reduces to a retrieval
ACTIVATION(a) each node starts with an initial activa- problem: among the things that I could say, what should I
tion level between 0.0 and 1.0. say?
For example, if the user input mentions birds, the
PA-lTERNS(p) one or more patterns (with weights) are response strings are searched for matches to birds, includ-
matched against the user input. If the ing parrots, canaries, etc., and those nodes have their ac-
pattern succeeds, the activation of the tivation level raised. The code was borrowed from one of
node is raised by that amount. our information retrieval engines.
By encoding information about the user in the activa-
RESPONSE(r) a single text string used as the response tion levels, the program tracks the judge’s responses. For
if this node has the highest activation. example, if the judge answers negatively to the question
about whether he has pets, the other nodes that ask about
ENHANCEMENT (+) if this node is used for a response, the pets are inhibited.
named nodes have their activation in-
creased. Tricks
INHIBITION(-) if this node is used for a response, the Shieber has criticized the Loebner competition as reward-
named nodes have their activation in- ing tricks (Shieber, 1992). This sort of qualitative assess-
hibited. ment of programmed knowledge is exactly what the Turing
test is supposed to avoid, replacing the question “Can
These figures show a small portion of the pet domain machines think?” with a performable test.
network. Additional world knowledge is encoded in the Here we unashamedly describe some of the better
ontology used during pattern matching. The program has a tricks, confident in the belief that when someday a com-
typical type hierarchy that allows a pattern to match just puter program does pass the Turing test, it will use many of
DOG,BIRJJ,PET,WILD,OrANIMAL, fOreXample. them, for the simple reason that people already use them.

18 The Arts
ELIa’s Tricks Transcripts
ELIzA's main trick was to use questions to draw a conver- This transcript is taken from the 1993 Loebner Competi-
sation out of the user himself, with little or no actual con- tion (Epstein, 1993). It contains the entire record of Judge
tribution from the program. This works because most #4’s first of two turns. The elapsed time is about 9 minutes
people like to talk about themselves, and are happy to and 29 seconds. This judge rated the program behind the
believe the program is listening. To quote Weizenbaum other two computers, but this fragment represents the
(Weizenbaum, 1976): longest string of reasonable responses during the competi-
tion. This judge’s low score is most likely explained by
What I had not realized is that extremely short ex-
the fact that during his second turn, the program repeated
posures to a relatively simple computer program two responses verbatim, and completely ignored several
could induce powerful delusional thinking in quite direct questions. The judge’s questions are marked “J”
normal people. and the computer’s responses are marked “C.”
The illusion of listening is fostered by including substr- J: DO you know anything about miniature
ings of the user’s input in the program’s output schnauzers?
Ueer : You hate me. c: I don't know anything ahout miniature
Eliza: Doe6 it please YOU to believe that schnauzers, Judge.
I hate you? Admitting Ignorance, repeating user input

A further trick is the use of the Rogerian mode, which J: well, I have one. We call him Pucci. He
provides unimpeachable cover for the computer. Since the ie a crazy cat--have YOU ever had a cat?
c: What a coincidence1 I have two cats
program never says anything declaratively, it cannot con-
tradict itself later. direct answer to direct question from activation net
J: I prefer doge myself. Cat0 are too
PARRY'S Tricks independent and they always jump up on
things. By the way, are you a h8D Cat?
PARRY has a much larger collection of tricks (Colby, c: People don't own cats...it's the other
1975): way around.
0 admitting ignorance, unlike ELIZA, PARRY can say “I controversial statement with expectedfollowups
don ‘t know. ” in response to a question. J: Soma people have enakee for pets-- do
@changing the level of the conversation, “Why do you you think they can be of any value as
ask that?, ” which is really a way of changing the sub- a pet?
ject. c: I think cat people are very self-aeeured.
0 rigidly continuing previous topic, PARRY includes Cata are very independent, but they can
also be very loving. Dogs are juet too
small stories about the Mafia, and tries to tell these
eubeervient and quick to pleaee for my
stories in a fixed order. taste.
e introducing new topic, launching into a new story. rigidly following up on previous statement about cats,
This simulates paranoid behavior well, but can also work but reference to snakes increases activation of snake topic.
to a limited extent in regular conversation.
J: Okay- - but how do you "pet" a snake?
c: Wow, that's unuaual. I don't know many
CHATTERBOT'S Tricks make owners.
The CHATIERBOT implements all of the above tricks from snake topic activated after two references
ELIZAand PARRY. It also uses several other tricks: J: How about turtles? All they do ie eit in
e Having many fragments of directed conversation stored their little artificial ponds, eat and
in activation network, much like the opening book of a shit, am I right?
c: Turtle6 just aren't very satisfying a0
chess playing program. These fragments enhance the D@btB. They don't seem to have emotione.
“connectedness” of the program’s own responses, and if direct response from activation network
the user plays along, the whole conversation can hang
together for a while. J: Ahal But where do we draw the line a6
humana? Who are we to decide which
0 Controversial statements, “People don ‘t own cats..., ” are animala have worth and which do not?
a way to drag the user into the program’s conversation, c: 00 on.
rather than letting the user direct the discourse. default, non-committal response, given no hot topic keywords
e Humorous statements, “Guinea pigs are about as smart
J: Take bees for exarpgls. I happen to like
as cucumbers, *’also make the program seem more human. them.
e Agreeing with the user, in certain cases, the program can C: He too.
safely agree with the user rather than being non-committal. TINYMUD level lo-priority response to user’s liking something
0 Excerpting USENET News, many of the fragments in the
activation network were gleaned from the newsgroup By our rough estimate, 7 of these 8 replies are respon-
[Link] patterns were added by hand. sive, although the last 2 are somewhat weak. The “self-
8 Simulated typing, by including realistic delays between assured” reply we judge unresponsive. A similar analysis
of the entire 3 hour log showed that only 45 of the 137
characters, we imitate the rhythm of a person typing. (34%) replies were responsive, so the average performance
Slowing down the interaction also gives the program fewer was not as good as this particular dialog. We also found
chances to make a big mistake.

Believable Agents 19
another 34 cases (25%) where the activation network did Does The Loebner Prize Further AI?
contain a responsive reply that could have been used if the
input patterns were more complete. We strongly believe that the current work on the Loebner
competition will hasten the passing of an unrestricted
Turing test. Any such program will have to be more than
Simulating Human Typing
just a general problem solver: some of its knowledge and
One observation made during the first Loebner Prize was tactics will be based on long experience with typed com-
that although many programs attempted to simulate human munication with real people. The best two examples of
typing, most failed miserably (Epstein, 1992). Although typed conversational interaction today are network com-
our first program did attempt to simulate human typing, munications (TINYMUD and INTERNETRelay Chat), and
this module was replaced for the second and third annual the Loebner Prize, and only the Loebner Prize competition
competitions. In the last two competitions, all output from is committed to providing unbiased judges and a double-
programs was buffered, but even so, by simulating human blind judging scheme to validate the results.
typing at all points, we obtain realistic delays in the ap
pearance of the response to the judge. And if character- A Restricted Turing Test
mode is used in future competitions. we have a realistic
model available. Because current programs are a long way from passing an
The basic method is to use a Markov model of the unrestricted Turing test, the Loebner prize competition
intercharacter delay based on character trigrams. We ob- allows the programme r to restrict the discussion to a single
tained the real-time logs of the 1991 competition from the domain. But this can be problematic; for example, here
Cambridge Center for Behavioral Studies, and sampled the Judge #2 is clearly trying to avoid talking directly about
typing record of judge #lO (chosen because he was the the topic of “Pets:”
slowest typist of all 10 judges). The average delay be- J: Have you ever kept a penthouae pet?
tween two characters is 330 milliseconds, with a standard C: Do you have any pets?
deviation of 490 milliseconds (these values were computed J: I think you're avoiding my question.
from a total of 9,183 characters typed by that judge during C: Do you think that animals can think?
a three hour period). We also determined that the average
delay between the terminal’s last output and the judge’s J: Do you think it ethical for one species
first typed character was 12.4 seconds with a standard to enslave another?
deviation of 11.4 seconds. C: What kind8 of pet8 did you have ae a
child?
J: I'm asking the questions. what ia a pet
if it ie not a slave of epeciee-iem?
C: Oh, that's so sad. I think pets enrich
a childhood immneely.
J: How can we keep 8ome animals a.8 pets,
yet eat others?
C: Guinea pigs are about as smart a6
cucumbers.
We do not believe that the restriction to a domain
weakens the test in any important way. If a program can
pass the test in a single domain, then 50 or 100 groups
making similar programs for different domains would
Om6 2SOm8 5OOms 750mo
make a program broad enough that the judge and the
Intercharacter Delay in Milliseconds program could find common ground for a discussion. By
taking the initiative, the program could lead the judge into
Figure 4: Judge #lo: Intercharacter delay distribution an area of its own expertise. Neither ELIZA nor PARRY
restricted their users to a single domain.
After the response has been generated by the conver- Lastly we note that in 1995 the Loebner Competition
sational components, the typing simulator inserts random will hold its first unrestricted Turing test, with no limits on
errors (1 in 68 characters), such as mishits, transpositions, the topics discussed.
insertions and deletions, and case errors. A physical model
of key positions on the standard QWERTY keyboard is Chosing Judges
used to generate mishits. After the error is inserted, back- The slipperiest term in Turing’s description of the imitation
spaces and the correct letters are inserted to simulate back- game is “average questioner.” The 1991 and 1992 com-
ing up to correct the error (5% of mistakes are left uncor- petitions chose judges from the general public using a
rected). All of these probabilities were estimated by newspaper classified ad. For the 1993 competition, judges
analyzing the log of judge #lo. were members of the national press. It’s clear from the
Once the errorful string with corrections is generated, it transcripts and scores that the press corps are a much less
is displayed to the judge one character at a time with docile and cooperative group of questioners.
random delays between the characters. The delay times are Another unforeseen difficulty faced by our program in
generated by using the smoothed probability distribution 1993 was that each judge was given two turns, instead of
shown in Figure 4, with the mean and standard deviation one turn as in the first two competitions. We had just
modified to match the trigram statistics (for trigrams with 3 weeks before installed code to detect a change of judges
or fewer occurrences in the input log, bigram or single based on the timing of the input (the Colbys’ program also
character statistics are used). used timing to detect new judges). When a new judge was

20 The Arts
detected, the activation levels of the nodes were reset so We would like to see increased participation in the
that each judge started out with a fresh network. This Loebner Prize. We hope by dissecting one of the three best
greatly improves the coherence of the dialog for each programs in the competition to spur others to conclude “I
judge, but it meant that during their second conversations could have written something better than that!” and then
most of the judges had one or more responses repeated. do so.
Some even co mmmented on it during the test:
J: What is your favorite dog breed? Acknowledgments
c: we had to give her away. We only had a All transcripts from the Loebner Competition were used
fifth of an acre of land, and auseiee
love to run.
with the permission of the Cambridge Center for Be-
havioral Studies, 675 Massachusetts Avenue, Cambridge,
J: You're giving me the exact taame answers Mass., 02139.
you gave me first time around. What ie This work was supported in part by the Center for
your favorite dog breed? NIachine Translation, and benefited from the collaboration
In effect, this change in the methodology turns what and inspiration of many people, including: Jim Aspnes,
had been a 15 minute test into a 3 hour long test, making it Joseph Bates, Stewart Clamen, Scott Dickenshied, Guy
much more difficult for the program. Turing’s original Jacobson, Ben Jackson, Eric Nyberg, John Ockerbloom,
game was only 5 minutes long. Indeed, by asking the Russ and Jennifer Smith, Conrad Wong, and Bennet Yee.
judges’ final rankings after they’ve had 3 hours of inter-
action with a collection of people and computers, they may More Infomaation
have learned enough about computers to no longer qualify
as average questioners. The scores would probably be The entry deadline for the 1994 competition is November
quite different if each judge spent only 5 minutes with each 1. Entrants must submit up to 10 double-spaced pages of
program. One of the judges, Judith Gunther of Popular logs of their program interacting with human beings. The
Science magazine, told me that her rankings might have Loebner Prize committee will select no more than 8
been different if she had been asked to score each terminal finalists from the submissions, and finalists will be notified
by November 21. The competition itself will be held in
immediately after each conversation.
real-time in San Diego on December 12, 1994. To obtain
Trick or AI? an entry form, write the Cambridge Center at the above
address.
The Turing test in general and the Loebner prize in par- To converse with Julia yourself, TELNET to host
ticular reward tricks, and the winning program the last [Link], and enter usemame “julia” with
three years clearly has some good tricks. But adding real no password. Type one or more lines of English, followed
world knowledge and deeper understanding in addition to by two carriage returns to end your input.
the tricks helped our program perform better (we credit our
improvement between 1992 and 1993 in part to the ad- eferences
dition of the activation network and the ontology of
animals, and not at all to better tricks). It may be amazing Colby, K. Artificial Paranoia: A Computer Simulation of
how far a program can get on tricks alone, but our current Paranoid Process. Pergamon Press, New York, 1975.
improvements come from modeling the world and the con-
versation, and that will be our focus in coming competi- Epstein, R. The Quest for the Thinking Computer. AAAl
tions. Magazine 13(2):80-95, Summer, 1992.
But suppose that simply increasing the size of ELIzA’s
script or the CHATIERBOT’Sactivation net could achieve Epstein, R. 1993 Loebner Prize Competition in Artificial
Turing’s prediction of fooling 70% of average questioners Intelligence: OJicial Transcripts and Results. Technical
5 minutes. After all, the CHAITERBOThas already fooled Report, Cambridge Center for Behavioral Studies, Decem-
“average” questioners in the TINYMUD domain for a few ber, 1993.
minutes. If a larger collection of “tricks” sufficed, would
you redefine “artificial intelligence,” “average ques- Rheingold, H. Virtual Reality. Summit Books, New York,
tioner,” or “trick?” 1991.

Conclusion Shieber, S. Lessons from a Restricted Turing Test. Tech-

nical Report TR-19-92, Harvard University, Sept., 1992.
Perhaps the biggest obstacle to improvement in this area is Revision 4.
that there aren’t very many uses for fooling people besides
the Turing test. This tension is present in our own Turing, A.M. Computing Machinery and Intelligence.
program: in the TINYMUD world, the robot is most useful Mind 54(236):433-460, October, 1950.
when answering stylized questions in a somewhat
computer-like fashion. These TINYMuD-specific services Weizenbaum, J. Computer Power and Human Reason.
are disabled during actual competitions. The only funded WI-I. Freeman and Co., New York, 1976.
research we know of in Turing systems is for entertain-
ment: providing agents for interactive fiction. In such Wired. Wired Magazine. Louis Rossetto, San Francisco,
works, the reader wishes to be fooled; it becomes a posi- Dec. 1993.
tive part of the experience.

Believable Agents 21