Software Agent
- Overview -
Outline
• Overview of agents
– Definition
– Characteristics
– A typology of agents
– Comparison
– Related research areas
– Applications
• Summary
2/41
Internet
Telephone
Face to Face
3/41
Attributes of Intelligent Behavior
• Think and reason
• Use reason to solve problems
• Learn or understand from experience
• Acquire and apply knowledge
• Exhibit creativity and imagination
• Deal with complex or perplexing
situations
• Respond quickly and successfully to
new situations.
• Recognize the relative importance of
elements in a situation
• Handle ambiguous, incomplete, or
erroneous information
4/41
AI Application in Software Agents
Inference
Decision
Planning
Modeling
AI Application Learning
in Software Agents
Interaction
5/41
What is the Agent?
• “ An agent is anything that can be viewed as
perceiving its environment through sensors
and acting upon that environment through
effectors.”
• “ Autonomous agents are computational
systems that inhabit some complex dynamic
environment, sense and act autonomously in
this environment, and by doing so realize a
set of goals or tasks for which they are
designed.” [Maes, 1995]
6/41
What is the Agent?
• “A hardware or (more usually) software-based computer
system that enjoys the following properties: autonomy, social
ability, reactivity, pro-activeness.” [Wooldridge and Jennings,
1995]
• “Intelligent agents continuously perform three functions:
perception of dynamic conditions in the environment; action to
affect conditions in the environment; and reasoning to
interpret perceptions, solve problems, draw inferences, and
determine actions.” [Hayes-Roth, 1995]
• “Intelligent agents are software entities that carry out some
set of operations on behalf of a user or another program with
some degree of independence or autonomy, and in so doing,
employ some knowledge or representation of the user's goals
or desires.” [IBM]
7/41
Software Agent
• A formal definition of “Agent” [Wooldridge, 2002]
– An agent is a computer system that is situated in some environment, and that is
capable of autonomous action in this environment in order to meet its design
objectives
AGENT
input output
ENVIRONMENT
• Key attributes
– Autonomy: capable of acting independently, exhibiting control over their internal state
and behavior
– Situateness
• Place to live: it situates in some environment
• Perception capability: its ability to perceive the environment
• Effector capability: its ability to modify the environment
– Persistent: it functions continuously as long as it is alive
8/41
Characteristics of Software Agents
Cooperation (Proactive)
Collaborative
Smart agent learning
agent
Collaborative
agent Interface
agent
Autonomy Adaptation
(Learning)
[Nwana, 1996]
9/41
Software Agent
Adaptation
• Agents adapt to their
environment and users
and learn from
Cooperation experience.
– Via machine learning,
knowledge discovery,
data mining, etc.
– Via exchange of
metadata, brokering, and
Autonomy Adaptation facilitation.
– Interface agents acquire
and use user models
– Situated in and aware of
their environment
10/41
Software Agent
Cooperation
• Agents use standard
languages and protocols to
cooperate and collaborate
Cooperation to achieve common goals.
– Cooperate with human
agents and other software
agents
– Supported by agent
communication languages
and protocols.
Autonomy Adaptation
– Consistent with human
conventions and intuition.
– Toward team formation and
agent ensembles
11/41
Software Agent
Autonomy
• Agents act
autonomously to
pursue their agenda.
Cooperation
– Proactive and reactive
– Goal directed behavior
– Appropriately
persistent
Adaptation
– Multi-threaded
Autonomy
behavior
– Encourage viewing
from an “intentional
stance”
12/41
Intelligent Agents
• An intelligent agent is one that is capable of flexible autonomous action in
order to meet its design objectives
– Agent + flexibility
• Properties on flexibility
– Reactivity: agents are able to perceive their environment, and respond in a timely
fashion to changes that occur in it in order to satisfy its design objectives
– Pro-activeness: intelligent agents are able to exhibit goal-directed behavior by taking
the initiative in order to satisfy its design objectives
– Social ability: intelligent agents are capable of interacting with other agents (and
possibly humans) in order to satisfy its design objectives
• Open research issue
– Purely reactive is easy
– Purely proactive is not hard
– But designing an agent that can balance the two remains open
• Because most environments are dynamic rather than fixed
• Taking into account possibility of program failure rather than blindly executing program
13/41
Intelligent Agents
Reactivity
• If a program’s environment is guaranteed to be fixed, the program
need never worry about its own success or failure – program just
executes blindly
– Example of fixed environment: compiler
• The real world is not like that: things change, information is incomplete.
Many (most?) interesting environments are dynamic
• Software is hard to build for dynamic domains: program must take into
account possibility of failure – ask itself whether it is worth executing!
• A reactive system is one that maintains an ongoing interaction with its
environment, and responds to changes that occur in it (in time for the
response to be useful)
14/41
Intelligent Agents
Proactiveness
• Reacting to an environment is easy (e.g., stimulus
response rules)
• But we generally want agents to do things for us
• Hence goal directed behavior
• Pro-activeness = generating and attempting to achieve goals;
not driven solely by events; taking the initiative
• Recognizing opportunities
Example: Autonomous Vehicles, Smart Thermostats, Intelligent
Personal Assistants
15/41
Intelligent Agents
Balancing Reactive and Goal-Oriented Behavior
• We want our agents to be reactive, responding to
changing conditions in an appropriate (timely) fashion
• We want our agents to systematically work towards
long-term goals
• These two considerations can be at odds with one
another
• Designing an agent that can balance the two remains
an open research problem
16/41
Intelligent Agents
Social Ability
• The real world is a multi-agent environment: we cannot go
around attempting to achieve goals without taking others into
account
• Some goals can only be achieved with the cooperation of
others
• Similarly for many computer environments: witness the
Internet
• Social ability in agents is the ability to interact with other
agents (and possibly humans) via some kind of agent-
communication language, and perhaps cooperate with others
17/41
Intelligent Agents
Other Properties
• Mobility: the ability of an agent to move around an electronic
network
• Veracity – honesty: an agent will not knowingly communicate false
information
• Benevolence – kind: agents do not have conflicting goals, and that
every agent will therefore always try to do what is asked of it
• Rationality: agent will act in order to achieve its goals, and will not
act in such a way as to prevent its goals being achieved - at least
insofar as its beliefs permit
• Learning/adaption: agents improve performance over time
18/41
Summary of Agents’ Features
[Brenner et al., 1998]
19/41
A Typology of Agents
• This section attempts to place existing agents
into different agent classes
• There are several dimensions to classify
existing software agents.
• Firstly, agents may be classified by their
mobility, i.e. by their ability to move around
some network. This yields the classes of static
or mobile agents.
20/41
• Secondly, they may be classed as either deliberative or
reactive. Deliberative agents derive from
• the deliberative thinking paradigm: the agents possess
an internal symbolic, reasoning model and they engage
in planning and negotiation in order to achieve
coordination with other agents.
• These reactive agents on the contrary do not have any
internal, symbolic models of their environment, and
they act using a stimulus/response type of behaviour
by responding to the present state of the environment
in which they are embedded (Ferber, 1994).
21/41
• Thirdly, agents may be classified along several
ideal and primary attributes which agents
should exhibit.
At BT Labs, we have identified a minimal list
of three: autonomy, learning and
cooperation.
22/41
Agents vs. Objects
• Are agents just objects by another name?
• Object:
– encapsulates some state
– communicates via message passing
– has methods, corresponding to operations that may be performed on
this state
• Main differences:
– agents are autonomous: agents embody stronger notion of autonomy
than objects, and in particular, they decide for themselves whether or
not to perform an action on request from another agent
– agents are smart: capable of flexible (reactive, pro-active, social)
behavior, and the standard object model has nothing to say about such
types of behavior
– agents are active: a multi-agent system is inherently multi-threaded, in
that each agent is assumed to have at least one thread of active control
23/41
Agents vs. Expert Systems (1)
• Aren’t agents just expert systems by another name?
• Expert systems typically disembodied ‘expertise’ about some (abstract)
domain of discourse (e.g., blood diseases)
• Example: MYCIN knows about blood diseases in humans
– It has a wealth of knowledge about blood diseases, in the form of rules
– A doctor can obtain expert advice about blood diseases by giving MYCIN facts,
answering questions, and posing queries
• Main differences:
– agents situated in an environment: MYCIN is not aware of the world — only
information obtained is by asking the user questions
– agents act: MYCIN does not operate on patients
• Some real-time (typically process control) expert systems are agents
24/41
Intelligent Agents vs. AI
• Aren’t agents just the AI project?
Isn’t building an agent what AI is all about?
– AI aims to build systems that can (ultimately) understand natural language, recognize
and understand scenes, use common sense, think creatively, etc. - all of which are
very hard
– So, don’t we need to solve all of AI to build an agent…?
• When building an agent, we simply want a system that can choose the right
action to perform, typically in a limited domain
• We do not have to solve all the problems of AI to build a useful agent:
– a little intelligence goes a long way!
• Oren Etzioni, speaking about the commercial experience of NETBOT, Inc:
“We made our agents dumber and dumber and dumber…until finally they
made money.”
25/41
Accessible vs. Inaccessible
• An accessible environment is one in which
the agent can obtain complete, accurate, up-
to-date information about the environment’s
state
• Most moderately complex environments
(including, for example, the everyday physical
world and the Internet) are inaccessible
• The more accessible an environment is, the
simpler it is to build agents to operate in it
26/41
Deterministic vs. Non-deterministic
• A deterministic environment is one in which
any action has a single guaranteed effect —
there is no uncertainty about the state that
will result from performing an action
• The physical world can to all intents and
purposes be regarded as non-deterministic
• Non-deterministic environments present
greater problems for the agent designer
27/41
Episodic vs. Non-episodic
• In an episodic environment, the performance of an
agent is dependent on a number of discrete
episodes, with no link between the performance of
an agent in different scenarios
• Episodic environments are simpler from the agent
developer’s perspective because the agent can
decide what action to perform based only on the
current episode — it need not reason about the
interactions between this and future episodes
28/41
Static vs. Dynamic
• A static environment is one that can be assumed to
remain unchanged except by the performance of
actions by the agent
• A dynamic environment is one that has other
processes operating on it, and which hence changes
in ways beyond the agent’s control
• Other processes can interfere with the agent’s actions
(as in concurrent systems theory)
• The physical world is a highly dynamic environment
29/41
Discrete vs. Continuous
• An environment is discrete if there are a fixed, finite
number of actions and percepts in it
• Russell and Norvig give a chess game as an example
of a discrete environment, and taxi driving as an
example of a continuous one
• Continuous environments have a certain level of
mismatch with computer systems
• Discrete environments could in principle be handled by
a kind of “lookup table”
30/41
Related Research Areas
Distributed Biological analogies
Systems
Computational linguistics Cognitive
agents AI & Science
Database & Machine
Knowledge base Learning
Technology
31/41
Related Area
Database and Knowledge-base
• Intelligent agents need to be able to
represent and reason about a number of
things, including:
– metadata about documents and collections of
documents
– linguistic knowledge (e.g., thesauri, proper name
recognition, etc)
– domain-specific data, information and knowledge
– models of other agents (human or artificial): their
capabilities, performance, beliefs, desires,
intentions, plans, etc.
– tasks, task structures, plans, etc.
32/41
Related Area
Distributed Computing
• Concurrency
– analyzing and specifying protocols, e.g., deadlock
and livelock prevention, fairness
– achieving and preserving consistency
• Performance evaluation
– visualization
– debugging
• Exploit the advantages of parallelism
– multi-threaded implementation
33/41
Related Area
Cognitive Science
• BDI agent model
– Beliefs: Recognitions for the self and
environments
– Desires: Goal state
– Intentions: Actions and plans to achieve the goal
• The advantage of the BDI model
– Designing static agent models
– Designing the communication between agent
– Inferring the internal state of the agent
– Expecting the other agent’s actions
34/41
Related Area
Computational Linguistics
• We draw on research in (computational) linguistics
for an underlying communication model
• Speech act theory is a high level framework
developed by philosophers and linguists to
account for human communication
– Speakers do more than just utter sentences which
have a logical (truth-theoretic) meaning
• “The cat is on the mat”
– Speakers perform speech acts: requests, suggestions,
promises, warnings, threats, criticisms, praise, etc.
• “I hereby promise to buy you lunch”
– Every utterance is a speech act
Tell
start asked Deny told
Ask-one
Untell
35/41
Sorry
Related Area
Econometric Models
• Economics studies how to model predict and control the
aggregate behavior of large collections of independent
agents.
• Conceptual tools include:
– game theory
– general equilibrium market mechanisms
– protocols for voting and auctions
• An objective is to design good artificial markets and protocols
to result in the desired behavior of our agents
• Michigan’s Digital Library project uses a market-based
approach to control its agents.
36/41
Related Area
Biological Analogies
• One way that agents could adapt is via an
evolutionary process.
• Individual agents use their own strategies for
behavior with “death” as the punishment for poor
performance and “reproduction” the reward for
good.
• Artificial life techniques include:
– genetic programming
– natural selection
– sexual reproduction
37/41
Related Area
Machine Learning
• Techniques developed for machine learning
are being applied to software agents to allow
them to adapt to users and other agents.
• Popular techniques
– reasoning with uncertainty
– decision tree induction
– neural networks
– reinforcement learning
– memory-based reasoning
– genetic algorithms
38/41
Applications of Intelligent Agents (1)
• E-mail Agents
– Beyond Mail, Lotus Notes, Maxims
• Scheduling Agents
– ContactFinder
• Desktop Agents
– Office 2000 Help, Open Sesame
• Web-Browsing Assistants
– WebWatcher, Letizia
• Information Filtering Agents
– Amalthaea, Jester, InfoFinders, Remembrance agent, PHOAKS,
SiteSeer
39/41
Applications of Intelligent Agents (2)
• News-service Agents
– NewsHound, GroupLens, FireFly, Fab, ReferralWeb, NewT
• Comparison Shopping Agents
– Mysimon, BargainFinder, Bazzar, Shopbor, Fido
• Brokering Agents
– PersonalLogic, Barnes, Kasbah, Jango, Yenta
• Auction Agents
– AuctionBot, AuctionWeb
• Negotiation Agents
– DataDetector, T@T
40/41
Summary
• An agent is a computer system that is
situated in some environment, and that is
capable of autonomous action in this
environment in order to meet its design
objectives
– Key characteristics: autonomy, social ability,
reactivity, pro-activeness, situateness,
persistent, cooperation, adaptation
– Other characteristics: mobility, veracity,
benevolence, rationality
41/41