Ethical Concerns in Affective Computing and
Emotionally Intelligent Agents
Introduction
As emotionally intelligent virtual agents gain ground in real-world applications such as education,
therapy, and workplace assistance, it is imperative to consider the ethical implications of deploying
such systems. This lecture outlines key ethical concerns related to privacy, emotional manipulation,
bias, dependence, deception, social isolation, and liability.
1. Privacy Concerns
Emotionally intelligent agents collect personal data to personalize responses and build context.
Examples:
Mental health sessions
Classroom interactions
Workplace well-being tools
Risks:
Unauthorized data collection
Commercial exploitation without user consent
Solution:
Implement strict data usage regulations
Ensure user consent for data collection and storage
2. Emotional Manipulation
Agents can influence user emotions without explicit consent.
Can nudge users to adopt new behaviors or shift belief systems.
Raises questions about the ethical limits of emotional persuasion.
Applicable across all affective computing domains, not just virtual agents.
3. Bias and Discrimination
AI models trained on non-diverse datasets may not generalize well.
Risks of discriminatory outcomes based on:
Page 1 of 5
Ethnicity
Language
Region or culture
Example:
Dataset from Western users may fail on Indian populations.
Language-specific data may not transfer across regions.
Solution:
Diversify datasets
Evaluate models on varied demographics
4. Addiction and Dependence
Users may develop emotional dependence on virtual agents.
Example: Turning to an agent for solace instead of social support
Risks:
Reduced real-world human interaction
Social withdrawal and mental health deterioration
Particularly concerning in vulnerable populations (e.g., users with depression)
5. Legal Liability
If an agent’s interaction leads to harm, who is responsible?
Example: Misguidance in a healthcare setting
Raises the need for legal frameworks defining:
Developer/manufacturer liability
Accountability for agent behavior
6. Deception and Anthropomorphism
Users may wrongly perceive agents as sentient or emotionally aware.
Emotional mimicry ≠ genuine feeling.
Anthropomorphism can create false emotional bonds.
Example:
Public grief over a bomb-defusing robot’s destruction in war despite it being a non-living
machine
Recommendation:
Clearly inform users that the agent lacks real emotions
Page 2 of 5
7. Social Isolation
Emotional reliance on agents can lead to reduced human interaction.
Example:
Patients with PTSD preferring agent interactions over human therapy
Consequences:
Compromised recovery
Reinforced social withdrawal
Case Scenarios and Ethical Implications
Scenario 1: Voice-Based Emotional Support
Agent collects voice patterns to offer personalized support.
Concern: Privacy
Consent may be missing
Risk of commercial data use without approval
Scenario 2: VR Therapy for PTSD
Virtual agent helps users relive trauma in controlled setting
Concerns:
Intensification of symptoms
Poor emotional control by the agent
Risk of addiction
Scenario 3: Customer Service Virtual Agent
Personalized responses and product recommendations
Concerns:
Manipulative marketing practices
Violating user trust
Ethical advertising boundaries
Expert Interview: Dr. Aniket Bera (Purdue University)
Inspiration
Addressing lack of emotion modeling in traditional computing
Designing systems that incorporate human emotional behavior
Page 3 of 5
Research Focus
Emotion-aware robots, virtual agents, and therapists
Modeling non-verbal communication: tone, facial expressions, body posture
Notable Projects
1. Virtual Therapist:
Bridges gaps between real therapy sessions
Provides support in rural areas with fewer professionals
2. Metaverse Interaction Agents:
Emotionally intelligent avatars for immersive experiences
3. Robotic Collaboration with Military:
Emotion-aware joint task execution
4. Police Training Simulators:
Virtual characters with cultural/emotional diversity for improved empathy
Research Example: "Emotions Don’t Lie" (Deep Fake Detection)
Problem
Deep fakes often have emotion mismatches between audio and video
Real human expressions are naturally synchronized across modalities
Solution
Build a fusion-based classifier comparing audio and visual emotion cues
Detect discrepancies indicating fakeness
Method
Use Facebook-released datasets (real vs. deep fake videos)
Extract emotion cues using:
Speech features
Facial expressions
Apply multi-modal fusion techniques
Results
Outperformed baseline deep fake detectors by 30–40%
Highlighted emotion as a robust feature for fake detection
Page 4 of 5
Challenges
Cultural variation in emotional expression
Sarcasm and irony detection
Need for better performance metrics beyond accuracy
Advice to Young Researchers
Start with real-world problems, not abstract models
Add human behavior as a second layer after identifying the need
Consider interdisciplinary tools: NLP, psychology, robotics, graphics
Focus on individual and cultural variability in both users and agents
Conclusion
Emotionally intelligent systems raise several ethical concerns, including privacy, manipulation,
addiction, and liability. Developers must actively mitigate risks through transparent design, consent-
driven data practices, and culturally adaptive models. At the same time, innovative use cases—from
therapy to deep fake detection—demonstrate the vast potential of affective computing when
approached ethically and thoughtfully.
Page 5 of 5