AI in Marketing

Prof. Zillur Rahman

Department of Management Studies
Indian Institute of Technology, Roorkee
Week-12
Lec 56-Navigating Ethical Challenges in AI-I

Welcome to this NPTEL online certification course on artificial intelligence in marketing

and this is module 56. So, as you can see from this slide, these four modules are
dedicated to understanding the ethical challenges in AI. Now, that ethical challenges in
AI are the biggest challenges that the society and the world faces at this point in time.
So, this is part 1 and these are the things that we will cover in this module. So, we will
start with understanding what is the role of ethics in the use of different artificial
intelligence tools. Then we will identify different ethical concerns in using AI and the
third is what are the concerns about privacy and the potential for bias in AI.

So, now let us look at what are the ethical issues of AI. So, companies are leveraging
data and artificial intelligence to create scalable solutions and they are also scaling their
reputational, regulatory and legal risks. For instance, Los Angeles is suing IBM for
allegedly misappropriating data it collected with its Obiquitious Weather App. Optum is
being investigated by regulators for creating an algorithm that allegedly recommended
that doctors and nurses pay more attention to white patients than to sicker black patients.

Facebook infamously granted Cambridge Analytica, a political firm access to personal

data of more than 50 million users. Now, let us look at the case of Goldman Sachs and
Apple credit cards. Entrepreneur David Heinemer Hansen says in a tweet that his credit
limit for Apple credit cards which are issued by Goldman Sachs was 20 times that of his
wife even though she has the higher credit scores. So, this is the tweet. So, he says that
the Apple card is such a success program.

My wife and I filed joint tax returns, live in a community property state and have been
married for a long time. Yet Apple's black box algorithm thinks I deserve 20 times the
credit limit she does. No appeals work. The tweets immediately gained traction and
received a comment from Apple co-founder Steve Vosnak who alleged that Apple card
gave him 10 times the credit limit that his wife got. The New York Department of
Financial Services, a Wall Street regulator has opened an investigation.

Hansen wrote further, it does not matter what the intent of individual Apple reps are, it
matters what the algorithms they have placed their complete faith in does and what it
does is discriminate. Now, let us look at the trust and fairness in AI systems. So, there
are many contexts in which one can use the word trust. For example, one might ask if
they trust their spouses, if they trust a company such as Facebook or Google with their
data, if they trust the government with their health data or if they trust their housekeepers
or babysitters. When a human trust another human, the trustor knows they are making
themselves vulnerable to another person's actions.

When a person trust an AI agent, it is unclear if the machine is making its own decision
or following some predefined scripted behavior. So, here we know when we are we are
trusting another human, we are vulnerable to that person’s actions, but when we are
trusting an AI agent, it becomes difficult to understand. It is unclear if the machine is
making its own decision or following some predefined scripted behavior. Furthermore,
people have experiences with other people and have certain expectations about their
behavior, norms and values. We may trust a pilot to safely land an aircraft.

When a user interacts with a robo or an AI system, these experiences may become
useless or even misleading. So, we cannot say that AI will also behave like another
human. We cannot be certain that a system will act in our best interest. Here in this
section, we will discuss trust and fairness in the context of human trusting machines such
as AI and robots. Now, these are the three ethical concerns in using AI.

First is concerns about privacy. The second is concerns about the potential for bias and
the third is concerns for appropriateness. Now, let us start with the first one that is
concerns about privacy. Privacy has always meant minimizing the collection of data to
what is fit for purpose and then limiting the access and retention of that data only to what
is required. Artificial intelligence promising exciting opportunities for new services and
product has always been hungry for data from any source in order to derive insights.

So, they it use data to derive insight. These two worlds are rapidly colliding. As they
do, they have the potential to address both the promise and the risk of data privacy and
artificial intelligence. Now, let us look at the privacy concerns with AI voice assistants.
Privacy is another major concern for voice assistant users.

By their very nature, these devices must be listening at all times so that they can respond
to the users. Amazon, Apple, Google and Microsoft all insist that their devices are not
recording unless users speak the command to wake the assistant. But there has been at
least one case where a malfunctioning device was recording at all times and sending
those recording back to the Google servers. Even if the companies developing these
voice assistants are being careful and scrupulous, there is a potential for data to be stolen,
leaked or used to incriminate people. A murder investigation in Arkansas led to the
authorities issuing a warrant to Amazon to retrieve the suspects' Alexa history.
 Children privacy when using these devices is also a concern. Metal recently shelved
plans to build a voice assistant aimed at children code named Aristotle. After complaints
were raised that the device would be invasive of children privacy. To enable a robust
layer of governance, your organization need to take several steps. The first step is to
consider what governance your organization needs to move past regulatory-driven
privacy protection into data and data use ethics.

Data, data use and AI ethics involve more than privacy. Some organizations are
adapting principles around explainability, societal benefits and fairness among other
principles. Identify which principles are relevant and more importantly what these
principles mean to your organization. Get wide executive agreement on these principles
and translate them into concrete standards and procedures for each practice within your
organization to enact trust-driven approaches. The second is appreciate the difference
between AI and software.

Find the best connection points with the AI development teams given their
experimentation-driven approach. There should be oversight as to where and when.
Within the development process, data is accessed, cleaned, manipulated, augmented and
protected. Doing so can enable more thorough analysis of broader ethical consideration,
privacy and beyond. The third is build off what you have.

No one wants burdensome governance and compliance, but expanding governance

practices unnecessarily can lead to poor compliance down the line. Building of existing
processes can help reduce friction and change management required to gain effective
oversight. Most use of personal data already requires a privacy impact assessment or a
GDPR. Require data protection, impact assessment in higher risk data scenarios.
Augment these requirements with additional questions relating to AI and mandate their
use across all AI development.

Place particular focus on questions to identify and assess the likelihood and significance
of benefits, risks and mitigation controls. Recognize this action to be a benefit-risk
decision-making tool and bring a broad cross-section of internal stakeholders to the
decision input. The fourth is align on language. And don't go it alone, anticipate any
differences in how terminology is used between privacy and AI teams. Educate and
collaborate to develop a shared understanding of these terms.

Collaborative governance with a multilayer approach can help reduce friction and
provide the broad range of perspective necessary for your organization to develop robust
and lasting mechanisms for ethical AI. The fifth is use AI techniques to address privacy
issues. When AI introduces a number of complications in dealing with privacy, it also
provides a few innovative ways of solving privacy issues. New techniques like
homomorphic encryption and differential privacy enable data sharing through encryption
or noise induction. Federated learning allow insights to be generated locally and
aggregated without revealing private data.

As an active space for research, AI will undoubtedly yield new applications that may
enable privacy protection. Now, let us look at the concerns about the potential for bias.
There is often an assumption that technology is neutral. But the reality is far from it.
Green learning algorithms are created by people who all have biases.

They are never fully objective, rather they reflect the worldview of those who build
them and the data they are fed. The authority that administers A-level college entrance
exam in the UK recently found itself mired in a scandal. Unable to hold life exam
because of COVID-19, it designed and employed an algorithm that based scores partly on
the historical performance of the schools students attended. The outcry was immediate as
students who were already disadvantaged found themselves further penalized by
artificially deflated scores, their efforts disregarded and their future thrown into disarray.
This is far from an isolated incident.

Even the world's most sophisticated technology companies have faced similar problems.
In 2018, Amazons recruiting algorithm was flagged for penalizing applications that
contained the word woman. Certainly, this is unfair. Just as important, these biases
undermine efficiency and productivity. If we see a value in putting the best student in the
best schools, the inherent bias in algorithm undermine that purpose.

Similarly, a bias recruiting algorithm undermines a firm's ability to attract the best talent
and a biased credit rating algorithm undermines the ability to make smart credit
decisions. Just as the cost of bias are substantial, the benefits of eliminating biases can be
just as significant. In fact, one econometric study at Stanford University found that at
least 25% of growth in US GDP between 1960 and 2010 can be attributed to greater
gender and racial balance in the workplace and that the figure could be as high as 40%.
Now, we will look at the case of COMPAS algorithm and its bias against black people.
COMPAS which stands for Correctional Offender Management Profiling for Alternative
Sanctions is an artificial intelligence algorithm created by Northpoint and used in the US
to predict which criminals are more likely to reoffend in the future.

Based on these forecasts, judges make decisions about the future of these criminals
ranging from their jail sentence to the bail amount for release. However, ProPublica, a
Pulitzer Prize winning non-profit news organization found that COMPAS was biased.
Black criminals were judged to be much more likely to recommit crimes in the future
than they committed. On the other hand, white criminals were judged less risky than they
were by COMPAS. Even for violent crimes, black criminals were misclassified as more
dangerous about double the time as compared to the white criminals.

This discovery in COMPAS proved that it had somehow learned the inherent bias that is
frequent in human, which is black people commit many more crimes than white people
on average and more likely to commit crimes in the future as well. Now, what are the
sources of AI biases? So biased human judgment can affect AI systems in two different
ways. The first is bias in the data that systems learn from. You can see this play out for
yourself. Do a Google image search for professional haircut and another for
unprofessional haircut.

Professional haircut turns up results that are exclusively white men while unprofessional
haircut has much more gender and racial diversity. This issue was originally surfaced by
Twitter users back in 2016. The second is bias occurs in the way algorithms are
designed. Machine learning algorithms generally rely on correlation rather than causal
relationships. For example, it is relatively easy to design an algorithm that will find a
relationship between temperature readings and ice cream consumption, showing that as
the thermometer readings go up, people eat more ice cream.

Of course, high temperature readings themselves do not actually cause increased ice
cream sales. Putting a thermometer next to a heater will not elicit a sudden rush for
something cold. Human get that, algorithms often do not. The problem is that there is an
intermediate factor called confounding factor that affects both thermometer readings and
ice cream sales, namely hot weather. That algorithms are not typically designed to
consider.

Now there are six potential ways for organizations to minimize AI bias. So the first is
be aware of context in which AI can help correct for biases and those in which there is
high risk for AI to increase bias. The second way is establish processes and practices to
test for and mitigate bias in AI systems. The third is to engage in fact-based conversation
about potential biases in human decisions. The fourth is fully explore how human and
machine can best work together.

The fifth is invest more in bias research, make more data available for research while
respecting privacy and adopt a multidisciplinary approach. And the sixth is to invest
more in diversifying the AI field itself. So let us look at the first one, be aware of the
context in which AI can help correct for biases as well as where there is a high risk that
AI could increase bias. When deploying AI, it is important to anticipate domains
potentially prone to unfair biases such as those with previous examples of biased systems
or with skewed data. Organization will need to stay up to date to see how and where AI
can improve fairness and where AI systems have struggled.

The second is to establish processes and practices to test for and mitigate bias in AI
systems. Tackling unfair bias will require drawing on a portfolio of tools and procedures.
The technical tools described above can highlight potential sources of bias and reveal the
trait in the data that most heavily influence the output. Operational strategies can include
improving data collection through more cognizant sampling and using internal red teams
or third parties to audit data and models. Finally, transparency about processes and
metrics can help observers understand the steps taken to promote fairness and any
associated trade-offs.

The third is to engage in fact-based conversation about potential biases in human

decisions. As AI reveals more about human decision making, leaders can consider
whether the proxies used in the past are adequate and how AI can help by surfacing long-
standing biases that may have gone unnoticed. When models trained on recent human
decisions or behavior show bias, organizations should consider how human-driven
processes might be improved in the future. The fourth is fully explore how humans and
machine can work best together. This includes considering situations and use cases when
automated decision making is acceptable and indeed ready for the real world versus when
human should always be involved.

Some promising systems use a combination of machines and humans to reduce bias.
Techniques in this vein include human in the loop decision making, where algorithms
provide recommendations or options which humans double check or choose from. In
such systems, transparency about the algorithms' confidence in the recommendation can
help humans understand how much weight to give it. The fifth is to invest more in biased
research, make more data available for research while respecting privacy and adopt a
multidisciplinary approach. While significant progress has been made in recent years in
technical and multidisciplinary research, more investment in these efforts will be needed.

Business leaders can also help support progress by making more data available to
researchers and practitioners across organizations working on these issues while being
sensitive to privacy concerns and potential risks. More progress will require
interdisciplinary engagement including ethicists, social scientists and experts who best
understand the norms of each application area in the process. A key part of the
multidisciplinary approach will be to continually consider and evaluate the role of AI
decision making as the field progresses and practical experience in real applications
grow. The sixth is to invest more in diversifying the AI field itself. Many have pointed
to the fact that the AI field itself does not encompass society’s diversity including on
gender, race, geography, class and physical disabilities.

A more diverse AI community will be better equipped to anticipate, spot and review
issues of unfair bias and better able to engage communities likely affected by these
biases. This will require investments on multiple fronts, but especially in AI education
and access to tools and opportunities. So, to conclude in this module we have, and we
have understood what is the role of ethics in the use of different artificial intelligence
tools. Further we have identified different ethical concerns in using AI. We have
discussed what are the concerns about privacy and the potential for bias and
appropriateness of use.

And lastly, we understood how to make AI transparent for all the stakeholders and the
challenges that come with it. And these are the five sources from which the material for
this module was taken. Thank you.
 AI in Marketing
Prof. Zillur Rahman
Department of Management Studies
Indian Institute of Technology, Roorkee
Week-12
Lec 57-Navigating Ethical Challenges in AI -II

Welcome to this NPTEL online certification course on Artificial Intelligence and Marketing.
And now we will talk about module 57. So, as you can see from this slide, we are continuing
to discuss about the ethical challenges in AI and this is part 2 of that. And these are the
things that we will cover in this module. So, we will start with what are the concerns about
appropriateness of use. Understand how to make AI transparent for all the stakeholders
and the challenges that come with it.

Building accountability in AI, keeping in mind all the ethical concerns. Now, let us start
with the first one about the concerns about appropriateness. While most technical
literature is devoted to improving the performance of machine learning algorithms, there is
a little guidance available regarding how an algorithm should be assessed and evaluated
prior to deployment in a real world operational context. To meet this need, we include here
an additional criteria termed appropriateness, which addresses how well the machine
learning algorithm is matched to the specific context and the specific population.

So this appropriateness is about specific context and specific population. So, as the
population change, the context will change and as the context change, the population will
change and therefore the appropriateness will change. Now, we will look at the case of
Facebook's new algorithm. One ethical issue to consider is whether it is appropriate for
firms to use different appeals for consumers based on their level of resources. For instance,
Facebook is now developing algorithm that would allow their data scientists to estimate
their users' socioeconomic status.

Based on factors such as their education, travel history and number of devices owned. This
would allow them to target them with different advertisements as a function of socio-
economic status, making modern AI transparent and the challenges that comes with it.
Technical transparency. Revealing the source code, inputs and outputs of the algorithm can
build trust in many situations. But most algorithms in the world today are created and
managed by for-profit companies.

And many businesses regard their algorithms as highly valuable forms of intellectual
property that must remain in a black box. Some lawmakers have proposed a compromise
suggesting that the source code be revealed to regulators or auditors in the event of a
serious problem and this educator will assure consumers that the process is fair. One
problem with technical transparency is that it makes algorithms vulnerable to gaming. If an
instructor releases the complete source code of an algorithm grading a student's essay, it
becomes easy for students to exploit loopholes in the code. For example, the algorithm sees
evidence that the student have done research by looking for phrases such as according to
publicized research.

A student might then deliberately use this language at the start of every paragraph in her
essay. But the biggest problem is that modern AI is making source code transparent or not
less relevant compared with other factors in algorithmic functioning. Specifically, machine
learning algorithms and deep learning algorithms in particular are usually built on just a
few hundred lines of code. The algorithm's logic is mostly learnt from training data and is
rarely reflected in its source code. Which is to say, some of the today's best performing
algorithms are often the most opaque.

High transparency must involve getting our heads around reams and reams of data and
then still only being able to guess at what lessons the algorithm has learnt from it. Now, let
us look at the benefits of transparent AI. The first is adopting responsible AI practices help
establish trust with your customers and the general public. Consumers are increasingly
more aware of the potential harms that AI systems can cause. They demand more
transparency into the inner working of such technologies that may have catastrophic effect
on individual's life like denying them a loan, rejecting a job application or quoting a higher
health insurance price.

AI transparency enables predictable and consistent regulation. As AI tools are deployed

across many critical domains ranging from banking to employment to criminal justice
systems, regulators are paying particular attention to the potential ill effects of AI on
fundamental rights and taking actions. AI transparency can streamline legal compliance
efforts and preemptively shield your organization from possible legal actions in the future.
The next benefit is to assess to protected class data can help ML practitioners understand
where to root out biases. While perhaps counterintuitive at the surface giving ML
practitioners access to protected class data can help mitigate model biases.

By understanding which segment a model exhibit bias against, ML practitioners can take
proactive steps to reduce or eliminate those biases by employing certain techniques. For
example, in the pre-processing stage techniques such as counterfactual fairness and path
specific counterfactual methods can be used to remedy inaccuracies in the data and to
ensure that the sensitive attributes do not change the output. How to build accountability
in AI? The first is to understand the entire AI life cycle. Too often oversight questions are
asked about an AI system after it is built and already deployed but that is not enough.
Assessments of an AI or machine learning system should occur at every point in its life
cycle.

This will help identify system wide issues that can be missed during narrowly defined
point in time assessments. Building on work done by the Organization of Economic
Cooperation and Development that is OECD and others we have noted that the important
stages of AI systems life cycle include. First is the design, articulating the system's
objectives and goals including any underlying assumptions and general performance
requirements. The second is development, defining technical requirements, collecting and
processing data, building the model and validating the system. The third is deployment,
piloting, checking compatibility with other systems, ensuring regulatory compliances and
evaluating user experience.

The fourth is monitoring, continuously assessing the system's output and impact both
intended and unintended, refining the model and making decisions to expand or retire the
system. So, the four dimensions of AI accountability as organizations, leaders and third
party assessors focus on accountability over the entire life cycle of AI system. There are
four dimensions to consider. First is governance, second is data, the third is performance
and fourth is monitoring. Within each area there are many important actions to take and
things to look for.

So, let us start with the first governance. So, the one thing that is important here is to
assess governance structure. A healthy ecosystem for managing AI must include
governance processes and structures. Appropriate governance of AI can help manage risk,
demonstrate ethical values and ensure compliance. Accountability for AI means looking for
solid evidence of governance at the organization level including clear goals and objectives
for AI system, well-defined roles and responsibilities, lines of authority and a
multidisciplinary workforce capable of managing AI systems.

A broader set of stakeholders and risk management processes. Additionally, it is vital to

look for system level governance elements such as documented technical specifications of
the AI system, governance and stakeholder access to system design and operation
information. The second is data. Understand the data. Most of us know by now that data is
the lifeblood of many AI and machine learning systems.

But the same data that gives AI system their power can also be a vulnerability. It is
important to have documentation of how data is being used in two different stages of the
system. One is when it is being used to build the underlying model and while AI system is in
actual operation. Good AI oversight includes having documentation of the sources and
origin of data used to develop the AI models. Technical issues around variable selection and
use of altered data also need attention.

The reliability and representativeness. of the data needs to be examined including the
potential for bias, inequity or other societal concerns. Accountability also includes
evaluating an AI system's data security and privacy. The third is performance. Define
performance goals and metrics.

After an AI system has been developed and deployed, it is important not to lose sight of the
question why did we build this system in the first place and how do we know it is working.
Answering these important questions requires robust documentation of all AI systems
stated purpose along with definition of performance metrics and the methods used to
assess that performance. And those evaluating these systems must be able to ensure an AI
application meets its intended goals. It is crucial that these performance assessments take
place at the broad system level but also focus on the individual components that support
and interact with the overall system.

The fourth is monitoring. Review monitoring plans. AI should not be considered a set it and
forget its system. It is true that many of AI benefits stems from its automation of certain
tasks often at scale and speed beyond human ability. But at the same time, continuous
performance monitoring by people is essential. This includes establishing a range of model
drift that is acceptable, sustained monitoring to ensure that the system produces the
expected results.

Long term monitoring must also include assessments of whether the operating
environment has changed and to what extent conditions support scaling up or expanding
the system to other operational settings. Other important question to ask are whether the
AI system is still needed to achieve the intended goals and what metrics are needed to
determine when to retire a given system. Now how to build accountability in AI? So there
are three features that provide expanded ways to measure or assess performance in order
to improve accountability. And there are first the audit, the second the ledger and the third
is the reasoning. So what is this audit? When you have accountability in AI, you are able to
show the provenance of your data.

In an audit, auditor strays the trail of data back to its sources. Where did it came from?
How was it aggregated? How was it trained? Who was responsible for it? Have we done a
meaningful review of first order data use as well as all secondary and subsequent uses. As
all of these applications clearly defined and are all of these applications clearly defined and
communicated. Organizations must should follow the same process for auditing algorithms,
models and ensembles. As data is aggregated into more complex forms rendered in such
ways as neural networks, some people use its complexity as an excuse not to explain its
origin.

But that is not acceptable. And audit ensures we understand each link in the data chain.
The second is ledger. In technology, we often focus on technical debt or the time, money
and resources your business will have to expend in the future to make up for the limited
solution or shortcut you opted for today. We should do far better to talk less about
technical debt and talk more about technical margins and not about technical debt. What
are you doing now to be ahead of your competition in the future? What steps are you taking
to create a margin instead of always operating in the red? To develop greater
accountability, create a ledger to measure your technical debt versus technical margin,
treating data as a strategic asset.
 So, this is to be measured. Getting started with a simple ledger that collects your accounts.
For each, note a starting balance and record transactions, either debit or credit and a
closing balance. This can include risk, data health, decisioning speed, readiness or
otherwise. You have already tracked the data, processes, algorithms, personas, decisions
and consequences in the audit. The ledger helps you understand the interaction among
these systems regarding what is happening and what the outcomes are in the real time.

The third is reasoning. A third feature for enhancing accountability is to provide control.
Standards and processes for documenting the logic for how conclusions are drawn, what
we can call reasoned reasons or simply reasoning. In a responsible AI framework, you need
strong checks, counterbalances and consequences and rewards. With the complexity and
velocity of AI, we cannot or should not understand reasoning purely by outcome. It is best
to evaluate our reasoning construct in AI by expected value and how these are deduced,
considered and acted upon.

Because of this, I do not believe we are likely to be successful by striving for balance. A
ballet dancer never balances herself. She is always counterbalancing herself and the same is
true for AI. We are constantly counterbalancing against risk and other environmental
effects. How we reason and what we do to understand that reasoning is what will make us
more or less accountable.

Now let us move on to implementing AI. Just as when new talent joins a team, AI must be
set up to succeed rather than to fail. A smart employee train new hires by giving them
simple tasks that build hands-on experience in a non-critical context and assign them
mentors to offer help and advice. This allows the newcomers to learn while other focuses
on higher value tasks. As they gain experience and demonstrate that they can do the job,
their mentors increasingly rely on them as sounding boards and entrust them with more
substantive decisions. Over time, an apprentice becomes a partner, contributing skills and
insights.

This approach can work for artificial intelligence as well. So in the following slides, we later
on we can draw on our own and other research and consulting on AI and information
systems implementation along with organizational studies of innovation and work
practices to present a four-phase approach to implementing AI. So this four-phased
approach will come later. So these are the four phases. The first is assistant, the monitor,
the coach and the teammate.

Now let us start with the first one that is the assistant. The first phase is very similar to
onboarding a new human assistant. After the new hire has been taught a few fundamental
rules and procedures, he is capable of assuming basic but time-consuming tasks. For
example, filing online forms or summarizing documents, enabling the trainer to focus on
other more strategic aspects of the job. Positioning the technology in this slide can help
assuage the perennial AI is robbing jobs fears and also help human employees feel more in
control of the technology.

The second phase is the monitor, setting up the system to provide real-time feedback is the
second phase of implementation. As the HBR article puts it, AI is very helpful during high
volume decision making when humans may be tired. Of course, the technology is not
always accurate. Some things we have written about before here at the apex of innovation.
The HBR piece elaborates, often its suggestions do not take into account some reliable
private information to which the human decision makers have access.

So the AI might steer an employee off course rather than simply correct for behavioral
biases. The authors of that article stresses AI should be like a dialogue in which any
override is explained by an employee so that the system learns from human intelligence
and experience. Not only does this improve the usefulness of technology, it also gives
employees autonomy. The phase 3 is the coach. The challenges with providing performance
feedback have been well documented.

But this is a problem AI could address. The third phase involves deploying AI systems to
identify key behaviors that impact performance and provide personalized feedback to help
employees change their behavior over time. Of course, such an implementation could easily
play into people's man versus machine fears. And the HBR piece stresses that employees
must be involved in the design and roll out to lessen the likelihood of such issues. The
fourth phase is the team mate. As the authors put it, in the final phase of AI implementation
journey, which to our knowledge no organization has yet adopted, companies should
develop a coupled network of humans and machines in which both contribute expertise.

They acknowledge that a number of challenges must be overcome in order for this phase
to truly be a possibility but stress that it is certainly something we will see as the
technology matures. So, to conclude in this module, we have understood what are the
concerns about appropriateness of use. Further, we have discussed how to make AI
transparent for all the stakeholders and the challenges that comes with it. Lastly, we have
understood building accountability in AI keeping in mind all the ethical concerns. And these
are the 5 sources from which the material for this module was taken. Thank you.
 AI in Marketing
Prof. Zillur Rahman
Department of Management Studies
Indian Institute of Technology, Roorkee
Week-12
Lec 58-Navigating Ethical Challenges in AI-III

Welcome to this NPTEL online certification course on Artificial Intelligence and

Marketing. And now we will talk about module 58. So, as you can see that these four
modules are dedicated for understanding the ethical challenges in AI. So, this is the third
part of navigating ethical challenges in AI. And these are the things that we will talk about
in this module. So, we will start with understanding the implementing AI and managing
AI's unintended consequences.

Then we will move on to understanding privacy and surveillance in the digital age, ethical
concerns in AI surveillance and manipulating behavior through information use and then
we will understand artificial moral agents. So, now let us start with managing AI's
unintended consequences. So, companies are increasingly using reinforcement learning
agents that is a type of AI that rapidly improves through trial and error as it single-mindedly
pursue its goals often with unintended and even dangerous consequences. So, this is a cause
of concern.

One such example is the use of RL agents that is reinforced learning agents on self-driving
cars by a company such as Tesla. The weaponization of polarizing content on social media
platform is an extensive example of what can happen when RL agents are not properly
constrained. To prevent their RL agents from causing harm, leaders should abide by five
rules as they integrate this AI into their strategy execution. So, these are the five important
things. So, following are the five rules leaders should abide by as they integrate RL agents
into their strategy execution.

The first is assume that your RL agent will affect behavior in unforeseen ways. The second
is systematically evaluate deviations from the expected. The third is interview users,
customers and others about their responses to the RL agents outputs. The fourth is if an
agent is promoting undesirable behavior, modify its policies and the fifth is if undesirable
behavior persist, change the reward function. So, now let us look at each one of them in
some more detail.

So, let us start with the first one that is assume that your RL agent will affect behavior in
unforeseen ways. So, my company built an agent to speed the quality assurance of account
and transactions by flagging anomalies that is potential errors that the algorithm scored as
high risk and putting these first in the queue for evaluation by an analyst. By dramatically
reducing the total number of anomalies, the analyst needed to review the algorithm
substantially cut the overall review time as we had hoped it would. But we were surprised
to see suspiciously fast review times for even the most complex anomalies. This could have
taken the analyst more time not less.

The second is systematically evaluate deviations from the expected. To date, few
companies methodically assess how their RL agents are influencing people’s behavior.
Start by regularly asking your data scientist for data on behavior changes that may be
associated with the agent's activities. If you see a departure from what is expected, dig
deeper. In our case, the fact that the analyst were flying through the riskiest anomalies was
the red flag that the algorithm was causing unexpected knockout effect.

We knew we had a problem. The third is interview users, customers and others about their
response to the RL agent’s output. While those at the receiving end of an RL agents action
may be unaware that they are being influenced by an AI, you can still gauge their responses
to it. Because we are concerned about our accounting analysts to speedy reviews, we spoke
with them about their responses to the algorithms compelling anomalies for them to assess.
It turned out that they wrongly assumed that the agent was doing more of the quality
assurance on these anomalies than it was.

They were over relying on agents expertise and so paid less attention in their own
investigation of the anomaly. Incidentally, such over reliance on AI is one reason people
are crashing self-driving cars. They assume the AI is more capable than it is and hand over
too much control. That is a dangerous knock-on effect. The fourth is if an agent is
promoting undesirable behavior, modify its policies.

To optimize agents pursuit of their reward, most AI team constantly adjust the agent’s
policies, generally modifying state action pairs. For example, the time in the billing cycle
that is the state in which an agent will send a payment prompt that is the action. In our
accounting example, we made various policy changes including redefining the state to
include the time spent by analyst on each anomaly and adding actions that challenged an
analyst's conclusion if they were reached too quickly and evaluated selected anomalies to
a supervisor. These policy changes substantially cut the number of serious anomalies that
the analyst dismissed as false positives. The fifth is if undesirable behavior persist, change
the reward function.

Adjusting an agent’s state action pairs can often curb undesirable knock-on behaviors but
not always. The biggest stick available to leaders when other intervention fails is to change
the agent's goal. Generally, changing a reward function is not even considered because it
is presumed to be sacrosanct. But when the agent's pursuit of its goal is prompting harmful
behavior and adjusting the states or action available to the agent cannot fix the problem, it
is time to examine the reward itself. The next issue is that of privacy and surveillance.

So, what is privacy and surveillance in the digital age? Discussions on the privacy and
surveillance encompasses access to personal data with implications for individual's right
and data control. Traditional privacy aspects like the right to solitude and information
control extend to the digital realm. Surveillance includes state and corporate monitoring
exploiting the evolving technology landscape. And at the same time, we also have to
exploit the evolving technology landscape. So, data collection has shifted to digital
domains encompassing AI-driven analysis and trade.

Anonymization and privacy-preserving techniques exist but enforcement and legal

challenges persist. Dominant tech entities profit from data collection compromising
autonomy and fostering a surveillance capitalism model. As AI advances concerns about
algorithms understanding us better than ourselves emerge. Balancing technological
progress with privacy protection remains a challenge. The next thing that we will
understand is the manipulation of behavior.

The ethical issues of AI in surveillance go beyond the mere accumulation of data and
direction of attention. They include the use of information to manipulate behavior online
and offline in a way that undermines autonomous rational choices. To manipulate behavior
are ancient but they may gain a new quality when they use AI systems. Given users intense
interaction with data systems and the deep knowledge about individuals this provides they
are vulnerable to nudges, manipulation and deception. With sufficient prior data,
algorithms can be used to target individuals or small groups with just the kind of input that
is likely to influence these particular individuals.

A nudge changes the environment such that it influences behavior in a predictable way
that is positive for the individual but easy and cheap to avoid. This is a slippery slope from
here to parentalism and manipulation. Many advertisers, marketers and online sellers will
use any legal means at their disposal to maximize profits including extrapolation of
behavior biases, deception and addiction generation. Such manipulation is the business
model in much of the gambling and gambling industry, but it is spreading. For example, to
low-cost airlines in interfaith design on web pages or in games.

This manipulation uses what is called dark patterns. At this moment, gambling and the
sale of addictive substances are highly regulated but online manipulation and addiction are
not. Even though manipulation of online behavior is becoming a core business model of
the internet. Furthermore, social media is now the prime location for political propaganda.
This influence can be used to steer voting behavior as in the case of Facebook, camera,
Analytica scandal and if successful it may harm the autonomy of individuals.
 Improved AI faking technologies make that once was reliable evidence into unreliable
evidence. This has already happened to digital photos, sound recordings and videos. It will
soon be quite easy to create rather than alter deep fakes, text, photos and video material
with any desired content. Soon sophisticated real-time interaction with persons over text,
phone or video will be faked too. So, we cannot trust digital interactions while we are at
the same time increasingly dependent on such interactions.

One more specific issue is that machine learning technique in AI rely on training with vast
amounts of data. This means there will often be a trade-off between privacy and the rights
to data versus technical quality of the product. This influences the consequentialist
evaluation of privacy violating practices. The policy in this field has its ups and downs.
Civil liberties and the protection of individual rights are under intense pressure from
businesses lobbying, secret services and other state agencies that depend on surveillance.

Privacy protection has diminished massively compared to the pre-digital age when
communication was based on letters, analog telephone communication and personal
conversation and when surveillance operated under significant legal constraints. While the
EU General Data Protection Regulation, Regulation EU 2016/679 has strengthened privacy
protection. The US and China prefer growth with less regulation likely in the hope that this
provides a competitive advantage. It is clear that state and business actors have increased
their ability to invade privacy and manipulate people with the help of AI technology and
will continue to do so to further their interest unless leaned in by policy in the interest of
general society. Now let us look at the stakeholders in AI ethics.

Each of these actors play an important role in ensuring less bias and risk for AI
technologies. The first one is academics. Researchers and professors are responsible for
developing theory-based statistics, research and ideas that can support governments,
corporations and non-profit organizations. The second are the government. Agencies and
committees within the government can help facilitate AI ethics in a nation.

A good example of this is the preparing for the Future of AI report that was developed by
National Science and Technology Council in 2016 which outlines AI and its relationship
to public outreach, regulations, governance, economy and security. The third stakeholder
is intergovernmental entities. Entities like the United Nations and the World Bank are
responsible for raising awareness and drafting agreements for AI ethics globally. For
example UNESCO 193 member states adopted the first ever global agreement on the ethics
of AI in November 2021 to promote human rights and dignity. Now the fourth stakeholder
is non-profit organizations.
 Non-profit organizations like Black in AI and Cure in AI help diverse groups gain
representation within AI technology. The Future of Life Institute created 26 guidelines
that are now the Asilomar AI principles which outlines specific risks, challenges, outcomes
for AI technologies. The fifth stakeholder in all this is are the private companies.
Executives at Google, Mita and other tech companies as well as banking, consulting,
healthcare and any other industry that uses AI technology are responsible for creating
ethics teams and codes of conduct. This often creates a standard for companies to follow
suit.

Now let us look at the artificial moral agents. If one takes machine ethics to concern moral
agents in some substantial sense, then these agents can be called artificial moral agents
having rights and responsibilities. However, the discussion about artificial entities
challenge a number of common notions in ethics and it can be very useful to understand
these in abstraction from the human case. Several authors use artificial moral agent in a
less demanding sense borrowing from the use of agent in software engineering in which
case matters of responsibility and rights will not arise.

There are four type of machine agents. First is ethical impact agent, for example Robo-
Joki's. The second are implicit ethical agents like safe autopilot. The third are explicit
ethical agents for example using formal methods to estimate utility and the fourth is full
ethical agents who can make explicit ethical judgment and generally is competent to
reasonably justify them. An average adult human is the full ethical agent. Several ways to
achieve explicit or full ethical agents have been proposed via programming it in operational
morality via developing the ethics itself that is functional morality and finally full blown
morality with full intelligence and sentience.

Programmed agents are sometimes not considered full agents because they are competent
without comprehension just like the neurons in the brain competent without
comprehension. The notion of moral patient plays a role ethical agents have
responsibilities while ethical patients have rights because harm to them matters. It seems
clear that some entities are patient without being agents. For example, simple animals that
can feel pain but cannot make justified choices. On the other hand it is normally understood
that all agents will also be patients.

Usually being a person is supposed to be what makes an entity a responsible agent

someone who can have duties and be in the object of ethical concerns. Such personhood
is typically a deep notion associated with phenomenal consciousness intention and free
will. Artificial or machine ethics could be defined as designing machines that do things
that when done by human are indicative of the possession of ethical status in those humans
which he takes to be ethical productivity and ethical receptivity his expressions for moral
agents and patients. Now let us look at what is the responsibility of robots. There is a broad
consensus that accountability, liability and the rule of law are basic requirement that must
be upheld in the face of new technologies European group on ethics and science and new
technologies 2018.

But the issue in the case of robo is how this can be done and how responsibility can be
allocated. Is the robo act they will themselves be responsible liable or accountable for their
actions or should the distribution of risk perhaps take precedence over discussions of
responsibility. Traditional distribution of responsibility already occurs. A car maker is
responsible for the technical safety of the car.

A driver is responsible for driving. A mechanic is responsible for proper maintenance and
the public authorities are responsible for the technical conditions of the road etc. The effect
of decisions on actions based on AI are often the results of countless interaction among
many actors including designers. Developers, users, software, hardware with distributing
agency comes distributed responsibility. How this distribution might occur is not a
problem that is specific to AI, but it gains particularly but it gains particular urgency in this
context. In the classical control engineering distributed control is often achieved through a
control hierarchy plus control loops across these hierarchies.

How this distribution might occur is not a problem that is specific to AI. But it gains
particular urgency in this context. In classical control engineering distributed control is
often achieved through a control hierarchy plus control loops across these hierarchies.
Rights for robots some authors have indicated that it should be seriously considered
whether current robots must be allocated rights. This position seems to rely largely on
criticism of the opponent and on the empirical observation that robots and other non-
person are sometimes treated as having rights.

In this way a relationship turns has been forced. If we relate to robots as though they had
rights then we might be well advised not to search whether they really do have such rights.
This raises the question how far such anti-realism or quasi-realism can go and what it
means then to say that robots have rights in a human centered approach. On the other side
of the debate Branson has insisted that robots should not enjoy rights though she considers
it a possibility. There is wholly separate issue whether robots or other AI systems should
be given the status of legal entities or legal persons. In a sense natural persons but also
states businesses or organizations are entities namely they can have legal rights and duties.

The European Parliament has considered allocating such status to robots in order to deal
with civil liability EU parliament 2016 but not criminal liability which is reserved for
natural persons. It should also be possible to assign only a certain subset of rights and duties
to robots. It has been said that such legislative action would be morally unnecessary and
legally troublesome because it would not serve the interest of humans. In environmental
ethics there is a long-standing discussion about the legal rights for natural objects like trees.
It has also been said that the reasons for developing robots with rights or artificial moral
patience in the future are ethically doubtful.

In the community of artificial consciousness researchers there is a significant concern

whether it should be ethical to create such consciousness since creating it would
presumably imply ethical obligations to a sentient being that is not to harm it and not to
end its existence by switching it off. Such authors have called for a moratorium on synthetic
phenomenology. So, to conclude in this module we have discussed how to implement AI
and managing AI’s unintended consequences. Further we have explained privacy and
surveillance in the digital age. We have understood ethical concerns in AI surveillance and
manipulating behavior through information use and lastly we have focused on artificial
moral agents.

And these are the 7 sources from which the material for this module was taken. Thank you.
 AI in Marketing
Prof. Zillur Rahman
Department of Management Studies
Indian Institute of Technology, Roorkee
Week-12

Lec 59-Navigating Ethical Challenges in AI IV

Welcome to this NPTEL online certification course on artificial intelligence in marketing and
now we are talking of module 59. So, as you can see from this slide, this is the last topic that
we will talk about related to navigating ethical challenges in AI. So, this is part 4 and we are
talking of module 59 and these are the things that we will cover in this module. So, we will
start with examining AI enabled Mobile health apps, then we will talk about ethical
concerns for AI enabled mHealth and then move on to operationalizing ethical values in the
agile AI development life cycle. So, to start with, AI's roles in healthcare is growing due to
users data and computing power. AI enhanced medical devices and mobile apps show
promise improving disease management.

However, ethical concerns like privacy and fairness need addressing. To address this, a
recent article focused on ethical issues in AI-enabled mobile health apps. These apps
leverage AI for personalized health info and monitoring. Also, integrating AI ethics into the
agile development process is important for effective implementation, considering relational
aspects and balancing benefits with risks.

Practical examples demonstrate these ethical principles in AI-enabled mHealth apps. Now,
let us start about talk about examining AI-enabled mHealth app. The first is dietary
assessment. Traditional dietary assessment methods face bias and inaccuracies. Mobile
devices with AI offer better privacy and efficiency.

Vision-based techniques for food recognition and volume estimation are rising. Both
handcrafted and deep learning based methods are explored. Mobile tool assist type 2
diabetes patient in making informed dietary choices. Deep learning like CNN that is
Correlational Neural Networks extract features automatically. The second is patient
monitoring.

AI powered medical apps empower patients and improve personalized decision making.
Variable devices monitor conditions like artificial fibrillation . Real-time monitoring faces
challenges in less developed areas. On-device, lightweight algorithms are crucial. The third
is mental health management.

AI aids mental health through chatbots-based mental health apps. They provide non-
genmental support and cost-effective interventions. AI bots shows reduction in depressive
symptoms. Careful transparency, professional supervision, and addressing social
implications are essential. The fourth is healthcare administration.

AI reduces administrative burdens in healthcare. Chatbots handles inquiries and

appointments. Natural language processing streamlines patients' record. Allocation of
resources and pandemic measures benefit from AI. Data privacy and bias issues must be
addressed.

Now, what are the ethical concerns for AI-enabled mHealth? The first is fairness, ensure
equitable treatment and empowerment, addressing biases in training data and model
choices for marginalized users like blind individuals, offer alternative navigation methods.
The second is agility, AI systems should capture temporal changes in health conditions,
allowing for real-time adaptation and dynamic predictions. The third is precision, balance
accuracy and interpretability in AI models, complex medical conditions require careful
consideration of input variables. The fourth is safeguarding humanity, prevent Prioritize
community well-being, uphold human rights, and adhere to legal regulations like GDPR. The
fifth is respect for others.

Show compassion and care, provide human oversight in complex decision making, and
prioritize user privacy. The sixth is trust and accountability. Build long-term trust through
ethical actions, establish mechanism for accountability when AI systems fail, and foster
community involvement. The seventh is robustness and reproducibility. Ensure reliability
throughout an AI system's life cycle, generate locks for transparency, and focus on external
validation and standardization for reproducibility.

Now, how to operationalize AI ethics? Addressing ethical challenges in AI system

development is complex. Code of ethics alone are insufficient due to the unpredictable
nature of AI outcomes. To effectively integrate ethics, comprehensive measures are
required. Operationalizing ethical values in the agile AI development life cycle is key. The
agile development life cycle comprises four phases that will be shown in figure 1.

So, the phase 1 is ethical requirements elicitation, design and development, testing and
deployment. So, these are the, this is figure 1, embedding ethical principles into the agile
software development process. So, this is the agile software development process and these
are the four stages, requirement elicitation, design and development, testing and
deployment. Now, you see that in requirement elicitation ethical concern, it starts with
identifying and prioritizing ethical principles, then collect representative data for fairness
involve users in data curation. Then move on to transparent reporting of data used that is
transparency, establish a data governance framework, establish mechanism for
accountability, align system requirements with identified ethical principles that is ethics by
design approach.

Then move on to design and development, design and development ethical considerations.
Design with the user, trust, respect user autonomy, human in the loop, balance system
performance with ethical principles such as privacy, ensure the system is designed to be
robust and develop human-centric explanations. The next is testing, testing ethical
considerations. Evaluate the system with users. Evaluate the accuracy of the system, that is
precision, evaluate system explanation, evaluate transparent reporting of the system,
evaluate privacy and consent concerns, evaluate the robustness and reproducibility, assess
equity that is fairness and evaluate the reproducibility and robustness of the system.

The fourth is deployment that is deployment of ethical concern, ethical considerations,

periodical evaluation of system to ensure that it functions as an in an ethical manner.
operationalize accountability mechanism, continuous evaluation of ethical principles,
transparent reporting system failures that is system logs, system should ensure well-being
of users and report all harm caused that is safeguard humanity. So, let us start with the first
one that is requirement felicitation. In the early stages of developing AI systems such as
during the requirements gathering It is essential to identify and align ethical principles
with the design of the solution. This involves taking into account specific human values,
biases, and real-life experience of users.

These ethical principles are unique to each use case. In the context of AI-enabled mobile
health applications, we have pinpointed seven ethical principles, fairness, agility, precision,
safeguarding humanity, respect for others, trust and accountability, and robustness and
reproducibility. For fairness, when creating AI-enabled mHealth solutions, it is crucial to
ensure that the AI system treat all patients equally. This means considering both the current
level of privilege and the potential empowerment of individuals. Bias is the concern that can
emerge at various stages of development, from training data to algorithms and
implementation. To address this in AI-enabled mHealth apps, it is important to study
cultural values, curate diverse datasets and ensure representation. For instance, a dataset
for recognizing various foods should include images from diverse communities the app
intends to serve. Similarly, algorithms for personalizing blood pressure management
should account for diverse patients in terms of factors like race, gender, age and physical
fitness. Data collection is another crucial aspect.

The development team need to ensure that data is not only fairly representative, but also
establish a robust data governance strategy. Data governance is essential to manage data
relationships that can inadvertently amplify social relations and potentially harm users.
This strategy should also protect data workers and outline clear guidelines for outsourced
data curation, ensuring fairness, compensation and proper working conditions. Privacy
concerns must be addressed by engaging with different communities early on to
understand their privacy perspective and cross-cultural differences. This can help shape the
privacy design of AI technologies.

Developers should engage with communities using collaborative methods like ideation,
hackathons or workshops to understand individual privacy differences during the
requirements phase. To ensure precision, Correctness and safety of software systems
formal method should be applied, especially in critical applications like healthcare
solutions. These methods provide a rigorous approach to specifying software behavior and
verifying its correctness. For instance, an AI-based mental health application using natural
language processing can benefit from formal methods to establish strict rules for input
apply mathematical models for processing and verify precision and safety properties. Given
the complexity of healthcare systems and the involvement of diverse stakeholders,
prioritizing ethical concerns is crucial.

For AI-enabled mHealth apps, putting patients at the center with an equity goal is
recommended, especially for vulnerable populations. Effective communication between AI,
developers and healthcare stakeholder is vital for understanding and alignment. This
involves bridging the gap between technical and healthcare language, managing
expectations and conducting user experience research to ensure clarity and comprehension
at all levels. The second is design and development. The proliferation of generative AI
technologies introduce potential risk to mHealth applications.

One concern rises from generative AI tools' ability to create realistic images or videos of
food and other objects, potentially leading to the misuse of vision-based mHealth solutions
for malicious purposes, including the manipulation of visual content. Developers of such
applications must be proactive in anticipating the risk of deepfakes and implementing
safeguarding against them. Additionally, the expanding use of large language models like
ChatGPT and BARD, accessible through mobile devices such as smartphones and tablets,
presents its own set of challenges. Like LLMs, that is large language models, must be utilized
for diagnostic and treatment recommendations.

Their accuracy is not guaranteed. Inaccurate information or advice from LLMs could have
severe and irreversible consequences for users. A tragic case involving a Belgian man's
interaction with an AI chatbot serves as a sobering reminder of the potential impact. To
address this, cautious measures must be taken, including limiting access to these tools for
users with certain mental health conditions. like generalized anxiety disorder or
hypochondria. Furthermore, LLMs are trained on vast datasets from the internet, which
often reflects dominant perspective and biases.

As a result, these models may perpetuate inequalities and biases, disproportionately

affecting marginalized communities. Developing health-oriented LLMs requires special
considerations to ensure equitable and unbiased care provisions by clinicians. Developers
should steer clear of creating general-purpose AI tools for healthcare and instead
concentrate on crafting specialized AI tools tailored to address specific health issues. Lastly,
the implementation of AI-driven mHealth solution prompts ethical inquiries regarding the
delegation of authority and autonomy in healthcare decision-making as well as concerns
about the potential erosion of human empathy and interpersonal connections in patient
care. To address these issues, ethical considerations must be woven into the design and
development phases of AI-driven systems.
 It is imperative that AI systems are designed to foster mutually beneficial relationships,
upholding ethical standards and ensuring that the use of technology enhances rather than
detracts from human-centric care. The third is testing. Following the completion of
development, diverse iterations of the AI-driven mHealth application should undergo a
series of testing phases involving real users. The integration of a feedback loop ensures that
user input informs subsequent iterations of the system, aligning with stakeholder
expectation, while emphasizing patient centricity and equitable treatment for marginalized
groups. The testing phase should encompass a systematic integration of the ethical
framework and the testing process.

This approach ensures that both white box and black box AI algorithms adheres to ethical
fostering the creation of dependable and socially responsible AI systems. For white box
models, the internal mechanism and structure of the system should be rigorously tested.
However, given that most healthcare algorithms employ deep learning techniques and are
consequently black box models, alternative methods should be employed. Developers
should employ model interpretability techniques such as local interpretable model agnostic
explanation as heavily addictive explanation or textual explanation for visual models. These
methods shed light on the decision-making process and inner logic of a given model,
enhancing transparency and facilitating the interpretation of incorrect predictions.

Furthermore, testing scenarios should be meticulously designed to evaluate compliance

with ethical requirements. These scenarios should assess how the system responds to
diverse input data that is robustness, its handling of sensitive patient information and its
ability to make unbiased and fair decisions. Ensuring consistent output for the same inputs,
reproducibility is essential, biases related to factors like age, race and gender should be
rigorously tested to prevent results that reinforce existing health inequalities. The
application of causality in ML approach for healthcare is gaining traction. Causality entails
establishing relationships between cause and effects in the analyzed data.

In healthcare, using diverse datasets like images and text poses a challenge in extracting
meaningful information for ML. Developers should design a silico scenario to establish
causal relationships between variables while accounting for confounding factors. This is
particularly pertinent for complex applications like mHealth solutions for remote blood
pressure management. Evaluating the accuracy of AI-powered conversational mHealth
application is pivotal for their reliability and safety. Simulation experiments with expert
evaluation including clinicians and patients help assess the real-world performance of these
systems.

Furthermore, benchmarking AI-powered mHealth solution against clinical gold standards

is essential for validation. Data security remains a concern. Rigorously testing under various
conditions is imperative to identify vulnerabilities like data leaks and ensure the protection
of user-sensitive health data. The application of explainable AI tools aid in testing the
transparency and comprehensibility of the system, particularly in healthcare, where
accuracy, fairness and biases have significant repercussions on patients' well-being. In
gauging the integration of human values into the AI system, a combination of qualitative and
quantitative metrics should be developed, tested and validated.

Evaluations rooted in human-centered explanation should be conducted with potential

users to determine the degree to which the AI solution aligns with their values. The fourth is
deployment. In the deployment phase, the establishment of continuous monitoring
mechanism is essential. This ensures that the developed AI system consistently adheres to
ethical requirements and functions as intended. A continuous feedback loop should also be
integrated enabling users to provide input and raise concerns such as biases, inaccuracies
and security issues.

Developers should guarantee transparent reporting of both ethical and technical aspects of
system failures, incorporating elements like system logs and sustainability reports. Such
reports clarify the AI system's decision-making rationale. Operational methods and
limitation, mitigating the risk of misunderstanding and mistrust. Transparent reporting
further showcases developers' commitment to ethical principles and offer reassurance to
the public about the responsible development and deployment of the AI system. To
establish accountability for system failures, robust mechanism and clear guidelines should
be put in place.

When a failure occurs, the development team must One, ascertain the root cause of the
failure. Two, implement corrective measures to rectify the erroneous action, often through
model updates. And third is, transparently communicate with stakeholders. The
communication should encompass an explanation of the failure, one, corrective steps taken,
two, and the potential application for patients and their broader ecosystems. Lastly,
organizations deploying AI should implement ethics-based adjusting using structured
processes.

This allows them to evaluate the extent to which their AI systems align with ethical
principles and identifying any gaps that require corrective action. In addition put
mechanism in place to address identified gaps, therefore improving user satisfaction and
fostering trust in AI systems. So, to conclude, in this module, we have examined the AI-
enabled mHealth apps. Further, we focus on ethical concerns for AI-enabled mHealth. And
lastly, we have operationalized ethical values in the agile AI development lifecycle.

And these are the 6 references from which the material for this module was taken. Thank
you.
 AI in Marketing
Prof. Zillur Rahman
Department of Management Studies
Indian Institute of Technology, Roorkee
Week-12

Lec 60-AI and Sustainability

Welcome to this NPTEL online certification course on Artificial Intelligence and

Marketing and now we will talk about module 60. So, as you can see from this module,
this is, this is the last module of this course and we are talking about AI and
sustainability. These are the things that we will talk about in this module. So, we will
delve into the integrate relationships that define the contemporary business landscape.
We will begin by navigating the multifaceted realms of corporate social responsibility
where companies embark on a commitment to uphold the highest standard across their
operations. But what lies beneath this commitment? We uncover the connection between
CSR, corporate law and the pursuit of sustainability.

Now, let us navigate the landscape of CSR and corporate law. Within the realms of
corporate business, corporate social responsibility stands as a multifaceted concept that
encapsulate sustainability development, corporate governance, stakeholder protection
and socially responsible investment. So, these are the 4 things that this covers. It
represents a commitment to uphold exemplary standards across a company's dealings.

Central to this is the process of identifying and neutralizing any detrimental effect that
corporate actions and operations might inflict upon society. This pursuit has showcased
the interplay between ethical conduct towards non-mandated stakeholders and the
adherence to stakeholders' primary mandates ingrained within the corporate law
bolstered by judicial guidance. In essence, CSR oblige directors to act in ways that
benefit the company and serve the interest of broader society. Core CSR elements
encompasses social, environmental and human rights, all of which intervene with the
fabric of sustainable corporate operations. Corporate law and governance mechanism
have assumed the mantle of endorsing and propagating CSR, often realized through the
disclosure of information and the alignment of directors' duties with social and
environmental issues.

The very definition of CSR remains a dynamic and contested subject. Variances arise
from the ever-evolving expectations of diverse stakeholders, adapting to the rapid
mutations in the business landscape. This has led to the proliferation of CSR's reach
from its early focus on mitigating negative societal impacts and philanthropy to its
contemporary embodiment emphasizing strategic collaborations, scalability and
responsiveness. So, these are the new concerns. Crafting sustainable future through CSR
and corporate decisions.

Sustainability is the key part of CSR. Touching on economic, environmental and social
aspects, it means meeting today's needs without harming future generations. The United
Nations agenda 2030 and its 17 sustainable development goals aim to balance economic
growth with environmental and social health. In CSR, decisions must balance what's
good for society and benefits stakeholders. Laws like section 172 of the UK Companies
Act 2006 highlights that directors should think about stakeholders beyond shareholders.

Moving from shareholders to stakeholders. This way business decisions connect

business goals and responsibilities while also considering impacts outside the company.
Even though CSR can boast a company's reputation and success, meeting a solid CSR
plan requires understanding and analyzing sustainability. With the rise in technology like
AI, making a start choice choices becomes easier. AI including machine learning and
robots can use lots of data to spot patterns and trends, helping to come up with creative
CSR plans and smart business decisions.

Now let us look at how AI is reshaping corporate governance and CSR practice. The
advent of AI presents an opportunity to revolutionize corporate governance by
suggesting informed actions, solving complex challenges and aligning with sustainable
objectives. Machine learning is embedded within AI broad scope, which derives insights
from data to guide directors towards effective strategies. For corporations, AI
development serves to augment CSR programs, propelling economic value, long-term
interest and solutions to multifaceted challenges. But using AI comes with challenge like
dealing with rules, guidelines and ethical concerns.

In the midst of this digital change, good data is crucial for AI to work well. Combining
AI with CSR has great, has big possibilities but companies must follow rules and ethics
carefully. How to use AI in the boardroom? The potential application of artificial
intelligence in the boardroom are vast and promising, offering opportunities to enhance
decision making processes. With access to high quality big data, AI can uncover hidden
insights and valuable knowledge, leading to improved efficacy and quality of decision
making. However, the integration of AI into the boardroom presents challenges such as
managing biases in data-driven decision, ensuring transparency in the decision making
process and addressing the monopolization of data and expertise.

This module explores the various roles AI can assume in corporate management,
analyzing the potential benefits and challenges and emphasizes the importance of a
regulated and sustainable AI environment. AI's integration into corporate management
can be classified into three roles. First is assisted AI, the second is advisory or
augmented AI and the third is autonomous AI. Each role reflects a different level of AI
independence and collaboration with human decision makers. So, the first is assisted AI.

At the simplest level, AI acts as an assistant performing administrative tasks with

minimal autonomy. Decision rights remain with human board members, allowing them
to focus on strategic decisions while AI handles tasks like data analysis and monitoring.
The second is advisory or augmented AI. In the advisory role, AI supports decision
making in complex scenarios by asking relevant questions, identifying opportunities,
detecting irregularities and mitigating risks. While AI provide recommendations, the
final decision remain in the hand of the human board members.

The third is autonomous AI. AI is empowered to make decisions autonomously based

on data analysis and pattern recognition. However, human oversight and validation are
crucial as algorithms can learn independently but still require human direction.
Challenges of integrating AI in the boardroom. Despite the potential benefits, the
adoption of AI in the boardroom comes with challenges that need to be addressed.

The first is data bias. Data-driven decisions may inadvertently perpetuate societal biases
present in the data. Both inherent data biases and those introduced by designers can lead
to discriminatory or unethical outcomes. Offers are required to standardize data
collection, detect and eliminate biases and establish shared databases accessible to all.
The second is lack of transparency.

The opacity of AI decision making processes can lead to difficulties in explaining

decisions and justifying outcomes. A transparent AI approach involves ensuring the
transparency of both the outcomes and the process. Developing auditable measures and
following best practices for transparency can help mitigate this challenge. The third is
monopolization of data and expertise. AI implementation demands substantial
investment in data infrastructure and talent acquisition, facing large companies and
creating imbalances in market power.

SMEs may be left behind due to limited access to resources. Addressing this challenge
involves promoting sustainable and regulated AI deployment to ensure fair access for all
businesses. The fourth is regulatory uncertainty. The lack of clear regulations for AI can
hinder adoption due to fears of legal and ethical issues. A harmonized, risk-based
regulatory framework is needed to ensure AI systems' ethical and lawful use.

This framework should accommodate various technologies and promote a sustainable

and consensual AI environment. What is AI's contribution towards more sustainable
decisions? The contribution of AI towards fostering more sustainable decisions is
perceived with optimism, driven by its innovative potential to benefit both businesses
and society. However, alongside the challenge of ensuring transparency and
accountability in AI decision-making, designers and users must also consider the long-
term and transformative impacts on individual and society. Recognizing the urgency of
aligning corporate actions with sustainable development, it becomes imperative to
embed the three core pillars of sustainability. They are environment, society and
economy into the ethics of AI.

A consensus emerges that effectively harness AI and other advanced technologies,

necessitates collaboration among various stakeholders, particularly corporate directors
and the public sector. The European Commission's suggestion that AI engineers should
be answerable for the societal, environmental and human health consequences of AI
decisions, underscores the significance of ethical and sustainable AI development. In
this context, the following section dwells into how AI-driven decisions can be ethically
and sustainably oriented to serve the interest of companies and broader societal well-
being while mitigating potential associated cost. Even though we are just starting to look
at how AI and sustainability connect, we should make sure AI helps us reach important
goals for society. AI can help with all kind of important goals.

When we talk about sustainable AI, we mean thinking about AI at every step from how
it is made to how it is used. When integrating specific AI systems, the board of directors
should consider their environmental impact. The training and development of algorithms
can contribute significantly to greenhouse gas emission. Hence, the environmental
footprints of such processes needs to be justified by the benefits they offer. Companies
must allocate resources and budget allowances to ensure the sustainability of the
infrastructure used for training or fine-tuning AI algorithms.

Once a sustainable AI infrastructure is established, the focus shift to leveraging AI for

broader environmental and socio-economic goals. Basic algorithms can be programmed
to steer AI towards ethical and sustainable corporate action. Through data science for
social good, AI can address societal challenges and unsolved issues in a measurable
manner. By fostering transparency, AI can measure disclosure against required standards
and guidelines, thereby enhancing sustainability. Furthermore, AI can recommend
sustainable policies and decisions, augmenting the capacities of human directors.

It can predict the efficacy of sustainable oriented corporate policies with precision,
aiding in formulating and optimizing CSR programs for distributive justice. AI can
facilitate automated planning of environmental, social and governance investment
strategies and complementary tasks, identify stakeholders' networks and assessing
variables for strategic clarity. AI's roles extend to prevention, where it acts as a barrier
against human, corporate harm to society and the environment. Smart technology driven
by AI can predict and prevent discrimination, fraud, conflict of interest, thereby
mitigating social and environmental risks. AI in cooperation to decision making process
can enhance both adaptive capabilities and expedite responses to environmental changes.

The alignment of AI with organization's core values is crucial. AI ethical programming

must occur in the goal setting phase, striking a balance between the organization's
interests, features and available data. However, while AI presents numerous benefits, it
also poses challenges such as data dependence and biases, potential ethical conflicts and
exhibition of societal inequalities. For effective AI driven sustainability, it is vital to
establish a harmonized regulatory framework. The current scenario categorized by
fragmentary and inconsistent approaches, necessitates comprehensive and coordinated
strategies.

The complexity of AI and its potential impact requires a harmonized framework as

evidenced by three critical issues. The first is uncertainty stemming from multiple ethical
principles, the influence of private sector and the lack of enforcement mechanism.
Recognizing these challenges, the European Commission introduced a proposal for AI
regulation in 2021. This regulation based on the concept of trustworthy AI aims to foster
a trustworthy AI environment while establishing transparent and proportionate
obligations for AI systems. Similarly, the UK has also engaged in exploring the
implication of AI and has called for international norms to regulate AI's design,
development and deployment.

However, the journey towards harmonized AI regulation is a long one and any
regulatory model should be grounded in a risk based assessment to ensure proportionate
responses to potential harms. Now, let us look at the risk based regulatory approach. The
argument for a consistent legal framework for AI has been established. The next
consideration revolves around determining the most effective approach. AI, a domain
encompassing a multitude of technologies with distinct attributes and potential risks,
inherently lacks a universal solution due to its diversity.

Furthermore, the rapid evolution of technology renders traditional regulatory methods

inadequate for promptly addressing new challenges. In light of these factors, a novel
strategy is proposed, one predicated on the inherent risk levels of each specific AI
solution. This approach refers to as de minimis regulation outlines a set of overarching
principles and minimal standards adaptable to each situation. This innovative regulatory
model strives for harmonized application across diverse market actors while
simultaneously fostering technological advancement and innovation. The next we will
look at the proportion of a risk based approach.

The introduction of a risk based approach to AI regulation is advocated. This approach

already prevalent across various domains such as environment, finance, food and legal
services involves the use of specific strategies and techniques by regulators. This entails
constructing decision making frameworks that prioritize regulatory actions and risk
assessment. The methodology starts by identifying risk as its foundation and delves into
the various attributes of these risks, including their nature, type, magnitude and
probability. Subsequently, a hierarchical arrangement of risk based on these evaluations
is established.

Although space constraints limit an in-depth exploration of this approach in this present
module, it is anticipated that adopting such a strategy will facilitate the deployment the
development of AI technologies in a secure and advantageous direction. This approach is
particularly pertinent for regulators tasked with addressing potential AI related accidents
or security breaches. A global collaboration to mitigate risk and cultivate AI for the
collective benefit necessitates the involvement of regulatory bodies. The harmonization
of legal framework holds potential to offer clear and consistent AI risk assessment
standard. The salient advantage of this risk based regulatory stance lies in incentivizing
companies to incorporate meticulous risk evaluations into their corporate decisions.

Ultimately, this regulatory paradigm contributes to a robust governance structure that

optimizes the allocation of regulatory resources based on the gravity of risks. Next we
will talk about the principle of proportionality and gradation of regulation. The regulatory
measures for AI systems differ in accordance with the associated risk of adverse
outcomes. Four scenarios involved, either negligible or low risk, a flexible approach
suffices. This might encompass a voluntary ethical code of conduct or adherence to
international AI principles.

In instance, where risks range from medium to high, the implementation of business
standards or guidelines are complete by transparent disclosures and compliance
mechanism becomes requisite. For high risk systems, comprehensive regulatory
measures could be introduced. Consequently, the degree of regulatory intervention is
correlated with the potential harm posed, underscoring the regulator's role in
safeguarding public interest. European Commission's validation and proposal. The
European Commission has recently endorsed this model in its proposal for an AI
regulation, acknowledging the adverse consequences of AI on stakeholders, workers and
individuals.

This regulatory framework seems to harmonize the diverse objectives and interests of
involved parties. The proposals are aligned with the open data directive. The regulation
on European Data Governance, the Data Act, GDPR and other pertinent legislations
emphasizes the safeguarding of privacy, personal data and sensitive information. While
some constraints on business operations might ensure, they align with the objective of
ensuring the market only accommodates safe products. Such constraints are justified by
the overriding public interest in avoiding substantial safety risks and fundamental rights
infringement.

The proposed regulatory framework harmoniously integrates with existing sectoral

safety laws, underscoring consistency and minimal additional burdens. Tier regulations
and enforcement. The regulatory proposal categorizes AI systems into different risk tiers.
Unacceptable risk, high risk, low and minimal risk. Systems constituting unacceptable
risk via violating fundamental rights are prohibited.

High risk systems necessitates adherence to specific rules including high quality data,
transparency, human oversight, accuracy, reverseness and ex-ante conformity
assessment. For AI solutions in lower risk categories, limited transparency obligations
are proposed. Encouraging voluntary code of conduct for non-high risk AI providers
further enhances the regulatory framework. The next is the vitality of effective
enforcement and monitoring. The efficacy of all regulatory endeavors hinges upon
robust enforcement mechanism.

A comprehensive monitoring and evaluation scheme are vital to ensuring consistent

implementation of the regulation. The proposal recommends establishing a European
Artificial Intelligence Board to aid national authorities in executing and applying the
European regulation. This collaborative arrangement necessitates the designation of
Nationalist Supervisory Authority within member states. Additionally, AI providers must
promptly inform authorities about serious incident of malfunctions constituting
fundamental rights breaches as well as any recall or withdrawal of AI systems. The next
comes stakeholder participation and multidisciplinary involvement.

The proposed regulatory framework underscores stakeholder involvement in policy

formulation. Emphasizing multidisciplinary contributions, recognizing the importance of
minimizing administrative burdens and clarifying existing regulations, this strategy
aligns with AI regulations needs. Stakeholder possessing insights into AI applications,
particularly those versed in AI, big data and robotics play a pivotal role in crafting
accessible and comprehensible regulations. This collaborative effort between
stakeholders and data scientists ensure appropriate data labeling and effective regulation.
This approach hinges on a cooperative endeavor that encompasses various disciplines
delineating roles in AI deployment within the corporate sphere.
 In the pursuit of a common good, AI, the role of AI as a catalyst. The regulatory
framework extends beyond addressing AI's risks and challenges to encompass societal
well-being and environmental preservation. Expressions like common good or the
commoners signifying granting all individuals unrestricted access to communal
resources for social equity. These concepts underline cooperation, collaboration and
coordination aligned with sustainable corporate practices and AI applications. The UK
Parliament's proposition that AI should be developed for humanity's common good
further enforces this principle.

In essence, the commerce represents the drive for corporate sustainability while the
common good emerges as the ultimate aspiration for AI deployment. In the corporate
landscape, stakeholders collectively share common spool resources within an
organization contributing to its sustainability journey. AI, when employed, seriously
serves as a catalyst for these efforts. Its benefits are to extend to all constituencies
aligning with the concept of common spool resources. Corporations' utilization of AI to
affect positive societal changes ultimately culminates in the attainment of the common
good.

In conclusion, AI's myriad benefits and challenges necessitates a risk-based regulatory

approach. This approach should aptly balance intervention with innovation, is
instrumental in achieving a harmonious and ethical AI deployment landscape. By
upholding human values and mitigating potential AI-related harm, a coherent regulatory
framework can usher in AI for the common good. The integrate logic underpinning the
trajectory is visualized in figure 1.

So, this is the figure 1. Artificial intelligence and corporate decision making, then
promote SGD and CSR that leads to sustainable decisions informed by AI which leads to
mitigating risks associated with AI. Then that goes to risk-based regulatory approach, AI
as a common good and this leads to a harmonious approach and then AI as a common
good. So, to conclude, in this module, firstly we have focused on navigating the
landscape of CSR and corporate law. Secondly, we have studied, we have studied how
AI is reshaping corporate governance and CSR practice. Further, we have studied how
AI contributes towards sustainable decisions.

And lastly, we have discussed what is risk-based regulatory approach and the principle
of proportionality and gradation of regulation. And these are the 6 sources from which
the material for this module was taken. Thank you.