DOCUMENT L – AI Risks
What Are the Potential Risks of AI?
While AI can present significant benefits, it also presents risks and challenges to our
business. Data sourcing, technology, integration and process issues, program bias into
decision-making algorithms, security challenges and the protection of personal privacy could
impair the adoption and acceptance of AI solutions.
o Inaccurate output from Artificial Intelligence (AI) could result in brand and reputation
damage. If the output from AI solutions are deemed to be inaccurate or
questionable, our brand and reputation may be harmed and we may potentially be
subject to legal liability claims.
o Invasion of Data Privacy
One of the potential dangers of AI is that you will no longer have privacy in everything you
do online. If you’ve noticed, your Facebook timeline is bombarded with ads that relate to
the item you purchased on an e-commerce platform earlier. Every person’s online as well as
their daily behaviour are being tracked. Because of AI, facial recognition algorithms are
aware of your identity as well as CCTVs everywhere.
o Self-Crashing Vehicles
AI integrations are already moving towards self-driving vehicles. However, the Uber case
showcased that it’s not the best option yet. Safety is still the main priority for people, and AI
developments for self-driving vehicles may take decades or more before it can be completely
safe for passengers and pedestrians alike.
o Loss of Skills
With the emergence of AI technology, the loss of skills is one of the most significant threats
AI posts. People are losing jobs because smartphones, computers, and robots have replaced
them.
o Discrimination
Through AI integration, machines have the ability to gather, track, and analyze sensitive
data about you. As a result, those machines can utilize sensitive data against you. It’s a
likely occurrence that insurance companies will reject your application since the camera
detected you numerous of times talking to your smartphones while driving on the highway.
Also, another potential danger of AI in terms of discrimination is that a potential employer
will withhold an employment contract based on a person’s social credit score.
o Autonomous Weapons
Since AI technology is getting more and more user-friendly, cheaper, and accessible for
everyone, it’s adding more potential harm especially those people who have malicious
intent.
For example, a criminal can buy a drone to autonomously fly. With facial recognition
technology, they can recognize the person’s identity and track the person’s whereabouts. In
terms of the military, it could be a fateful stepping stone towards supplying AI technology
for automating deadly drone attacks. Imagine this, a drone flying above your rooftop has
the potential to kill you without you knowing who is responsible for flying the drone.
Indeed, it’s one of the potential dangers of AI that needs to be addressed.
o Social Manipulation
Through AI-powered algorithms, social media become an effective platform for target
marketing. They have the power to reveal our identity, our preferences, and personalities
through predictive analysis and programming. A prime example is the Cambridge Analytica
scandal that disrupted the 2016 presidential elections. While it’s still not confirmed, AI has
already imposed its potential danger in terms of social manipulation. By creating
propagandas to individuals identified through sensitive data and AI-powered algorithms,
Artificial Intelligence has the power to target specific individuals and spread rumours about
them regardless of the format they want.
o Strategy Risk — As I wrote in an earlier post, it is not simple to craft AI strategy. A
mistake at the early stage sets the stage for other downstream problems.
Unfortunately strategy is often in the hands of those who don’t have a thorough
understand of AI capabilities. This category includes risk from selecting wrong
(infeasible) initiative (relative to the organization’s ground reality), lack of executive
support, uncoordinated policies, friction among business groups, overestimation of
AI/ML potential (contrary to the hype, ML is not the best answer for all analytics or
prediction problems), lack of clarity or unclear goals or success metrics. The most
obvious mitigation approach is to have alignment between the AI leadership and
executive team on the strategy and what risks are associated with it. Understand
exactly how AI will impact people and processes and what to do when things don’t go
well.
o Financial Risk — One of the common, but not often talked about assumption is that
the cost of model development is the main financial factor for AI/ML. Only if we had
the best data/AI scientists then we would be all set. As discussed in this paper, the full
lifecycle of AI is way more complex and includes data governance and management,
human monitoring and infrastructure costs (cloud, containers, GPUs etc.) There’s
always the uncertainty factor with AI work which means that compared with software
development, the process will be more experimental and non-linear and even after all
the expensive development work the end-result is not always positive. Make sure your
finance leadership understand this and not treat AI/ML as just another tech project
o Technical Risk — This is the most visible challenge. The technology is not mature or
ready but the technologists or the vendors want to push it. Or perhaps the business
leadership want to impress the media or their competitors with the forward looking
vision. Technical risks come in many forms: Data: Data quality, suitability,
representativeness, rigid data infrastructure and more. Model risks: capabilities,
learning approach, poor performance or reliability in real world. Concept Drift, that is,
change in the market or environment over time or due to an unexpected event.
Competence: lack of right skills or experience, implementation delays, errors, not
thorough testing, lack of stable or mature environments, lack of DataOps, MLOps, IT
team not up to speed with ML/AI deployment or scaling needs, security breach and IP
theft. The best way to mitigate these is to work on your team’s skills, invest in modern
data infrastructure and follow best practices for ML .
o People and Process Risk — It should go without saying but the right organizational
structure, people and culture are critical to the success of your work. With competent
people and supportive culture, you can catch problems and face challenges. Sign of a
bad culture is that people hide problems and avoid ownership. When you’ve politics
and tensions between teams, problems get amplified. Skills gap, rigid mindset,
miscommunications, old school IT lacks operational knowledge to scale AI. Gaps in
processes, coordination issues between IT and AI, fragmented or inconsistent
practices/tools, vendor hype. Lack of data/knowledge sharing (org structure), missing
input/review from domain experts, lack of oversight / policy controls & fallback plans,
third-party model dependency, insufficient human oversight, learning feedback loop.
Weak tech foundations.
o Trust and Explainability Risk — You did all the work but the end users of your AI-
powered application are hesitant to use or adopt the model. IT is a common
challenge. Reasons include poor performance of the model under certain conditions,
opaqueness of the model (lack of explanation of results), lack of help when questions
arise, poor user experience, lack of incentive alignment, major disruption to people’s
workflow or daily routine. As ML/AI practitioners know, the best models such as deep
neural networks are the least explainable. This leads to difficult questions such as,
what is more important: model performance or its adoption by intended users?
o Compliance and Regulatory Risk — AI/ML can cause major headaches for use cases
or verticals that need to comply with rules and regulations. There’s a fine line here —
if you don’t take some action the competitors may get too far ahead. When you do
take action, you must protect against unforeseen consequences and investigations or
fines by regulators. Financial and healthcare industries are good examples of such
tensions. The explainability factors discussed above are key here. Mitigation: ensure
that risk management teams are well-aware of the AI/ML work and its implications.
Allocate resources for human monitoring and corrective actions.
o Ethical Risk — Your AI/ML project has great data and a superstar technical team,
benefits are clear and there are no legal issues — but is it ethical? Take the example
of Facial Recognition for police work. Vendors pushed this as a revolutionary way to
improve policing but the initial models lacked the robustness needed to make fair and
accurate predictions and resulted in a clear bias against certain minority groups.
Credit scoring and insurance models have suffered from bias for a long time — with
the growth of ML-powered applications, it has become a much bigger problem.