Article 11

1. The technical documentation of a high-risk AI system shall be drawn up before that

system is placed on the market or put into service and shall be kept up-to date.
The technical documentation shall be drawn up in such a way as to demonstrate that the
high-risk AI system complies with the requirements set out in this Section and to provide
national competent authorities and notified bodies with the necessary information in a
clear and comprehensive form to assess the compliance of the AI system with those
requirements. It shall contain, at a minimum, the elements set out in Annex IV. SMEs,
including start-ups, may provide the elements of the technical documentation specified in
Annex IV in a simplified manner. To that end, the Commission shall establish a simplified
technical documentation form targeted at the needs of small and microenterprises.
Where an SME, including a start-up, opts to provide the information required in Annex IV
in a simplified manner, it shall use the form referred to in this paragraph. Notified bodies
shall accept the form for the purposes of the conformity assessment.
2. Where a high-risk AI system related to a product covered by the Union harmonisation
legislation listed in Section A of Annex I is placed on the market or put into service, a
single set of technical documentation shall be drawn up containing all the information set
out in paragraph 1, as well as the information required under those legal acts.
3. The Commission is empowered to adopt delegated acts in accordance with Article 97
in order to amend Annex IV, where necessary, to ensure that, in light of technical
progress, the technical documentation provides all the information necessary to assess
the compliance of the system with the requirements set out in this Section.

Introduction
In this lecture, we will dive into the requirements for the technical documentation of
high-risk AI systems. We will break down the essential components of this
documentation, ensuring you understand the level of detail needed.
Our sub-lectures will cover:
Part 1: General System Description of AI system
● In this section, you will learn how to effectively document the purpose, provider
information, system interactions, forms of distribution, and hardware
specifications of your AI system.
Part 2: AI System Development Details
● This section will explore what aspects of your development methodology, design
decisions, system architecture, and dataset choices need to be thoroughly
documented.
Part 3: Monitoring, Risk Management, and Beyond
 ● We'll cover how to document your AI system's post-deployment performance
monitoring, evolving risk assessments, human oversight safeguards, system
updates, and conformity with relevant standards.
IMPORTANT NOTE: Within each section, you'll find real-life examples provided to
enhance your understanding and application of each requirement. Examples
focused on writing technical documentation for a Fraud Detection system are
highlighted in green.
Additionally, you will find examples highlighted in blue that dive into various other
AI systems. These are intended to showcase the application of regulation's
requirements in a range of different AI system scenarios.

1.1 Intended Purpose

Begin by stating the primary objective of your AI system. Describe what the system is
designed to do, including the tasks it performs and the value it aims to provide. Specify
who will benefit from using the system and how it addresses the problems it's designed
to solve.

Key requirements to address:

● the main goal of the AI system

● the specific tasks it accomplishes
● the problems it aims to solve
● the system's beneficiaries

Example of Technical Documentation

1. Intended Purpose:

The Fraud Detection AI System's primary goal is to detect and signal potentially
fraudulent financial transactions in real time. It leverages pattern recognition and
behavioral analysis to help financial entities, such as banks, credit card companies, and
online retailers, enhance security, prevent unauthorized access, and minimize financial
losses due to fraud.

1.2 Provider Identification

Clearly identify the provider of the AI system. Include the provider's name, contact
information, and the department responsible for the system's development,
maintenance, and compliance. This ensures accountability and facilitates
communication for regulatory inquiries or user support.

Key requirements to address:

● the name of the organization providing the AI system

● contact details for inquiries or support
● the department overseeing the AI system's lifecycle

2. Provider Identification:

● Provider Name: SafeGuard AI Technologies

● Contact Information: Contact details
● Responsible Department: AI Systems Development and Compliance

1.3 Versioning

Document the version history of the AI system, detailing each major release, update,
and patch. Include a detailed listing of all software or firmware versions your AI system
depends on at the time of the documentation. This should encompass both the AI
system's own versions and those of external dependencies critical to its operation.
Detail the procedures and requirements for version updates, how the AI system handles
updates, the expected update frequency, compatibility issues, and the steps taken to
ensure updates maintain or enhance the performance of AI system.

Key requirements to address:

● each version's release number and date

● a brief description of what each version introduces or changes
● the significance of updates or patches in terms of features or bug fixes
● a current versions of the AI system's software or firmware (including
dependencies)
● update protocols (managing updates, frequency, compatibility)

3. Versioning:
 ● Initial Release: v1.0.0: The launch version that introduces basic capabilities for
detecting fraud using machine learning to analyze transaction data.
● Update v1.1.0: Enhanced detection models to better identify sophisticated
phishing attempts.
● Patch v1.1.1: Fixed a memory leak issue in the data processing module,
improving system stability.
● Update v1.2.0: Added real-time transaction monitoring features, greatly reducing
the latency in fraud detection.

1.4 Interactions with Other Systems

In this section we should provide specific details about how the AI system interacts or
has the potential to interact with external hardware or software, including other AI
systems. This includes listing all external hardware or software that feeds data into the
AI system and describing the mechanisms through which the outputs from AI system
are used or could be used to influence or trigger processes in external systems. We also
should identify any dependencies on external systems that could affect the functionality
of AI system. The documentation should detail the strategies implemented to mitigate
risks associated with interdependent system failures.

Key requirements to address:

● input data sources and their origin

● influence of AI system outputs on other systems
● risk mitigation strategies for interdependent failures

4. Interactions with Other Systems:

The Fraud Detection AI System receives transaction data from the financial institution's
transaction processing software and customer databases. Detected fraud alerts are
sent to both the customer relationship management (CRM) system for follow-up
actions and the security incident management system for further investigation. This
interconnectedness necessitates robust data exchange protocols and error handling
mechanisms to prevent potential cascade failures and ensure seamless integration
across different platforms.
1.5 Forms of Distribution

In this section, we need to cover how the system is made available to users and any
associated operational, compliance, security, and maintenance requirements. This
should include detailing the technical prerequisites users must meet for each
distribution form, such as software packages, embedded systems, cloud-based APIs, or
downloadable models. Additionally, address the compliance and regulatory
considerations specific to these forms. Highlight the security and privacy measures
integral to each distribution method to protect data integrity and user privacy. Finally,
outline the protocols for updates and maintenance, ensuring users understand their
roles and responsibilities in keeping the AI system current and functioning optimally.

Key requirements to address:

● specify distribution methods

● detail technical needs for users across distribution methods
● identify applicable compliance and regulatory standards
● outline measures for safeguarding data and ensuring privacy
● describe procedures for keeping the system up-to-date and operational

5. Forms of Distribution:

The Fraud Detection AI System is made available to clients through cloud-based APIs,
allowing for real-time transaction analysis without the need for extensive on-premise
hardware. This distribution method requires clients to have internet access and the
capability to integrate API calls into their existing transaction processing systems. The
choice of a cloud-based API emphasizes the need for clear documentation on API
usage, data formatting requirements, and the handling of updates or changes to the API
to ensure clients can maintain optimal fraud detection capabilities without interruption.

1.6 Hardware Requirements

Detail the specific hardware environment required for your AI system. This should
include the minimum hardware specifications necessary to support the system, such as
processing power, memory, and, if applicable, specialized requirements like GPUs.

Key requirements to address:

● minimum hardware specifications

● specialized hardware requirements

6. Hardware Requirements and Product Integration:

The Fraud Detection AI System is designed for cloud-based deployment, minimizing the
need for extensive on-premise hardware. However, clients looking to integrate the
system within their existing IT infrastructure should ensure they have servers capable of
handling high-volume transaction data processing, with a minimum of 16 GB RAM and
quad-core processors. For clients utilizing the system for large-scale data analytics,
GPU support for deep learning algorithms is recommended to accelerate processing
times. The system is compatible with standard financial transaction processing
software and CRM systems, requiring minimal adjustments for integration. Compliance
with data protection regulations is ensured through encryption and secure data
handling, with all necessary compliance markings included in the system's
documentation.

1.7 AI System Integrated in Product

When the AI system is a component of a product, such as a smart thermostat, the

documentation should contain visual components of how the AI system integrates
within this product. These should include photographs or illustration that show the
external appearance of the product and the internal arrangement of the product. We
should document any external markings on the product, such as labels, serial numbers,
or regulatory compliance marks.

Key requirements to address:

 ● photographs or illustrations of the product
● marking details
● internal layout images
● detailed explanation of visuals

7. AI System Integrated in Product:

Let's consider a real-life example with a smart thermostat that incorporates an AI

system to optimize home heating and cooling efficiency based on user behavior,
weather forecasts, and energy prices. Alongside each photograph or illustration, the
documentation should provide descriptions. These could include information on how
the AI uses data from the sensors, connectivity modules for receiving external data (like
weather forecasts), and how user interactions with the control buttons or mobile app
influence the learning of AI system.

1.8 User Interfaces and Instructions

This part should outline how users interact with the AI system, providing necessary
instructions for setup, monitoring, and day-to-day management. Start with a succinct
description of the UI - highlight its main components and layout. Briefly explain how to
navigate through the UI, focusing on accessing essential features and settings. Provide
concise steps for the initial setup and emphasize critical configurations that affect
system performance and security. Include a direct line or link to technical support for
more complex issues. Outline the primary tasks deployers will undertake in daily system
management

Key requirements to address:

● provide a brief yet thorough overview of the UI

● detail essential navigation paths to access critical functionalities
● offer clear, step-by-step guidance for initial configuration
● highlight key daily management and monitoring tasks
● describe solutions for common issues and direct support contacts for complex
problems

8. User Interfaces and Instructions:

The primary deployers of the Fraud Detection AI System are financial analysts and IT
specialists at financial institutions. The system provides a web-based dashboard
interface for deployers to monitor transaction analysis in real time, set alert thresholds,
and review flagged transactions. Instructions for setting up the system include API
integration guidelines with existing transaction processing platforms, detailed steps for
configuring alert parameters, and best practices for responding to potential fraud alerts.
The documentation offers a comprehensive guide on daily system monitoring, periodic
evaluations for threshold adjustments, and procedures for manual overrides, ensuring
deployers can manage the system efficiently and effectively.

Part 2: AI System Development Details

2.1 Development Methodology and Third-Party Tools

When documenting the development process of high-risk AI systems, we should include

comprehensive details on the methodologies and tools employed throughout the
system's lifecycle.

Key requirements to address:

● selected development methodology

● used pre-trained models
● integrated third-party tools

Let's discuss each requirement:

1. Selected Development Methodology:

Detail the chosen development framework (e.g., Agile, Waterfall) and provide a rationale
for this choice. Focus on its alignment with project goals and flexibility in responding to
changing requirements. Discuss the impact of this methodology on efficiency, reliability,
and adaptability of the project - this way you offer insights into how the approach
supports the overall objectives.
 2. Used Pre-trained Models:

If your AI system utilizes pre-trained models, explain its selection, advantages, risks, and
any modifications that have been made. Outline the criteria used for choosing these pre-
trained models, such as performance benchmarks, compatibility with project goals, and
considerations for data privacy and security. Detail how these models were adapted to
fit the specific needs of your AI system - highlight the balance between leveraging
existing resources and customizing them to address distinct requirements.

3. Integrating Third-Party Tools:

Document any third-party tools or libraries integrated into the AI project - include their
contributions, versions, and dependencies. If you faced integration difficulties, describe
these situations and the strategies you employed to resolve them.

9. Development Methodology and Tools:

1. Development Methods:

The Fraud Detection AI System was developed using the Agile methodology. This
choice was justified by the project's need for flexibility and the ability to adapt to new
types of financial fraud as they emerge. Agile allows for iterative development and
continuous feedback, ensuring the system remains effective against evolving fraud
tactics.

2. Pre-trained Models:

We leveraged pre-trained models for initial pattern recognition tasks, capitalizing on

their ability to identify unusual transaction patterns quickly. These models were
modified to better recognize specific fraud scenarios relevant to our financial sector
clients. The modifications included retraining the models with datasets comprising
transactions flagged as fraudulent, enhancing their accuracy in real-world applications.

3. Third-Party Tools:
Several third-party tools were integrated for data processing and analysis, including
TensorFlow for machine learning tasks and Apache Kafka for streaming transaction
data. TensorFlow's version 2.4.0 was selected for its robust support for deep learning
models, and Kafka 2.8.0 was chosen for its scalability and reliability in processing high-
volume data streams. Documentation includes detailed guidelines on integrating these
tools within the system's architecture, ensuring seamless operation and data flow.

2.2 Design Specifications

This section of your technical documentation is pivotal in shedding light on the thought
process behind the AI system's architectural framework and the critical decisions that
shaped its development. Articulate these elements clearly, showcase the system's
robustness and its alignment with intended functionalities and user requirements.

Key requirements to address:

● outline of the AI system's architecture

● detail assumptions and target user group
● describe classification choices and parameters
● add rationale behind design decisions and trade-offs
● detail optimization goals and performance metrics
● describe expected output and output quality
● discuss technical solution trade-offs and compliance

Now let's deep dive into each requirement:

1. Blueprint of AI System Design: Outline the architecture of your AI system - focus
on selected models and their configurations. Detail how these elements are
structured to meet the project's goals.
2. Assumptions and Intended User Group: Describe the assumptions made during
the design phase - those that affect design choices, and explicitly state the target
users. This should include considerations made for users' specific needs or
constraints.
3. Main Classification Choices and Parameters: Clarify the principal classification
strategies employed in your system and discuss the significance of parameters.
This includes the rationale for choosing certain classification algorithms and how
various parameters were weighted or prioritized to optimize system
performance.
 4. Rationale Behind Design Decisions and Trade-offs: Enumerate significant design
decisions, such as opting for certain algorithms over others. Discuss the trade-
offs involved, such as balancing accuracy with interpretability. Explain how these
decisions and trade-offs were navigated to best serve the system's purpose and
enhance user experience.
5. Optimization Goals and Performance Metrics: Define your system's primary
optimization targets (e.g., maximizing precision or enhancing recall) and the
reasons for selecting specific performance metrics. This section should make
clear to stakeholders the performance priorities and how they align with the
overall objectives.
6. Expected Output and Output Quality: Offer a comprehensive description of the
intended output and the standards used to evaluate its quality. This part should
align expectations with the system's capability to deliver reliable and accurate
results for its users.
7. Technical Solution Trade-offs and Compliance: Discuss the compromises made
in choosing technical solutions, especially those related to meeting regulatory
requirements and ethical considerations. Detail how these trade-offs were
essential in aligning the AI system with both internal objectives and external
standards.

10. Design Specifications:

The core architecture of the Fraud Detection AI System is built around a convolutional
neural network (CNN) model, chosen for its efficacy in identifying patterns within
complex data sets. The CNN is complemented by decision tree algorithms to classify
transactions based on learned patterns, providing a balanced approach to fraud
detection that maximizes accuracy while minimizing false positives.

A significant design decision was choosing the CNN model over simpler algorithms, like
logistic regression, due to its superior performance in pattern recognition tasks.
However, this choice required additional computational resources and introduced
complexity in model interpretation. The trade-off was deemed acceptable, given the
critical need for high accuracy in fraud detection. The system was designed with layers
of abstraction to mitigate interpretability issues, allowing users to understand decision
logic at a higher level.

The system is optimized for recall, prioritizing the identification of as many fraudulent
transactions as possible, even at the risk of higher false positives. This approach
reflects the high cost of missed fraud cases compared to the relatively lower
inconvenience of verifying false alerts. Performance metrics, including recall, precision,
and the F1 score, are used to continuously monitor and adjust the system, ensuring it
meets the evolving needs of financial institutions.

2.3 System Architecture and Computation

In this section you should describe the technical aspects of your AI system. This
includes detailing the components of the system, their interactions, and the
computational resources required.

Key requirements to address:

● provide a detailed description of the system architecture

● elaborate on integration of components
● describe flow of the data in the system
● describe required computational resources

Let's clarify each requirement that needs to be included in the documentation:

1. System Architecture Overview:

For each component of your AI system, describe its specific functionality and role. This
should include input processors, algorithms, data storage mechanisms, and any other
critical parts that contribute to your system's operation. Detail the technological stack -
this encompasses the programming languages, platforms, and tools employed in
creating each component. Explain the contribution of these elements.

For instance, if your AI system collects data from social media platforms, describe the
use of specific APIs for data retrieval. Detail the algorithm of the selected model and its
approach to analyzing the collected data. Specify the programming language used for
development (e.g., Python) and mention the libraries employed (e.g. pandas, scikit-
learn). Also, highlight the use of tools like Docker containers for deploying each
component.
 2. Component Integration:

In this section you should emphasize how different parts of the system communicate
and work together. Elucidate on the protocols or methods used for data exchange
between components - describe how data flows through the system, from input to
processing and finally to output including any intermediate steps or transformations.
Clearly identify where components integrate with one another within your system's
architecture - this includes specifying any APIs, services, or middleware that facilitate
communication and data transfer between components.

For instance, for a customer recommendation system, detail how user data is collected
via a web interface and transmitted using JSON over HTTP for processing. Explain how
this data then aids the recommendation engine in generating user-specific suggestions.
A simplified diagram could illustrate this data path. In a microservices setup, describe
the communication between services like user authentication and product catalog
management using RESTful APIs.

3. Data Flow and Processing:

Describe the protocols and formats for data exchange between system components.
For example, if your system uses RESTful APIs for component communication, detail
the API specifications and data formats (JSON, XML) used. Mention how data
encryption methods ensure secure data transmission. Provide a detailed walkthrough of
the data's journey through the system.

For instance, explain how initial data is ingested via web forms, processed through a
natural language processing (NLP) algorithm for sentiment analysis, and how the
results feed into a decision-making component that triggers specific user notifications.
You can use, for example, flowcharts or sequence diagrams to visually represent this
process, making it easier to follow. Clarify the interactions and dependencies. If the
output of a data cleaning component is required as input for a machine learning model,
describe how these two components are linked, including any data validation checks
performed before the handoff. If applicable, mention fallback mechanisms in case one
component fails to provide the needed output.

4. Computational Resources:

Specify the computational resources required at various stages of the AI lifecycle,

including development, training, testing, and validation. Distinguish the resource needs
at each stage: the types of processors (CPU, GPU), the quantity of memory required, the
storage for datasets, any version control systems, etc. Mention whether the
infrastructure is on-premises or cloud-based.

11. System Architecture:

The Fraud Detection AI System is composed of several key components: data ingestion
pipelines, a preprocessing module, the core convolutional neural network (CNN) model
for pattern recognition, and post-processing steps for decision making. Data flows from
the ingestion pipelines, where raw transaction data is collected and formatted, through
the preprocessing module that normalizes the data and extracts relevant features. The
processed data is then fed into the CNN model, with post-processing analyzing the
model's output to classify transactions as fraudulent or legitimate.

Computational Resources:

The system is deployed on a cloud-based infrastructure, utilizing both CPUs and GPUs
to balance cost and performance. CPUs handle data preprocessing and post-
processing, while GPUs accelerate the training and inference phases of the CNN model.
Key software libraries include TensorFlow for machine learning tasks and Apache Spark
for handling large-scale data processing.

2.4 Used Datasets

This section of the documentation should include details about used datasets - how
they are sourced, prepared, and utilized.

Key requirements to address:

 ● describe provenance, selection and key features of the datasets
● provide overview of dataset coverage
● detail labeling and cleaning procedures

Let's take a closer look at the mentioned requirements:

1. Dataset Provenance:

Expand on the origin of each dataset - include details on the methodologies used for
data collection, the entities involved in its collection and maintenance, and any ethical
considerations or permissions required for its use.

2. Data Obtaining and Selection Process:

Clarify the criteria and processes used for selecting data included in the dataset. This
should cover any sampling strategies or inclusion/exclusion criteria that guided the
data collection, highlighting how these decisions influence the dataset's
representativeness and the system's eventual performance.

3. Scope and Representativeness:

Provide an overview of the dataset's coverage - include geographical areas, time

periods, different groups of people, etc. Describe efforts made to ensure the dataset
accurately reflects the diversity of the intended application domain and address any
potential biases or limitations.

4. Key Characteristics:

Include a general description of the datasets, analyze their composition and how the
features align with your AI system's objectives. Discuss the relevance of specific data
types, such as demographic information or behavioral indicators, to your system's
functionality.

5. Labelling and Data Cleaning:

For labeling, detail the guidelines or standards used to define labels, the process for
ensuring label accuracy, and any tools or techniques employed to assist with labeling
(e.g., crowdsourcing, expert annotation). For data cleaning, describe specific
methodologies used, such as outlier detection techniques, methods for dealing with
missing or inconsistent data, and any automated processes for data normalization.

12. Data Requirements:

The primary dataset for the Fraud Detection AI System consists of transaction records
from various financial institutions, collected over the past five years. The dataset
includes transactions labeled as fraudulent or legitimate by financial experts, covering a
wide range of fraud types.

● Provenance: The data was sourced from collaborating financial institutions, with
consent for use in fraud detection research and development.
● Scope and Representativeness: The dataset is globally representative, including
transactions from multiple countries and currencies, ensuring the system's
effectiveness across diverse financial contexts.
● Key Characteristics: Demographics are not directly applicable; however,
transaction metadata such as transaction amount, location, and time are
included.
● Labelling and Data Cleaning Procedures: Transactions were labeled based on
investigations and confirmed fraud reports. Data cleaning involved removing
incomplete records and normalizing transaction amounts for consistency across
currencies.

2.5 Human Oversight

In this part of documentation, it's crucial to detail how human oversight is integrated to
manage risks and ensure the system's safe use. Consider the following aspects on
what should be covered in this section:

1. Identify Oversight Needs: Clearly state the human oversight requirements based
on the system’s potential risks and its operational context. Include an analysis of
why specific oversight measures are necessary. For instance, if the system is
used in healthcare for patient diagnosis, outline the necessity for healthcare
professionals to oversee its diagnoses.
2. Integrated Tools and User-implemented Oversight Protocols: Describe tools or
features within your AI system for monitoring and intervention. For example,
mention a feature that flags inconsistent diagnostic results for review by medical
staff. Provide any protocols for users to follow for effective oversight. An
 example could be a checklist for financial analysts to verify AI-generated
investment recommendations against market conditions.
3. Monitoring and Decision-making Support:
a) Output Interpretation - Include descriptions of tools aiding deployers in
interpreting AI outputs. For instance, a dashboard that visualizes AI predictions
with confidence intervals can help users assess reliability.
b) Mitigation of Automation Bias - Outline how the system alerts users to
automation bias risks. An example is a reminder prompt for users in a
recruitment tool to consider all candidate data alongside the AI's
recommendations.
c) Control and Intervention Mechanisms - Specify operational control
mechanisms - include procedures for individuals to directly take control or safely
shut down the AI system in response to unexpected conditions or safety
concerns. For example, in a manufacturing plant using AI-driven robotic arms for
assembly, a physical "emergency stop" button is available at each workstation.
This button, when pressed, immediately halts the robot's operations to prevent
accidents or address malfunctions, ensuring worker safety
4. Competence of Oversight Personnel: Enumerate the training programs or
materials provided to individuals responsible for oversight, emphasizing the
development of necessary skills and competence. Confirm that individuals
tasked with oversight have the authority to make decisions, intervene, and apply
changes to the AI system’s operations as needed for safety and compliance.
5. Dual Verification Process: For systems requiring additional scrutiny (Remote
biometric identification systems), describe the process for verification and
confirmation of AI identifications by at least two competent persons. Document
how this process is integrated into the system's workflow and any exceptions
based on the system's application area.
Note: The requirement for independent verification by two individuals is not
mandatory for high-risk AI systems deployed in law enforcement, immigration,
border control, or asylum contexts, when deemed excessive by applicable Union
or national legislation

2.6 Validation and testing procedures

In this segment, we will explore how to effectively document the scope, methodologies,
and results of validation and testing phases within the AI system lifecycle.

1. Scope of Validation and Testing: Clearly state at which stages of the AI system's
lifecycle validation and testing occur. Typically, this includes pre-deployment
testing, periodic testing during use, and validation checks after any significant
updates. Outline the primary goals of validation and testing for your AI system.
This might include ensuring accuracy, reliability, robustness against attacks, and
fairness in outcomes.
 2. Methodologies Employed:

A) VALIDATION PHASE:

In documenting our validation phase, the primary focus is on capturing information

about the data used, the specific methodology employed, and the performance metrics
that guide our understanding of the model.

Let's start with data:

● Thoroughly describe the dataset used for validation, including where it was
sourced, its overall size, types of data, composition, diversity (e.g., demographic
representation in data), relevant unique characteristics and any steps taken
during preprocessing.
● Specify if synthetic data or real-world data is used.
● Explain the reasoning behind selecting this particular dataset - ideally, it should
mirror the real-world data the AI system is expected to encounter.
Now, let's turn to methodology:

● List the tools and techniques employed for validation, including any software or
custom scripts developed to automate parts of the process.
● If you've used cross-validation, detail the implemented type (e.g., k-fold). Specify
any relevant configuration parameters that influence the cross-validation
process. This might include the value of 'k' in k-fold cross-validation, the criteria
for data shuffling, and any stratification methods used to maintain the
representation of different classes within each fold.
● Elaborate on how the validation dataset is prepared and processed for cross-
validation. Detail any preprocessing steps, such as normalization or feature
selection, ensuring consistency across all folds.
● Clearly define the performance metrics chosen (accuracy, precision, F1-score,
etc.) and why they're the best fit for evaluating your system.
The key to meaningful validation documentation lies in summarizing the results and
insights - List and explain the metrics used to evaluate the AI system's performance
across the cross-validation folds (e.g., accuracy, precision, recall, F1 score, area under
the ROC curve). Provide a rationale for choosing these metrics.

Discuss the aggregated performance metrics and what they tell you about the AI
system's capabilities and limitations. Conclude with how the findings from the cross-
validation process have influenced further development, adjustments, or optimizations
of the AI system. This could include modifications to the model, additional training data
requirements, or changes in feature selection.

2.7 Cybersecurity measures

In this section, we are required to detail the cybersecurity measures implemented within
our AI system. This involves outlining the strategies and technologies employed to
protect the system from cyber threats, ensuring its integrity and the safety of its users.

1. Cybersecurity Strategies: Begin by listing all cybersecurity measures in place for

your AI system. This list should include, but not be limited to, data encryption,
access control mechanisms, and intrusion detection systems. For instance, for
an online payment processing AI system, document the implementation of
secure socket layer (SSL) certificates for website encryption
2. Specifying Cybersecurity Measures: For easier navigation and understanding,
you can divide the measures into categories, such as network security,
application security, endpoint security, data security, etc.
3. Detailed Description of Measures: For each cybersecurity measure, provide a
detailed description of its purpose, how it is implemented within the AI system,
and its role in the overall security architecture. Discuss how these measures
integrate with the AI system's components and functionalities. For instance,
detail the use of a Web Application Firewall (WAF) to monitor and block
malicious web traffic, explaining how it’s configured to protect the AI system’s
web interface.
4. Implementation of measures and Operation: Document the step-by-step process
of implementing the identified cybersecurity measures, including any
configurations, customizations, and testing procedures. Describe the operational
protocols and procedures established to maintain and monitor these
cybersecurity measures over time. For a cloud-based AI service used in
education for personalized learning, document the process of integrating identity
and access management (IAM) solutions, setting up role-based access controls,
and conducting regular security training sessions for system administrators.
5. Assessing and Reporting Cybersecurity Efficacy: Explain the methodologies
used to assess the effectiveness of the cybersecurity measures, including both
automated systems and manual review processes. Also highlight the importance
of periodic reviews and the testing process to ensure ongoing protection against
new and evolving cyber threats. For instance, detail how a machine learning
model for fraud detection undergoes monthly security assessments to adjust to
new fraudulent techniques.
6. Incident Response and Recovery Plans: Outline the AI system's response
strategy for handling potential cybersecurity incidents, detailing roles,
responsibilities, and action steps. Document the procedures in place to restore
the system's integrity and functionality following a cybersecurity incident. For
 example, for an AI-powered health monitoring system, outline an incident
response plan that includes immediate data isolation, notification of healthcare
stakeholders, and a switch to manual monitoring protocols. Recovery procedures
might include a detailed forensic analysis to understand the breach, followed by
a system-wide security audit and the implementation of recommended security
enhancements.

Part 3: Monitoring, Functioning and Control of AI

system

3.1 AI System Performance

Technical documentation for your AI system requires a detailed exposition of the

system's performance capabilities, limitations, and anticipated accuracy. Let's sive into
how to effectively document these aspects to meet regulatory standards.

Document Performance Capabilities

Clearly articulate the primary functions and operational strengths of the AI system.
Describe what the system is designed to achieve, including its key performance
objectives and any innovative features that distinguish it from other solutions.

Include quantifiable metrics such as success rates, response times, and the volume of
data the system can process within a given timeframe. These metrics should be based
on results from testing phases or actual deployments.

The AI chatbot is designed to handle up to 10,000 simultaneous customer inquiries,

reducing response time from an average of 10 minutes to under 30 seconds. Key
performance objectives include resolving common customer issues autonomously and
escalating complex cases to human agents. During its pilot deployment, the chatbot
successfully resolved 85% of inquiries without human intervention, demonstrating a
significant improvement in operational efficiency. It has been trained to recognize and
respond to over 500 different customer queries related to account management, billing,
and technical support.

Detail Performance Limitations

Provide an honest assessment of the areas where the AI system's performance may fall
short of expectations. This could involve accuracy variances across different
demographics, environmental conditions, or specific operational scenarios.

Support these assessments with statistical data that highlights the performance
discrepancies. Detail the conditions under which these limitations become apparent
and discuss any steps taken to address them.

Despite the chatbot's robust design, it exhibits a decrease in resolution efficiency for
inquiries requiring complex, multi-step problem-solving, managing these with a 70%
success rate. Furthermore, its performance slightly diminishes during peak traffic hours,
noted by a 10% increase in response time. Analysis reveals that the chatbot's
performance dips to 65% efficiency when handling simultaneously received inquiries
beyond its optimal capacity of 10,000, particularly on days coinciding with promotional
events. Continuous monitoring is in place to identify patterns of such occurrences and
implement scaling solutions.

Expected Level of Accuracy

Offer a projection of the overall expected accuracy of the AI system in relation to its
intended purpose. This should be based on comprehensive testing, including validation
and field tests.

Explain the methodologies used to estimate expected accuracy, noting any

assumptions or external variables that could affect these projections. Discuss how the
system's accuracy is anticipated to improve with ongoing training and updates.

Based on comprehensive testing, including scenario-based simulations and real-world

deployments, the chatbot is projected to maintain an overall accuracy rate of 88% in
query resolution. This projection accounts for its current capabilities and the typical
variation in customer inquiry complexity. This accuracy projection derives from
aggregating performance metrics across diverse operational scenarios, from standard
inquiries to complex problem-solving tasks. Anticipated improvements in natural
language processing algorithms and an expanded knowledge base are expected to
elevate the accuracy rate to over 90% within the next 12 months, with particular emphasis
on enhancing the system's performance during peak traffic periods.

3.2 Foreseeable Unintended Outcomes and Risk Sources

This lecture focuses on how to anticipate, identify, and document foreseeable

unintended outcomes and sources of risks associated with AI systems.

Identifying Risks

The first step in risk documentation involves a comprehensive identification process.

Here's how you can approach it:

A) Systematic Analysis: Begin with a systematic analysis of your AI system's

components and functionalities. Consider scenarios where each component might not
work as intended or might interact in unexpected ways with others.

B)User Interaction: Think about how users interact with your AI system.
Misinterpretations or misuse can lead to unintended consequences, impacting health,
safety, or even infringing on privacy rights.

C) Historical Data Review: Look into historical data and case studies of similar AI
applications. Learning from past incidents can provide insight into potential risks your
system might face.

D) Expert Consultation: Engage with domain experts and ethicists to gain a broader
perspective on possible unintended outcomes. Their expertise can uncover risks that
might not be immediately apparent.
Documenting Risks

Once risks have been identified, documenting them involves several key elements. For
each identified risk, provide a detailed description. This should include how the risk
manifests, under what conditions, and its potential impact on users and stakeholders.
Assess and document the likelihood of each risk occurring and the severity of its
impact. This helps prioritize risk mitigation efforts based on their potential to cause
harm. For every risk identified, propose mitigation strategies. Document the steps you
plan to take to minimize these risks, including algorithm adjustments, user education,
and safety features.

Discrimination and Fundamental Rights

A critical aspect of risk documentation focuses on discrimination and the violation of

fundamental rights. Suppose an AI system designed for job applicant screening tends
to favor candidates from a specific demographic background, thus discriminating
against others. An analysis might reveal that the algorithm's training data predominantly
represents successful candidates from that demographic, skewing its decision-making
process. Documenting this bias, its impact assessment, and the implementation of
corrective measures, such as diversifying the training dataset or adjusting the algorithm,
is vital.

3.3 Input Data

This section of documentation should contain a detailed description of input data for
deployed AI models. As you compile your input data specifications, focus on clarity,
accuracy, and thoroughness.

Begin by enumerating the specific types of data your AI system processes. For
instance, a content recommendation AI might need user behavioral data, textual
descriptions, and metadata about content items. Be explicit about the nature of each
data type — whether numerical, textual, visual, or auditory.

Clearly indicate the formats your AI model accepts for each data type. If your system
processes images, specify acceptable formats (e.g., JPEG, PNG). For textual data, detail
whether your model requires plain text, JSON, or HTML formats.

Document the quality parameters that input data must meet. This could include
resolution for images, length or cleanliness of text, or sampling rate for audio files.
Establishing these standards ensures that the data fed into the AI system is of sufficient
quality to produce reliable results.

Describe the scope of data coverage required. Ensure that the input data encompasses
a comprehensive range of examples that reflect the variety of scenarios the AI will
encounter. For a financial fraud detection AI, this means including transactions of
different types, sizes, and originating from various channels.

Outline the protocols for input data validation. This section should detail methods for
identifying and handling missing values, outliers, or incorrect data entries. Explain how
you ensure the data's integrity before it's processed by the AI system.

It's essential to document the origins of your input data. Provide information on data
sourcing, including any partnerships with data providers or use of public datasets.
Mention any licenses or permissions required to use the data.

Address the steps taken to detect and mitigate biases in the input [Link] includes
detailing the continuous monitoring practices that ensure input data remains free of
bias over time. Also, describe the dynamic adjustments made to the AI's processing
logic when biases are detected, allowing for real-time corrections. Finally, highlight the
establishment of feedback loops that enable users to report observed biases in the AI's
outputs, which are then used to further refine and adjust the handling of input data.