Unit I MLOPS FUNDAMENTALS

Fundamentals of an MLOps Workflow

Machine learning (ML) is progressing from research to practical business applications.

However, according to [Link], only 2% of companies using ML have successfully
deployed a model in production to improve their business processes.

Why is this so difficult? What steps can be taken to improve the situation?
MLOps is the evolution and intersection of software development and ML in order to
understand these challenges and solutions.

THE EVOLUTION OF INFRASTRUCTURE AND SOFTWARE DEVELOPMENT

Around 1995, with the advent of the modern internet age, there was a significant rise in
software applications. This period saw the introduction of operating systems like Windows 95
and Linux and websites such as Google and Amazon, which have been online for over two
decades. This growth promoted a culture of continuously improving services by collecting,
storing, and processing vast amounts of user interaction data. These advancements have greatly
influenced the evolution of IT infrastructure and software development.

Since the early 2000s, the transformation of IT infrastructure has accelerated.

Businesses have increasingly adopted cloud computing, allowing them to outsource IT
infrastructure maintenance while providing essential resources like storage and computing
services to run and scale their operations.

Cloud computing offers on-demand provisioning and availability of IT resources, such

as data storage and computing power, without requiring active management by the user. For
example, businesses that use cloud services for computational and storage resources do not
need to manage these resources directly, as the cloud service provider handles the maintenance.

The benefits of cloud computing include:

• No need to purchase and maintain IT resources, avoiding the capital expenditure of
buying hardware and its maintenance costs.
• Reduced need for in-house IT expertise in IT resource maintenance.
• Cost and resource optimization, as businesses only pay for the resources they use.
• Scalability on demand, allowing businesses to scale their operations up or down as
needed.

As a result, many companies have adopted cloud computing as a core part of their business and
IT infrastructure.

Cloud Computing Evolution

Cloud computing gained significant industry traction in 2006. Sun Microsystems

launched Sun Grid, a hardware and data resource sharing service later acquired by Oracle and
renamed Sun Cloud. That same year, Amazon introduced Elastic Compute Cloud (EC2),
revolutionizing how businesses provision computation, storage, and scaling capabilities on
demand. This marked the beginning of widespread cloud adoption across various industries.
 Over the past decade, global and regional companies have accelerated the shift toward
cloud computing. Major players such as Google, IBM, Microsoft, UpCloud, and Alibaba have
heavily invested in research and development of cloud services. This transition from localized
computing, where companies maintained their own servers and data centers, to on-demand
cloud computing has been driven by the availability of robust and scalable cloud services.
Businesses and organizations can now efficiently provision resources on-demand to meet their
data processing needs. These advancements align with Moore’s Law, which suggests that the
number of transistors on a microchip doubles approximately every two years, leading to a
reduction in computer costs. This trend has generally held true, reflecting ongoing
improvements in computing technology.

The Rise of Machine Learning and Deep Learning

In the last decade, machine learning (ML) has become common in everyday
applications. Beyond specialized uses like Dota or AlphaGo, ML is now widely used in areas
such as machine translation, image processing, and voice recognition. This broad adoption
highlights the transformative impact of ML and deep learning on daily life and business
operations.

The adoption of ML and deep learning has been driven by advancements in

infrastructure, particularly the use of computational power. These developments have unlocked
the potential of deep learning, as shown in Figure 1.1. Deep learning breakthroughs are closely
linked to the exponential growth in computing power, which increases approximately 35 times
every 18 months. However, as demands continue to grow, there may be limits to scaling up
central computing resources like CPUs, GPUs, or TPUs. This challenge has led to exploring
alternatives like distributed learning, where data processing is spread across multiple nodes.
Distributed learning methods, including federated learning and edge computing, have shown
promise in meeting the growing demands of deep learning. These approaches offer scalable
solutions to address the limits of central computing resources and continue to drive
advancements in the field.

The End of Moore’s Law

Before 2012, advancements in artificial intelligence (AI) followed Moore’s Law, which
predicted that compute power would double approximately every two years. However, after
2012, compute power has been doubling approximately every 3.4 months.
Figure 1.1 – Demand for Deep Learning and High-Performance Computing (HPC)

As shown in Figure 1.1, the demand for deep learning and HPC has been increasing
exponentially, with computing power growing around 35 times every 18 months. This rate of
growth has outpaced Moore’s Law, which traditionally predicted a doubling every 18 months.
Moore’s Law remains applicable to CPUs (single-core performance) but does not hold true for
newer hardware architectures such as GPUs and TPUs. This discrepancy renders Moore’s Law
obsolete in the context of current computing demands and trends.

AI-CENTRIC APPLICATIONS

Applications across various industries are increasingly becoming AI-centric. Virtually

every application is now incorporating AI and is often running on distributed workloads such
as HPC, microservices, and big data systems.

By combining High-Performance Computing (HPC) and AI, we can leverage the

computational power required to train deep learning and ML models. The integration of big
data with AI enables the extraction and processing of large-scale data for AI model training.
Additionally, combining microservices with AI allows for the efficient serving of AI models
for inference, thereby enhancing business operations and impact. This synergy has led to the
rise of distributed applications as the new norm. Developing AI-centric applications at scale
necessitates a coordinated approach involving distributed applications (HPC, microservices,
and big data), which in turn requires a new paradigm in software development.

Applications across various industries are increasingly AI-centric. Almost every

application now incorporates AI and often runs on distributed workloads such as HPC,
microservices, and big data systems.

By combining High-Performance Computing (HPC) and AI, the computational power

needed to train deep learning and ML models can be leveraged. Integrating big data with AI
enables the extraction and processing of large-scale data for AI model training. Additionally,
combining microservices with AI allows for the efficient serving of AI models for inference,
thereby enhancing business operations and impact. This synergy has led to the rise of
distributed applications as the new norm. Developing AI-centric applications at scale requires
a coordinated approach involving distributed applications (HPC, microservices, and big data),
necessitating a new paradigm in software development.

Figure 1.3 – Software Development Evolution

Waterfall Method

The Waterfall method, used since the onset of the internet age (1995), is a non-iterative
approach to software development. It follows a sequential process where each stage is pre-
organized and executed in order, starting from requirements gathering to software design,
development, and testing. This method is suitable when requirements are well-defined,
specific, and unlikely to change over time. However, it is not well-suited for dynamic projects
where requirements evolve. The major disadvantages of the Waterfall method include:

• The entire set of requirements must be defined before development begins.

• Modifications to requirements during or after development are not feasible.

Disadvantages of the Waterfall Method

• Limited Reusability: There are fewer opportunities to create or implement reusable

components.
• Testing Constraints: Testing occurs only after development is completed. It is not
iterative; once a stage is finished, it is not possible to revisit and fix issues.
• Impact on Delivery: Customer acceptance tests often introduce changes, causing
delays in delivery and increased costs. This can negatively affect the project timeline
and budget.
• User-Centric Issues: Systems are typically provisioned based on the developer’s
understanding, which may not align with user needs and can fall short in meeting them.
The Agile Method

The Agile method offers an iterative and progressive approach to software development,
contrasting with the Waterfall method. Agile emphasizes precision and user-centricity,
incorporating end users or customers throughout the development and testing phases. This
iterative feedback loop allows for continuous improvement. The Agile method has several
advantages over Waterfall:

• Flexible Requirements: Requirements are defined before development begins but can
be modified at any stage.
• Reusable Components: It facilitates the creation and implementation of reusable
components.
• Modular Solutions: Projects can be modularized, with different modules delivered
periodically.
• User Involvement: Users or customers can participate in testing and evaluating
developed modules, ensuring that business needs are met. This user-centric approach
enhances the quality and relevance of the final product.

The DevOps Method

The DevOps method builds on Agile development practices by enhancing the speed and
efficiency of software development throughout the build, test, deploy, and delivery stages. Key
components of DevOps include:

• Continuous Integration (CI): Regularly integrating code changes into a shared

repository, allowing for early detection of issues.
• Continuous Deployment (CD): Automating the deployment of code changes to
production environments, ensuring that updates are released swiftly and reliably.
• Continuous Delivery: Ensuring that software can be released to production at any time
with minimal manual intervention.

DevOps fosters collaboration between software developers and IT operations teams,

promoting integration and automation to enhance:

• Efficiency: Streamlining the development process to reduce time and effort.

 • Speed: Accelerating the release cycle to deliver updates faster.
• Quality: Improving the quality of software through continuous testing and feedback.

Overall, DevOps facilitates rapid and consistent software releases, enabling organizations
to deploy and maintain software in production within minutes.

• Flexible Requirements: Requirements are defined before development begins but can
be modified at any stage.
• Reusable Components: It facilitates the creation and implementation of reusable
components.
• Modular Solutions: Projects can be modularized, with different modules delivered
periodically.
• User Involvement: Users or customers can participate in testing and evaluating
developed modules, ensuring that business needs are met. This user-centric approach
enhances the quality and relevance of the final product.

TRADITIONAL SOFTWARE DEVELOPMENT CHALLENGES

In the previous section, the evolution of software development from the traditional
waterfall model to Agile and DevOps practices was explored. Despite the advancements of
these methods, they do not fully apply to machine learning (ML) applications.

Understanding why involves recognizing the significant differences between ML and

traditional software development. Unlike conventional software development, which focuses
primarily on code, ML involves both code and data. This distinction matters for the following
reasons:

• Data is Fundamental: In ML, data is a core component of the model. The quality and
quantity of data directly influence the model's performance and accuracy.
• Code + Data: The code in ML processes and fits the data, enabling the model to derive
valuable insights. This integration of data with code introduces unique challenges that
traditional methods may not effectively address.
 Therefore, while Agile and DevOps practices have greatly advanced traditional software
development, ML requires specialized approaches to handle the interplay between data and
code.

Bridging Code and Data in ML Development

The critical relationship between code and data in ML necessitates systematic integration
and evolution of both elements to achieve a robust and scalable ML system. As data for
training, testing, and inference changes over time and across different sources, it must be
matched with evolving code. Without a systematic MLOps approach, inconsistencies between
code and data can lead to several issues:

• Production Problems: Divergence in code and data evolution can cause problems in
production environments.
• Deployment Challenges: Inconsistent code and data make deployment difficult.
• Reproducibility Issues: Results become hard to trace and reproduce if code and data
are not aligned.

MLOps addresses these challenges by streamlining the development, deployment, and

monitoring pipeline for ML applications. It ensures that contributions from various teams are
unified and that every step in the process is recorded and repeatable. The following sections
will explore how MLOps facilitates collaboration between data science and IT teams, leading
to the development and maintenance of effective ML systems.

Trends of ML Adoption in Software Development

Before discussing MLOps methods and workflows, it is useful to understand the broader
trends in ML adoption within software development:

• Increasing Integration: ML is increasingly being integrated into software

development, driven by several key factors:
o Investments: In 2019, global investments in AI surpassed $70 billion,
including:
▪ Over $37 billion in start-up investments
▪ $34 billion in mergers and acquisitions (M&A)
▪ $5 billion in initial public offerings (IPOs)
▪ Around $2 billion in minority stakes
o Market Growth: AI's market value reached $9.5 billion in 2018 and is
projected to grow to $118 billion by 2025. The economic impact of AI is
expected to be substantial by 2030.
o Regional Funding Distribution: The US receives approximately 50% of
global venture capital (VC) funding, China about 39%, and Europe around 11%.
 These trends highlight the significant and growing role of AI and ML in software
development, emphasizing the need for effective MLOps practices to manage and leverage
these advancements.

TRENDS DRIVING ML ADOPTION IN SOFTWARE DEVELOPMENT

Several key trends are influencing the integration of ML into software development:

• Big Data: The volume, velocity, veracity, and variety of data are growing
exponentially. For example, in Europe, data volume is increasing at a rate of 61% per
year, and it is projected that four times more data will be created by 2025 than exists
today. Data is an essential raw material for AI development.
• Infrastructural Developments and Adoption: Moore's law, which predicted the
doubling of compute power every two years, was closely followed until 2012. Since
then, compute power has been doubling approximately every 3.4 months.
• Increasing Research and Development: AI research has seen significant growth in
both quality and quantity. The volume of peer-reviewed AI papers increased by 300%
from 1998 to 2018, accounting for 9% of conference papers and 3% of journal
publications.
• Industry Adoption: According to a survey, 47% of large companies had adopted AI
in at least one function or business unit. This figure rose to 58% in 2019 and is expected
to continue increasing.

FUNDAMENTALS OF MLOPS WORKFLOW

Understanding MLOps

Software development is multidisciplinary and is evolving to accommodate machine

learning (ML). MLOps is a new approach that merges ML with software development by
integrating various domains. MLOps combines ML, DevOps, and data engineering to build,
deploy, and maintain ML systems in production reliably and efficiently. This fusion is best
understood through the intersection of these fields.

Fig.1.1 MLOps Intersection

 To make this intersection (MLOps) operational, Wieringa proposed a workflow to bring
together Data Engineering, Machine Learning, and DevOps. Design science applies design to
problems and context. It involves designing and investigating artifacts in a context. In this case,
the artifact is the MLOps workflow, which is designed iteratively by interacting with problem
contexts (industry use cases for the application of AI).

In a structured and iterative approach, two cycles (the design cycle and the empirical
cycle) are implemented for qualitative and quantitative analysis of the MLOps workflow design
through iterations. As a result of these cycles, an MLOps workflow is developed and validated
by applying it to multiple problem contexts. These contexts include tens of ML use cases (for
example, anomaly detection, real-time trading, predictive maintenance, recommender systems,
virtual assistants, etc.) across multiple industries (for example, finance, manufacturing,
healthcare, retail, automotive, energy, etc.). This MLOps workflow has been applied and
validated successfully in various projects across multiple industries to operationalize ML.

CONCEPTS AND WORKFLOW OF MLOPS

MLOps Workflow Overview

The MLOps workflow has evolved through numerous design iterations, integrating data
engineering, machine learning (ML), and DevOps in a streamlined manner. This workflow,
illustrated in Figure 1.10, is both modular and flexible, suitable for building proofs of concept
or operationalizing ML solutions in various industries.
The workflow is divided into two main modules:

• MLOps Pipeline (Build, Deploy, Monitor): Upper layer

• Drivers (Data, Code, Artifacts, Middleware, Infrastructure): Mid and lower layers

The MLOps pipeline is powered by an array of services, drivers, middleware, and

infrastructure, facilitating the creation of ML-driven solutions. This pipeline enables rapid
prototyping, testing, validation, and large-scale, efficient deployment of models.

MLOPS PIPELINE STAGES

Build

The build Stage comprises 5 modules mainly, Data ingestion, Model training Model
testing, Model Packaging and Model registering.

1. Data Ingestion:
o Function: Initiates the ML pipeline by managing data characteristics like
volume, velocity, veracity, and variety. Data is extracted from sources such as
databases, data warehouses, or data lakes.
o Process: Establishes robust data pipelines for Extract, Transform, and Load
(ETL) operations, preparing data for ML training, splitting it into training and
testing sets, and enabling data versioning.
o Use Case: Extracts 100,000 labeled images of cats and dogs from a data lake,
splits them into training (80%) and testing (20%) sets, and versions the data
for traceability.
2. Model Training:
o Data Preprocessing: Transforms raw data into a suitable format for the
model, involving tasks like resizing images and normalizing pixel values.
o Feature Engineering: Creates new features from existing data to improve
model performance, such as extracting edges or shapes from images.
o Feature Scaling: Ensures all features are on a common scale for equal
contribution during training.
o Model Training with Hyperparameter Tuning: Trains the model and
optimizes its configuration settings for best performance.
3. Model Testing:
o Evaluation on Test Data: Uses a separate test dataset to assess model
performance using metrics like precision and recall.
o Decision and Iteration: If results are satisfactory, proceed; if not, refine
previous stages.
4. Model Packaging: Packages the model for deployment, often using serialization or
containerization tools like Docker.
 5. Model Registering:
o Function: Stores the trained model in a central repository, including files like
vectorizers, model weights, and serialized models.
o Benefits: Centralized storage, versioning, and collaboration.
o Use Case: Registers the serialized model in a model registry for easy
deployment in the production environment.
Deploy

The two phases of Deploy are Application Testing & Production Release

1. Application Testing:
o Purpose: Tests the trained model's performance in a simulated production
environment.
o Process:
▪ Deployment: Deploys the model as an API or streaming service in a
test environment.
▪ Data Acquisition: Uses test data to simulate real-world predictions.
▪ Model Inference: Assesses model performance on test data.
▪ Performance Review: Analyzes performance results.
▪ Decision: Approves model for production if performance standards are
met.
o Use Case: Deploys the model as an API service connected to a CCTV camera
for real-time video data testing in a pet park.
2. Production Release:
o Function: Deploys the model into the production environment for real-world
predictions.
o Process:
▪ Model Deployment: Approves and deploys the model to production
servers or infrastructure.
▪ Model Inference: Makes predictions on real-world data.
▪ CI/CD Pipeline Integration: Automates deployment tasks through
Continuous Integration and Continuous Delivery pipelines.
o Use Case: Deploys the model as an API service to classify cats and dogs in
real-time from a CCTV camera, automated by the CI/CD pipeline.
Monitoring

The Three phases of Monitoring Stage is Monitor, Analysis and Govern.

1. Monitor:
o Function: Continuously monitors the deployed model's performance and
health.
o Process:
▪ Data Integrity and Model Drift: Ensures data integrity and tracks
model performance degradation over time.
▪ Application Performance: Monitors system health using telemetry
data.
▪ Analysis and Governance: Analyzes data to identify issues and
initiates corrective actions.
o Use Case: Monitors data integrity, model drift, and application performance,
triggering actions like retraining if performance falls below a threshold.
2. Analysis:
o Importance: Ensures optimal results and alignment with business goals.
o Techniques: Uses explainability techniques to understand model predictions.
o Focus Areas: Fairness, trust, bias, transparency, and error analysis.
o Model Drift: Monitors and addresses performance degradation over time.
o Course of Action: Alerts stakeholders and initiates model updates or
retraining.
o Use Case: Regularly monitors performance metrics and uses explainability
techniques to understand model classifications, initiating retraining if needed.
3. Govern:
o Function: Ensures the model performs well, meets business needs, and
adheres to regulations.
o Process:
▪ Monitoring and Analysis: Continuously monitors and analyzes
performance metrics.
▪ Alerting and Actions: Triggers alerts and corrective actions if issues
are detected.
▪ Compliance: Ensures regulatory compliance and documents model
lineage.
o Use Case: Regularly monitors performance metrics, alerts stakeholders if
issues arise, and retrains the model to maintain fairness and accuracy in
classifying cats and dogs.
MLOPS DRIVERS

The essential components for the efficient functioning of an MLOps pipeline include:

• Data:
o Formats: Data may be structured (e.g., tables) or unstructured (e.g., text,
images, video).
o Management: Data management in ML involves stages such as acquisition,
annotation, cataloging, preparation, quality checking, sampling, and
augmentation.
o Segmentation and Versioning: Training, testing, and production data
(including inputs, outputs, and telemetry) need to be segmented and versioned
for effective pipeline operation.
• Code:
o Types: The MLOps pipeline is driven by three key code modules: training
code, testing code, and application code.
o Management: A source code management system (e.g., Git) is essential for
version control, orchestration, and integration with CI/CD and data pipelines.
• Artifacts:
o Types: Artifacts include data, serialized models, code snippets, system logs,
and training/testing metrics, all vital for the pipeline's success.
o Management: Middleware services (e.g., model registry, workspaces,
logging) handle these artifacts to ensure traceability and sustainability.
• Middleware:
o Function: Middleware services automate and orchestrate processes within the
MLOps pipeline.
o Examples: Includes Git (for source code management), VNets (for network
configuration), Docker (for containerization), and Kubernetes (for container
orchestration).
• Infrastructure:
o Compute Resources: Required for training, deploying, and monitoring
machine learning models.
o Storage Resources:
▪ Central Storage: Used for storing logs, artifacts, and
training/testing/monitoring data.
▪ Feature Store (Optional): Stores features that have been extracted,
transformed, and are ready for training or inference.
 o Deployment Options: Resources may be on-premises or cloud-based,
utilizing Infrastructure as a Service (IaaS) provided by various cloud
providers.

CHARACTERIZING YOUR MACHINE LEARNING PROBLEM

ML Operations (MLOps) aims to bridge the gap between academia and industry by
using advanced engineering principles. To gain a comprehensive understanding and awareness
of the possibilities, different elements from both academia and industry will be explored.

The ML Solution Development Process

Machine Learning (ML) offers numerous opportunities to enhance and automate

business operations. To maximize the benefits of ML, teams and individuals involved in ML-
driven business transformation need to understand both ML and the business itself. Effective
business transformation starts with a basic understanding of the business, including aspects
such as value-chain analysis, use-case identification, data mapping, and business simulations
for validating the transformation.

Figure 2.1 – ML solution development process

Figure 2.1 presents a process for developing ML solutions to enhance or automate

business operations. The process begins with business understanding. Once a solid grasp of the
business is achieved, the next step involves data analysis. This includes acquiring, versioning,
and storing the appropriate data. Data is then utilized for ML modeling through data pipelines,
where feature engineering is performed to obtain the features needed for training the model.
The trained models are evaluated and packaged for deployment. Deployment and monitoring
are managed using a pipeline that leverages Continuous Integration/Continuous Deployment
(CI/CD) features, enabling real-time and continuous deployment of trained ML models to
users. This process ensures the development of robust and scalable ML solutions.

TYPES OF ML MODELS

A variety of ML and deep learning models are available to address the same business
problem. Understanding the landscape of ML models is crucial for selecting the most efficient
algorithm. There are approximately 15 types of ML techniques, categorized into four main
groups: learning models, hybrid models, statistical models, and Human-In-The-Loop (HITL)
models, as illustrated in Figure 2.2. It is important to note that there are other ways to categorize
ML models, and no single categorization is entirely complete. These categorizations are
appropriate for some scenarios but not for others. The recommended categorization for
examining ML models includes:

Learning Models

Two types of standard learning models are considered: supervised learning and
unsupervised learning.
Supervised Learning

Supervised learning models or algorithms are trained using labeled data. In this
approach, the result of the input is known or marked in the training data. A model is trained to
predict outcomes based on the labeled data it learns from, with the system being informed about
which output corresponds to a given input.

Supervised learning models are effective for narrow AI cases and well-defined tasks
but require sufficient and comprehensive labeled data. Figure 2.3 illustrates how a supervised
learning model learns to predict and classify an input. For example, an image classification
model is trained on labeled data of thousands of correctly labeled images of cats and dogs. The
trained model then learns to classify a new image as either a cat or a dog.

Unsupervised Learning

Unsupervised learning does not involve a machine operating without human

supervision. Unsupervised learning models or algorithms learn from unlabeled data. These
models can be used to extract insights and identify patterns from such data. Unsupervised
algorithms are commonly used for clustering or anomaly detection without relying on labels.
They can identify patterns and use them to label new, unlabeled data inputs.

Unsupervised learning is primarily used for analytics, though it can also be applied to
automation and ML. These algorithms are not recommended for production use due to their
dynamic nature, which can result in changing outputs with each training cycle. However, they
can be useful for automating processes like segmenting incoming data or identifying anomalies
in real-time.

For example, consider clustering news articles into relevant groups. If thousands of
news articles are available without labels, unsupervised learning can be used to identify article
types or categories. By inputting these articles into the algorithm, they can be clustered into
groups such as politics, sports, science, and health. This approach helps in discovering patterns
within the data.

Hybrid Models in Machine Learning

Recent advancements in machine learning (ML) have led to the development of hybrid
models that combine traditional methods to address a variety of business and research
challenges. These hybrid models integrate different learning paradigms to leverage their
strengths and overcome their limitations. Below are some notable hybrid models and their
applications:

Figure 2.4 – Types of hybrid models)

Semi-supervised Learning

Semi-supervised learning combines supervised and unsupervised learning techniques,

particularly useful when only a few data samples are labeled while many remain unlabeled.
This approach allows for efficient utilization of available data, including unlabeled samples.
For instance, in the task of classifying text documents, it is often impractical to manually label
a large corpus of documents. Semi-supervised learning addresses this issue by enabling the
algorithm to learn from a limited set of labeled documents and generalize its learning to classify
a much larger set of unlabeled documents. This approach is ideal in scenarios where labeling
is costly or time-consuming.

Self-supervised Learning

Self-supervised learning falls within the realm of unsupervised learning but transforms
problems into a supervised learning framework. This method is employed to create supervisory
signals from the data itself, allowing algorithms to train without explicit labels. Self-supervised
algorithms often solve auxiliary tasks that generate supervisory signals. An example is
Generative Adversarial Networks (GANs), which can generate synthetic data by learning from
both labeled and unlabeled data. GANs, once trained, can produce outputs such as human faces
from text descriptions (e.g., "male, 30 years old, brown hair"). This self-supervised approach
facilitates the creation of high-quality synthetic data that can be used in various applications.

Multi-instance Learning

In multi-instance learning, data is labeled not by individual samples but by collections

or categories of samples. Unlike traditional supervised learning, where each data point is
labeled, multi-instance learning assigns labels to sets of instances. For example, in categorizing
news articles, rather than labeling each article individually, groups of articles are labeled by
their broader category (e.g., politics, science, sports). Supervised learning algorithms then use
these categorical labels to make predictions. This method is particularly useful in situations
where individual data labeling is impractical or where data samples naturally occur in groups.

Multitask Learning

Multitask learning involves training a model on a single dataset to perform multiple

tasks simultaneously. This approach leverages shared representations and improves the model's
generalization capabilities across different tasks. For example, in natural language processing,
models like Bidirectional Encoder Representations from Transformers (BERT) are pre-trained
on large text corpora and then fine-tuned for various tasks such as text classification, keyword
extraction, and sentiment analysis. By training on a comprehensive dataset, multitask learning
models can efficiently handle diverse linguistic tasks.

Reinforcement Learning

Reinforcement learning involves training an agent to perform tasks within a defined

environment through sequential decision-making. The agent learns from feedback and rewards
received from the environment, which helps in shaping its behavior and decision-making
process. For example, Google's AlphaGo employed reinforcement learning to outperform the
world's top Go player. Through 40 days of self-training using feedback and rewards, AlphaGo
mastered the game and defeated the best human player. This type of learning is crucial for tasks
where the agent must make a series of decisions and learn from the consequences of its actions.

Ensemble Learning

Ensemble learning combines multiple models to make predictions, improving overall accuracy
and robustness. In this approach, each model is trained on the same data, and their predictions
are aggregated to produce a final result. For example, the random forest algorithm uses an
ensemble of decision trees to perform classification or regression tasks. Each decision tree
makes an individual prediction, and the final prediction is determined by averaging the
predictions of all the trees. This method enhances performance by leveraging the diverse
strengths of multiple models.

Transfer Learning

Transfer learning involves using knowledge gained from one model to improve another
model's performance on a different but related task. This approach is prevalent in deep learning,
where pre-trained models are adapted to new tasks. For instance, a pre-trained BERT model,
originally trained on a vast corpus of text, can be fine-tuned for specific tasks such as sentiment
analysis or text classification. Transfer learning allows models to benefit from prior knowledge,
reducing the need for extensive training data and computational resources.

Federated Learning

Federated learning is a collaborative ML approach that trains models across multiple

devices while keeping data localized. Instead of aggregating data from different sources,
federated learning shares model updates to create a global model. This method maintains data
privacy, as illustrated by its use in healthcare where patient data remains confidential. Local
models are trained on individual devices, and the updates are combined to refine the global
model. This approach ensures that sensitive data is not exposed while benefiting from
collective learning.

Statistical Models

Statistical models offer different methods for data analysis and prediction. The main
types include:
 Figure 2.7 - Relationship between the three types of statistical models

• Inductive Learning: This method generalizes from specific examples to derive general
rules. For instance, fitting a machine learning model involves creating a general
function from observed data points.
• Deductive Learning: Deductive learning applies general rules to specific cases,
leading to deterministic outcomes. It is the reverse of inductive learning, where specific
observations are generalized.
• Transductive Learning: This method makes predictions based on specific training
samples. Unlike inductive learning, which generalizes predictions, transductive
learning compares specific samples to predict outcomes. For example, the k-nearest
neighbors algorithm uses specific data points to predict results.

HITL Models

Human-in-the-loop (HITL) models include human-centered reinforcement learning and

active learning models. These models incorporate human feedback to enhance machine
learning processes.

• Human-Centered Reinforcement Learning: This model involves humans providing

feedback to an agent during its learning process, improving its ability to mimic human
behavior. Feedback from human experts shapes the agent's learning, making it effective
in scenarios where human-like behavior is required.
 Figure 2.8-Workflow of human-centered reinforcement learning
• Active Learning: In active learning, the trained model queries human users to resolve
uncertainties during the learning process. For example, a chatbot might ask users yes/no
questions to validate predictions.
These various hybrid models illustrate the range of possibilities for developing ML
solutions to address real-world challenges. To effectively implement MLOps, it is essential to
categorize and align them with specific business and technological needs. The next step
involves identifying the appropriate requirements, tools, methodologies, and infrastructure to
support both business goals and MLOps. The following section will focus on structuring
MLOps to meet these needs.

STRUCTURING YOUR MLOPS

The main aim of MLOps is to ensure efficient collaboration within an organization or
among individuals to build data and ML-driven assets, solving business problems effectively.
This enhances overall performance and transparency. Working in isolation or duplicating
functionalities can be very costly and time-consuming.

Figure 2.9 Categories of MLOps

 MLOps can be structured within organizations in 4 Categories. It is crucial to get the
MLOps process right. By selecting the appropriate process and tools for MLOps, a robust,
scalable, economical, and sustainable MLOps process can be implemented. For example, a
healthcare client experienced a 76% cost optimization in storage and compute resources after
optimizing their MLOps compared to traditional operations. The client's data scientists had
30% of their time freed from repetitive tasks like data wrangling, ML pipeline management,
and hyperparameter tuning. Efficient MLOps can assure your team of efficiency, high
performance, and excellent collaboration that is repeatable and traceable within your
organization.
MLOps can be categorized into small data ops, big data ops, large-scale MLOps, and
hybrid MLOps. This categorization is based on team size, ML applications, business models,
data scale, tools, and infrastructure.
Small Data Ops
A small start-up with data scientists building ML models for narrow and well-defined
problems can be agile and collaborative. Typically, models are trained locally on data scientists'
computers and then deployed on the cloud for inference. However, pitfalls such as a lack of
streamlined CI/CD approach for deploying models can occur. Central or distributed data
sources may be managed carefully by the team, and training code can be versioned and
maintained in a central repository. Scaling operations can lead to repeated work, working in
silos with minimal understanding of teammates' parallel work, and incurring higher-than-
expected costs. If the team consists only of data scientists, works with Python environments,
handles small data (<10 GB), and supports open-source technologies like PyTorch,
TensorFlow, and scikit-learn, the operations can be categorized as small data ops.
Big Data Ops
This involves a team of experienced data scientists and engineers in a start-up or SME
requiring large-scale big data processing for ML training or inference. Tools like Kafka, Spark,
or Hadoop are used for data pipelines, with high-powered processors like GPUs or TPUs. ML
model development is led by data scientists, with deployment managed by data/software
engineers. However, challenges such as a lack of traceability for model training and
monitoring, non-reproducible artifacts, and independent growth of code and data can arise. If
the team requires big data processing capacity, uses frameworks like Databricks for
collaboration, and conducts ML model development in the cloud, the operations can be
categorized as big data ops.
Hybrid MLOps
Hybrid teams include experienced data scientists, data engineers, and DevOps
engineers. They work with big data and open-source tools like PyTorch, TensorFlow, and
scikit-learn, requiring efficient collaboration. They often implement robust and scalable
software engineering practices. However, challenges such as high costs due to repeated work
and inefficient model monitoring and retraining mechanisms can occur. If the team has high
collaboration and big data processing requirements and supports open-source technologies for
any type of ML, the operations can be categorized as hybrid MLOps.
Large-Scale MLOps
Large-scale operations are common in big companies with large or medium-sized
engineering teams. They handle data operations on a big data scale, with various types of data
and multiple legacy systems. These teams are prone to high costs due to repeated work and
independent growth of code and data. In bureaucratic and highly regulated environments with
stringent processes and quality checks, the complexity and interdependence of systems become
critical factors. If the team handles large-scale inference and operations, manages ML models
across multiple resources, works with big or multiple teams, and handles multiple use cases
and models, the operations can be categorized as large-scale MLOps.
Categorizing MLOps according to business and technological needs allows for a more
efficient and robust implementation. For instance, a fintech start-up processing 0-1,000
transactions a day would require small-scale data ops, whereas a larger financial institution
would need large-scale MLOps. Once characterized, a solid implementation roadmap ensures
smooth development and deployment of a robust and scalable MLOps solution for the
organization.

AN IMPLEMENTATION ROADMAP FOR YOUR SOLUTION

This general implementation roadmap can facilitate MLOps for any ML problem in
detail. The goal of this roadmap is to solve the problem with the right solution. It gives an well-
defined method and clear milestones that ensures the successful delivery of the desired ML
solution using MLOps methods.

By following the preceding roadmap, the transition from ML development to MLOps

can be achieved with clear milestones, outlined in three phases of MLOps implementation.
These phases will be discussed in more detail. After this theoretical section, practical
implementation of the roadmap will be demonstrated through a real-world business use case.
Phase 1 - ML Development
This phase marks the beginning of implementing the MLOps framework for a problem.
Before starting, it is essential to clearly understand the problem and the solution. In this phase,
system requirements are considered to design and implement a robust and scalable MLOps
framework. The initial step involves selecting the appropriate tools and infrastructure needed,
such as storage and compute resources.
Once the infrastructure is set up, the necessary workspace and the development and test
environments should be provisioned to execute ML experiments, including training and testing.
ML models are trained in the development environment and tested for performance and
functionality using test data, depending on the workflow or requirement. When the
infrastructure is established and the first ML model is trained, tested, serialized, and packaged,
Phase 1 of the MLOps framework is set up and validated for robustness. Serializing and
containerizing are crucial processes to standardize and prepare the models for deployment.
Phase 2 - Transition to Operations
Phase 2 focuses on transitioning to operations, which involves serializing and
containerizing the models trained in Phase 1 to prepare them for deployment. This process
ensures standardized and efficient deployments. The models are served as APIs or independent
artifacts for batch inference. Once a model is packaged and ready to be served, it is deployed
in the production environment using streamlined CI/CD pipelines after passing quality
assurance checks. By the end of Phase 2, packaged models are served and deployed in the
production environment, performing inference in real time.
Phase 3 - Operations
Phase 3 is the core operations phase for the models deployed in Phase 2. In this phase,
the performance of the deployed models is monitored in terms of model drift, bias, and other
metrics. Based on the model's performance, continual learning is enabled through periodic
model retraining, and alerts and actions are implemented. Simultaneously, logs and telemetry
data for the production environment are monitored to detect and resolve any possible errors,
ensuring the uninterrupted operation of the production system. Data pipelines, the ML
platform, and security are managed continuously. Successful implementation of this phase
allows for the monitoring and retraining of deployed models in a robust, scalable, and secure
manner.
In most cases, all three phases need to be implemented for an ML solution. However,
in some cases, Phases 1 and 2 may be sufficient, such as when ML models perform batch
inferences and do not require real-time inference. By achieving these milestones and
implementing all three phases, a robust and scalable ML lifecycle for applications is established
systematically and sustainably.

Procuring Data, Requirements, and Tools

Implementing successful MLOps relies on procuring appropriate training data,
maintaining high standards, and having suitable requirements, tools, and infrastructure. This
section delves into these factors that contribute to robust and scalable MLOps.
Data
Understanding data involves more than just mastering tools like Python, SQL, and
regression. The context and domain matter at every stage, from data cleaning to modeling to
interpretation. The best tools in the world won’t fix a poorly defined problem or the lack of
one. Knowing what problem to solve is a context-driven and business-dependent decision.
Once the problem and its context are understood, the right training data needed to solve it can
be identified.
Training data is vital for ML systems and plays a more significant role compared to
traditional software systems. Both code and training data work in parallel to develop and
maintain an ML system. Ensuring the right data for algorithm training involves providing both
the right quantity and quality of data:
• Data Quantity: Often, data scientists argue that model performance is subpar due to
insufficient data, claiming that more data would improve performance. However, more
data might not always help as expected since quality is also a critical factor. Models
can learn more insights from data with more samples for each class. For example,
analyzing anomalous financial transactions with many samples reveals more types of
anomalies. Conversely, if there is only one anomalous case, ML is not very useful. The
data requirements for ML projects should focus on data diversity rather than just
quantity. However, constraints on data quantity may exist, such as limited time frames
for available data. The goal is to acquire data of maximum possible quantity and quality
to train the best-performing ML models.
 • Data Quality: Data quality significantly impacts model performance. The more
comprehensive and higher the quality of the data, the better the ML model will work.
Therefore, preprocessing stages—cleaning, augmenting, and scaling the data—are
crucial. Important dimensions of data quality include consistency, correctness, and
completeness:
o Data Consistency: Refers to the correspondence and coherence of the data
samples throughout the dataset.
o Data Correctness: Measures the degree of accuracy and the extent to which
the data reliably reflects real events, depending on how the data was collected.
o Data Completeness: Reflects the sparsity of data for each characteristic,
ensuring comprehensive coverage of possible values to reflect an event.
With an appropriate quantity of high-quality data, ML models and applications can
perform above the required standards, solving business problems efficiently.
Requirements
The product or business/tech problem owner plays a key role in facilitating the efficient
building of a robust ML system by identifying requirements and tailoring them concerning the
scope of data, collection of data, and required data formats. These requirements are vital inputs
for developers of ML systems, such as data scientists or ML engineers, to start architecting the
solution to address the problem by analyzing and correlating the given dataset based on the
requirements.
ML solution requirements should consist of comprehensive data requirements,
including information about data quality and quantity. Requirements can also include
estimations about anticipated or expected predictive performance, expressed in terms of
performance metrics determined during requirements analysis and elicitation.
Meticulous specifications, such as expected or anticipated performance on the training
data, can be rapidly validated after model training. Based on the training performance,
inference or runtime performance (including production and operations) can be assessed during
operations.
Requirements should also consider important factors such as ethical considerations and
explainability. Issues like discrimination or bias are critical for applications, and certain
properties should be preserved as part of data privacy. Explainability requirements must be
explicitly accounted for to explain the situations and decisions of the ML solution or system to
the users. Requirements must also stipulate regulations and restrictions concerning data use
and validation of decisions made by the ML system.

The table in Figure 2.11 illustrates the flow of the requirements characterization
process, from elicitation to analysis to specifications to verification and validation of the
system. This process ensures that the best-fit resources are procured to build and deploy an
efficient ML system to solve the problem. When requirements are well defined, selecting the
right tools and infrastructure to support the established process and ensure standards are met is
crucial.
Tools and Infrastructure
The MLOps landscape has been rapidly developing, with many tools and frameworks
evolving as part of the infrastructural offering. Many mainstream options have been developed
to orchestrate ML, deep learning, reinforcement learning, development environments, data
pipelines, model management, explainable AI, security, and distributed computing.
There is a surge in services provided by popular cloud service providers such as
Microsoft, AWS, and Google, complemented by data processing tools like Airflow,
Databricks, and Data Lake. These tools enable ML and deep learning with frameworks such as
scikit-learn, Spark MLlib, PyTorch, TensorFlow, MXNet, and CNTK. With many tools and
frameworks available, selecting the right ones depends on the context of the ML solution and
operations setup. Having the right tools ensures high efficiency and automation for the MLOps
workflow.
The options are numerous, but it is important to start somewhere to reach the sky.
Therefore, hands-on experience is provided in the next section through a real-life business
problem.