Introduction to fully
automated MLOps
F U L LY A U T O M AT E D M L O P S
Arturo Opsetmoen Amador
Senior Consultant - Machine Learning
MLOps in an industrial setting
What are common goals companies have when using machine learning?
Develop ML tools and products that use data to
Better serve customers
Optimize processes
FULLY AUTOMATED MLOPS
Optimizing for value generation
Companies aim to maximize profits
Machine learning can be used to increase
profitability
By analyzing costs and revenues, a
company can estimate potential profits
Deploying the right number of ML use-
cases can result in profits
FULLY AUTOMATED MLOPS
Costs in software development
Expected costs in a traditional software development project:
Development costs
Project management
UI/UX design
Quality assurance
FULLY AUTOMATED MLOPS
Technical debt in software development
Technical debt or design debt:
Cost of rework caused by poor design
1 [Link]
FULLY AUTOMATED MLOPS
Hidden technical debt in ML systems
Machine Learning: "The high-interest credit card of technical debt"[1].
Hidden technical debt can be related to:
1. The data used to train the ML models
2. The models powering the ML system
3. The infrastructure used by the ML system
4. The monitoring of the ML system
1 [Link]
FULLY AUTOMATED MLOPS
Costs of machine learning projects
Reduced profit due to technical debt in ML systems:
FULLY AUTOMATED MLOPS
The high-interest credit card of technical debt
ML systems can be complex and become unruly.
FULLY AUTOMATED MLOPS
MLOps: The best-known way to pay
If ML is the high-interest credit card of technical debt, MLOps is the best way to pay for it.
MLOps can include:
Automated testing
Automated experiment tracking
Automated monitoring
To keep the technical debt to a minimum
FULLY AUTOMATED MLOPS
Let's practice!
F U L LY A U T O M AT E D M L O P S
MLOps lifecycle
stages
F U L LY A U T O M AT E D M L O P S
Arturo Opsetmoen Amador
Senior Consultant - Machine Learning
The MLOps lifecycle
The MLOps lifecycle includes three core stages:
The three stages are iterative
The three stages are interconnected and rely on each other
It is normal to go back and forth between stages
FULLY AUTOMATED MLOPS
MLOps in the ML lifecycle - Design
The Design stage includes:
BUSINESS UNDERSTANDING
DATA UNDERSTANDING
DESIGNING THE ML SOLUTION
FULLY AUTOMATED MLOPS
MLOps in the ML lifecycle - Design
The Design stage includes:
BUSINESS UNDERSTANDING
Understanding the business context
Establishing the business goals
FULLY AUTOMATED MLOPS
MLOps in the ML lifecycle - Design
The Design stage includes:
DATA UNDERSTANDING
Data exploration
Data visualization
FULLY AUTOMATED MLOPS
MLOps in the ML lifecycle - Design
The Design stage includes:
SYSTEM DESIGN
System architecture design
Data security
Data privacy
FULLY AUTOMATED MLOPS
MLOps in the ML lifecycle - Design
The Design stage includes:
BUSINESS UNDERSTANDING
DATA UNDERSTANDING
DESIGNING THE ML SOLUTION
Much of the design phase can't be
automated
FULLY AUTOMATED MLOPS
MLOps in the ML lifecycle - Development
The ML experimentation and development
stage includes:
DEVELOPING PROOF-OF-CONCEPTS
(POCS)
DATA ENGINEERING
MODEL DEVELOPMENT
FULLY AUTOMATED MLOPS
MLOps in the ML lifecycle - Development
The ML experimentation and development
stage includes:
DEVELOPING PROOF-OF-CONCEPTS
(POCS)
Combination of process & automation
Use of frameworks
FULLY AUTOMATED MLOPS
MLOps in the ML lifecycle - Development
The ML experimentation and development
stage includes:
DATA ENGINEERING (POCS)
Use of automation --> Quality assurance
FULLY AUTOMATED MLOPS
MLOps in the ML lifecycle - Development
The ML experimentation and development
stage includes:
MODEL DEVELOPMENT (POCS)
Experiment tracking
Automated training
Automated hyperparameter tuning
FULLY AUTOMATED MLOPS
MLOps in the ML lifecycle - Deployment
The ML deployment and operations phase
include:
Productionization
Testing
Versioning
Continuous delivery
Monitoring
FULLY AUTOMATED MLOPS
Building for scale: Automation first
We use process streamlining, best practices, and automation
When no automation, process streamlining:
CRISP-DM
TDSP [1]
1 [Link]
FULLY AUTOMATED MLOPS
Process streamlining and best practices
DESIGN PHASE DEVELOPMENT PHASE
Best practices Best practices
Include domain expertise Write clean code
Involve business stakeholders Document our work
Get feedback from end-users
FULLY AUTOMATED MLOPS
Let's practice!
F U L LY A U T O M AT E D M L O P S
Reference
architecture: Fully
automated MLOps
F U L LY A U T O M AT E D M L O P S
Arturo Opsetmoen Amador
Senior Consultant - Machine Learning
What is a reference architecture?
A reference architecture is a template we can use to design solutions in different IT domains.
A set of documents with:
Structures and integrations of IT elements
Patterns commonly present in IT systems, including ML systems
Leverage experience and best practice from industry players
FULLY AUTOMATED MLOPS
Fully automated MLOps architecture
1 [Link]
learning
FULLY AUTOMATED MLOPS
Reference architecture - Orchestrated experiments
FULLY AUTOMATED MLOPS
Reference architecture - Source code & CI
FULLY AUTOMATED MLOPS
Reference architecture - Artifacts & CD
FULLY AUTOMATED MLOPS
Reference architecture - ML pipeline deployment
FULLY AUTOMATED MLOPS
Reference architecture - The metadata store
FULLY AUTOMATED MLOPS
Reference architecture - The model registry
FULLY AUTOMATED MLOPS
Reference architecture - Prediction services
FULLY AUTOMATED MLOPS
Reference architecture - Continuous monitoring
FULLY AUTOMATED MLOPS
Reference architecture - Automated trigger
FULLY AUTOMATED MLOPS
Reference architecture - Automated retraining
FULLY AUTOMATED MLOPS
Reference architecture
FULLY AUTOMATED MLOPS
Reference architecture - The feature store
FULLY AUTOMATED MLOPS
Reference architecture - The feature store
FULLY AUTOMATED MLOPS
Reference architecture - The feature store
FULLY AUTOMATED MLOPS
Reference architecture - The feature store
FULLY AUTOMATED MLOPS
Fully automated MLOps architecture
FULLY AUTOMATED MLOPS
Let's practice!
F U L LY A U T O M AT E D M L O P S