Scribd
Upload
26 views6 pages

Automated ML Solutions with Azure

The document outlines the steps for designing and implementing a machine learning solution using Azure Machine Learning, covering aspects such as compute specifications, model deployment, and workspace management. It details the processes for managing data, exploring data, training models, and preparing models for deployment, along with MLOps practices for automation. Additionally, it highlights resources for learning and instructor-led training to prepare for the DP-100 exam.

Uploaded by

Revathy P
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
26 views6 pages

Automated ML Solutions with Azure

The document outlines the steps for designing and implementing a machine learning solution using Azure Machine Learning, covering aspects such as compute specifications, model deployment, and workspace management. It details the processes for managing data, exploring data, training models, and preparing models for deployment, along with MLOps practices for automation. Additionally, it highlights resources for learning and instructor-led training to prepare for the DP-100 exam.

Uploaded by

Revathy P
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Design and prepare a machine learning solution (20–25%)

Design a machine learning solution

 Determine the appropriate compute specifications for a training


workload (Microsoft Documentation: compute targets in Azure
Machine Learning)

 Describe model deployment requirements (Microsoft


Documentation: Deploy machine learning models to Azure)

 Select which development approach to use to build or train a


model (Microsoft Documentation: Train models with Azure Machine
Learning)

Manage an Azure Machine Learning workspace

 Create an Azure Machine Learning workspace (Microsoft


Documentation: Create workspace resources you need to get started
with Azure Machine Learning)

 Manage a workspace by using developer tools for workspace


interaction (Microsoft Documentation: Manage Azure Machine
Learning workspaces in the portal or with the Python SDK (v2))

 Set up Git integration for source control (Microsoft


Documentation: Source control in Azure Data Factory)

 Create and manage registries

Manage data in an Azure Machine Learning workspace

 Select Azure Storage resources (Microsoft


Documentation: Introduction to Azure Storage)

 Register and maintain datastores (Microsoft Documentation: Create


datastores)

 Create and manage data assets (Microsoft Documentation: Create


data assets)

Manage compute for experiments in Azure Machine Learning

 Create compute targets for experiments and training (Microsoft


Documentation: Configure and submit training jobs)

 Select an environment for a machine learning use case (Microsoft


Documentation: What are Azure Machine Learning environments?)
 Configure attached compute resources, including Azure Synapse Spark
pools and serverless Spark compute (Microsoft
Documentation: Apache Spark pool configurations in Azure Synapse
Analytics)

 Monitor compute utilization

Explore data, and train models (35–40%)

Explore data by using data assets and data stores

 Access and wrangle data during interactive development (Microsoft


Documentation: What is data wrangling?)

 Wrangle interactive data with attached Synapse Spark pools and


serverless Spark compute (Microsoft Documentation: Interactive
Data Wrangling with Apache Spark in Azure Machine Learning)

Create models by using the Azure Machine Learning designer

 Create a training pipeline (Microsoft Documentation: Create a build


pipeline with Azure Pipelines)

 Consume data assets from the designer (Microsoft


Documentation: Create data assets)

 Use custom code components in designer (Microsoft


Documentation: Add code components to a custom page for your
model-driven app)

 Evaluate the model, including responsible AI guidelines (Microsoft


Documentation: What is Responsible AI?)

Use automated machine learning to explore optimal models

 Use automated machine learning for tabular data (Microsoft


Documentation: What is automated machine learning (AutoML)?)

 Use automated machine learning for computer vision

 Use automated machine learning for natural language


processing (Microsoft Documentation: Set up AutoML to train a
natural language processing model)

 Select and understand training options, including preprocessing and


algorithms
 Evaluate an automated machine learning run, including responsible AI
guidelines (Microsoft Documentation: What is Responsible AI?)

Use notebooks for custom model training

 Develop code by using a compute instance (Microsoft


Documentation: Create and manage an Azure Machine Learning
compute instance)

 Track model training by using MLflow (Microsoft


Documentation: Track ML experiments and models with MLflow)

 Evaluate a model (Microsoft Documentation: Evaluate Model


component)

 Train a model by using Python SDKv2

 Use the terminal to configure a compute instance (Microsoft


Documentation: Access a compute instance terminal in your
workspace)

Tune hyperparameters with Azure Machine Learning

 Select a sampling method (Microsoft Documentation: Sampling in


Application Insights)

 Define the search space

 Define the primary metric (Microsoft Documentation: Set up


AutoML training with the Azure ML Python SDK v2)

 Define early termination options (Microsoft


Documentation: Hyperparameter tuning a model (v2))

Prepare a model for deployment (20–25%)

Run model training scripts

 Configure job run settings for a script (Microsoft


Documentation: Configure and submit training jobs)

 Configure compute for a job run

 Consume data from a data asset in a job (Microsoft


Documentation: Create data assets)
 Run a script as a job by using Azure Machine Learning (Microsoft
Documentation: Azure Machine Learning in a day, Configure and
submit training jobs)

 Use MLflow to log metrics from a job run (Microsoft


Documentation: Log metrics, parameters and files with MLflow)

 Use logs to troubleshoot job run errors (Microsoft


Documentation: Review logs to diagnose pipeline issues)

 Configure an environment for a job run (Microsoft


Documentation: Create and target an environment)

 Define parameters for a job (Microsoft Documentation: Runtime


parameters)

Implement training pipelines

 Create a pipeline (Microsoft Documentation: Create your first


pipeline, What is Azure Pipelines?)

 Pass data between steps in a pipeline (Microsoft


Documentation: How to use parameters, expressions and functions in
Azure Data Factory)

 Run and schedule a pipeline (Microsoft Documentation: Configure


schedules for pipelines)

 Monitor pipeline runs (Microsoft Documentation: Visually monitor


Azure Data Factory)

 Create custom components (Microsoft Documentation: Create your


first component)

 Use component-based pipelines (Microsoft Documentation: Create


and run machine learning pipelines using components with the Azure
Machine Learning CLI)

Manage models in Azure Machine Learning

 Describe MLflow model output (Microsoft Documentation: Track ML


experiments and models with MLflow)

 Identify an appropriate framework to package a model (Microsoft


Documentation: Model management, deployment, and monitoring
with Azure Machine Learning)
 Assess a model by using responsible AI guidelines (Microsoft
Documentation: What is Responsible AI?)

Deploy and retrain a model (10–15%)

Deploy a model

 Configure settings for online deployment (Microsoft


Documentation: Configuration options for the Office Deployment
Tool)

 Configure compute for a batch deployment (Microsoft


Documentation: Deploy applications to compute nodes with Batch
application packages)

 Deploy a model to an online endpoint (Microsoft


Documentation: Deploy and score a machine learning model by using
an online endpoint)

 Deploy a model to a batch endpoint (Microsoft Documentation: Use


batch endpoints for batch scoring)

 Test an online deployed service (Microsoft Documentation: Testing


the Deployment)

 Invoke the batch endpoint to start a batch scoring job (Microsoft


Documentation: Use batch endpoints for batch scoring)

Apply machine learning operations (MLOps) practices

 Trigger an Azure Machine Learning job, including from Azure DevOps or


GitHub (Microsoft Documentation: Trigger Azure Machine Learning
jobs with GitHub Actions)

 Automate model retraining based on new data additions or data


changes

 Define event-based retraining triggers (Microsoft


Documentation: Create a trigger that runs a pipeline in response to a
storage event)

Microsoft Learning Platform

Microsoft offers various learning path for DP-100 exam, that is available on
the official website of Microsoft. For Designing and Implementing a Data
Science (DP-100) exam, you will find many learning paths and
documentation as mentioned above. And, finding relatable content on the
Microsoft website is quite an easy task. From here, you directly go
on Designing and Implementing a Data Science (DP-100) Page.

Instructor-Led Training

The instructor-led training is an essential resource in order to prepare and


clear all doubts for Designing and Implementing a Data Science (DP-100)
exam. You can find the instructor-led training on the page of the particular
exam on the Microsoft website. In this context, you will engage in acquiring
expertise in operating machine learning solutions on a large scale within the
Azure Machine Learning environment. This entails leveraging your existing
proficiency in Python and machine learning to oversee tasks such as data
ingestion and preparation, model training and deployment, as well as
monitoring machine learning solutions within Microsoft Azure.

 Designing and Implementing a Data Science Solution on Azure

Common questions

Powered by AI

Pipelines in Azure Machine Learning orchestrate and automate multiple stages of model deployment, from data processing through to model training and evaluation. They facilitate reproducibility, scalability, and efficiency by allowing modular development, where each component can be adjusted independently. Pipelines also support conditional execution and retraining triggers, promoting continuous integration and delivery practices. The advantages include streamlined workflows, reduced manual errors, and enhanced collaboration .

The benefits of using AutoML in Azure include acceleration of the model development process, the ability to handle diverse datasets without extensive pre-processing, and the automatic selection of algorithms and feature engineering techniques. This leads to potentially better model performance by exploring a broader range of model options more efficiently. However, challenges may include reliance on the summarization heuristics used by AutoML, which could overlook some model nuances, and the need for adequate compute resources to explore various models and configurations systematically .

Batch endpoints in Azure Machine Learning allow for processing large volumes of data in a non-interactive way, which is highly efficient for scenarios that do not require real-time predictions. They are most effective in data-heavy tasks like analyzing periodic logs or processing large datasets overnight. However, they are limited in use cases demanding low-latency predictions. Scenarios where batch endpoints excel include large-scale churn analysis and financial transaction batch processing. The main critique is their unsuitability for applications requiring immediate responses .

Configuring a compute instance for model training in Azure involves several steps: creating the compute instance with required specifications, setting up necessary tools and libraries within the instance, and ensuring secure access to data assets and compute resources. This setup supports custom model training by providing an isolated, scalable environment tailored for developing, testing, and deploying machine learning models with greater control over dependencies and runtime settings .

Azure supports various development approaches, such as using the graphical Azure ML designer, code-based development with Python SDKs, and automated machine learning (AutoML), which enhances model building and training by offering flexibility tailored to user expertise and project requirements. For instance, the designer is ideal for users who prefer a visual interface, while the Python SDKs offer full control and customization for developers comfortable with coding. AutoML caters to those seeking rapid development without deep knowledge of machine learning algorithms .

Integrating Git for source control in Azure Machine Learning allows for better version control of machine learning projects. This integration involves setting up a connection between an Azure Machine Learning workspace and a Git repository, which enables tracking changes, collaborating with others, and maintaining a history of modifications. The benefits include improved collaboration, rollback capabilities, and a more structured development environment .

Key considerations when deploying machine learning models to Azure include ensuring the model meets deployment requirements, selecting suitable compute resources for the deployment, and adhering to responsible AI guidelines. Deployment methods, such as online or batch endpoints, should align with the use case needs for latency and throughput. Also, monitoring and updating capabilities are critical to maintain model performance over time .

Determining the appropriate compute specifications for a training workload in Azure Machine Learning involves considering factors such as the size of the dataset, the complexity of the model, and the speed at which results are needed. Azure provides various compute targets like Azure Machine Learning Compute, Azure Databricks, and Azure HDInsight, which can be selected based on these requirements. For example, larger datasets might benefit from using a distributed processing system like Azure Databricks .

Azure Machine Learning ensures responsible AI practices by integrating tools and guidelines that assess model fairness, interpretability, and ethical impacts. During model evaluation, it offers capabilities to evaluate the transparency and biases within models. Deployment involves checks for data privacy and security measures. These practices help maintain trust and adherence to ethical standards in AI solutions .

MLflow in Azure Machine Learning is used to track experiments, log metrics, and manage model lifecycle. It facilitates reproducibility and transparency in model training by keeping a record of training parameters, code versions, and evaluation metrics. Advantages include improved collaboration among teams, easier debugging through detailed logs, and simplified deployment processes through consistent tracking .

You might also like