Enhancing Data Science

Outcomes With Efficient

Workflow
NVIDIA

 Please create your NVIDIAxx Deep Learning Institute (DLI) account

 Go to [Link] and enter the access code:

FTLV_RAPIDS2_AMBASSADOR_AP26

1
ENHANCING DATA
SCIENCE OUTCOMES
WITH EFFICIENT
WORKFLOW
NVIDIA DLI
 WORKSHOP SUMMARY
Enhancing Data Science Outcomes With Efficient Workflow

• Full-day, workshop consisting of three hands-on labs

• Develop an end-to-end data processing pipeline with a single node multi-GPU configuration for large
tabular datasets. The workshop covers ETL (extract-transform-load), feature engineering, model
training, feature selection, and deployment

• Learn ways data science can be improved when the software and hardware enable exploration and
rapid experimentation

• The tools we will utilize include NVIDIA RAPIDs, Dask, plotly, NVTabular, PyTorch, and NVIDIA Triton
Inference Server

3
 WORKSHOP SUMMARY
Agenda
• • 09:00 – 10:15: Workshop
Session
• 10:15 – 10:30: Coffee Break
• 10:30 – 12:00: Workshop
Session
• 12:00 – 14:00: Lunch Break
• 14:00 – 15:15: Workshop
Session
• 15:15 – 15:30: Coffee Break
• 15:30 – 17:00: Exam
Session
4
 WHAT DOES THIS WORKSHOP COVER

• Overview of GPU/multi-GPU computing and the problems they solve

• Potential slowdowns that may occur when working with distributed computing

• Extract, transform, and load data pipelines that support data analytics

• Exploratory data analysis to identify patterns in the data

• Various feature engineering techniques to help machine learning algorithms learn patterns

• Build and fine-tune machine learning models

• Tune GPU-accelerated inference to meet production requirements

5
 WORKSHOP OUTLINE

Lab 1 Advanced ETL

• Focuses on developing pipeline for ETL
• Discuss Parquet file format
• Perform EDA on real behavior dataset
• Explore various feature engineering techniques for categorical and continuous features
• Demonstrate interoperability

Lab 2 Advanced Analytics

• Build tree-based classification models
• Perform feature selection and fine-tune models with hyperparameter optimization
• Create additional features using neural network embeddings

Lab 3 Deployment
• Deploy data processing pipeline to Triton Inference Server
• Experiment various inference configurations to optimize performance

Assessment
6
AGENDA

Lab 1 – Advanced ETL

• Focuses on developing pipeline for ETL
• Discuss Parquet file format
• Perform EDA on real behavior dataset
• Explore various feature engineering techniques for categorical and
continuous features
• Demonstrate interoperability

Lab 2 – Advanced Analytics

Lab 3 – Deployment

Assessment
7
 CRISP-DM DATA SCIENCE LIFE CYCLE
Emphasizes Processing Data Efficiently

Business Data
Understandi Understandi
ng ng

Data
Preparation
Deployment Data

Modeling

Evaluation

8
 Data Science Is the Key to Unlocking the Future of Modern
Business
Enables data-driven decision making

Retail Telecom Healthcare Manufacturing

Supply Chain & Inventory Management Detect Network/Security Anomalies Improve Clinical Care Remaining Useful Life Estimation
Price Management / Markdown Optimization Forecasting Network Performance Drive Operational Efficiency Failure Prediction
Promotion Prioritization And Ad Targeting Network Resource Optimization (SON) Speed Up Drug Discovery Demand Forecasting

Financial Services Consumer Internet Oil & Gas Automotive

Claim Fraud Ad Personalization Sensor Data Tag Mapping Personalization & Intelligent Customer
Customer Service Chatbots/Routing Click Through Rate Optimization Anomaly Detection Interactions
Risk Evaluation Churn Reduction Robust Fault Prediction Connected Vehicle Predictive Maintenance
Forecasting, Demand, & Capacity Planning
 Data Science Is More Than Machine Learning and Analytics
Non-analytical tasks related to data science are equally important

Analytical tasks include:

●Machine learning model

development
●Measuring key metrics
●Statistical analysis
●Data visualizations

Non-analytical tasks (upstream)

include:

●Extract, transform, and load

(ETL)
○ Data collection
○ Data cleaning and preprocessing
○ Data modeling
●Data engineering
 Data Science Is More Than Machine Learning and Analytics
Non-analytical tasks related to data science are equally important

Analytical tasks include:

40% of data
●Machine learning model scientist’s time
development
●Measuring key metrics
●Statistical analysis
●Data visualizations

Non-analytical tasks (upstream)

include:

• Extract, transform, and load

(ETL)
o Data collection
o Data cleaning and preprocessing
o Data modeling

• Data engineering
Challenges in Data Science Today
What needs to be solved to empower data scientists

well
 Less Waiting. More Building.
The right solution enables data scientists to focus on delivering value
 Less Waiting. More Building.
The right solution enables data scientists to focus on delivering value
 Modern Applications Need Accelerated Computing
Petabyte scale data | Massive models | Real-time performance

1000X
by 2025

• Recommend LLMs • Forecasting

ers

• Fraud • Genomic • Cybersecurit

Detection Analysis y
 CPU and GPU Comparison
GPUs leverage parallel processing to handle large datasets

[Link]
 DATA SCIENCE WORKFLOW WITH RAPIDS
End-to-end GPU-accelerated Workflow Build on CUDA

DAT PREDICTIO
A NS
DATA PREPARATION MODEL TRAINING VISUALIZATION

17
 DATA SCIENCE WORKFLOW WITH RAPIDS
End-to-end GPU-accelerated Workflow Build on CUDA

DAT PREDICTIO
A NS

DATA PREPARATION
GPUs accelerated compute for in-memory data
preparation Simplified implementation using familiar
data science tools
Python drop-in Pandas replacement built on CUDA 18
C++. GPU-accelerated Spark (in development)
 DATA SCIENCE WORKFLOW WITH RAPIDS
End-to-end GPU-accelerated Workflow Build on CUDA

DAT PREDICTIO
A NS

MODEL TRAINING
GPU-acceleration of today’s most popular ML
algorithms
XGBoost, PCA, K-means, k-NN, DBScan, tSVD …

19
 DATA SCIENCE WORKFLOW WITH RAPIDS
End-to-end GPU-accelerated Workflow Build on CUDA

DAT PREDICTIO
A NS

VISUALIZATION
Effortless exploration of datasets, billions of records in
milliseconds Dynamic interaction with data = faster ML
model development
Data visualization ecosystem (Graphistry & OmniSci), 20
integrated with RAPIDS
USE CASES
21
IMPROVING
DEMAND
FORECASTS
• Predict demand for over 500
million item-by-store combinations
every day
• 1 GPU = 20 CPUs with 20x faster
training of machine learning
algorithms
• Improved forecast accuracy, for
real-time reaction to shopper
trends

22
AI HELPS DOMINO’S PREDICT
WHEN THREE BILLION PIZZAS
ARE READY
FOR PICKUP
 Domino’s has over 17,000 stores worldwide
and delivers more than 3 billion pizzas a year

 AI is used to improve operational efficiencies

and enhance the customer experience
including:
- predicting order readiness
- curbside pickup
- routing deliveries
- determining ideal timing for marketing
campaigns

 Leveraging NVIDIA DGX-1 servers and

NVIDIA RAPIDS data science libraries
to power their data science platform

 Model training time has

improved 72x and order
readiness prediction has
improved from 75% to 95%

23
24
DATA SCIENCE IS THE KEY TO MODERN BUSINESS
Accelerated Data Science Use Cases in Every Industry

25
 GPU ADOPTION BARRIERS
GPUs Are Fast, but…

Challenges:

• Too much data movement

• Too many makeshift data

formats require
transformation

• Writing CUDA C/C++ is

hard

• No Python API for data

manipulation

26
 GPU ADOPTION BARRIERS
GPUs Are Fast, but…

Challenges:

• Too much data movement

• Too many makeshift data

formats require
transformation

• Writing CUDA C/C++ is

hard

• No Python API for data

manipulation

Solutions:

• Shared, standard data

structures are essential to
building an efficient 27
 RAPIDS
GPU Accelerated Data Science Libraries

Data Preparation Model Training Visualization

Dask

PyTorch, cuxfilter, pyViz,

cuDF cuIO cuML cuGraph
TensorFlow plotly
Analytics Machine Learning Graph Analytics
Deep Learning Visualization

GPU Memory

28
 CUDF
GPU Accelerated Data Wrangling and Feature Engineering

• Python interface to CUDA C++ libraries for loading, joining,

aggregating, filtering, and otherwise manipulating data using a
DataFrame style API

• Accelerated file readers for prevalent formats in data science

• Powerful aggregation functions using GroupBy

• Various string and list manipulation

• Ability to write user-defined functions (UDFs), without needing

to write CUDA code

• ▸ JIT compilation of User-Defined Functions (UDFs) using Numba

29
 CUDF AND CUPY
Interoperability of the GPU PyData Ecosystem

Using DLPack and __cuda_array_interface__ – the CUDA array interface is a standard format that describes
a GPU array to allow sharing GPU arrays between different libraries without needing to copy or convert
data

30
 DASK
Parallel Computing Library

• Dask is a parallel computing library built to scale Python workloads from single
machine to supercomputer clusters

• Dask reduce reliance on high-cost infrastructure like supercomputers for scalability

• Light-weight framework for working with chunked data across a variety of

computational backends.

• Executes operations on chunks of data and aggregates results, thus enabling working with
datasets that are larger than memory

• Uses familiar APIs as those found in the PyData ecosystem

• Support for multi-GPU, multi-node

• Tools that enable better memory monitoring

32
 RAPIDS + DASK
Distributed Parallel Computing on Multi-GPU

• Data is read into memory

• Partitioned across a
cluster
Cluster
• Operated on in parallel Worker

Client Scheduler Worker

Client creates the task Scheduler manages
graph, sends instructions state and sends tasks to
to the scheduler, and workers for execution.
collects results from the Furthermore, scheduler
workers helps workers coordinate Worker
with each other
Workers execute
computation tasks, as
well as store and serves
results to other workers
or client
34
 RAPIDS + DASK
Scale Up and Scale Out Workloads

• RAPIDS is used to scale up workloads by reimplementing low-level parts of common open-source APIs to
run on GPUs for faster execution

• Dask is used to scale out workloads to multiple processors and multiple machines

When to use RAPIDS with Dask:

• Distribute workloads across multiple GPUs

• Out-of-core computations on large datasets

• Production requires fast execution

• Cloud computing that is accessible from anywhere

35
DASK DASHBOARD DIAGNOSTICS
Profiling Parallel Computing and Monitoring Cluster

36
 DATA SCIENCE IS NOT A LINEAR PROCESS
Faster Data Processing Leads to Better Data Science Outcomes

Cross Validate
Exhaustive Feature Selection
Benefits of Accelerated Data Science Workflow Grid Search

• Higher productivity

• More explorations and iterations can improve model accuracy and optimize inference performance

• Lower time to insight and total cost of ownership

37
 ACCELERATING DATA WRANGLING
Dask-cuDF for Fast Data Processing

• ETL – Extract, Transform, and Load

• Dask-cuDF extends Dask where necessary to allow DataFrame partitions to be processed using cuDF GPU DataFrames

• Mitigates running out of memory when dataset is larger than GPU memory

• Reads one or more files in parallel

• Uses cuDF primitives underneath in map-reduce style operations with the same high-level API

• EDA – Exploratory Data Analysis

• Visualizations can be used to facilitate the learning of the characteristics of the dataset, but the benefits are
diminished when working with large datasets

38
 OPTIMIZING DISTRIBUTING COMPUTING
High Performance Computing Requires Efficient Worker Utilization

• Unlike the CPU, memory is more expensive and limited on GPU. To fully utilize the benefits of parallel
computing and maximize throughput, users need to:

• Efficiently load and distribute data onto GPU memory

• Be intentional about partitions and ensure saturation of GPU memory

• Keep data on the GPU at all stages of the pipeline and avoid hidden slowdowns from:

• Shuffle – the transfer of data between partitions, workers, and other forms of data movement as it diminishes the
benefits of GPU-acceleration

• Worker communication and memory spilling, which are slow

• Idle workers

• Monitor GPU memory utilization and remove bottlenecks

39
 GROUPBY AGGREGATION IS EASY
Apply, Concat, Apply Method

aggregat
apply_concat_apply() method breaks the partitions chunk
outputs a
concat
e
operation into: outputs a
reduction
reduction

group value
• chunk() – applies aggregation to each group value
a 1
partition a 2
a 3
b 2
b 2 group value
c 3
• combine() – results are concatenated and c 3 a 3 group value
combined b 2 a 6
c 3 b 6

• aggregate() – applies aggregation to results a 3 c 3

group value b 4 d 1
group value
a 3 d 1
Other algorithms like var() require different b 1
a 3
operations at different stages b 3
b 4
d 1
d 1
Results returned in 1 partition, which may groupby(‘group’)[‘value’].sum()
lead to out-of-memory issues if there are
many groups
40
 USE PARQUET FILE FORMAT
Advanced Analytics on Big Data Require Data Format for Performant Queries

• Columnar file format and stores chunks of columns File

Row Group

• Row group allow for vertical partitioning Column Chunk

Page

• Statistical metadata for each chunk allows for skipping blocks of data by Page
comparing with the provided predicates – this is useful when data is sorted in
an order that allows omitting the read of many row groups/column strips
Column Chunk
Page
• Provides efficient storage and fast read speed

• Enables better compression as data within each column is more homogenous Footer
(reduce size on disk and memory) File Metadata
|--Row Group Metadata
| |--Column Chunk Metadata
• Enables high performance retrieval and filtering of data, which is useful since | |--Column Chunk Metadata
|
much of the data processing requires subsetting both rows and columns. |--Row Group Metadata

Structure: root > parquet file > row-group >

• Increases data throughput and performance using techniques like data column chunk > data page
skipping, whereby queries that fetch specific column values (column
pruning) need not read the entire row of data (predicate pushdown)
41
 FEATURE ENGINEERING
The Best Features Are Those That Capture the Underlying Signals

Categorical Features Continuous Features

Categorify | Label Encoding | Factorize Clipping
Combining Features Binning
One-Hot Encoding Normalize
Count Encoding Time-Series Features
Target Encoding
Embedding

42
 NVTABULAR
Faster and Easier GPU-Based ETL

• Feature engineering and preprocessing library for tabular data designed to quickly and easily
manipulate terabyte scale datasets

• High-level API building upon Dask-cuDF

• Allows users to think about what they want to do with the data, not how they have to do it or how to scale it, for
common operations

• Prepare datasets quickly and easily for experimentation so that more models can be trained

• Accelerates dataloading for deep learning frameworks

• A component of the NVIDIA Merlin, and open-source framework for building and deploying recommender
systems

43
Hands-On Lab1: Advanced
ETL

44
 LAB 1 SUMMARY

• Focuses on developing pipeline for ETL

• Discuss Parquet file format

• Perform EDA on real behavior dataset

• Explore various feature engineering techniques for categorical and continuous features

• Demonstrate interoperability

45
 WHAT’S NEXT (LAB 2)

• Build tree-based classification models

• Perform feature selection and fine-tune models with hyperparameter optimization

• Create additional features using neural network embeddings

46
AGENDA

Lab 1 – Advanced ETL

Lab 2 – Advanced Analytics

• Build tree-based classification models
• Perform feature selection and fine-tune models with
hyperparameter optimization
• Create additional features using neural network embeddings

Lab 3 – Deployment

Assessment

47
 MACHINE LEARNING PROBLEMS
Use Cases in Every Industry

Supervised Unsupervised
• Spam / Not Spam • Group Articles in Categories
Online Services
• Predict Probability Customer Will Click on Ad • Similar Search Results

Financial Services • Credit Card Fraud • Anomaly Detection

Telecom • Predict Customer Churn

• Probability of Readmission
Health Care • Patient Similarity
• Predict Days of Hospital Stay

Real Estate • Predict House Price

• Similar Customers
• Predict Price
• Product Similarity
Retail • Forecast Sales
• Customer Group
• Sentiment Analysis
• Products Which Are Purchased Together

48
 PROBLEM
Data Sizes Continue to Grow

49
 TREE-BASED ENSEMBLE METHODS
Using Multiple Decision Trees

(Random Forest) Bagging is an ensemble technique used to reduce (Gradient Boosting Decision Tree) Boosting is another type of
the variance of predictions by considering results from multiple ensemble learning that combines weak learners to achieve improved
decision tree models that were trained on different sub-samples of the model performance. Weak learners are simple models that predict
same data set relatively poorly. The concept of boosting is to sequentially train
models, each time trying to fit better than before using residual errors
51
 XGBOOST + RAPIDS (LINK)
Optimized Distributed Gradient Boosting Machine Learning Algorithm

• Extreme Gradient Boosting, is a scalable, distributed gradient-boosted decision

tree machine learning algorithm

• Builds upon supervised machine learning, decision trees, ensemble

learning, and gradient boosting concepts

• Gradient boosting is an extension of boosting where the process of additively generating weak models
is formalized as a gradient descent algorithm over an objective function. Gradient boosting sets
targeted outcomes for the next model to minimize errors

• With XGBoost, trees are built in parallel. It follows a level-wise strategy, scanning across gradient values
and using these partial sums to evaluate the quality of splits at every possible split in the training set

• XGBoost includes seamless GPU acceleration, including loading data from cuDF DataFrames and cuPy
arrays. This significantly speeds up model training and improves accuracy for better predictions. The
Dask API makes it easy to scale to multiple nodes or multiple GPUs. Furthermore, the RAPIDS Memory
Manager (RMM) integrates with XGBoost to enable a single, shared, high-speed memory pool

52
 CLASSIFICATION MODEL PERFORMANCE
AUC – ROC Curve

• The AUC (Area Under The Curve) ROC (Receiver Operating Characteristics) curve is an important
evaluation metric for classification models

• ROC – probability curve created by plotting with TPR (True Positive Rate) against FPR (False Positive
Rate)

• AUC – indicates the model’s ability to distinguish classes, a.k.a. separability

• An excellent model has AUC close to 1. When AUC is 0.5, it indicates the model has low class separation
capacity

53
 MODEL TUNING
Low Data Processing Cycle Time Translates to More Experimentation and Higher
Productivity

• Faster runtime means more iterations to improve accuracy

• Faster runtime shortens development time and improves productivity

• Cross validation – resampling method to protect against overfitting

• Feature selection – finding the optimal subset of features

Select Features
• Reduce noise and irrelevant features
and Model Training Evaluation
Parameters
• Reduce model complexity and training time
Iterate
• Improve model inference performance

• Hyperparameter Tuning

• Grid Search

54
 FEATURE SELECTION
Some Methods Require Iterative Model Training

Filter Method Wrapper Method Embedded Method

Definition Generic set of methods Evaluates on a specific Embeds (fix) features during
which do not incorporate a learning algorithm to find model building process.
specific machine learning optimal features Feature selection is done by
algorithm observing each iteration of
model training phase
Time Much faster compared to High computation time Between
Requirement wrapper methods
Strengths Less prone to overfitting High chance of overfitting Reduce overfitting by
because it involves training penalizing the coefficients of
machine learning models a model being too large
with different combination of
features
Methods Correlation, Chi-Square Test, Forward Selection, Backward LASSO, Elastic Net, Ridge
ANOVA, Information Gain Selection, Stepwise Regression
Selection
55
 ENTITY EMBEDDING
Representing Categorical Data With Multiple Dimensions

Entity embedding is a technique for applying deep learning to tabular data. It involves representing the
categorical variables in a continuous way while retaining the relationship between different data values

• Product embedding

• Customer embedding

56
 FINDING THE EMBEDDING SPACE
Applying Deep Learning to Tabular Data

• The embeddings are found by

Embeddin
Categoric
setting up a surrogate problem

al

g
• The embeddings retain semantic
information between categories

Embeddin
Categoric

Dropout
al

g
• Can be used to encode categorical

Dense Linear

Dense Linear
Batch Norm

Batch Norm
variables

Dropout

Dropout

Output
Linear
Embeddin
Categoric
• Dask can be used to enable

al
distributed training:

g
• Data parallelism

• Model parallelism
Continuo

Batch
Norm
us

57
 BUILD DATA PROCESSING PIPELINE AND FINAL
MODEL
Optimize Pipeline Throughput By Eliminating Noisy Features

Cross Validate
Exhaustive Feature Selection
Benefits of Accelerated Data Science Workflow Grid Search

• Higher productivity

• More explorations and iterations can improve model accuracy and optimize inference performance

• Lower time to insight and total cost of ownership

61
Hands-On Lab2: Advanced
Analytics

62
 LAB 2 SUMMARY

• Build tree-based classification models

• Perform feature selection and fine-tune models with hyperparameter optimization

• Create additional features using neural network embeddings

63
 WHAT’S NEXT (LAB 3)

• Deploy data processing pipeline to Triton Inference Server

• Experiment various inference configurations to optimize performance

64
AGENDA

Lab 1 – Advanced ETL

Lab 2 – Advanced Analytics

Lab 3 – Deployment
• Deploy data processing pipeline to Triton Inference Server
• Experiment various inference configurations to optimize
performance

Assessment

65
 NVIDIA TRITON INFERENCE SERVER
Deploy, Run, and Scale Trained ML or DL Models

• Executes multiple models on single or multi-GPU configurations

• Supports all popular AI frameworks, including Forest Inference Library

• Dynamically batches requests to increase throughput

• Provides latency and health metrics

• Auto scaling and load balancing

66
 FOREST INFERENCE LIBRARY
FIL Enables High Performance Inference

• cuML’s Forest Inference Library accelerates prediction

(inference) for random forests and boosted decision trees

• Works with existing saved models (XGBoost, LightGBM, and cuML

RF)

• Can be deployed on the Triton Inference Server

• Improve inference performance for:

• Low-latency case – where inference is applied to each instance

arriving in a streaming context

• High-throughput case – where inference is applied to many samples

67
RAPIDS INTEGRATED INTO CLOUD ML FRAMEWORKS
Focused on Robust Functionality, Deployment, and User Experience

69
 SUMMARY
Faster Data Processing Leads to Better Data Science Outcomes

Accelerating data science workflow enable rapid experimentation:

• Create and discover useful features for model training

• Hyperparameter tuning to find the most accurate possible model(s)

• Select features for optimal inference performance

Close integration with major cloud and OSS solution providers to appropriately scale workflows and be
cost efficient

70
Hands-On Lab3: Deployment

71
 LAB 3 SUMMARY

• Deploy data processing pipeline to Triton Inference Server

• Experiment various inference configurations to optimize performance

72