The Llama fine-
tuning libraries
FINE-TUNING WITH LLAMA 3
Francesca Donadoni
Curriculum Manager, DataCamp
When to use fine-tuning
Pre-trained model Improve accuracy
Uses specialized data Reduce bias
Improve knowledge base
FINE-TUNING WITH LLAMA 3
How to use fine-tuning
Quality of the data
Model's capacity
Task definition
Fine-tuning process
FINE-TUNING WITH LLAMA 3
How to use fine-tuning
Quality of the data
Model's capacity
Task definition
Fine-tuning process
FINE-TUNING WITH LLAMA 3
How to use fine-tuning
Quality of the data
Model's capacity
Task definition
Fine-tuning process
FINE-TUNING WITH LLAMA 3
How to use fine-tuning
Quality of the data
Model's capacity
Task definition
Fine-tuning process
FINE-TUNING WITH LLAMA 3
How to use fine-tuning
Quality of the data
Model's capacity
Task definition
Fine-tuning process
New model
Evaluation
FINE-TUNING WITH LLAMA 3
The Llama fine-tuning libraries
Several libraries for fine-tuning
TorchTune for Llama fine-tuning
Launching a fine-tuning task with TorchTune
FINE-TUNING WITH LLAMA 3
Options for Llama fine-tuning
TorchTune SFTTrainer from Hugging Face
Based on configurable templates Access to other LLMs
Ideal for: scaling quickly Ideal for: fine-tuning multiple models
Unsloth Axolotl
Efficient memory usage Modular approach
Ideal for: limited hardware Ideal for: no extensive reconfiguration
FINE-TUNING WITH LLAMA 3
TorchTune and the recipes for fine-tuning
TorchTune recipes:
Modular templates
Configurable to be adapted to different
projects
Keep code organized
Ensure reproducibility
FINE-TUNING WITH LLAMA 3
TorchTune list
Run from a terminal
Environment with Python
Install TorchTune
pip3 install torchtune
List available recipes
tune ls
! if using IPython
!tune ls
FINE-TUNING WITH LLAMA 3
TorchTune list
!tune ls
Output:
RECIPE CONFIG
full_finetune_single_device llama3/8B_full_single_device
llama3_1/8B_full_single_device
llama3_2/1B_full_single_device
llama3_2/3B_full_single_device
full_finetune_distributed llama3/8B_full
llama3_1/8B_full
llama3_2/1B_full
...
FINE-TUNING WITH LLAMA 3
TorchTune run
Use recipe + --config + configuration
Run fine-tuning
tune run full_finetune_single_device --config \
llama3_1/8B_lora_single_device
Parameters device=cpu or device=cuda
epochs=<int> ( <int> is 0 or a positive integer)
FINE-TUNING WITH LLAMA 3
Let's practice!
FINE-TUNING WITH LLAMA 3
Preprocessing data
for fine-tuning
FINE-TUNING WITH LLAMA 3
Francesca Donadoni
Curriculum Manager, DataCamp
Using datasets for fine-tuning
Quality of the data is key
Training Set:
For model training
Majority of the data
FINE-TUNING WITH LLAMA 3
Using datasets for fine-tuning
Quality of the data is key
Training Set:
For model training
Majority of the data
Validation Set:
For selecting the best model version
FINE-TUNING WITH LLAMA 3
Using datasets for fine-tuning
Quality of the data is key
Training Set:
For model training
Majority of the data
Validation Set:
For selecting the best model version
Test Set:
For evaluating model's performance
FINE-TUNING WITH LLAMA 3
Preparing data using the datasets library
Datasets library
Preprocessing
Split
Load
Manage memory
FINE-TUNING WITH LLAMA 3
Loading a customer service dataset
from datasets import load_dataset
ds = load_dataset(
'bitext/Bitext-customer-support-llm-chatbot-training-dataset',
split="train"
)
print(ds.column_names)
['flags', 'instruction', 'category', 'intent', 'response']
FINE-TUNING WITH LLAMA 3
Peeking into the data
import pprint
[Link](ds[0])
{'category': 'ORDER',
'flags': 'B',
'instruction': 'question about cancelling order {{Order Number}}',
'intent': 'cancel_order',
'response': "I've understood you have a question regarding canceling order "
"{{Order Number}}, and I'm here to provide you with the "
'information you need. Please go ahead and ask your question, and '
"I'll do my best to assist you."}
FINE-TUNING WITH LLAMA 3
Filtering the dataset
from datasets import load_dataset, Dataset
ds = load_dataset(
'bitext/Bitext-customer-support-llm-chatbot-training-dataset',
split="train")
print([Link])
(26872, 5)
first_thousand_points = ds[:1000]
ds = Dataset.from_dict(first_thousand_points)
FINE-TUNING WITH LLAMA 3
Preprocessing the dataset
def merge_example(row):
row['conversation'] = f"Query: {row['instruction']}\nResponse: {row['response']}"
return row
ds = [Link](merge_example)
print(ds[0]['conversation'])
Query: question about cancelling order {{Order Number}}
Response: I've understood you have a question regarding canceling order {{Order Number}},
and I'm here to provide you with the information you need. Please go ahead and ask your
question, and I'll do my best to assist you.
FINE-TUNING WITH LLAMA 3
Saving the preprocessed dataset
ds.save_to_disk("preprocessed_dataset")
Saving the dataset (1/1 shards): 100%
26872/26872 [00:00<00:00, 383823.33 examples/s]
from datasets import load_from_disk
ds_preprocessed = load_from_disk("preprocessed_dataset")
FINE-TUNING WITH LLAMA 3
Using Hugging Face datasets with TorchTune
Can use Hugging Face dataset with TorchTune
Set a dataset path and configurations
tune run full_finetune_single_device --config llama3/8B_full_single_device \
dataset=preprocessed_dataset [Link]=train
FINE-TUNING WITH LLAMA 3
Let's practice!
FINE-TUNING WITH LLAMA 3
Fine-tuning with
TorchTune
FINE-TUNING WITH LLAMA 3
Francesca Donadoni
Curriculum Manager, DataCamp
The components of TorchTune fine-tuning
Model
Defines the architecture and pre-trained
weights to fine-tune
Different versions and number of
parameters available
Dataset
Specifies the data used for training
Recipe
Central configuration file combining the
model, dataset, and training parameters
Ensures consistency and reproducibility
FINE-TUNING WITH LLAMA 3
The components of TorchTune fine-tuning
Model
!tune ls
llama3/8B_full
llama3_1/8B_full
llama3_2/1B_full ...
Dataset
ds.save_to_disk("new_dataset")
Recipe
custom_recipe.yaml
FINE-TUNING WITH LLAMA 3
The components of a TorchTune recipe
General Settings and ouput directory batch_size: 4
Batch size, device and epochs device: cuda
epochs: 20
output_dir: /tmp/full-llama3.2-finetune
Model model:
Specifies architecture and model _component_:
[Link].llama3_2.llama3_2_1b
configurations
optimizer:
_component_: [Link].PagedAdamW8bit
Optimizer lr: 2.0e-05
Includes learning rate
dataset:
Dataset _component_: [Link].alpaca_dataset
Defines preprocessing and dataset path
FINE-TUNING WITH LLAMA 3
Configuring TorchTune recipes
More parameters available
Configurable in Python using yaml
import yaml
config_dict = {"batch_size": 4,
"device": "cuda",
"model": {
"_component_": "[Link].llama3_2.llama3_2_1b"
},
...
}
yaml_file_path = "custom_recipe.yaml"
with open(yaml_file_path, "w") as yaml_file:
[Link](config_dict, yaml_file)
FINE-TUNING WITH LLAMA 3
Running custom fine-tuning
tune run --config custom_recipe.yaml
INFO:[Link]:Running
Writing logs to /tmp/full-llama3.2-finetune/log_1732815689.txt
INFO:[Link]:Model is initialized with precision torch.bfloat16.
INFO:[Link]:Tokenizer is initialized from file.
1|52|Loss: 2.3697006702423096: 0%|? | 52/25880
Saved logs
Successful initialization
Epoch and step count progress
Loss metrics
FINE-TUNING WITH LLAMA 3
Let's practice!
FINE-TUNING WITH LLAMA 3