Introduction to Machine Learning

Welcome to Introduction to Machine Learning! This course introduces machine learning (ML)
concepts. This course does not cover how to implement ML or work with data.

What is Machine Learning?

 On this page
 Types of ML Systems
 Supervised learning
o Regression
o Classification
 Unsupervised learning
 Reinforcement learning
 Generative AI


Machine learning (ML) powers some of the most important technologies we use, from translation
apps to autonomous vehicles. This course explains the core concepts behind ML.

ML offers a new way to solve problems, answer complex questions, and create new content. ML can
predict the weather, estimate travel times, recommend songs, auto-complete sentences, summarize
articles, and generate never-seen-before images.

In basic terms, ML is the process of training a piece of software, called a model, to make
useful predictions or generate content (like text, images, audio, or video) from data.

For example, suppose we wanted to create an app to predict rainfall. We could use either a traditional
approach or an ML approach. Using a traditional approach, we'd create a physics-based
representation of the Earth's atmosphere and surface, computing massive amounts of fluid dynamics
equations. This is incredibly difficult.

Using an ML approach, we would give an ML model enormous amounts of weather data until the ML
model eventually learned the mathematical relationship between weather patterns that produce
differing amounts of rain. We would then give the model the current weather data, and it would predict
the amount of rain.

Check Your Understanding

What is a "model" in machine learning?
A model is a smaller representation of the thing you're studying.
A model is a piece of computer hardware
A model is a mathematical relationship derived from data that an ML system uses to make predictions
Types of ML Systems

ML systems fall into one or more of the following categories based on how they learn to make
predictions or generate content:

 Supervised learning
 Unsupervised learning
 Reinforcement learning
 Generative AI

Supervised learning

Supervised learning models can make predictions after seeing lots of data with the correct answers
and then discovering the connections between the elements in the data that produce the correct
answers. This is like a student learning new material by studying old exams that contain both
questions and answers. Once the student has trained on enough old exams, the student is well
prepared to take a new exam. These ML systems are "supervised" in the sense that a human gives
the ML system data with the known correct results.

Two of the most common use cases for supervised learning are regression and classification.

Regression
A regression model predicts a numeric value. For example, a weather model that predicts the amount
of rain, in inches or millimeters, is a regression model.

See the table below for more examples of regression models:

Scenario Possible input data Numeric prediction

Future Square footage, zip code, number of bedrooms and The price of the home.
house bathrooms, lot size, mortgage interest rate, property tax
price rate, construction costs, and number of homes for sale in
the area.

Future ride Historical traffic conditions (gathered from smartphones, The time in minutes and
time traffic sensors, ride-hailing and other navigation seconds to arrive at a
applications), distance from destination, and weather destination.
conditions.
Classification
Classification models predict the likelihood that something belongs to a category. Unlike regression
models, whose output is a number, classification models output a value that states whether or not
something belongs to a particular category. For example, classification models are used to predict if
an email is spam or if a photo contains a cat.

Classification models are divided into two groups: binary classification and multiclass classification.
Binary classification models output a value from a class that contains only two values, for example, a
model that outputs either rain or no rain. Multiclass classification models output a value from a
class that contains more than two values, for example, a model that can output
either rain, hail, snow, or sleet.

Check Your Understanding

If you wanted to use an ML model to predict energy usage for commercial buildings, what type of model
would you use?
Classification
Regression

Unsupervised learning

Unsupervised learning models make predictions by being given data that does not contain any
correct answers. An unsupervised learning model's goal is to identify meaningful patterns among the
data. In other words, the model has no hints on how to categorize each piece of data, but instead it
must infer its own rules.

A commonly used unsupervised learning model employs a technique called clustering. The model
finds data points that demarcate natural groupings.
Figure 1. An ML model clustering similar data points.

Figure 2. Groups of clusters with natural demarcations.

Clustering differs from classification because the categories aren't defined by you. For example, an
unsupervised model might cluster a weather dataset based on temperature, revealing segmentations
that define the seasons. You might then attempt to name those clusters based on your understanding
of the dataset.

Figure 3. An ML model clustering similar weather patterns.

Figure 4. Clusters of weather patterns labeled as snow, sleet, rain, and no rain.
Check Your Understanding
What distinguishes a supervised approach from an unsupervised approach?
An unsupervised approach knows how to label clusters of data.
A supervised approach is given data that contains the correct answer.
A supervised approach typically uses clustering.

Reinforcement learning

Reinforcement learning models make predictions by getting rewards or penalties based on actions
performed within an environment. A reinforcement learning system generates a policy that defines the
best strategy for getting the most rewards.

Reinforcement learning is used to train robots to perform tasks, like walking around a room, and
software programs like AlphaGo to play the game of Go.

Generative AI

Generative AI is a class of models that creates content from user input. For example, generative AI
can create unique images, music compositions, and jokes; it can summarize articles, explain how to
perform a task, or edit a photo.

Generative AI can take a variety of inputs and create a variety of outputs, like text, images, audio, and
video. It can also take and create combinations of these. For example, a model can take an image as
input and create an image and text as output, or take an image and text as input and create a video
as output.

We can discuss generative models by their inputs and outputs, typically written as "type of input"-
to-"type of output." For example, the following is a partial list of some inputs and outputs for
generative models:

 Text-to-text
 Text-to-image
 Text-to-video
 Text-to-code
 Text-to-speech
 Image and text-to-image

The following table list examples of generative models, their input, and an example of their possible
output:
Model Input Example output

Text- Who invented the Le The 24 Hours of Le Mans was founded by the Automobile
to-text Mans race? Club de l'Ouest (ACO), a motoring organization founded in
1906. The first race was held on 26 and 27 May 1923 on the
Sarthe Circuit, a 13.426-kilometre (8.345-mile) road circuit
located near the town of Le Mans in France. The race was
originally conceived as a way to promote the ACO and the
Sarthe region.
Source: Bard

Text- An alien octopus floats

to- through a portal while
image reading a newspaper.

Source: Imagen

Text- A photorealistic teddy

to- bear is swimming in the
video ocean at San Francisco.
The teddy bear goes
under water. The teddy
bear keeps swimming
under the water with
Source: Phenaki
colorful fishes. A panda
bear is swimming under
water.

Text- Write a Python loop that for number in numbers:

to- loops over a list of # Check if the number is prime.
code numbers and prints the is_prime = True
prime numbers. for i in range(2, number):
if number % i == 0:
is_prime = False
 Model Input Example output

break
# If the number is prime, print it.
if is_prime:
print(number)

Source: Bard

Image- This is a flamingo. They are found in the Caribbean.

to-text Source: Google DeepMind

How does generative AI work? At a high-level, generative models learn patterns in data with the goal
to produce new but similar data. Generative models are like the following:

 Comedians who learn to imitate others by observing people's behaviors and style of speaking
 Artists who learn to paint in a particular style by studying lots of paintings in that style
 Cover bands that learn to sound like a specific music group by listening to lots of music by that group

To produce unique and creative outputs, generative models are initially trained using an unsupervised
approach, where the model learns to mimic the data it's trained on. The model is sometimes trained
further using supervised or reinforcement learning on specific data related to tasks the model might
be asked to perform, for example, summarize an article or edit a photo.

Generative AI is a quickly evolving technology with new use cases constantly being discovered. For
example, generative models are helping businesses refine their ecommerce product images by
automatically removing distracting backgrounds or improving the quality of low-resolution images.

Key Terms:

 classification model
 clustering
 model
 policy
 prediction
 regression model
 reinforcement learning
 reward
 supervised learning
  training
 unsupervised learning

Supervised Learning

bookmark_border
 On this page
 Foundational supervised learning concepts
o Data
o Dataset characteristics
o Model
o Training
o Evaluating
o Inference


Supervised learning's tasks are well-defined and can be applied to a multitude of scenarios—like
identifying spam or predicting precipitation.

Foundational supervised learning concepts

Supervised machine learning is based on the following core concepts:

 Data
 Model
 Training
 Evaluating
 Inference

Data
Data is the driving force of ML. Data comes in the form of words and numbers stored in tables, or as
the values of pixels and waveforms captured in images and audio files. We store related data in
datasets. For example, we might have a dataset of the following:

 Images of cats
 Housing prices
 Weather information
Datasets are made up of individual examples that contain features and a label. You could think of an
example as analogous to a single row in a spreadsheet. Features are the values that a supervised
model uses to predict the label. The label is the "answer," or the value we want the model to predict.
In a weather model that predicts rainfall, the features could
be latitude, longitude, temperature, humidity, cloud coverage, wind direction, and atmospheric
pressure. The label would be rainfall amount.

Examples that contain both features and a label are called labeled examples.

Two labeled examples

In contrast, unlabeled examples contain features, but no label. After you create a model, the model
predicts the label from the features.

Two unlabeled examples

Dataset characteristics
A dataset is characterized by its size and diversity. Size indicates the number of examples. Diversity
indicates the range those examples cover. Good datasets are both large and highly diverse.

Datasets can be large and diverse, or large but not diverse, or small but highly diverse. In other
words, a large dataset doesn't guarantee sufficient diversity, and a dataset that is highly diverse
doesn't guarantee sufficient examples.
For instance, a dataset might contain 100 years worth of data, but only for the month of July. Using
this dataset to predict rainfall in January would produce poor predictions. Conversely, a dataset might
cover only a few years but contain every month. This dataset might produce poor predictions because
it doesn't contain enough years to account for variability.

Check Your Understanding

What attributes of a dataset would be ideal to use for ML?
Small size / High diversity
Large size / Low diversity
Large size / High diversity
Small size / Low diversity

A dataset can also be characterized by the number of its features. For example, some weather
datasets might contain hundreds of features, ranging from satellite imagery to cloud coverage values.
Other datasets might contain only three or four features, like humidity, atmospheric pressure, and
temperature. Datasets with more features can help a model discover additional patterns and make
better predictions. However, datasets with more features don't always produce models that make
better predictions because some features might have no causal relationship to the label.

Model
In supervised learning, a model is the complex collection of numbers that define the mathematical
relationship from specific input feature patterns to specific output label values. The model discovers
these patterns through training.

Training
Before a supervised model can make predictions, it must be trained. To train a model, we give the
model a dataset with labeled examples. The model's goal is to work out the best solution for
predicting the labels from the features. The model finds the best solution by comparing its predicted
value to the label's actual value. Based on the difference between the predicted and actual values—
defined as the loss—the model gradually updates its solution. In other words, the model learns the
mathematical relationship between the features and the label so that it can make the best predictions
on unseen data.

For example, if the model predicted 1.15 inches of rain, but the actual value was .75 inches, the
model modifies its solution so its prediction is closer to .75 inches. After the model has looked at
each example in the dataset—in some cases, multiple times—it arrives at a solution that makes the
best predictions, on average, for each of the examples.

The following demonstrates training a model:

1. The model takes in a single labeled example and provides a prediction.

 Figure 1. An ML model making a prediction from a labeled example.

2. The model compares its predicted value with the actual value and updates its solution.

Figure 2. An ML model updating its predicted value.

3. The model repeats this process for each labeled example in the dataset.
 Figure 3. An ML model updating its predictions for each labeled example in the training
dataset.

In this way, the model gradually learns the correct relationship between the features and the label.
This gradual understanding is also why large and diverse datasets produce a better model. The
model has seen more data with a wider range of values and has refined its understanding of the
relationship between the features and the label.

During training, ML practitioners can make subtle adjustments to the configurations and features the
model uses to make predictions. For example, certain features have more predictive power than
others. Therefore, ML practitioners can select which features the model uses during training. For
example, suppose a weather dataset containstime_of_day as a feature. In this case, an ML
practitioner can add or remove time_of_day during training to see whether the model makes better
predictions with or without it.

Evaluating
We evaluate a trained model to determine how well it learned. When we evaluate a model, we use a
labeled dataset, but we only give the model the dataset's features. We then compare the model's
predictions to the label's true values.
Figure 4. Evaluating an ML model by comparing its predictions to the actual values.

Depending on the model's predictions, we might do more training and evaluating before deploying the
model in a real-world application.

Check Your Understanding

Why does a model need to be trained before it can make predictions?
A model doesn't need to be trained. Models are available on most computers.
A model needs to be trained so it won't require data to make a prediction.
A model needs to be trained to learn the mathematical relationship between the features and the label in a
dataset.

Inference
Once we're satisfied with the results from evaluating the model, we can use the model to make
predictions, called inferences, on unlabeled examples. In the weather app example, we would give
the model the current weather conditions—like temperature, atmospheric pressure, and relative
humidity—and it would predict the amount of rainfall.

Key Terms:

 example
 feature
 inference
 labeled example
 label
 loss
 prediction
 training

Large language models

bookmark_border
 On this page
 What is a language model?
o N-gram language models
 Context
o Recurrent neural networks
  Exercise: Check your understanding

Estimated module length: 45 minutesLearning objectives

 Define a few different types of language models and their components.

 Describe how large language models are created and the importance of context and parameters.
 Identify how large language models take advantage of self-attention.
 Reveal three key problems with large language models.
 Explain how fine-tuning and distillation can improve a model's predictions and efficiency.
Prerequisites:

This module assumes you are familiar with the concepts covered in the following modules:

 Introduction to Machine Learning

 Linear regression
 Working with categorical data
 Datasets, generalization, and overfitting
 Neural networks
 Embeddings

What is a language model?

A language model estimates the probability of a token or sequence of tokens occurring within a
longer sequence of tokens. A token could be a word, a subword (a subset of a word), or even a single
character.

Click the icon to learn more about tokens.

Consider the following sentence and the token(s) that might complete it:

When I hear rain on my roof, I _______ in my kitchen.

A language model determines the probabilities of different tokens or sequences of tokens to complete
that blank. For example, the following probability table identifies some possible tokens and their
probabilities:

Probability Token(s)

9.4% cook soup

5.2% warm up a kettle

 3.6% cower

2.5% nap

2.2% relax

In some situations, the sequence of tokens could be an entire sentence, paragraph, or even an entire
essay.

An application can use the probability table to make predictions. The prediction might be the highest
probability (for example, "cook soup") or a random selection from tokens having a probability greater
than a certain threshold.

Estimating the probability of what fills in the blank in a text sequence can be extended to more
complex tasks, including:

 Generating text.
 Translating text from one language to another.
 Summarizing documents.

By modeling the statistical patterns of tokens, modern language models develop extremely powerful
internal representations of language and can generate plausible language.

N-gram language models

N-grams are ordered sequences of words used to build language models, where N is the number of
words in the sequence. For example, when N is 2, the N-gram is called a 2-gram (or a bigram);
when N is 5, the N-gram is called a 5-gram. Given the following phrase in a training document:

you are very nice

The resulting 2-grams are as follows:

 you are
 are very
 very nice

When N is 3, the N-gram is called a 3-gram (or a trigram). Given that same phrase, the resulting 3-
grams are:

 you are very

 are very nice
Given two words as input, a language model based on 3-grams can predict the likelihood of the third
word. For example, given the following two words:

orange is

A language model examines all the different 3-grams derived from its training corpus that start
with orange is to determine the most likely third word. Hundreds of 3-grams could start with the two
words orange is, but you can focus solely on the following two possibilities:

orange is ripe
orange is cheerful

The first possibility (orange is ripe) is about orange the fruit, while the second possibility (orange
is cheerful) is about the color orange.

Context

Humans can retain relatively long contexts. While watching Act 3 of a play, you retain knowledge of
characters introduced in Act 1. Similarly, the punchline of a long joke makes you laugh because you
can remember the context from the joke's setup.

In language models, context is helpful information before or after the target token. Context can help a
language model determine whether "orange" refers to a citrus fruit or a color.

Context can help a language model make better predictions, but does a 3-gram provide sufficient
context? Unfortunately, the only context a 3-gram provides is the first two words. For example, the
two words orange is doesn't provide enough context for the language model to predict the third
word. Due to lack of context, language models based on 3-grams make a lot of mistakes.

Longer N-grams would certainly provide more context than shorter N-grams. However, as N grows,
the relative occurrence of each instance decreases. When N becomes very large, the language
model typically has only a single instance of each occurrence of N tokens, which isn't very helpful in
predicting the target token.

Recurrent neural networks

Recurrent neural networks provide more context than N-grams. A recurrent neural network is a type
of neural network that trains on a sequence of tokens. For example, a recurrent neural network
can gradually learn (and learn to ignore) selected context from each word in a sentence, kind of like
you would when listening to someone speak. A large recurrent neural network can gain context from
a passage of several sentences.

Although recurrent neural networks learn more context than N-grams, the amount of useful context
recurrent neural networks can intuit is still relatively limited. Recurrent neural networks evaluate
information "token by token." In contrast, large language models—the topic of the next section—can
evaluate the whole context at once.

Note that training recurrent neural networks for long contexts is constrained by the vanishing
gradient problem.
Exercise: Check your understanding

Which language model makes better predictions for English text?

 A language model based on 6-grams

 A language model based on 5-grams

The language model based on 6-grams.

The language model based on 5-grams.
The answer depends on the size and diversity of the training set.
Key terms:

 Bigram
 Language model
 N-gram
 Neural network
 Recurrent neural network
 Token
 Trigram
 Vanishing gradient problem

LLMs: What's a large language model?

bookmark_border
 On this page
 What's a Transformer?
o What is self-attention?
o What is multi-head self-attention?
 Why are Transformers so large?
 But how does an LLM generate text?
 Benefits of LLMs
 Problems with LLMs
 Exercise: Check your understanding


A newer technology, large language models (LLMs) predict a token or sequence of tokens,
sometimes many paragraphs worth of predicted tokens. Remember that a token can be a word, a
subword (a subset of a word), or even a single character. LLMs make much better predictions than N-
gram language models or recurrent neural networks because:

 LLMs contain far more parameters than recurrent models.

 LLMs gather far more context.
This section introduces the most successful and widely used architecture for building LLMs: the
Transformer.

What's a Transformer?

Transformers are the state-of-the-art architecture for a wide variety of language model applications,
such as translation:

Figure 1. A Transformer-based application that translates from English to French.

Full transformers consist of an encoder and a decoder:

 An encoder converts input text into an intermediate representation. An encoder is an enormous neural
net.
 A decoder converts that intermediate representation into useful text. A decoder is also an enormous
neural net.

For example, in a translator:

 The encoder processes the input text (for example, an English sentence) into some intermediate
representation.
 The decoder converts that intermediate representation into output text (for example, the equivalent
French sentence).

Figure 2. A full Transformer contains both an encoder and a decoder.

Click the icon to learn more about partial Transformers.

What is self-attention?
To enhance context, Transformers rely heavily on a concept called self-attention. Effectively, on
behalf of each token of input, self-attention asks the following question:

"How much does each other token of input affect the interpretation of this token?"

The "self" in "self-attention" refers to the input sequence. Some attention mechanisms weight
relations of input tokens to tokens in an output sequence like a translation or to tokens in some other
sequence. But self-attention only weights the importance of relations between tokens in the input
sequence.

To simplify matters, assume that each token is a word and the complete context is only a single
sentence. Consider the following sentence:

The animal didn't cross the street because it was too tired.

The preceding sentence contains eleven words. Each of the eleven words is paying attention to the
other ten, wondering how much each of those ten words matters to itself. For example, notice that the
sentence contains the pronoun it. Pronouns are often ambiguous. The pronoun it typically refers to a
recent noun or noun phrase, but in the example sentence, which recent noun does it refer to—the
animal or the street?

The self-attention mechanism determines the relevance of each nearby word to the pronoun it. Figure
3 shows the results—the bluer the line, the more important that word is to the pronoun it. That
is, animal is more important than street to the pronoun it.

Figure 3. Self-attention for the pronoun it. From Transformer:

A Novel Neural Network Architecture for Language Understanding.

Conversely, suppose the final word in the sentence changes as follows:

The animal didn't cross the street because it was too wide.
In this revised sentence, self-attention would hopefully rate street as more relevant than animal to
the pronoun it.

Some self-attention mechanisms are bidirectional, meaning that they calculate relevance scores for
tokens preceding and following the word being attended to. For example, in Figure 3, notice that
words on both sides of it are examined. So, a bidirectional self-attention mechanism can gather
context from words on either side of the word being attended to. By contrast, a unidirectional self-
attention mechanism can only gather context from words on one side of the word being attended to.
Bidirectional self-attention is especially useful for generating representations of whole sequences,
while applications that generate sequences token-by-token require unidirectional self-attention. For
this reason, encoders use bidirectional self-attention, while decoders use unidirectional.

What is multi-head self-attention?

Each self-attention layer is typically comprised of multiple self-attention heads. The output of a layer
is a mathematical operation (for example, weighted average or dot product) of the output of the
different heads.

Since each self-attention layer is initialized to random values, different heads can learn different
relationships between each word being attended to and the nearby words. For example, the self-
attention layer described in the previous section focused on determining which noun the
pronoun it referred to. However, other self-attention layers might learn the grammatical relevance of
each word to every other word, or learn other interactions.

Click the icon to learn about Big O for LLMs.

Click the icon to learn more about how LLMs are trained.

Why are Transformers so large?

Transformers contain hundreds of billion or even trillions of parameters. This course has generally
recommended building models with a smaller number of parameters over those with a larger number
of parameters. After all, a model with a smaller number of parameters uses fewer resources to make
predictions than a model with a larger number of parameters. However, research shows that
Transformers with more parameters consistently outperform Transformers with fewer parameters.

But how does an LLM generate text?

You've seen how researchers train LLMs to predict a missing word or two, and you might be
unimpressed. After all, predicting a word or two is essentially the autocomplete feature built into
various text, email, and authoring software. You might be wondering how LLMs can generate
sentences or paragraphs or haikus about arbitrage.
In fact, LLMs are essentially autocomplete mechanisms that can automatically predict (complete)
thousands of tokens. For example, consider a sentence followed by a masked sentence:

My dog, Max, knows how to perform many traditional dog tricks.

___ (masked sentence)

An LLM can generate probabilities for the masked sentence, including:

Probability Word(s)

3.1% For example, he can sit, stay, and roll over.

2.9% For example, he knows how to sit, stay, and roll over.

A sufficiently large LLM can generate probabilities for paragraphs and entire essays. You can think of
a user's questions to an LLM as the "given" sentence followed by an imaginary mask. For example:

User's question: What is the easiest trick to teach a dog?

LLM's response: ___

The LLM generates probabilities for various possible responses.

As another example, an LLM trained on a massive number of mathematical "word problems" can give
the appearance of doing sophisticated mathematical reasoning. However, those LLMs are basically
just autocompleting a word problem prompt.

Benefits of LLMs

LLMs can generate clear, easy-to-understand text for a wide variety of target audiences. LLMs can
make predictions on tasks they are explicitly trained on. Some researchers claim that LLMs can also
make predictions for input they were not explicitly trained on, but other researchers have refuted this
claim.

Problems with LLMs

Training an LLM entails many problems, including:

 Gathering an enormous training set.

 Consuming multiple months and enormous computational resources and electricity.
 Solving parallelism challenges.

Using LLMs to infer predictions causes the following problems:

 LLMs hallucinate, meaning their predictions often contain mistakes.

  LLMs consume enormous amounts of computational resources and electricity. Training LLMs on larger
datasets typically reduces the amount of resources required for inference, though the larger training
sets incur more training resources.
 Like all ML models, LLMs can exhibit all sorts of bias.

Exercise: Check your understanding

Suppose a Transformer is trained on a billion documents, including thousands of documents containing at least one
instance of the word elephant. Which of the following statements are probably true?
The Transformer will associate the word elephant with various idioms that contain the word elephant.
The Transformer will gradually learn to ignore any sarcastic or ironic uses of the word elephant in training data.
Acacia trees, an important part of an elephant's diet, will gradually gain a high self-attention score with the
word elephant.
Key terms:

 Decoder
 Embedding layer
 Encoder
 Hallucination
 Instruction tuning
 Large language model (LLM)
 Neural net
 Parameter
 Self attention
 Tensor Processing Unit (TPU)
 Tensor Processing Unit (TPU) Pod
 Unsupervised learning

LLMs: Fine-tuning, distillation, and prompt

engineering

bookmark_border
 On this page
 Fine-tuning
 Distillation
 Prompt engineering
  Offline inference
 Use LLMs responsibly
 Exercise: Check your understanding


The previous unit described general-purpose LLMs, variously known as:

 foundation LLMs
 base LLMs
 pre-trained LLMs

A foundation LLM is trained on enough natural language to "know" a remarkable amount about
grammar, words, and idioms. A foundation language model can generate helpful sentences about
topics it is trained on. Furthermore, a foundation LLM can perform certain tasks traditionally called
"creative," like writing poetry. However, a foundation LLM's generative text output isn't a solution for
other kinds of common ML problems, such as regression or classification. For these use cases, a
foundation LLM can serve as a platform rather than a solution.

Transforming a foundation LLM into a solution that meets an application's needs requires a process
called fine-tuning. A secondary process called distillation generates a smaller (fewer parameters)
version of the fine-tuned model.

Fine-tuning

Research shows that the pattern-recognition abilities of foundation language models are so powerful
that they sometimes require relatively little additional training to learn specific tasks. That additional
training helps the model make better predictions on a specific task. This additional training,
called fine-tuning, unlocks an LLM's practical side.

Fine-tuning trains on examples specific to the task your application will perform. Engineers can
sometimes fine-tune a foundation LLM on just a few hundred or a few thousand training examples.

Despite the relatively tiny number of training examples, standard fine-tuning is often computationally
expensive. That's because standard fine-tuning involves updating the weight and bias of every
parameter on each backpropagation iteration. Fortunately, a smarter process called parameter-
efficient tuning can fine-tune an LLM by adjusting only a subset of parameters on each
backpropagation iteration.

A fine-tuned model's predictions are usually better than the foundation LLM's predictions. However, a
fine-tuned model contains the same number of parameters as the foundation LLM. So, if a foundation
LLM contains ten billion parameters, then the fine-tuned version will also contain ten billion
parameters.
Distillation

Most fine-tuned LLMs contain enormous numbers of parameters. Consequently, foundation LLMs
require enormous computational and environmental resources to generate predictions. Note that
large swaths of those parameters are typically irrelevant for a specific application.

Distillation creates a smaller version of an LLM. The distilled LLM generates predictions much faster
and requires fewer computational and environmental resources than the full LLM. However, the
distilled model's predictions are generally not quite as good as the original LLM's predictions. Recall
that LLMs with more parameters almost always generate better predictions than LLMs with fewer
parameters.

Click the icon to learn how distillation works.

Prompt engineering

Prompt engineering enables an LLM's end users to customize the model's output. That is, end
users clarify how the LLM should respond to their prompt.

Humans learn well from examples. So do LLMs. Showing one example to an LLM is called one-shot
prompting. For example, suppose you want a model to use the following format to output a fruit's
family:

User inputs the name of a fruit: LLM outputs that fruit's class.

A one-shot prompt shows the LLM a single example of the preceding format and then asks the LLM
to complete a query based on that example. For instance:

peach: drupe
apple: ______

A single example is sometimes sufficient. If it is, the LLM outputs a useful prediction. For instance:

apple: pome

In other situations, a single example is insufficient. That is, the user must show the
LLM multiple examples. For instance, the following prompt contains two examples:

plum: drupe
pear: pome
lemon: ____

Providing multiple examples is called few-shot prompting. You can think of the first two lines of the
preceding prompt as training examples.

Can an LLM provide useful predictions with no examples (zero-shot prompting)? Sometimes, but
LLMs like context. Without context, the following zero-shot prompt might return information about the
technology company rather than the fruit:
apple: _______
Note: Prompt engineering doesn't alter the model's parameters. Prompts leverage the pattern-recognition
abilities of the existing LLM.

Offline inference

The number of parameters in an LLM is sometimes so large that online inference is too slow to be
practical for real-world tasks like regression or classification. Consequently, many engineering teams
rely on offline inference (also known as bulk inference or static inference) instead. In other words,
rather than responding to queries at serving time, the trained model makes predictions in advance
and then caches those predictions.

It doesn't matter if it takes a long time for an LLM to complete its task if the LLM only has to perform
the task once a week or once a month.

For example, Google Search used an LLM to perform offline inference in order to cache a list of over
800 synonyms for Covid vaccines in more than 50 languages. Google Search then used the cached
list to identify queries about vaccines in live traffic.

Use LLMs responsibly

Like any form of machine learning, LLMs generally share the biases of:

 The data they were trained on.

 The data they were distilled on.

Use LLMs fairly and responsibly, following the guidelines presented in the data modules and
the Fairness module.

Exercise: Check your understanding

Which of the following statements is true about LLMs?

A distilled LLM contains fewer parameters than the foundation language model it sprung from.
As users perform more prompt engineering, the number of parameters in an LLM grows.
A fine-tuned LLM contains fewer parameters than the foundation language model it was trained on.
Key terms:

 Backpropagation
 Distillation
 Few-shot prompting
 Fine-tuning
 Offline inference
 One shot prompting
 Online inference
 Parameter efficient tuning
 Prompt engineering
 Zero-shot prompting