Prompt Engineering Implementation Partner

Foundations of LLMs

Dr. Satya Jayadev P

Principal Data Scientist at Gyan Data

Guest Faculty at IIT Madras
Recap and Roadmap
Quick Recap: What you’ll Learn now:

● Artificial Intelligence enables machines ● How is text generated: Language

to perform useful intelligent tasks models
● Machine Learning enables machines or
● Large Language Models
computers learn patterns from data
● What is a Prompt?
● Deep learning uses neural networks to
enable machines learn more complex ● Text Generation Workflow
patterns
● Generative AI enables creation of new
content such as text, code, images, audio,
and video
What is a Language Model?

● A language model is a system that can generate text

● It is designed to work with language by producing likely continuations based on the
text it has seen
● In simple terms, if we give it some text, it tries to predict what text should come
next
● Why is it called a language model: Models patterns in language
● It learns how words, phrases, and sentences are commonly used together
● Based on these learned patterns, it estimates what text is likely to come next in a
sequence
From Language models to Large Language models
What kinds of tasks need text generation
● Early language models had more
capability?
limited capability and scale
answering
● Techniques to model language summarizing
questions in
long passages
evolved to learn from much large natural
or documents
amount of text language
● This led to Large Language
drafting emails, rewriting text for
Models (LLMs), which are trained
reports, and clarity, tone, or
on very large amounts of text and messages length
can handle a much wider range of
tasks needing text generation generating Translating text
explanations for from one
different language to
audiences another
LLMs are general-purpose text engines

● LLMs are not built separately for every language related task
● The same model can be used for many different tasks
● That is why LLMs are general purpose text engines
● Q: How does the model know which task to perform?
● A: Task to be performed is communicated to the model through an input text from
the user
● The user given input is referred to as Prompt
What is a prompt?
● A prompt is the input we give to the model

● Prompt may include:

○ a question
○ an instruction
○ some background or relevant context
○ an example
○ an expected output style
○ Combination of any of the above
● For example, a prompt could be:
● As simple as “Summarize this paragraph in 3 bullet points” (or)
● As specific as “Explain this concept to a beginner using one analogy and two examples”
Tokenisation and Text Generation
How can Computers handle Text?

● Computers cannot work with raw text in the same way humans read and
understand language
● For a computer, text has to be represented in a numerical form so that it can
be processed inside a model
● This conversion is necessary both when the model reads the input and when it
generates a response
● So, before understanding how LLMs generate text, we need to understand
how text is handled internally
Numerical Form must be Language Dependant

● This numerical form should not be random

● It should help the model capture useful aspects of language such as:
○ words and word parts
○ meaning and usage patterns
○ relationships between words
○ grammar and sentence structure
○ repeated patterns seen across large amounts of text
● In other words, the model needs a systematic way to work with language, not
just text as isolated symbols
Motivation: Text to Tokens

● To process language effectively, the model needs standard units of text that it
can use throughout the workflow
● These units should be suitable both for understanding the input and for
generating the output
● In language models, these units are called tokens
● A token is therefore a basic unit of text used by the model while reading and
generating language
● Note: Generally, for any model, the set of tokens to be used are fixed and
cannot be changed post-training
Tokenization and examples

● Tokenization is the process of breaking text into tokens

● A token may be:
○ a whole word
○ part of a word
○ punctuation
○ a number or symbol
● Example sentence:
○ Prompt engineering is useful.
● Possible tokenization:
○ “Prompt”, “engineering”, “is”, “useful”, “.”
Overall Workflow of Text Generation
The model processes
The model receives The input text is
these tokens and
the prompt tokenized into tokens
predicts the most
Eg: The capital of Eg: “The”, “capital”, likely next token
India is “of”, “India”, “is”
Eg: “New”

That predicted token The updated

is added to the sequence is This continues
sequence processed again to repeatedly until the
Eg: The capital of predict the next token response is complete
India is New Eg: “Delhi”

● Core idea: Language generation in LLMs is built around repeated

next-token prediction
Why understanding tokenization matters
● Both prompts and responses are ultimately handled in terms of tokens
● Long prompts use more tokens and leave less room for the model's response
● Long outputs also consume tokens
● Many systems define input limits and output limits in terms of tokens
● The cost of running LLMs is also often linked to the number of input and
output tokens
● So, understanding tokenization helps us think more clearly about prompt
length, response length, context limits, and usage cost
Key Takeaways & What’s Next
Key takeaways

● LLMs enabled computers perform a wide range of langugage related tasks through text
generation capabilities

● LLMs generate text token by token

● Prompts provide the task for which text is to be generated
● Tokenization enables computers understand and generate text through numerical
encoding and decoding

Up-Next
● Next, we will look more closely on context in prompts
● Understand how LLMs use context while generating text
● Understand transformers, attention, and context windows at high level
Thank you