Scribd
Upload
14 views2 pages

Selecting and Using Pre-trained LLMs

This document outlines the steps for setting up and using pre-trained models from the Hugging Face library, including the installation of necessary libraries and the selection of models based on task requirements and hardware constraints. It emphasizes the importance of analyzing model architecture and training data for effective fine-tuning, as well as the computational resources needed for large models. Additionally, it addresses privacy concerns related to using external LLM providers and provides a formula for calculating memory requirements for model parameters.

Uploaded by

milonpch
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
14 views2 pages

Selecting and Using Pre-trained LLMs

This document outlines the steps for setting up and using pre-trained models from the Hugging Face library, including the installation of necessary libraries and the selection of models based on task requirements and hardware constraints. It emphasizes the importance of analyzing model architecture and training data for effective fine-tuning, as well as the computational resources needed for large models. Additionally, it addresses privacy concerns related to using external LLM providers and provides a formula for calculating memory requirements for model parameters.

Uploaded by

milonpch
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Ensure that all necessary software and libraries are installed.

Then Import the required


libraries in your script or notebook. Common libraries include transformers from
Hugging Face, torch for PyTorch, and other utility libraries.

We should decide which model we want to use based on our Task requirements (e.g.,
text generation, text classification, etc.), Hardware constraints (some models are large
and may not fit in memory without a GPU or specialized hardware), Availability on
model hubs like Hugging Face Model Hub.
from transformers import AutoModel, AutoTokenizer

model_name = "bert-base-uncased" # BERT is an encoder-only model


tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

AutoModel.from_pretrained(model_name) can load every type of model


without any predefined head for a specific task. We need to manually add a
classification, generation, or other output layers if required. from_pretrained(...)
methods, automatically download the model weights and configuration from the
repository.

Once downloaded, the model and tokenizer are ready to be used. At this point, you may also
want to move the model to a GPU (if available) for faster processing.
# Check if a GPU is available
device = [Link]("cuda" if [Link].is_available() else "cpu")

# Move model to the chosen device


model = [Link](device)
For very large models, you might need more advanced strategies like model parallelism, 8-bit
quantization, or using libraries like Accelerate or DeepSpeed.

Challenges:
Before selecting a model, it's crucial to analyze its architecture, capabilities, limitations, and
training data. Without this, fine-tuning may not be effective. Examine the datasets used for
pre-training to gauge the model’s understanding of language. These are important as there are
models available specifically for performing code generation, and we do not want to use those
models for finance text classification

Each model has strengths and weaknesses, so choosing one that aligns with the specific
task is essential for the best results.

Pre-trained LLMs demand high computational resources, including powerful CPUs/GPUs


and significant storage. For example, LLaMA 3 8B requires at least 16GB of RAM for
inference.

Privacy concerns make many businesses avoid external LLM providers. Hosting models
locally or using private cloud solutions helps protect sensitive data.

GB Calculation:
To calculate the memory required to store the model’s parameters: Memory (GB) = Model
Parameters × Precision (bytes) / 109 . FP32 (32-bit floating point) → 4 bytes per parameter.​
8-bit quantized model → 1 byte per parameter. But during training this storage increases 4
times as optimizer state and gradients are stored.​

You might also like