Scribd
Upload
11 views4 pages

Qwen2.5-VL Deployment on SageMaker Guide

This document provides a step-by-step guide for deploying the Qwen2.5-VL model on Amazon SageMaker, starting with AWS setup and IAM role configuration. It details the process of creating a SageMaker notebook instance, installing required dependencies, loading and deploying the model, performing inference, executing training tasks, and finally cleaning up resources. Each step includes specific commands and configurations necessary for successful deployment and operation.

Uploaded by

Haris Shahnawaz
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
11 views4 pages

Qwen2.5-VL Deployment on SageMaker Guide

This document provides a step-by-step guide for deploying the Qwen2.5-VL model on Amazon SageMaker, starting with AWS setup and IAM role configuration. It details the process of creating a SageMaker notebook instance, installing required dependencies, loading and deploying the model, performing inference, executing training tasks, and finally cleaning up resources. Each step includes specific commands and configurations necessary for successful deployment and operation.

Uploaded by

Haris Shahnawaz
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Step-by-Step Deployment Guide for Qwen2.

5-VL on Amazon SageMaker


Step 1: AWS Setup and Prerequisites
1. Create an AWS Account (if you don’t have one).
2. Set Up IAM Roles and Permissions:
o Go to IAM Console → Create a role for SageMaker.

o Attach policies like:

 AmazonSageMakerFullAccess
 AmazonS3FullAccess (if using S3 for model storage)
 AmazonEC2ContainerRegistryFullAccess (if using ECR for custom
containers)
o Note down the IAM role ARN.

3. Set Up a VPC (if needed):


o If working in a restricted environment, configure a VPC, security
groups, and subnets.

Step 2: Start a SageMaker Notebook Instance


1. Go to AWS SageMaker Console → Notebook Instances.
2. Click Create notebook instance.
3. Choose an appropriate instance type:
o ml.g5.12xlarge or ml.p4d.24xlarge for Qwen2.5-VL-72B.

o ml.g5.4xlarge or ml.p4d.24xlarge for Qwen2.5-VL-7B.

o ml.g5.2xlarge for Qwen2.5-VL-3B.

4. Assign the IAM role created earlier.


5. Enable Git Repositories if you want to use version control.

Step 3: Install Required Dependencies


1. Open a Jupyter Notebook.
2. Run the following command to install dependencies:
bash
CopyEdit
!pip install torch torchvision torchaudio transformers accelerate sagemaker

Step 4: Load and Deploy Qwen2.5-VL Model


Option 1: Load Model from Hugging Face
python
CopyEdit
from transformers import AutoModelForVision2Seq, AutoProcessor
import torch

model_name = "Qwen/Qwen2.5-VL-7B-Instruct" # Change model as needed

processor = AutoProcessor.from_pretrained(model_name)
model = AutoModelForVision2Seq.from_pretrained(model_name,
torch_dtype=torch.float16, device_map="auto")
Option 2: Deploy Model on SageMaker Endpoint
1. Create a SageMaker Model
python
CopyEdit
from [Link] import HuggingFaceModel

huggingface_model = HuggingFaceModel(
model_data=f"s3://your-bucket/{model_name}",
role="your-sagemaker-role",
transformers_version="4.33",
pytorch_version="2.1",
py_version="py39",
)
2. Deploy the Model to an Endpoint
python
CopyEdit
predictor = huggingface_model.deploy(
initial_instance_count=1,
instance_type="ml.g5.4xlarge"
)

Step 5: Perform Inference


python
CopyEdit
from PIL import Image

image_path = "your_image.jpg"
image = [Link](image_path)

inputs = processor(images=image, return_tensors="pt").to("cuda")


output = [Link](**inputs)

print(processor.batch_decode(output, skip_special_tokens=True))

Step 6: Execute Training Tasks


If you need fine-tuning, you’ll need:
 A labeled dataset (in JSON or CSV format)
 A SageMaker Training Job
Example training script:
python
CopyEdit
from [Link] import PyTorch

estimator = PyTorch(
entry_point="[Link]",
role="your-sagemaker-role",
instance_count=1,
instance_type="ml.p4d.24xlarge",
framework_version="2.1",
py_version="py39",
hyperparameters={"epochs": 3, "batch_size": 8}
)

[Link]({"training": "s3://your-dataset-bucket/train"})

Step 7: Cleanup
After finishing your work:
 Delete the endpoint:
python
CopyEdit
predictor.delete_endpoint()
 Shut down SageMaker Notebook to avoid unnecessary charges.

You might also like