Create a MIG with single-host TPU slices

This document describes how to create a managed instance group (MIG) that forms single-host TPU slices. A single-host TPU slice maps to one TPU virtual machine (VM) instance.

You can also create a single-host TPU slice as a standalone TPU instance. For more information, see Create a TPU VM instance.

Before you begin

• Review the limitations for creating MIGs with TPU instances.
• If you haven't already, set up authentication. Authentication verifies your identity for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

  1. Install the Google Cloud CLI. After installation, initialize the Google Cloud CLI by running the following command:

     gcloud init

     If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

  2. Set a default region and zone.

Prerequisites

Before you proceed to create a single-host TPU slice, you must do the following:

1. Choose the TPU version: select the TPU version that is suitable for your workload. For a list of TPU versions by workload type, see Recommended TPU versions by workload types.

2. Validate TPU availability in your preferred location: TPUs are available in specific Google Cloud regions. To use a TPU version, ensure its availability in your preferred region. For a list of TPU locations, see TPU availability.

3. Ensure that your project has sufficient TPU quota: if you are creating a single-host TPU slice with on-demand or Spot VMs, you must have sufficient TPU quota available in the region that you want to use. Creating a single-host TPU slice that consumes a TPU reservation doesn't require any TPU quota as quota is used when the reservation was created. For a list of TPU quota names, see TPU quota and for instructions on how to view the quota, see View and manage quotas

4. Choose a TPU consumption option: select a consumption option that best fits your workload, its duration, and your cost needs. For a list of consumption option availability by TPU versions, see TPU consumption options.

5. Choose a topology: select a topology supported for the selected TPU version. For a list of topologies available for each TPU version, see TPU topology.

Create a MIG with multiple single-host TPU slices

To create multiple, independent TPU VMs in a MIG:

1. Create an instance template.
2. Create the MIG.

Create an instance template

Instance template configurations vary based on the consumption option you use: on-demand, Spot, reservations, or Flex-start. For more information about consumption options, see About VM provisioning models .

Create an instance template for an on-demand TPU VM

The following command creates an instance template that uses the on-demand consumption option:

gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --machine-type=MACHINE_TYPE \
    --maintenance-policy=TERMINATE \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT

Replace the following placeholders:

• INSTANCE_TEMPLATE_NAME: The name of your instance template.
• MACHINE_TYPE: The machine type for the TPU VM (for example, ct6e-standard-8t).
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.

Create an instance template for a TPU Spot VM

The following command creates an instance template that uses the Spot consumption option:

 gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --machine-type=MACHINE_TYPE \
    --maintenance-policy=TERMINATE \
    --instance-termination-action=STOP \
    --provisioning-model=SPOT \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT

Replace the following placeholders:

• INSTANCE_TEMPLATE_NAME: The name of your instance template.
• MACHINE_TYPE: The machine type for the TPU VM (for example, ct6e-standard-8t).
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.

Create an instance template for a TPU reservation-bound VM

The following command creates an instance template that uses the reservation-bound consumption option:

 gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --machine-type=MACHINE_TYPE \
    --maintenance-policy=TERMINATE \
    --instance-termination-action=DELETE \
    --reservation-affinity=specific \
    --provisioning-model=reservation-bound \
    --reservation=RESERVATION_NAME \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT

Replace the following placeholders:

• INSTANCE_TEMPLATE_NAME: The name of your instance template.
• MACHINE_TYPE: The machine type for the TPU VM (for example, ct6e-standard-8t).
• RESERVATION_NAME: The name of a specific reservation that you want to consume.
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.

Create an instance template for a TPU Flex-start VM

The following command creates an instance template that uses the Flex-start consumption option:

 gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --machine-type=MACHINE_TYPE \
    --maintenance-policy=TERMINATE \
    --instance-termination-action=DELETE \
    --provisioning-model=FLEX_START \
    --max-run-duration=DURATION \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT

Replace the following placeholders:

• INSTANCE_TEMPLATE_NAME: The name of your instance template.
• MACHINE_TYPE: The machine type for the TPU VM (for example, ct6e-standard-8t).
• DURATION: The maximum run duration for the TPU VM (for example, 1h).
• IMAGE_FAMILY: The OS image family for the TPU VM. If you want to install a specific OS version, use the --image flag. For more information about OS images, see OS images.
• IMAGE_PROJECT: The project that contains the OS image. For TPU images, this is ubuntu-os-accelerator-images.

Create a MIG

Create a zonal or a regional MIG by using the gcloud compute instance-groups managed create command as follows:

• To create a zonal MIG containing a single-host TPU slice, use the following command:

  gcloud compute instance-groups managed create MIG_NAME \
      --size=MIG_SIZE \
      --template=INSTANCE_TEMPLATE_URL \
      --zone=ZONE
  

• To create a regional MIG containing a single-host TPU slice, use the following command:

  gcloud compute instance-groups managed create MIG_NAME \
      --size=MIG_SIZE \
      --template=INSTANCE_TEMPLATE_URL \
      --region=REGION \
      --target-distribution-shape=ANY_SINGLE_ZONE \
      --instance-redistribution-type=none
  

Replace the following placeholders:

• MIG_NAME: The name of your MIG.
• MIG_SIZE: The number of VMs in the MIG.
• INSTANCE_TEMPLATE_URL: the URL of the instance template that you want to use to create instances in the MIG. The URL can contain either the ID or name of the instance template. Specify one of the following values:
  • For a regional instance template: projects/PROJECT_ID/regions/REGION/instanceTemplates/INSTANCE_TEMPLATE_ID
  • For a global instance template: INSTANCE_TEMPLATE_ID
• ZONE: The zone where you want your MIG to be provisioned.
• REGION: The region where you want your MIG to be provisioned.

Create VMs with custom names in a MIG

You can create VMs in a MIG by specifying custom names for each VM. This is useful for debugging and ensuring instances are created in a specific order.