/

Console

• English
• Deutsch
• Español
• Español – América Latina
• Français
• Indonesia
• Italiano
• Português
• Português – Brasil
• עברית
• 中文 – 简体
• 中文 – 繁體
• 日本語
• 한국어

Sign in

• Compute Engine

Start free

GPU machines in accelerator-optimized machine family

This document describes the Compute Engine instances that have pre-attached NVIDIA GPUs in the accelerator-optimized machine family. These instances are designed specifically for artificial intelligence (AI), machine learning (ML), high performance computing (HPC), and graphics-intensive applications.

The accelerator-optimized machine family includes the following machine series: A4X Max, A4X, A4, A3, A2, G4, and G2. Each machine type within a series has a specific model and number of NVIDIA GPUs attached. You can also attach some GPU models to N1 general-purpose machine types.

For information about accelerator-optimized instances with attached TPUs, see TPU machines in accelerator-optimized machine family.

Recommended machine series by workload type

The following section provides the recommended machine series based on your GPU workloads:

Pricing and consumption options

Consumption options refers to the ways to get and use compute resources. Google Cloud bills accelerator-optimized machine types for their attached GPUs, predefined vCPU, memory, and bundled Local SSD (if applicable). Discounts for accelerator-optimized instances vary based on the consumption option you use. For more pricing information for accelerator-optimized instances, see the Accelerator-optimized machine type family section on the VM instance pricing page.

Discounts for accelerator-optimized instances vary based on the consumption option you choose:

• On-demand: You can receive committed use discounts (CUDs) for some resources by purchasing resource-based commitments. However, GPUs and Local SSD disks that you use with the on-demand option are ineligible for CUDs. To receive CUDs for GPUs and Local SSD disks, use one of the reservation options instead.
• Spot: Spot VMs automatically receive discounts through Spot VMs pricing.
• Flex-start: Instances provisioned by using the Flex-start consumption option automatically receive discounts through Dynamic Workload Scheduler pricing.
• Reservations: You can receive CUDs for your accelerator-optimized machine type resources by purchasing resource-based commitments. Commitments for GPUs and Local SSD disks require attached reservations for those resources.

Consumption option availability by machine type

The following table summarizes the availability of each consumption option by machine types. For more information about how to choose a consumption option, see Choose a consumption model in the AI Hypercomputer documentation.

Maintenance experience for accelerator-optimized machine types

During the lifecycle of a Compute Engine instance, the host machine that your instance runs on undergoes multiple host events. A host event can include the regular maintenance of Compute Engine infrastructure, or in rare cases, a host error. Compute Engine also applies some non-disruptive lightweight upgrades for the hypervisor and network in the background.

The following table describes the host maintenance features for accelerator-optimized machine types:

The maintenance frequencies shown in the previous table are approximations, not guarantees. Compute Engine might occasionally perform maintenance more frequently.

The A4X Max and A4X machine series

The A4X Max and A4X machine series runs on an exascale platform based on NVIDIA's rack-scale architecture and is optimized for compute and memory-intensive, network-bound ML training and HPC workloads. A4X Max and A4X differ primarily in their GPU and networking components. A4X Max is available only as bare metal instances, which provide direct access to the host server's CPU and memory, without Compute Engine's hypervisor in the middle.

All machine types in the A4X Max and A4X series have two sockets with NVIDIA Grace™ CPUs with Arm® Neoverse™ V2 cores. These CPUs connect to four GPUs with fast chip-to-chip NVLink-C2 communication.

NVLink domain

Both A4X Max and A4X machine series are built on NVIDIA's NVL72 rack-scale architecture, which uses NVLink domains to enable large-scale, high-performance GPU computing. An NVLink Domain is a group of interconnected NVIDIA NVSwitch chips and the GPUs that connect to them, forming a high-speed network fabric that allows for direct and fast communication between GPUs. For A4X Max and A4X machine types, a single NVL72 (NVLink) Domain is composed of 18 instances and 72 GPUs.

A4X Max and A4X comparison

The following table provides a detailed comparison of the A4X Max and A4X machine types:

A4X Max machine type (bare metal)

A4X Max accelerator-optimized machine types use NVIDIA GB300 Grace Blackwell Ultra Superchips (nvidia-gb300) and are ideal for foundation model training and serving. A4X Max machine types are available as bare metal instances.

A4X Max is an exascale platform based on NVIDIA GB300 NVL72. Each machine has two sockets with NVIDIA Grace CPUs with Arm Neoverse V2 cores. These CPUs are connected to four NVIDIA B300 Blackwell GPUs with fast chip-to-chip (NVLink-C2C) communication.

A4X machine type

A4X accelerator-optimized machine types use NVIDIA GB200 Grace Blackwell Superchips (nvidia-gb200) and are ideal for foundation model training and serving.

A4X is an exascale platform based on NVIDIA GB200 NVL72. Each machine has two sockets with NVIDIA Grace CPUs with Arm Neoverse V2 cores. These CPUs are connected to four NVIDIA B200 Blackwell GPUs with fast chip-to-chip (NVLink-C2C) communication.

A4X Max and A4X limitations

The following limitations apply to A4X Max and A4X instances:

Supported disk types for A4X Max and A4X instances

Disk and capacity limits

You can attach a mixture of different Hyperdisk types to an instance, but the maximum total disk capacity (in TiB) across all disk types can't exceed 512 TiB for all Hyperdisks.

For details about the capacity limits, see Hyperdisk size and attachment limits.

The A4 machine series

The A4 machine series offers machine types with up to 224 vCPUs, and 3,968 GB of memory. A4 instances provide up to 3x performance of previous GPU instance types for most GPU accelerated workloads. A4 is recommended for ML training workloads especially at large scales—for example, hundreds or thousands of GPUs. The A4 machine series is available in a single machine type.

VM instances created by using the A4 machine type provide the following features:

• GPU acceleration with NVIDIA B200 GPUs: NVIDIA B200 GPUs are automatically attached to A4 instances, which offer 180 GB GPU memory per GPU.

• 5th Generation Intel Xeon Scalable Processor (Emerald Rapids): offers up to 4.0 GHz sustained single-core max turbo frequency. For more information about this processor, see CPU platform.

• Industry-leading NVLink scalability: NVIDIA B200 GPUs provide GPU NVLink bandwidth of 1,800 GBps, bidirectionally per GPU.

  With all-to-all NVLink topology between 8 GPUs in a system, the aggregate NVLink Bandwidth is up to 14.4 TBps.

• Enhanced networking with RoCE: RDMA over Converged Ethernet (RoCE) increases the network performance by combining NVIDIA ConnectX-7 network interface cards (NICs) with Google's datacenter-wide four-way rail-aligned network. By leveraging RDMA over Converged Ethernet (RoCE), A4 instances achieve much higher throughput between instances in a cluster compared to most A3 instances, except those running on the A3 Ultra machine type.

• Increased network speeds: Offers up to 4x networking speeds when compared to the previous generation A2 instances.

  For more information about networking, see Network bandwidths and GPUs.

• Virtualization optimizations for data transfers and recovery: the Peripheral Component Interconnect Express (PCIe) topology of A4 instances provides more accurate locality information that workloads can use to optimize data transfers.

  The GPUs also expose Function Level Reset (FLR) for graceful recovery from failures and atomic operations support for concurrency improvements in certain scenarios.

• Disk support: A4 instances support Local SSD for fast scratch disks, which is useful for feeding data into GPUs while preventing I/O bottlenecks. For durable storage, you can attach Hyperdisk volumes.

  12,000 GiB of Local SSD is automatically added to A4 instances. For workloads that require durable block storage, you can also attach up to 512 TiB of Hyperdisk to A4 instances. For more information about disk types, see Choose a disk type.

• Dense allocation and topology aware scheduling support: When you provision A4 instances, you can request blocks of densely allocated capacity. Your host machines are allocated physically close to each other, provisioned as blocks of resources, and are interconnected with a dynamic ML network fabric to minimize network hops and optimize for the lowest latency. Additionally, you can get topology information at node and cluster level that can be used for job placement.

A4 machine type

A4 accelerator-optimized machine types have NVIDIA B200 Blackwell GPUs (nvidia-b200) attached and are ideal for foundation model training and serving.

A4 limitations

• You can only request capacity by using the supported consumption options for an A4 machine type.
• You don't receive sustained use discounts and flexible committed use discounts for instances that use an A4 machine type.
• You can only use an A4 machine type in certain regions and zones.
• You can't use Persistent Disk (regional or zonal). You can only use Google Cloud Hyperdisk.
• The A4 machine type is only available on the Emerald Rapids CPU platform.
• You can't change the machine type of an instance to or from A4 machine type. You must create a new instance with this machine type.
• A4 machine types don't support sole-tenancy.
• You can't run Windows operating systems on an A4 machine type.
• For A4 instances, when you use ethtool -S to monitor GPU networking, physical port counters that end in _phy don't update. This is expected behavior for instances that use the MRDMA Virtual Function (VF) architecture. For more information, see MRDMA functions and network monitoring tools.
• You can't attach Hyperdisk ML disks that were created before February 4, 2026 to A4 machine types.

Supported disk types for A4 instances

A4 instances can use the following block storage types:

• Hyperdisk Balanced (hyperdisk-balanced): this is the only disk type that is supported for the boot disk
• Hyperdisk Extreme (hyperdisk-extreme)
• Hyperdisk ML (hyperdisk-ml)
• Local SSD: which is automatically added to instances that are created by using any of the A4 machine types

Disk and capacity limits

You can attach a mixture of different Hyperdisk types to an instance, but the maximum total disk capacity (in TiB) across all disk types can't exceed 512 TiB for all Hyperdisks.

For details about the capacity limits, see Hyperdisk size and attachment limits.

The A3 machine series

The A3 machine series has up to 224 vCPUs, and 2,944 GB of memory. This machine series is optimized for compute and memory intensive, network bound ML training, and HPC workloads. The A3 machine series is available in A3 Ultra, A3 Mega, A3 High, and A3 Edge machine types.

VM instances created by using the A3 machine types provide the following features:

A3 Ultra machine type

A3 Ultra machine types have NVIDIA H200 SXM GPUs (nvidia-h200-141gb) attached and provides the highest network performance in the A3 series. A3 Ultra machine types are ideal for foundation model training and serving.

A3 Ultra limitations

• You can only request capacity by using the supported consumption options for an A3 Ultra machine type.
• You don't receive sustained use discounts and flexible committed use discounts for instances that use an A3 Ultra machine type.
• You can only use an A3 Ultra machine type in certain regions and zones.
• You can't use Persistent Disk (regional or zonal). You can only use Google Cloud Hyperdisk.
• The A3 Ultra machine type is only available on the Emerald Rapids CPU platform.
• Machine type changes aren't supported for A3 Ultra machine type. To switch to or from this machine type, you must create a new instance.
• You can't run Windows operating systems on an A3 Ultra machine type.
• A3 Ultra machine types don't support sole-tenancy.
• For A3 Ultra instances, when you use ethtool -S to monitor GPU networking, physical port counters that end in _phy don't update. This is expected behavior for instances that use the MRDMA Virtual Function (VF) architecture. For more information, see MRDMA functions and network monitoring tools.

A3 Mega machine type

A3 Mega limitations

• You can only request capacity by using the supported consumption options for an A3 Mega machine type.
• You don't receive sustained use discounts and flexible committed use discounts for instances that use an A3 Mega machine type.
• You can only use an A3 Mega machine type in certain regions and zones.
• You can't use regional Persistent Disk on an instance that uses an A3 Mega machine type.
• The A3 Mega machine type is only available on the Sapphire Rapids CPU platform.
• Machine type changes aren't supported for A3 Mega machine type. To switch to or from this machine type, you must create a new instance.
• You can't run Windows operating systems on an A3 Mega machine type.

A3 High machine type

A3 High limitations

• You can only request capacity by using the supported consumption options for an A3 High machine type.
• You don't receive sustained use discounts and flexible committed use discounts for instances that use an A3 High machine type.
• You can only use an A3 High machine type in certain regions and zones.
• You can't use regional Persistent Disk on an instance that uses an A3 High machine type.
• The A3 High machine type is only available on the Sapphire Rapids CPU platform.
• Machine type changes aren't supported for A3 High machine type. To switch to or from this machine type, you must create a new instance.
• You can't run Windows operating systems on an A3 High machine type.
• For a3-highgpu-1g, a3-highgpu-2g, anda3-highgpu-4g machine types, you must create instances by using Spot VMs or Flex-start VMs. For detailed instructions on these options, review the following:
  • To create Spot VMs, set the provisioning model to SPOT when you create an accelerator-optimized VM.
  • To create Flex-start VMs, you can use one of the following methods:
    • Create a standalone VM and set the provisioning model to FLEX_START when you create an accelerator-optimized VM.
    • Create a resize request in a managed instance group (MIG). For instructions, see Create a MIG with GPU VMs.
• You can only use a Confidential VM with an a3-highgpu-1g machine type in limited regions and zones, and all the limitations for Confidential VM running on the A3 High machine type apply.

A3 Edge machine type

A3 Edge limitations

• You can only request capacity by using the supported consumption options for an A3 Edge machine type.
• You don't receive sustained use discounts and flexible committed use discounts for instances that use an A3 Edge machine type.
• You can only use an A3 Edge machine type in certain regions and zones.
• You can't use regional Persistent Disk on an instance that uses an A3 Edge machine type.
• The A3 Edge machine type is only available on the Sapphire Rapids CPU platform.
• Machine type changes aren't supported for A3 Edge machine type. To switch to or from this machine type, you must create a new instance.
• You can't run Windows operating systems on an A3 Edge machine type.
• A3 Edge machine types don't support sole-tenancy.

Supported disk types for A3 instances

Disk and capacity limits

If supported by the machine type, you can attach a mixture of Hyperdisk and Persistent Disk volumes to an instance, but the following restrictions apply:

• The combined number of both Hyperdisk and Persistent Disk volumes can't exceed 128 per instance.

• The maximum total disk capacity (in TiB) across all disk types can't exceed:

  • For machine types with less than 32 vCPUs:

    • 257 TiB for all Hyperdisk or all Persistent Disk
    • 257 TiB for a mixture of Hyperdisk and Persistent Disk

  • For machine types with 32 or more vCPUs:

    • 512 TiB for all Hyperdisk
    • 512 TiB for a mixture of Hyperdisk and Persistent Disk
    • 257 TiB for all Persistent Disk

For details about the capacity limits, see Hyperdisk size and attachment limits and Persistent Disk maximum capacity.

The A2 machine series

The A2 machine series is available in A2 Standard and A2 Ultra machine types. These machine types have 12 to 96 vCPUs, and up to 1,360 GB of memory.

VM instances created by using the A2 machine types provide the following features:

• GPU acceleration: each A2 instance has NVIDIA A100 GPUs. These are available in both A100 40GB and A100 80GB options.

• Industry-leading NVLink scale that provides peak GPU to GPU NVLink bandwidth of 600 GBps. For example, systems with 16 GPUs have an aggregate NVLink bandwidth of up to 9.6 TBps. These 16 GPUs can be used as a single high performance accelerator with unified memory space to deliver up to 10 petaFLOPS of compute power and up to 20 petaFLOPS of inference compute power that can be used for artificial intelligence, deep learning, and machine learning workloads.

• Improved computing speeds: the attached NVIDIA A100 GPUs offer up to 10x improvements in computing speed when compared to previous generation NVIDIA V100 GPUs.

  With the A2 machine series, you can get up to 100 Gbps network bandwidth.

• Disk support: A2 instances support Local SSD for fast scratch disks, which is useful for feeding data into GPUs while preventing I/O bottlenecks. For durable storage, you can attach Persistent Disk and Hyperdisk volumes.

  Local SSD is supported as follows:

  • For A2 Standard machine types, you can add up to 3,000 GiB of Local SSD when you create an instance.
  • For A2 Ultra machine types, Local SSD is automatically attached when you create an instance.

  For workloads that require durable block storage, you can attach up to 257 TiB of Persistent Disk and 512 TiB of Hyperdisk volumes to A2 instances. For more information about disk types, see Choose a disk type.

• Compact placement policy support: provides you with more control over the physical placement of your instances within data centers. This enables lower-latency and higher bandwidth for instances that are located within a single availability zone. For more information, see Reduce latency by using compact placement policies.

The following machine types are available for the A2 machine series.

A2 Ultra machine types

These machine types have a fixed number of A100 80GB GPUs. Local SSD is automatically attached to instances created by using the A2 Ultra machine types.

A2 Ultra limitations

• You can only request capacity by using the supported consumption options for an A2 Ultra machine type.
• You don't receive sustained use discounts and flexible committed use discounts for instances that use an A2 Ultra machine type.
• You can only use an A2 Ultra machine type in certain regions and zones.
• The A2 Ultra machine type is only available on the Cascade Lake platform.
• If your instance uses an A2 Ultra machine type, you can't change the machine type. If you need to use a different A2 Ultra machine type, or any other machine type, you must create a new instance.
• You can't change any other machine type to an A2 Ultra machine type. If you need a instance that uses an A2 Ultra machine type, you must create a new instance.
• You can't do a quick format of the attached Local SSDs on Windows instances that use A2 Ultra machine types. To format these Local SSDs, you must do a full format by using the diskpart utility and specifying format fs=ntfs label=tmpfs.

A2 Standard machine types

These machine types have a fixed number of A100 40GB GPUs. You can also add Local SSD disks when creating an A2 Standard instance. For the number of disks you can attach, see Machine types that require you to choose a number of Local SSD disks.

A2 Standard limitations

• You can only request capacity by using the supported consumption options for an A2 Standard machine type.
• You don't receive sustained use discounts and flexible committed use discounts for instances that use an A2 Standard machine type.
• You can only use an A2 Standard machine type in certain regions and zones.
• The A2 Standard machine type is only available on the Cascade Lake platform.
• If your instance uses an A2 Standard machine type, you can only switch from one A2 Standard machine type type to another A2 Standard machine type. You can't change to any other machine type. For more information, see Modify accelerator-optimized instances.
• You can't use the Windows operating system with the a2-megagpu-16g machine type. When using a Windows operating system, choose a different A2 Standard machine type.
• You can't do a quick format of the attached Local SSDs on Windows instances that use A2 Standard machine types. To format these Local SSDs, you must do a full format by using the diskpart utility and specifying format fs=ntfs label=tmpfs.

Supported disk types for A2 instances

A2 instances can use the following block storage types:

• Hyperdisk ML (hyperdisk-ml)
• Balanced Persistent Disk (pd-balanced)
• SSD (performance) Persistent Disk (pd-ssd)
• Standard Persistent Disk (pd-standard)
• Local SSD: which is automatically attached to instances created by using the A2 Ultra machine types.

If supported by the machine type, you can attach a mixture of Hyperdisk and Persistent Disk volumes to an instance, but the following restrictions apply:

• The combined number of both Hyperdisk and Persistent Disk volumes can't exceed 128 per instance.

• The maximum total disk capacity (in TiB) across all disk types can't exceed:

  • For machine types with less than 32 vCPUs:

    • 257 TiB for all Hyperdisk or all Persistent Disk
    • 257 TiB for a mixture of Hyperdisk and Persistent Disk

  • For machine types with 32 or more vCPUs:

    • 512 TiB for all Hyperdisk
    • 512 TiB for a mixture of Hyperdisk and Persistent Disk
    • 257 TiB for all Persistent Disk

For details about the capacity limits, see Hyperdisk size and attachment limits and Persistent Disk maximum capacity.

The G4 machine series

The G4 machine series uses the AMD EPYC Turin CPU platform and features NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs. This machine series offers significant improvements over the previous-generation G2 machine series, with considerably more GPU memory, increased GPU memory bandwidth, and higher networking bandwidth.

G4 instances have up to 384 vCPUs, 1,440 GB of memory, and 12 TiB of Titanium SSD disks attached. G4 instances also provide up to 400 Gbps of standard network performance.

This machine series is particularly intended for workloads such as NVIDIA Omniverse simulation workloads, graphics-intensive applications, video transcoding, and virtual desktops. The G4 machine series also provide a low-cost solution for performing single host inference and model tuning compared with A series machine types.

Instances that use the G4 machine type provide the following features:

• GPU acceleration with NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs: G4 instances automatically attach NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs, which offer 96 GB GPU memory per GPU.

• 5th Generation AMD EPYC Turin CPU Platform: this platform offers up to 4.1 GHz sustained max boost frequency. For more information about this processor, see CPU platform.

• Next generation graphics performance: the NVIDIA RTX PRO 6000 GPUs provide significant performance and feature upgrades over the NVIDIA L4 GPUs that are attached to the G2 machine series. Thesed upgrades are as follows:

  • 5th-Generation Tensor Cores: these cores introduce support for FP4 precision and DLSS 4 Multi Frame Generation. By using these 5th Generation tensor cores, NVIDIA RTX PRO 6000 GPUs offers improved performance to accelerate tasks like local LLM development and content creation, compared to NVIDIA L4 GPUs.
  • 4th-Generation RT Cores: these cores deliver up to twice the ray-tracing performance of the previous generation NVIDIA L4 GPUs, accelerating rendering for design and manufacturing workloads.
  • Core count: the NVIDIA RTX PRO 6000 GPU includes 24,064 CUDA cores, 752 5th-gen Tensor cores, and 188 4th-gen RT cores. This update represents a substantial increase over prior generations like the L4 GPU, which has 7,680 CUDA cores and 240 Tensor cores.

• GPU sharing: there are a number of options that you can use to allow multiple workloads to access a single physical GPU. GPU sharing is useful for workloads that don't require the resources of a full GPU, helping you optimize costs. The following GPU sharing options are available for G4 instances:

  • Fractional GPU (vGPU) support: this feature allows a single physical GPU to be shared by multiple virtual machine (VM) instances. vGPUs provide multi-tenant security isolation because each vGPU is a separate VM instance. To use vGPUs, Compute Engine provides the following VM instance shapes: g4-standard-6 (1/8 GPU), g4-standard-12 (1/4 GPU), and g4-standard-24 (1/2 GPU).
  • Multi-Instance GPU (MIG): this feature allows a single GPU to be partitioned into up to four fully isolated instances on a single virtual machine. This option doesn't offer multi-tenant security isolation as all partitions belong to a single VM.

• Peripheral Component Interconnect Express (PCIe) Gen 5 support: G4 instances supports PCI Express Gen 5, which improves the data transfer speed from CPU memory to GPU compared to PCIe Gen 3 used by G2 instances.

• Disk support: G4 instances support Titanium SSD for fast scratch disks, which is useful for feeding data into GPUs while preventing I/O bottlenecks. For durable storage, you can attach Hyperdisk volumes.

  G4 instances support attaching up to 12,000 GiB of Titanium SSD. For workloads that require durable block storage, G4 instances also support attaching up to 512 TiB of Hyperdisk. For more information about disk types, see Choose a disk type.

• GPU Peer-to-Peer (P2P) communication: G4 instances support GPU P2P communication, enabling direct data transfer between GPUs within the same instance. This can significantly improve performance for multi-GPU workloads by reducing data transfer latency and freeing up CPU resources. For more information, see G4 GPU peer-to-peer (P2P) communication.

G4 machine types

G4 accelerator-optimized machine types use NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs (nvidia-rtx-pro-6000) and are suitable for NVIDIA Omniverse simulation workloads, graphics-intensive applications, video transcoding, and virtual desktops. G4 machine types also provide a low-cost solution for performing single host inference and model tuning compared with A series machine types.

G4 limitations

• You can only request capacity by using the supported consumption options for a G4 machine type.
• You don't receive sustained use discounts and flexible committed use discounts for instances that use a G4 machine type.
• You can only use a G4 machine type in certain regions and zones.
• You can't use Persistent Disk (regional or zonal) on an instance that uses a G4 machine type.
• The G4 machine type is only available on the AMD EPYC Turin 5th Generation platform.
• You can't create Confidential VM instances that use a G4 machine type.
• You can't create G4 instances on sole-tenant nodes.
• You can't use Windows operating systems on g4-standard-384 instances.
• You can't attach Hyperdisk ML disks that were created before February 4, 2026 to G4 machine types.
• When creating G4 instances that have less than one GPU attached (fractional GPUs), don't use the --no-service-account or --no-scopes flags. To authenticate NVIDIA vGPU drivers, Compute Engine must verify the VM's identity. This process requires that service accounts are enabled.

Supported disk types for G4 instances

G4 instances can use the following block storage types:

• Hyperdisk Balanced (hyperdisk-balanced): this is the only disk type that is supported for the boot disk
• Hyperdisk Balanced High Availability (hyperdisk-balanced-high-availability)
• Hyperdisk Extreme (hyperdisk-extreme): this disk type is only supported on G4 instances that have two or more GPUs attached
• Hyperdisk ML (hyperdisk-ml)
• Hyperdisk Throughput (hyperdisk-throughput)
• Titanium SSD: you can add Titanium SSD to instances created by using the G4 machine types.

You can attach a mixture of different Hyperdisk types to an instance, but the maximum total disk capacity (in TiB) across all disk types can't exceed 512 TiB for all Hyperdisks.

For details about the capacity limits, see Hyperdisk size and attachment limits.

GPU peer-to-peer (P2P) communication

G4 instances enhance multi-GPU workload performance by using direct GPU peer-to-peer (P2P) communication, which is supported only on machine types with two or more GPUs. This approach allows GPUs that attach to the same G4 instance to exchange data directly over the PCIe bus, bypassing the need to transfer data through the CPU's main memory. This direct path reduces latency, lowers CPU utilization, and increases the effective bandwidth between GPUs. P2P communication significantly accelerates multi-GPU applications such as machine learning (ML) training and high performance computing (HPC).

This feature typically requires no modifications to your application code. You only need to configure NCCL to use P2P. To configure NCCL, before you run your workloads, set the NCCL_P2P_LEVEL environment variable on your G4 instance based on the machine type:

• For G4 instances with 2 or 4 GPUs (g4-standard-96, g4-standard-192): set NCCL_P2P_LEVEL=PHB
• For G4 instances with 8 GPUs (g4-standard-384): set NCCL_P2P_LEVEL=SYS

Set the environment variable using one of the following options:

• On the command line, run the appropriate export command (for example, export NCCL_P2P_LEVEL=SYS) in the shell session where you plan to run your application. To make this setting persistent, add this command to your shell's startup script (for example, ~/.bashrc).
• Add the appropriate setting (for example, NCCL_P2P_LEVEL=SYS) to the NCCL configuration file located at /etc/nccl.conf.

Key benefits and performance

• Accelerates multi-GPU workloads on G4 instances with two or more GPUs: provides faster runtimes for applications running on g4-standard-96, g4-standard-192, and g4-standard-384 machine types.
• Provides high-bandwidth communication: enables high data transfer speeds between GPUs.

• Improves NCCL performance: provides significant performance improvements for applications that use the NVIDIA Collective Communication Library (NCCL) when compared to communication that doesn't use P2P. Google's hypervisor securely isolates this P2P communication within your instances.

  • On four GPU instances (g4-standard-192), all GPUs are on a single NUMA node, allowing for the most efficient P2P communication. This can lead to performance improvements of up to 2.04x for collectives such as Allgather, Allreduce, and ReduceScatter.
  • On eight GPU instances (g4-standard-384), GPUs are distributed across two NUMA nodes. P2P communication is accelerated for traffic both within and between these nodes, with performance improvements of up to 2.19x for the same collectives.

The G2 machine series

The G2 machine series is available in standard machine types that have 4 to 96 vCPUs, and up to 432 GB of memory. This machine series is optimized for inference and graphics workloads. The G2 machine series is available in a single standard machine type with multiple configurations.

Instances created by using the G2 machine types provide the following features:

• GPU acceleration: each G2 machine type has NVIDIA L4 GPUs.

• Improved inference rates: the G2 machine type provides support for the FP8 (8-bit floating point) data type which speeds up ML inference rates and reduces memory requirements.

• Next generation graphics performance: NVIDIA L4 GPUs provide up to 3X improvement in graphics performance by using third-generation RT cores and NVIDIA DLSS 3 (Deep Learning Super Sampling) technology.

• High performance network bandwidth: with the G2 machine types, you can get up to 100 Gbps network bandwidth.

• Disk support: G2 instances support Local SSD for fast scratch disks, which is useful for feeding data into GPUs while preventing I/O bottlenecks. For durable storage, you can attach Persistent Disk and Hyperdisk volumes.

  You can add up to 3,000 GiB of Local SSD to G2 instances. For workloads that require durable block storage, you can attach Hyperdisk and Persistent Disk volumes to G2 instances. The maximum storage capacity depends on the number of vCPUs the instance has. For more information about disk types, see Choose a disk type.

• Compact placement policy support: provides you with more control over the physical placement of your instances within data centers. This enables lower-latency and higher bandwidth for instances that are located within a single availability zone. For more information, see Reduce latency by using compact placement policies.

G2 machine types

G2 accelerator-optimized machine types have NVIDIA L4 GPUs attached and are ideal for cost-optimized inference, graphics-intensive and high performance computing workloads.

Each G2 machine type also has a default memory and a custom memory range. The custom memory range defines the amount of memory that you can allocate to your instance for each machine type. You can also add Local SSD disks when creating a G2 instance. For the number of disks you can attach, see Machine types that require you to choose a number of Local SSD disks.

G2 limitations

• You can only request capacity by using the supported consumption options for a G2 machine type.
• You don't receive sustained use discounts and flexible committed use discounts for instances that use a G2 machine type.
• You can only use a G2 machine type in certain regions and zones.
• The G2 machine type is only available on the Cascade Lake platform.
• Standard Persistent Disk (pd-standard) isn't supported on instances that use the G2 machine type. For supported disk types, see Supported disk types for G2.
• You can't create Multi-Instance GPUs on an instance that uses a G2 machine type.
• If you need to change the machine type of a G2 instance, review Modify accelerator-optmized instances.
• You can't use Deep Learning VM Images as boot disks for instances that use the G2 machine type.
• The current default driver for Container-Optimized OS doesn't support L4 GPUs running on G2 machine types. Also, Container-Optimized OS only supports a select set of drivers. If you want to use Container-Optimized OS on G2 machine types, review the following notes:
  • Use a Container-Optimized OS version that supports the minimum recommended NVIDIA driver version 525.60.13 or later. For more information, review the Container-Optimized OS release notes.
  • When you install the driver, specify the latest available version that works for the L4 GPUs. For example, sudo cos-extensions install gpu -- -version=525.60.13.
• You must use the Google Cloud CLI or REST to create G2 instances for the following scenarios:
  • You want to specify custom memory values.
  • You want to customize the number of visible CPU cores.

Supported disk types for G2 instances

G2 instances can use the following block storage types:

• Balanced Persistent Disk (pd-balanced)
• SSD (performance) Persistent Disk (pd-ssd)
• Hyperdisk ML (hyperdisk-ml)
• Hyperdisk Throughput (hyperdisk-throughput)
• Local SSD: you can add Local SSD to instances created by using the G2 machine types.

If supported by the machine type, you can attach a mixture of Hyperdisk and Persistent Disk volumes to an instance, but the following restrictions apply:

• The combined number of both Hyperdisk and Persistent Disk volumes can't exceed 128 per instance.

• The maximum total disk capacity (in TiB) across all disk types can't exceed:

  • For machine types with less than 32 vCPUs:

    • 257 TiB for all Hyperdisk or all Persistent Disk
    • 257 TiB for a mixture of Hyperdisk and Persistent Disk

  • For machine types with 32 or more vCPUs:

    • 512 TiB for all Hyperdisk
    • 512 TiB for a mixture of Hyperdisk and Persistent Disk
    • 257 TiB for all Persistent Disk

For details about the capacity limits, see Hyperdisk size and attachment limits and Persistent Disk maximum capacity.

What's next