Console
Start free

Relocate buckets

This page describes the process for relocating buckets from one location to another. For information about bucket relocation, see Bucket relocation.

Before you begin

Before you can relocate buckets, complete the following steps:

  1. Configure Storage Intelligence.

  2. Enable soft delete.

  3. Check the quotas and limits to verify that the new location has sufficient quotas to accommodate the bucket's data.

  4. Determine the bucket relocation type to understand whether write downtime is required.

  5. Review the bucket relocation limitations.

  6. Remove any existing bucket tags.

  7. If you use inventory reports, save your configurations.

  8. Get required roles, which are described in the following section.

Get required roles

To get the permissions that you need to relocate buckets, ask your administrator to grant you the Storage Admin (roles/storage.admin) IAM role on the project. For more information about granting roles, see Manage access to projects, folders, and organizations.

This predefined role contains the permissions required to relocate buckets. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to relocate buckets:

You might also be able to get these permissions with custom roles or other predefined roles.

Relocate buckets

This section describes the process of relocating Cloud Storage buckets from one location to another.

To relocate a bucket, complete the following steps:

  1. Test the bucket relocation with a dry run (Optional)

  2. Initiate the bucket relocation process

  3. Initiate the final synchronization step

For more information about these steps, see Understand the bucket relocation process.

Test the bucket relocation with a dry run (Optional)

To minimize potential issues during the bucket relocation process, we recommend you perform a dry run to test the end-to-end process. A dry run simulates the bucket relocation process without moving data, helping you to catch and resolve issues early on. The dry run checks for the following incompatibilities:

While a dry run can't identify every possible issue as some issues might only surface during the live migration due to factors such as real-time resource availability, it reduces the risk of facing time-consuming issues during the actual relocation.

Console

  1. In the Google Cloud console, go to the Cloud Storage Buckets page.

    Go to Buckets

  2. In the list of buckets, click the name of the bucket you want to relocate.
  3. On the Bucket details page, click the Configuration tab.
  4. In the Overview section, click Edit next to the Location field.
  5. On the Relocate bucket page, enter your bucket's new location. After each of the following steps, click Continue to proceed to the next step:
    1. In the Confirm the bucket you plan to relocate section, review the bucket and its location.
    2. In the Before you continue section, review configurations that block the relocation. If a restriction applies to your bucket, consider Storage Transfer Service as an alternative workaround.

    3. In the Choose where to relocate your bucket section, do the following:

      1. Select a location type.

      2. Select a location where object data within your bucket will be stored.

        Based on the source and destination location, you'll be informed if a write downtime is required. For information about the types of relocation and downtime, see Relocation types.

    4. In the Choose how you'd like to proceed section, click Start with a dry run (recommended). A dry run simulates the relocation to identify potential issues without moving data.
    5. Click Continue.
  6. Click Start.
  7. On the Start with a dry run confirmation dialog, review the message that appears and click Start dry run.

After you initiate a dry run, a long-running operation starts. You can monitor the progress in the Operations tab of the bucket details page.

Command line

To initiate a dry run, run the gcloud storage buckets relocate command with the --dry-run flag:

gcloud storage buckets relocate gs://BUCKET_NAME --location=LOCATION --dry-run

Where:

After you initiate a dry run, a long-running operation starts. You'll receive an operation ID and a description of the operation. Track the progress and completion of the dry run by getting the details of the long-running operation.

If the dry run reveals any issues, address them before proceeding with the relocation step.

JSON API

Have gcloud CLI installed and initialized, which lets you generate an access token for the Authorization header.

  • Create a JSON file that contains the settings for the bucket, which must include the destinationLocation and validateOnly parameters. See the Buckets: relocate documentation for a complete list of settings. The following are common settings to include:

    {
  "destinationLocation": "DESTINATION_LOCATION",
  "destinationCustomPlacementConfig": {
    "dataLocations": [
      LOCATIONS,
        ...
        ]
    },
  "validateOnly": "true"
  }

    Where:

  • Use cURL to call the JSON API:

    curl -X POST --data-binary @JSON_FILE_NAME \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  "https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/relocate"

    Where:

    After you initiate a dry run, a long-running operation starts. The dry run succeeds when the following conditions are met:

    • Initiate the bucket relocation

    Console

    1. In the Google Cloud console, go to the Cloud Storage Buckets page.

      Go to Buckets

    2. In the list of buckets, click the name of the bucket you want to relocate.
    3. On the Bucket details page, click the Configuration tab.
    4. In the Overview section, click Edit next to the Location field.
    5. On the Relocate bucket page, enter your bucket's new location. After each of the following steps, click Continue to proceed to the next step:
      1. In the Confirm the bucket you plan to relocate section, review the bucket and its location.
      2. In the Before you continue section, review configurations that block the relocation. If a restriction applies to your bucket, consider Storage Transfer Service as an alternative workaround.

      3. In the Choose where to relocate your bucket section, do the following:

        1. Select a location type.

        2. Select a location where object data within your bucket will be stored.

          Based on the source and destination location, you'll be informed if a write downtime is required. For information about the types of relocation and downtime, see Relocation types.

      4. In the Choose how you'd like to proceed section, click Relocate now.
      5. Click Continue.
    6. To start the relocation process, click Start.
    7. On the Relocate now confirmation dialog, review the message that appears and click Start relocate.

    After you start the relocation process, a long-running operation starts. You can monitor the progress in the Operations tab of the bucket details page.

    Command line

    To initiate the bucket relocation, run the gcloud storage buckets relocate command:

    gcloud storage buckets relocate gs://BUCKET_NAME --location=LOCATION

    Where:

    JSON API

    Have gcloud CLI installed and initialized, which lets you generate an access token for the Authorization header.

  • Create a JSON file that contains the settings for the bucket. See the Buckets: relocate documentation for a complete list of settings. The following are common settings to include:

    {
  "destinationLocation": "DESTINATION_LOCATION",
  "destinationCustomPlacementConfig": {
  "dataLocations": [
    LOCATIONS,
    ...
    ]
    },
  "validateOnly": "false"
  }

    Where:

  • Use cURL to call the JSON API:

    curl -X POST --data-binary @JSON_FILE_NAME \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  "https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/relocate"

    Where:

    • Monitor the relocation process

    Console

    To monitor the progress of the dry run or the relocation process, complete the following steps:

    1. In the Google Cloud console, go to the Cloud Storage Buckets page.

      Go to Buckets

    2. In the list of buckets, click the name of the bucket that you are relocating.

    3. On the Bucket details page, click the Operations tab.

    4. In the list of bucket operations that appear, click the dry run or the relocation operation to view its details.

    The operation details page displays the relocation summary, key metrics, and an error summary.

    Command line

    After you initiate a bucket relocation, a long-running operation starts. You'll receive an operation ID and a description of the operation. Track the progress and completion of the relocation process by getting the details of the long-running operation.

    JSON API

    Bucket relocation is a long-running operation that requires monitoring. To check the process status, regularly review the long-running operation details. For information about how to check the relocation process status, see Get details of a long-running operation.

    The following example shows the output generated by a relocation operation:

      done: false
  kind: storage#operation
  metadata:
  '@type': type.googleapis.com/google.storage.control.v2.RelocateBucketMetadata
  commonMetadata:
    createTime: '2024-10-21T04:26:59.666Z
    endTime: '2024-12-29T23:39:53.340Z'
    progressPercent: 99
    requestedCancellation: false
    type: relocate-bucket
    updateTime: '2024-10-21T04:27:03.2892'
  destinationLocation: US-CENTRAL1
  finalizationState: 'READY'
  progress:
    byteProgressPercent: 100
    discoveredBytes: 200
    remainingBytes: 0
    discoveredObjectCount: 10
    remainingObjectCount: 8
    objectProgressPercent: 100
    discoveredSyncCount: 8
    remainingSyncCount: 0
    syncProgressPercent: 100
  relocationState: SYNCING
  sourceLocation: US
  validateOnly: false
  estimatedWriteDowntimeDuration: '7200s'
  writeDowntimeExpireTime: '2024-12-30T10:34:01.786Z'
  name: projects//buckets/my-bucket1/operations/Bar7-1b0khdew@nhenUQRTF_R-Kk4dQ5V1f8fzezkFcPh3XMvlTqJ6xhnqJ1h_QXFIeAirrEqkjgu4zPKSRD6WSSG5UGXil6w
  response:
    '@type': type.googleapis.com/google.storage.control.v2.RelocateBucketResponse
      selfLink: https://storage.googleusercontent.com/storage/v1_ds/b/my-bucket1/operations/Bar7-1b0khdew@nhenUQRTF_R-Kk4dQ5V1f8fzezkFcPh3XMvlTqJ6xhnqJ1h_QXFIeAirrEqkjgu4zPKSRD6WSSG5UGXil6w

    The following table provides information about the key fields in the output generated by the relocation operation:

    Field name Description Possible values
    done Indicates the completion of the bucket relocation operation. true, false
    kind Indicates that this resource represents a storage operation.
    metadata Provides information about the operation.
    metadata.@type Indicates the type of operation as bucket relocation.
    metadata.commonMetadata Metadata common to all operations.
    metadata.commonMetadata.createTime The time the long-running operation was created.
    metadata.commonMetadata.endTime The time the long-running operation ended.
    metadata.commonMetadata.progressPercent The estimated progress of the long-running operation, in percentage. Between 0 to 100%. A value of -1 means that the progress is unknown or not applicable.
    metadata.commonMetadata.requestedCancellation Indicates whether the user has requested cancellation of the long-running operation. true, false
    metadata.commonMetadata.type Indicates the type of the long-running operation.
    metadata.commonMetadata.updateTime The time the long-running operation was last updated.
    metadata.destinationLocation The destination location of the bucket.
    metadata.finalizationState Indicates the readiness for initiating the final synchronization step.
    • READY: Indicates that you can initiate the final synchronization step. However, we recommend you wait until the value of the progressPercent field reaches 99.
    • WAITING_ON_SYNC: Indicates that you cannot initiate the final synchronization step.
    • NOT_REQUIRED: Indicates that the final synchronization step isn't necessary for this bucket and you can skip it.
    • BLOCKED_ON_ERRORS: Indicates that the finalization step is temporarily paused due to errors. You'll need to resolve the errors to proceed with the step.
    • RUNNING: Indicates that the finalization step is in progress.
    • FINALIZED: Indicates that the finalization step has completed successfully.
    metadata.progress Progress details of the relocation operation.
    metadata.progress.byteProgressPercent Progress of bytes copied in percentage. Between 0 to 100%. A value of -1 means that the progress is unknown or not applicable.
    metadata.progress.discoveredBytes Number of bytes discovered in the source bucket.
    metadata.progress.discoveredObjectCount Number of objects discovered in the source bucket.
    metadata.progress.discoveredSyncCount Number of object metadata updates discovered in the source bucket.
    metadata.progress.objectProgressPercent Progress of objects copied in percentage. Between 0 to 100%. A value of -1 means that the progress is unknown or not applicable.
    metadata.progress.remainingBytes Number of bytes remaining to be copied from the source bucket to the destination bucket.
    metadata.progress.remainingObjectCount Number of objects remaining to be copied from the source bucket to the destination bucket.
    metadata.progress.remainingSyncCount Number of remaining object metadata updates to be synced.
    metadata.progress.syncProgressPercent Progress of object metadata updates to be synced in percentage. Between 0 to 100%. A value of -1 means that the progress is unknown or not applicable.
    metadata.relocationState Overall state of the bucket relocation operation.
    • SYNCING: Indicates that the bucket relocation step is actively copying objects from the source bucket to the destination bucket.
    • FINALIZING: Indicates that the finalization step has been initiated.
    • FAILED: Indicates that the bucket relocation step encountered an error and did not complete successfully.
    • SUCCEEDED: Indicates that the bucket relocation step has completed successfully.
    • CANCELLED: Indicates that the bucket relocation step was canceled.
    metadata.sourceLocation The source location of the bucket.
    metadata.validateOnly Indicates if a dry run of the bucket relocation has been initiated. true, false
    metadata.estimatedWriteDowntimeDuration The estimated write downtime duration; populated once the finalizationState is READY. Minimum value is 7200s.
    metadata.writeDowntimeExpireTime The time the write downtime expires.
    name The unique identifier for this relocation operation.
    Format: projects/_/buckets/bucket-name/operations/operation-id
    response The response of the operation.
    response.@type The type of the response.
    selfLink A link to this operation.

    If you encounter issues when interacting with other Cloud Storage features, see Limitations.

    Initiate the final synchronization step

    For relocations that require a write downtime, you'll need to initiate the final synchronization. The final synchronization step involves a period where you cannot perform write operations on the bucket. We recommend that you schedule the final synchronization step at a time that minimizes disruption to your applications.

    Console

    To initiate the final synchronization step, complete the following steps:

    1. In the Google Cloud console, go to the Cloud Storage Buckets page.

      Go to Buckets

    2. In the list of buckets, click the name of the bucket that you are relocating.

    3. On the Bucket details page, click the Operations tab.

    4. In the list of bucket operations, click the relocation operation to view the operation details page. On the operation details page, when the data copy is at least 99% complete, a message bar appears indicating it's the optimal time to start the final synchronization.

    5. Optional: To set the maximum allowed write downtime, click Set maximum allowed downtime (TTL) and specify the maximum downtime duration.

    6. Click Start final synchronization.

    7. On the confirmation dialog that appears, click Start to begin the final synchronization.

    The operation details page shows the relocation summary, key metrics, and an error summary.

    Command line

    Before you proceed, confirm that the bucket is fully prepared by checking the finalizationState value in the output of the Initiate the bucket relocation step.

    When the finalizationState value is READY, run the gcloud storage buckets relocate command to initiate the final synchronization:

    gcloud storage buckets relocate --finalize --operation=projects/_/buckets/BUCKET_NAME/operations/OPERATION_ID

    Where:

     `name: projects/_/buckets/my-bucket/operations/AbCJYd8jKT1n-Ciw1LCNXIcubwvij_TdqO-ZFjuF2YntK0r74`

    Set the ttl flag to have greater control over the relocation process. For example:

    gcloud storage buckets relocate --finalize --ttl TTL_DURATION --operation=projects/_/buckets/BUCKET_NAME/operations/OPERATION_ID

    Where:

    TTL_DURATION is the Time to live (TTL) for the write downtime phase during a relocation process. It is expressed as a string, such as 12h for 12 hours. The TTL_DURATION determines the maximum allowed duration for the write downtime phase. If the write downtime exceeds this limit, the relocation process automatically reverts to the bucket relocation step, and write operations to the bucket are re-enabled. The value must be within the range of 6h (6 hours) to 48h (48 hours). If not specified, the default value is 12h (12 hours).

    JSON API

    Before you proceed, confirm that the bucket is fully prepared by checking the finalizationState value in the output of the bucket relocation step. The finalizationState value must be READY to proceed.

    If you initiate the final synchronization step prematurely, the command returns an error message The relocate bucket operation isn't ready to advance to finalization running state but the relocation process continues.

    We recommend that you wait until the progressPercent value is 99 before initiating the final synchronization step.

    Have gcloud CLI installed and initialized, which lets you generate an access token for the Authorization header.

  • Create a JSON file that contains the settings for bucket relocation. See the Buckets: advanceRelocateBucket documentation for a complete list of settings. The following are common settings to include:

    {
"expireTime": "EXPIRE_TIME",
"ttl": "TTL_DURATION"
}

    Where:

  • Use cURL to call the JSON API:

    curl -X POST --data-binary @JSON_FILE_NAME \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
 "https://storage.googleapis.com/storage/v1/b/bucket/BUCKET_NAME/operations/OPERATION_ID/advanceRelocateBucket"

    Where:

    • Validate the bucket relocation process

    After initiating a relocation, verify its successful completion. This section provides guidance on confirming the successful transfer of data.

    Validate the success of the relocation process using the following methods:

    Complete the post bucket relocation tasks

    After you have successfully relocated your bucket, complete the following steps:

    1. Optional: Restore any tag-based access controls on your bucket.
    2. Existing inventory report configurations are not preserved during the relocation process and you'll need to manually recreate them. For information about creating an inventory report configuration, see Create an inventory report configuration.
    3. Update your infrastructure as code configurations such as Terraform and Google Kubernetes Engine configuration connector to specify the bucket's new location.
    4. Regional endpoints are tied to specific locations, and you'll need to modify your application code to reflect the new endpoint.

    How to handle failed bucket relocation operations

    Consider the following factors before handling failed bucket relocation operations:

    What's next