Python: How to move or copy Azure Blob from one container to another

Question

I am using Microsoft Azure SDK for Python in project. I want to move or copy Blob from one container to another. for exmaple

https://demostorage.blob.core.windows.net/image-container/pretty.jpg

I want to move this blob to

https://demostorage.blob.core.windows.net/demo-container/

I have found following method in python SDK but unable to understand it.

def copy_blob(self, container_name, blob_name,...):

How can I do this? Thank you

Solution 1

I have done in this way.

from azure.storage.blob import BlobService

def copy_azure_files(self):

        blob_service = BlobService(account_name='account_name', account_key='account_key')
        blob_name = 'pretty.jpg'
        copy_from_container = 'image-container'
        copy_to_container = 'demo-container'

        blob_url = blob_service.make_blob_url(copy_from_container, blob_name)
        # blob_url:https://demostorage.blob.core.windows.net/image-container/pretty.jpg

        blob_service.copy_blob(copy_to_container, blob_name, blob_url)

        #for move the file use this line
        blob_service.delete_blob(copy_from_container, blob_name)

I have not found any Blob Move method yet. So I have used the copy method and then execute Blob function.

This is my solution. If you have better way to handle all this please share with me.

Note: I have not used any custom method all these methods are included in SDK.

Solution 2

With the current version of azure-storage-blob (at the moment v12.3.2) you will get an ImportError:

cannot import name 'BlockBlobService' from 'azure.storage.blob'

This code is working in my case:

from azure.storage.blob import BlobServiceClient

# Azure
# Get this from Settings/Access keys in your Storage account on Azure portal
account_name = "YOUR_AZURE_ACCOUNT_NAME"
connection_string = "YOUR_AZURE_CONNECTION_STRING"

# Source
source_container_name = "sourcecontainer"
source_file_path = "soure.jpg"
blob_service_client = BlobServiceClient.from_connection_string(connection_string)
source_blob = (f"https://{account_name}.blob.core.windows.net/{source_container_name}/{source_file_path}")

# Target
target_container_name = "targetcontainer"
target_file_path = "target.jpg"
copied_blob = blob_service_client.get_blob_client(target_container_name, target_file_path)
copied_blob.start_copy_from_url(source_blob)

# If you would like to delete the source file
remove_blob = blob_service_client.get_blob_client(source_container_name, source_file_path)
remove_blob.delete_blob()

Solution 3

Using the most current Azure Blob Storage SDK.

from azure.storage.blob import BlockBlobService

account_name = "demostorage"
account_key = "lkjASDRwelriJfou3lkjksdfjLj349u9LJfasdjfs/dlkjfjLKSjdfi8ulksjdfAlkjsdfkL762FDSDFSDAfju==" 

source_container_name = "image-container"
source_file_path = "pretty.jpg"

target_container_name = "demo-container"
target_file_path = "pretty_copy.jpg"

service = BlockBlobService(account_name, account_key)

service.copy_blob(
    target_container_name, 
    target_file_path, 
    f"https://{account_name}.blob.core.windows.net/{source_container_name}/{source_file_path}",
)

Solution 4

sdk version 12.0.0b, they change the copy_blob to start_copy_from_url. You can see it here about the method

Here is a snapshot on how you can do the moving mechanism without downloading the data.

    from azure.storage.blob import BlobServiceClient

    blob_service_client = BlobServiceClient.from_connection_string(
            os.getenv("AZURE_STORAGE_CONNECTION_STRING")
        ).get_container_client(self.container)

    # azure storage sdk does not have _move method
    source_blob = blob_service_client.get_blob_client(source_blob_path)
    dest_blob = blob_service_client.get_blob_client(dest_blob_path)

    dest_blob.start_copy_from_url(source_blob.url, requires_sync=True)
    copy_properties = dest_blob.get_blob_properties().copy

    if copy_properties.status != "success":
        dest_blob.abort_copy(copy_properties.id)
        raise Exception(
            f"Unable to copy blob %s with status %s"
            % (source_blob_path, copy_properties.status)
        )
    source_blob.delete_blob()

View more solutions

Comments

[Gaurav Mantri]

Move is not natively supported in Azure Storage. You can accomplish move operation by performing copy operation, wait for the copy to finish (very important because copy is async operation) and then performing the delete operation on the source blob.

[Alex S]

Think you meant to write blob_url = blob_service.make_blob_url(copy_from_container,blob_name) & blob_service.delete_blob(copy_from_container, blob_name)

[pietroppeter]

Should the initial import (from azure.storage.blob import BlobService) be changed to from azure.storage.blob.baseblobservice import BaseBlobService? I cannot find any reference of BlobService object in Azure SDK for python so I would assume they refactored it to BaseBlobService...

[Amir Imani]

I think you need to import this from azure.storage.blob import BlockBlobService

[Achilles]

The copy operation on blob service messes up with the metadata keys case. so beware if your blob metadata keys have camel case or something else because it will be converted to lower case by copy operation.

[RB17]

I am getting "ImportError: cannot import name 'BlobServiceClient' from 'azure.storage.blob'" upon trying to use it in Azure Function. My version of azure-storage-blob is 12.3.2 Any idea how can I avoid this?

[gailmichi]

Maybe there are some problems with your python environment? Are you using a virtual enviroment? Open your function in VSCode and try 'pip freeze' in the VSCode Terminal. You will see the used library versions. Check if there is the current version.

[shary.sharath]

is there any way to do it in bulk? I would like to copy blobs in bulk from one folder to folder.

[mas]

is there a way to copy from one storage account to another?