AzureBlob Upload ERROR:The specified blob already exists

Question

I am trying to upload file to Azure container daily.

I got an Error:"The specified blob already exists" when uploading file with same file( I want to overwrite the file)

from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient

conn_str = yml['AZURE_BLOB']['CONN_STR']
container_name = yml['AZURE_BLOB']['CONTAINER_NAME']

# Create the BlobServiceClient that is used to call the Blob service for the storage account
blob_service_client = BlobServiceClient.from_connection_string(conn_str=conn_str)

# Create a blob client using the local file name as the name for the blob
blob_client = blob_service_client.get_blob_client(container=container_name, blob=destination_file_name)

# Upload the created file
data = fs.open(source_path,mode='rb').read()
blob_client.upload_blob(data)

print(destination_file_name+'\t......[DONE]')

Error message:

azure.core.exceptions.ResourceExistsError: The specified blob already exists.
RequestId:13d062cd-801e-00a4-77c7-a81c56000000
Time:2019-12-02T04:18:06.0826908Z
ErrorCode:BlobAlreadyExists
Error:None

Solution 1

If you want to overwrite the existing blob using Blob storage client library v12, just add overwrite=True in the upload_blob method.

Here is the sample code:

from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient

conn_str = "xxx"
container_name = "test6"

blob_service_client = BlobServiceClient.from_connection_string(conn_str=conn_str)
blob_client = blob_service_client.get_blob_client(container=container_name,blob="a1.txt")

with open("F:\\temp\\a1.txt","rb") as data:
    blob_client.upload_blob(data,overwrite=True)

print("**completed**")

After executing the code, the new blob is uploaded and the existing blob can be overwritten. Screenshot as below:

Solution 2

Check out this blog post about a known issue.

This is a known issue with development storage. This happens when there are multiple threads launched to upload the blocks (which constitute the blob). Basically what is happening is that development storage makes use of SQL Server as the data store. Now first thing it does is makes an entry into the table which stores blob information. If there are multiple threads working then all of these threads will try to perform the same operation. After the first thread succeeds, the subsequent threads will result in this exception being raised.

Comments

[EEEEH]

Do you get the message like this "Tuple timeout setting is deprecated"

[Ivan Yang]

@EEEEH, I didn't see this message. For which line?

[EEEEH]

"blob_client.upload_blob(data, overwrite=True)" The message show up after uploading file success. Is it just a warning message yet?

[EEEEH]

No matter why my promble sloved! tks

[marhyno]

its 2022 and we encountered this problem in Production , what should we do to solve this ? In my Visual Studio I dont have this problem but when we publish it to azure machine, the problem occurs

[furkanayd]

@marhyno Might be related with encryption key/token to upload, try some variations with this method docs.microsoft.com/en-us/python/api/azure-storage-blob/…

[marhyno]

thanks, it was only a bug on our side with folder naming