How do I get all blobs at all levels in a container?

Question

I found this thread about listing the blobs in a container.

from azure import *
from azure.storage import *

blob_service = BlobService(account_name='<accountname>', account_key='<accountkey>')
next_marker = None
while True:
    blobs = blob_service.list_blobs('<containername>', maxresults=100, marker=next_marker)
    next_marker = blobs.next_marker
    print(next_marker)
    print(len(blobs))
    if next_marker is None:
        break
print "done"

However, that would only list the blobs in a specific container. How would I go about getting all the blobs in subfolders? I have several levels of subfolders, and I would like to get the names of all data files from the parent container.

Solution 1

There is a function list_containers() we can see at https://github.com/Azure/azure-storage-python/blob/master/azure/storage/blob/baseblobservice.py#L470 which purposes to get all the containers in your storage

import azure
from azure.storage.blob import BlobService

blob_service = BlobService(account_name='<account_name>', account_key='<account_key>')
containers = blob_service.list_containers()

for c in containers:
    print(c.name)

Then you can call list_blob method in the loop with the containers’ name.

Additionally, if you have several subfolders defined in blob names, there is a thread list virtual folders in azure blob storage via python API on SO you can refer to.

Solution 2

Modifying a little bit in the above-approved solution. BlobService has been deprecated in the new version use BlockBlobService instead.

import azure
from azure.storage.blob import BlockBlobService

blob_service = BlockBlobService(account_name='<account_name>', account_key='<account_key>')
containers = blob_service.list_containers()

for c in containers:
    print(c.name)

Comments

[Andrew Moll]

What do you mean by subfolders? This should list all of the blobs within a container.

[juvchan]

Are you using Microsoft Azure SDK for Phython 2.7 or 3.4?

[juvchan]

Do you mean you want to get all the blob containers and all blobs instead of only blobs in a specific container?

[Gaurav Mantri]

The code above will list all the blobs in a blob container. I believe the OP wants to list blobs in a sub folder inside a blob container.

[Bart C]

This helped me but I could import BlobService, had to change for BlockBlobService

[R overflow]

@BartC; same issue here. I can not import BlobService (neither after unsintalling Azure and reinstalling azure). How did you solved it with BlocbBlobService?

[ericOnline]

This is good for listing all containers in a blob storage account, but how to list all blobs regardless of container and subfolder?