User does not have storage.objects.list access to bucket

Question

I created a service account and assigned these roles:

Owner
Storage Admin
Storage Object Admin
Tester

Tester is a role I created for learning purposes with these permissions:

storage.buckets.create
storage.buckets.delete
storage.buckets.get
storage.buckets.getIamPolicy
storage.buckets.list
storage.buckets.setIamPolicy
storage.buckets.update
storage.objects.create
storage.objects.delete
storage.objects.get
storage.objects.getIamPolicy
storage.objects.list
storage.objects.setIamPolicy
storage.objects.update
...

I know I'm unnecessarily overdoing permissions with these roles; but, this is just for testing purposes.

Considering the bucket only contains a single file and the account has the corresponding permissions, the Python code below should work (Running on my local computer):

from google.cloud import storage


if __name__ == '__main__':
    storage_client = storage.Client()
    bucket = storage_client.bucket('my-bucket-name')
    blobs = bucket.list_blobs()
    for blob in blobs:
        print(blob.name)

But it doesn't:

Traceback (most recent call last):
  File "gcloud/test.py", line 8, in <module>
    for blob in blobs:
  File "/home/user/.local/lib/python3.6/site-packages/google/api_core/page_iterator.py", line 212, in _items_iter
    for page in self._page_iter(increment=False):
  File "/home/berkay/.local/lib/python3.6/site-packages/google/api_core/page_iterator.py", line 243, in _page_iter
    page = self._next_page()
  File "/home/user/.local/lib/python3.6/site-packages/google/api_core/page_iterator.py", line 369, in _next_page
    response = self._get_next_page_response()
  File "/home/user/.local/lib/python3.6/site-packages/google/api_core/page_iterator.py", line 419, in _get_next_page_response
    method=self._HTTP_METHOD, path=self.path, query_params=params
  File "/home/user/.local/lib/python3.6/site-packages/google/cloud/_http.py", line 421, in api_request
    raise exceptions.from_http_response(response)
google.api_core.exceptions.Forbidden: 403 GET LINK: USER does not have storage.objects.list access to BUCKET.

The bucket uses uniform bucket-level access control. The service account I'm using is a member of this bucket and it inherits this membership from:

Storage Admin
Storage Object Admin
Tester

Can someone explain me the reason behind this behavior?

Thanks

Solution 1

I personally believe in development/testing there is no need to shy away from over-granting roles. But if you are definitely giving multiple roles, may as well give an admin role instead of multiple smaller ones (since essentially they are both doing the same job, but with lesser roles)

For your specific problem here, I would suggest

1. Delete your old service account
2. Create a new service account and grant the roles of storage.admin and storage.object.admin
3. Use this service account

There is a similar post on SO, and that seemed to be resolved in a similar way.

For future readers: If problem still persists, completely uninstall gcloud-sdk and reinstall (using this link) with the latest version.

Solution 2

That is why I asked if you switched this bucket to uniform bucket-level access control or created it. My theory is that you switched it to uniform bucket level and triggered this disclaimer.

Caution: If you enable uniform bucket-level access, you revoke access from users who gain their access solely through object ACLs. Be sure that you read considerations when migrating an existing bucket prior to enabling uniform bucket-level access.

That's why it is working when you are adding the role manually.

You can read more about how uniform bucket-level access permisions work, here.

Here is more relevant information to what was happening.

Additionally, if you enable uniform bucket-level access as part of creating a new bucket, the bucket automatically receives additional Cloud IAM roles. This behavior maintains the permissioning that objects inherited from the bucket's default object ACLs. If you enable uniform bucket-level access on an existing bucket, you must apply any such roles manually; you may want to apply a different set of roles if you have changed the bucket's default object ACLs.

Also this which I understand it as an explanation of the error you we're getting.

Once enabled, the following ACL functionality ceases:

Requests to set, read, or modify bucket and object ACLs fail with 400 Bad Request errors.

Hope this helps.

Solution 3

I faced this problem when trying to check if a bucket already existed in my project before creating it and in my case it wasnt per say an issue of roles because my service account had a storage admin, storage object admin and creator roles but it turned out that the bucket name I was trying to check existed/create (from inside my python code) wasn't globally unique. When I changed the name of the bucket to something globally unique the error went away.