gsutil ServiceException: 401 Anonymous caller does not have storage.objects.list access to bucket even though I'm loggedin in gcloud

Question

I am trying to create an internal app to upload files to google cloud. I don't want each individual user or this app to log in so I'm using a service account. I login into the service account and everything is ok, but when I try to upload it gives me this error: ServiceException: 401 Anonymous caller does not have storage.objects.list access to bucket

As you can see I am logged in with a service account and my account and(neither service or personal) works

Solution 1

I had similar problem, and as always, it took me 2 hours but the solution was trivial, if only it was written somewhere... I needed to login (or authorize, what suits you) to the gsutil in addition to being authorized to the gcloud. I thought they are linked or whatever, but nah. After I ran gsutil config and authorized via the provided link (and code that I pasted back to the console), it started working for me.

Note that I was also logged in to gcloud via a service account linked to my project and having the service account .json key saved locally (see gcloud auth activate-service-account --help). Hope this helps anyone struggling with this!

Solution 2

gcloud auth login solved my issue. You need both steps:

gcloud auth login
gcloud auth application-default login

Solution 3

It happened to me because I had an incomplete initialisation while running gcloud init. I reinitialised the configuration using gcloud init command and it worked fine.

Solution 4

I can only think of a few things that might cause you to see this error:

1. Maybe you have an alias set up to a standalone installation of gsutil (which doesn't share credentials with gcloud)?
   Edit: it's also possible you're invoking the wrong gsutil entry point - make sure you're using <path-to-cloud-sdk>/google-cloud-sdk/bin/gsutil, and not <path-to-cloud-sdk>/google-cloud-sdk/platform/gsutil/gsutil. The platform path will not automatically know about your configured gcloud auth options.

2. Maybe your service account credentials have moved/are invalid now? If your boto file is referring to a keyfile path and the keyfile was moved, this might happen.

3. Maybe the gcloud boto file (that gcloud created to use with gsutil when you ran gcloud auth login) is gone. You can run gsutil version -l to see if it's shown in your config path. If gcloud's boto file is present, you should see a line similar to this:

   config path(s): /Users/Daniel/.config/gcloud/legacy_credentials/[email protected]/.boto

You can run gsutil version -l to get a bit more info and look into the possibilities above. In particular, these attributes from the output will probably be the most helpful: using cloud sdk, pass cloud sdk credentials to gsutil, config path(s), and gsutil path.

Solution 5

Use this command to resolve some issues

gsutil config

Follow the browser to get a code then set it.

View more solutions

Comments

[Joshua Royalty]

It wouldn't say 'Anonymous caller' if the user was authenticated. It would say 'Caller' (or perhaps have the email address of the caller).

[Daniel Tranca]

Yes, it does. It has the same permissions as my personal account which I can use from the web but not from console

[Ying Li]

I see what you mean. In that case, it's likely an installation issue (multiple gsutil version). You can test that with a clean install or use a new machine. Detailed instructions in my edited answer.

[xaphod]

Similar: I had to run gcloud auth list to determine I had no user logged in (a surprise to me), then gcloud auth login '[email protected]'

[CpILL]

YES! you are right, not documented anywhere, they just suggest you use gcloud to authenticate!?!?

[mhouglum]

If you run the gcloud wrapper script (.../bin/gsutil), it should use your gcloud credentials for gsutil. If you invoke the packaged gsutil directly (.../platform/gsutil/gsutil), it won't, and you'll need to configure your credentials via gsutil config. See my answer below.

[Sandeep Anand]

Now This can be also done through google cloud auth gcloud auth login

[Ian Newland]

This was the key for me gcloud auth activate-service-account pointed to my service account .json file

[Tom N Tech]

I don't understand what the solution is. I have authenticated via gcloud auth login and still gcsfs does not use my account.

[Petr Krýže]

@rjurney It is possible Google has changed the authentication system for gcloud since this answer was posted. I recently installed gcloud on a new computer, I (for a change) had now problems with not being able to get Windows to find gsutil. I think I messed around with the PATH, but I honestly can't remember all too well.

[Shailen]

I did a pip install gcloud. After gcloud is installed, I run gcloud auth login and I get bash: gcloud: command not found...

[Timofey]

Despite gsutil was coming from the standard installation, I still had to perform gsutil config and it authenticated the tool. Although my problem was described by mhouglum's answer

[Slbox]

This worked perfectly, so much simpler than the accepted answer. (and also I didn't seem to need the second line, running this in the online command prompt.)

[raina]

This solution worked for me! Thanks. I tried only use "gcloud auth login", which doesn't work, after also run "gcloud auth application-default login" it finally worked

[Guilherme Felipe Reis]

solved my issue this one here 👍

[Canovice]

this solved my issue as well !

[bfontaine]

This is a duplicate of existing answers.

[bfontaine]

This is a duplicate of existing answers.

[Admin]

this method worked for me, in my case I was authenticate in CLI but couldn't read/write on gap bucket from pandas, using this method it worked! thanks

[roma2341]

didn't work at all