gcloud compute ssh returns Permission Denied (publickey)

Question

According to Google Cloud documentation, if I am a project member with the "compute instance admin" role, I should be able to connect to any instance in my project using the gcloud tool.

On the project IAM page in the Google Cloud console, I have explicitly added my username with the "Compute Instance Admin (v1)" role, yet I am still unable to connect to an instance created by some of our automation.

[username]:~/src/infrastructure$ gcloud compute ssh [instance id]
Unauthorized use is strictly prohibited. All access and activity
is subject to logging and monitoring.
Permission denied (publickey).
ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255].

The only instances I seem to be able to connect to are those created by me.

What might be going wrong here?

As a side note, according to this documentation, I should be able to add my RSA public key to the instance manually, then connect using SSH.

I added my public key from ~/.ssh/google_compute_engine.pub to the instance metadata, then tried using SSH, with no luck.

[username]:~/src/infrastructure$ ssh -i ~/.ssh/google_compute_engine [public ip of instance]
Unauthorized use is strictly prohibited. All access and activity is subject to logging and monitoring.
Received disconnect from 35.197.127.143 port 22:2: Too many authentication failures for matts
Connection to 35.197.127.143 closed by remote host.
Connection to 35.197.127.143 closed.

As Google recommends, I do not want to manually manage SSH keys for instance access. I want gcloud compute ssh to work, so I'm less focused on this second failure than I am on the first one (unless they're related).

Solution 1

I had this problem. I couldn't login with gcloud command, manual ssh with -i flag, or even using the web browser ssh client.

I also tried to manually add a new key in the ssh keys editor which seemed to go through fine but STILL didn't let me in.

Increasing the boot disk size and restarting the instance fixed the problem.

Solution 2

This problem may happen if you delete the .ssh/authorized_keys file, and you may be able to fix the connection for the user with the problem if you have access to the machine through another user, usually that can be done with the following command ran by that other user:

gcloud compute ssh <machine-name> --project <project> --zone <zone>

Create the user's .ssh/authorized_keys file by adding the .ssh/google_compute_engine.pub key from the machine you are trying to connect from.

sudo -i

cd /home/<misconfigured-user>

# Optional, verify the keys are not already set.
cat .ssh/authorized_keys

touch .ssh/authorized_keys

Edit the file with your favorite editor and paste the local machine value.

Then just set the proper permissions for the file, it may not be necessary, but these are the default permissions.

chown <user> ~/.ssh/authorized_keys
chgrp <user> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys