How to make s3fs work with IAM roles?

Question

We are using IAM roles to maintain the machines. We now planning to mount the s3 to our ec2 instances and do the processing as per our need. We been using s3fs tool for mounting.

But that doesn't seems to be working with the IAM roles. Are there any ways to make s3fs work with the IAM roles? Did anyone done with this?

Thanks in advance.

Solution 1

This worked for me iam_role=auto

Here is my /etc/fstab entry

s3fs#my_bucket /s3mount_path fuse _netdev,allow_other,iam_role=auto 0 0

Just make sure you IAM role has proper permissions

Solution 2

There seems to be an iam_role option in the man page:

iam_role (default is no role) - set the IAM Role that will supply the credentials from the instance meta-data.

A cursory overview of the source suggests that it will get the token and rotate as they expire...

also, see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#instance-metadata-security-credentials for details on getting these credentials, token, etc yourself

Solution 3

Maybe a little late to the party but since majority of the answers are very old; just wanted to share the command that works right now :-

So, to manually mount the s3 bucket using an IAM role you will need to fire the following command :-

$ sudo s3fs <bucket-name> /<folder-path>/<folder-name> -o iam_role="<Role-name>" -o url="https://s3-<region-name>.amazonaws.com" -o endpoint=<region-name> -o dbglevel=info -o curldbg

Now, for example if my bucket-name is "Test-Bucket" and the folder path is "/root/Test-Bucket" and my IAM Role name is "ec2_to_s3" and the region in which I have the s3 bucket is "Asia-Pacific Mumbai", then the above specified command will be used as :-

$ sudo s3fs Test-Bucket /root/Test-Bucket -o iam_role="ec2_to_s3" -o url="https://s3-ap-south-1.amazonaws.com" -o endpoint=ap-south-1 -o dbglevel=info -o curldbg

and the bucket will be mounted successfully.

Now, if you want the bucket to get mounted on system reboot by its own; you will need to add this string in "/etc/fstab".

<bucket-name> /<folder-path>/<folder-name> fuse.s3fs _netdev,allow_other,use_path_request_style,iam_role=auto 0 0

Now, as per the previous example, the string that needs to be added in "/etc/fstab" would be :-

Test-Bucket /root/Test-Bucket fuse.s3fs _netdev,allow_other,use_path_request_style,iam_role=auto 0 0

Hope this helps...

Solution 4

IAM roles require the use of a token in addition to the access key id and secret access key.

This means that s3fs source will need be to be modified from its current state in order to support IAM roles.

Fortunately, s3fs is released under the GPLv2, so you are free to do so.

View more solutions

Comments

[sriram]

Thanks for the answer. Are there any format in which I need to call the S3 bucktes along with the token? I couln't able to see any call in the code something like Accesskey:SecretKey.

[Eric Hammond]

@GroovyUser: Sorry, but I can't understand that question as it is phrased. The actual source code for s3fs needs to be enhanced to support calling the AWS API with additional information.

[sriram]

Oops! My question is What is the header format of the request to be along with the session token? The code that there already works with by creating the header part of the request with Accesskey:SecretKey. But I need the format for the session tokens too. Or else are there tools that does the same job?

[Nicholas Terry]

Excellent! Good to know that the iam stuff works for s3fs