NFS mount across EC2 instances as alternative to EFS

Question

I'm configuring an application to run in AWS in "full-framework" deployment. One aspect of this deployment requires each server to be able to see a common storage location. They recommend using a WebDAV server, but we'd prefer to use some kind of shared mounted drive.

EFS is the perfect solution here... but it isn't available in my region.

Alternatively we thought of setting up an NFS mount. Is it possible to mount an NFS on an EBS volume to act similarly to an EFS? Any advantages/disadvantages to doing so?

Solution 1

I would be cautious of the S3fs recommendation. S3 isn't designed to be a file system and has some limitations when an application expects it to be. S3 is eventually consistent, meaning an update may or may not be reflected in a timely fashion. S3 has no concept of partial changes, so using a log file as an example, if you're appending or otherwise altering, the entire file gets re-uploaded on every change. This can become costly. S3FS tries to utilize a local cache to avoid excessive transfers, and this adds another layer of potential inconsistency if something were to happen on that node.

While setting up NFS on an EC2 instance is fairly trivial, you are creating a single point of failure (SPOF) in your design (if the instance or EBS goes away, your application is impacted). Also, cross-AZ traffic from other instances to your NFS one will incur bandwidth charges.

If you're looking for an enterprise grade NFS and cannot use EFS, there are Marketplace offerings (aws.amazon.com/marketplace), but you can also go opensource and use GlusterFS mounted as NFS with multiple EC2 instances to avoid a SPOF and keep traffic within AZs: https://docs.gluster.org/en/latest/Install-Guide/Setup_aws/ (but, Gluster can be a beast in itself to manage)

Solution 2

You can try NetApp's CVO/CVS https://cloud.netapp.com/cloud-volumes-service-for-aws they provide both CIFS and NFS

Solution 3

You have a number of options, but what's best depends on your requirements, but you haven't given enough detail to recommend one option over another. I probably haven't covered every option.

NFS Server

You can set up an NFS server on an EC2 instance. This uses an EBS volume and shares it over NFS. There will be plenty of tutorials online to do this (one, two).

Use S3 Natively

Depending on your requirements you may be able to use S3 natively, calling the S3 API, or even using aws s3 sync via cron.

RSync

You could consider rsync from a master server, a little t2.nano would probably be plenty for that kind of thing.

S3FS

Another option is to use S3FS, which makes an S3 bucket look like a file system. You can use an S3 Endpoint to prevent the data going over the public internet.

You'd want to do some reading to learn about reliability and some careful testing before you did this. My understanding is if your S3 bucket is in the same region as your EC2 instance you don't pay traffic charges, but I may be wrong and this can change at any time. The S3 pricing page says "Transfers between S3 buckets or from S3 to any service(s) within the same region are free" - I assume EC2 is counted as a service.

The information I've read suggests that this is likely to be a reasonably reliable option, but you need to test the S3FS release carefully.

Comments

[ceejayoz]

My understanding is EFS is essentially just a managed NFS service. You use a NFS client to mount an EFS volume, for example. docs.aws.amazon.com/efs/latest/ug/how-it-works.html The cons of your situation are, of course, having to manage the thing.

[jmkmay]

The application had support for WebDAV so I just set up a small WebDAV server (micro instance backed by large IOPS volume) and that did the trick wonderfully. Was able to keep everything inside private VPC too. Thanks for your input though I will consider those in the future.

[Yaron]

The "two" link in NFS Server is down, 404.

[Tim]

I removed the broken link