is it safe to create a snapshot of a volume of a running ec2 instance

Question

I'm running a MongoDB Cluster on 3 Instances on AWS

Each instance has 2 EBS attached to it

- / [8GB]
- /data [50GB]

To backup a cluster MongoDB recommends to create a snapshot of the /data driver

I created an automated tool that uses AWS-SDK that create snapshot for each volume mounted in /data at a specific time.

I was wondering if creating snapshot using AWS-SDK or using the AWS Console (manually) can cause the instance to run slow or become not available during the process?

I know creating a snapshot of the root volume can cause the system to reboot but what about creating snapshot of a volume attached to a running instance under another directory like data ?

Solution

The snapshot process shouldn't be visible to the instance. The real problem would come when you try to restore from the "unclean" snapshot. If a process is writing to the volume (or rather, to an open file descriptor for a file on the volume), write requests in the instance's kernel cache may not be written to disk, or may only be partially written. These files would look corrupted on a volume made from the snapshot.

AWS recommends unmounting all volumes before taking a snapshot for this reason. http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-creating-snapshot.html

(Obviously, you can't unmount / on a running instance. You could stop the instance to make the snapshot, but I would try to find another way to back that up. Presumably you only need mongodb configuration files on that disk, which are normally under /etc, /var, and /opt.)

Comments

[Eltorrooo]

I only want to backup volumes mounted in /data for the / I created a whole AMI while the instance is off ... for the / I only need to create a backup once

[Karen B]

OK, but the issue with being unable to make a reliable backup snapshot of the data volumes while it's mounted and in use still applies.

[Eltorrooo]

MongoDB docs doesn't say I should unmount it ... docs.mongodb.com/ecosystem/tutorial/…

[Karen B]

They do tell you to lock the database to prevent writes, but that sounds dangerous. Even if MongoDB had stopped writing to the volumes, the Linux kernel may still have unwritten data in memory, or may be writing to other parts of the volume. The link I posted in the answer from AWS recommends unmounting the filesystem before taking EBS snapshots for this reason.