External backup - "cp: cannot create regular file" error

Question

I have always backed up my home directory to an external hard drive using the cp -r command.

Until recently, it worked fine, but now I regularly get the error message cp: cannot create regular file followed by invalid argument or cp: cannot create symbolic link followed by operation not permitted.

I've tried sudo cp -r but the issue persists. What am I doing wrong?

Solution 1

A conflict between the source filenames and the destination filesystem can cause your cannot create regular file error. If you are copying to a USB thumb drive, you are probably using the vfat or fat32 filesystem, which is subject to the usual Windows naming restrictions.

To observe this, try creating a file named :, which is a Windows reserved character.

$ cp /dev/null /path/to/dest/:
cp: cannot create regular file '/path/to/dest/:': Invalid argument

To see the Operation not permitted error separately, try creating a symlink without copying.

$ ln -s somesillysymlink /path/to/dest/symlink
ln: failed to create symbolic link '/path/to/dest/symlink': Operation not permitted

If you see these errors, that is likely the cause of your problem.

Likely the simplest approach is to create an archive that is free from the naming restrictions you have encountered, and also preserves symlinks. By default, tar and 7z preserve symlinks. zip preserves symlinks with the appropriate flag. Each of these can store files that have Windows reserved characters in their names. See also “How can I zip/compress a symlink?”

Replacing the vfat filesystem with something more Linux-friendly – such as ext4 – would alleviate your problem, but at the cost of reduced portability. Almost any Linux system would be able to mount the drive, but other common systems would require extra work. See “Creating ext4 partition from console” and the mke2fs manpage for details on the creation end of the process. See “How to read ext4 partitions on Windows?”, “How can I mount an ext4 file system on OS X?”, and “How do I mount Ext4 using OS X Fuse” if you need to move that drive to other operating systems.

Solution 2

cp -rv should at least tell you what file it's baulking out on.

The problem you're having sounds like you're copying a symlink (that is a file that just points to another) to a filesystem that doesn't support symlinks. There are three options for this:

• Turn your backup volume into a filesystem that accepts symlinks (eg reformat it from FAT or NTFS to EXT3 or EXT4). Might be a pain depending on how much data you've got there (that you might have to juggle while you do it).

• Just ignore symlinks and not copy them. This might break things if you try to restore as it'll be missing some files.

• Expand out symlinks so they contain a copy of the actual data. This takes up more space.

The first would be my choice but if you need the drive for another system that doesn't support EXTx volumes, that's an issue.

Whether or not the second two are realistic options, I'd run cp with its verbose flag to see what you're dealing with. If it's a single file, perhaps a plain copy would be okay, if it's not, and it's just a junky helper, perhaps omitting it would be okay.

But as a side note, most people seem to prefer using rsync to do backups. It has a whole lot of options that make it perfect for the job. You can read a version of its manpage here. It has various options (as outlined above) on how to handle symlinks.

Solution 3

See your dmesg. Usual problem is either filesystem problem or failing hard disk (and therefore filesystem problem).

You can umounting external disk and then running fsck. For example

umount /dev/sdb1
fsck -f /dev/sdb1

Comments

[aneeshep]

What is the file system of the external hdd.?.