preserving extended attributes with cp/rsync

Question

When copying with cp, the extended attributes are not preserved, even with explicit

cp -a --preserve=all /source /dest

or

cp -a --preserve=xattr /source /dest

The same is with rsync, i.e.

rsync -aq -A -X --delete /source /dest

However, on the destination filesystem, I can create extended attribute manually (with chattr). This means that the target filesystem supports xattr.

Why am I unable to preserve xattr with cp or rsync ?

Additional information:

• Both source and target filesystems are ext4
• Both source and target filesystems are local (not nfs)
• I am using Debian Wheezy

Solution

Update

After messing around with this some more and looking at the code for chattr and other e2fsprogs, it is clear that the attributes set by chattr and those set by libattr (eg with the command setfattr) are very different. chattr sets ext filesystem flags which simply do not map to an named attribute or namespace. None of them show up with any call to libattr's listxattr. They probably should map to named attributes in the system namespace as assumed below, but as of yet this is completely unimplemented. Also the system.posix_acl_access attribute I mistook for the mapping to one of these attributes below, is nothing to do with the ext filesystem flags and is rather to do with access control lists. The associated strace messages appear for any file and disappear when only cp --preserve=xattr is used.

It seems that the attributes set by chattr are specific to ext filesystems and that the only way to affect them is through e2fsprogs tools. In fact the man page does not actually use the term 'extended attributes' for them, but rather 'file attributes'. 'Real' extended attributes are name/value pairs which can be altered by libattr and are implemented on multiple filesystems. These are what cp and rsync look for and transfer over to copied files when the right options are given. It does however seem that system namespace exists to map the chattr attributes to names and ultimately to equivalent attributes on other filesystems, but for now this doesn't work.

I have left the original answer intact as there is some good information there, although it does go quite far wrong at points.

Update 2

I should have came back to this again before now, but as per this answer, chattr works on more than just ext filesystems. According to Wikipedia, it is equivalent to the chflags command on BSD based systems.

I wrote a script to test the setting and reading of these attributes on a few filesystems and got the following results:

ext4:
suS-iadAcj-t-e-- mnt/test_file
suSDiadAcj-tTe-- mnt/test_dir

reiserfs:
lsattr: Inappropriate ioctl for device While reading flags on mnt/test_file
lsattr: Inappropriate ioctl for device While reading flags on mnt/test_dir

xfs:
--S-iadA-------- mnt/test_file
--S-iadA-------- mnt/test_dir

btrfs:
--S-iadAc------C mnt/test_file
--SDiadAc------C mnt/test_dir

Note that all attempts to read/set reiserfs file flags gave the above error, despite it being listed on Wikipedia as having some functionality. I did not test reiser4. Also while the c flag can set on ext4 it is not honoured. There may also be tuning/mount options that affect these flags, but I couldn't find any.

It does however seem that currently chattr is the only utility on Linux capable of modifying these attributes and so no copy utility is capable of preserving them.

Original Answer

The reason for rsync seems to be that is doesn't even try. From the -X section of the rsync documentation:

For systems that support extended-attribute namespaces, a copy being done by a
super-user copies all namespaces except system.*.  A normal user only  copies
the user.* namespace.

It is difficult to map the attribute letters used by chattr and lsattr to the underlying named attributes used in the filesystem (for one there is no list on the internet). From my tests though, the A attribute maps to the system.posix_acl_access attribute and since this is the system namespace, rsync won't even try to copy it. The other two namespaces not mentioned in the man snippet are trusted and security, root privileges are required to set these (and rsync won't try without).

Most likely the attributes you have tried to set fall in the system namespace which rsync ignores (and probably wisely). Either that or you need to be root to get the ones that aren't.

As for cp, there appears to be bugs at play. Running strace on cp -a, I get the following two interesting lines:

fgetxattr(3, "system.posix_acl_access", 0x7fff5181c0e0, 132) = -1 ENODATA (No data available)

and

fsetxattr(4, "system.posix_acl_access", "\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x04\x00\x04\x00\xff\xff\xff\xff \x00\x04\x00\xff\xff\xff\xff", 28, 0) = 0

Firstly the fgetxattr call doesn't return any data (probably because there isn't any - the existence of the attribute is enough), yet somehow cp finds 28 bytes of (junk?) data to set as the attribute value in the destination file. This seems like a bug in cp, but rather what is causing the issues seems to be a bug in libattr as the fsetattr call returns 0 for success without actually setting the attribute.

I get this behaviour on ext4 regardless of whether I mount with user_xattr. I can't find any documentation on this other than to say that 'some systems' need this mount option for extended attributes to work. Seemingly mine (Debian Jessie) doesn't. Even there is a mounting issue I have missed, it is wrong for fsetattr and thus cp to fail silently.

Actually user_xattr is needed on ext2, ext3, reiserfs and possibly some others. It is not necessary for ext4

Note also that the attr tools setfattr, getfattr and attr (the latter is documented to be only for XFS only, but seems to work just as well as the others for ext4) have problems working in anything but the user namespace. I get Operation not supported if I try to use setfattr to put an attribute in the the system namespace (or no namespace as per this bug). setfattr appears to succeed in the trusted and security namespaces, but then getfattr fails to read anything back and also fails to read anything from the system namespace set by chattr. The reason that chattr succeeds is that it uses an ioctl call and not libattr.

What does work perfectly though, is setting extended attributes in the user namespace with setfattr and using rsync or cp to copy with them intact (there are even no issues with cp if you don't specify a value when creating the attribute). I think the bottom line is that using system namespace values is currently buggy and/or unsupported, at least in Debian and probably other distros too. Likely the rsync developers know this, which is why they ignore them.