copy filenames with special characters to an external ntfs volume

Question

I'm trying to copy the Documents and Settings folder of a Windows XP system over to an NTFS external disk using a USB Live of Puppy Linux.

I encountered an encoding problem for namefiles in which the system doesn't recognize italian special characters (part of utf-8) so that using cp or the GUI file manager will bring the error invalid or incomplete multibyte or wide character.

How can I copy the files whose names include the special characters to the NTFS drive?

Solution 1

Are you sure the file names are valid on the NTFS filesystem?

Do you require that the file names stay the same?

If not, you could remove the "strange" characters to make your live easier:

There is a tool for that, detox.

You can check what would get renamed without changing the filenames first:

$ detox -n somedir/*

And then, actually do it:

$ detox somedir/*

---

Another approach is to mount the NTFS filesystem in a way that it cleans up ('sanitizes') the file names itself.

There is a mount option to enable this, windows_names:

From man ntfs-3g:

  windows_names
              This option prevents files, directories and extended  attributes
              to be created with a name not allowed by windows, either because
              it contains some not  allowed  character  (which  are  the  nine
              characters  "  * / : < > ? \ | and those whose code is less than
              0x20) or because the  last  character  is  a  space  or  a  dot.
              Existing such files can still be read (and renamed).

Solution 2

rsync has an option to change filenames to fit the destination filesystem. from the man page :

--iconv=CONVERT_SPEC

Rsync can convert filenames between character sets using this option. Using a CONVERT_SPEC of "." tells rsync to look up the default character-set via the locale setting. Alternately, you can fully specify what conversion to do by giving a local and a remote charset separated by a comma in the order --iconv=LOCAL,REMOTE, e.g. --iconv=utf8,iso88591. This order ensures that the option will stay the same whether you’re pushing or pulling files.

Try

rsync --iconv=. --archive /source /destination 

Or to specify that the local filesystem is ISO-8859-1,

rsync --iconv=ISO-8859-1,. --archive /source /destination

I couldn't work out how it decides which filesystem is the source and which is the destination when I have both drives mounted locally. But this is the way that worked for me.

Solution 3

Add iocharset=utf8 to your mount options, e.g.

mount -o iocharset=utf8 /dev/sdd1 /mnt/external

Now you can copy files to an NTFS partition that violate MS Windows standards.

This will cause problems if you do a chkdsk under Windows on the external drive, but you can just be careful not to trash your filesystem. :-)

Comments

[maja]

the files are on a NTFS Windows system partition so I'd say they are valid. I don't really require them to stay the same, but I'm not the only owner of the files and it would be better if they wouldn't change. I think I'll try to get a full list of the problematic file and try a banal GUI copy/paste from windows, hoping that none of them is a system file.. I guess detox may be useful to obtain such a list, thank you.

[maja]

I guess something more then I can see now is going on, I'll try to overwrite the output folder without the `windows-names option but using manual mount and see what happens. If it doesn't work I guess I should find a command that can output which files are "different" between two folders..