Why is mv so much faster than cp? How do I recover from an incorrect mv command?

Question

I drag and drop a folder into another by mistake in FileZilla.

~/big_folder
~/some_other_folder

The folder got moved is a very huge one. It includes hundreds of thousands of files (node_modules, small image files, a lot of folders)

What is so weird is that after I release my mouse, the moving is done. The folder "big_folder" is moved into "some_other_folder".

~/some_other_folder/big_folder

(there is no big_folder in the ~/ after moving)

Then I realize the mistake and try move back but it fails both on FileZilla and terminal.

Then I have to cp -r to copy files back because there are server-side codes accessing those files in ~/big_folder

And it takes like forever to wait ...

What should I do?

BTW, here are the output from FileZilla (it's the failure of the moving back):

Status:       Renaming '/root/big_folder' to '/root/some_other_folder/big_folder'
Status:       /root/big_folder -> /root/some_other_folder/big_folder

Status:       Renaming '/root/some_other_folder/big_folder' to '/root/big_folder'
Command:  mv "big_folder" "/root/big_folder"
Error:          mv /root/some_other_folder/big_folder /root/big_folder: received failure with description 'Failure'

Solution 1

If a directory is moved within the same filesystem (the same partition), then all that is needed is to rename the file path of the directory. No data apart from the directory entry for the directory itself has to be altered.

When copying directories, the data for each and every file needs to be duplicated. This involves reading all the source data and writing it at the destination.

Moving a directory between filesystems would involve copying the data to the destination and removing it from the source. This would take about as long time as copying (duplicating) the data within a single filesystem.

---

If FileZilla successfully renamed the directory from ~/big_folder to ~/some_other_folder/big_folder, then I would revert that using

mv ~/some_other_folder/big_folder ~/big_folder

... after first making sure that there were no directory called ~/big_folder (if there was, the move would put big_folder from some_other_folder into the ~/big_folder directory as a subfolder).

Solution 2

The existing answer is great, but I'd like to expand on it a bit by showing exactly what is happening when you move versus when you copy a file. When you look at the syscalls during a copy, you see:

open("hello1.txt", O_RDONLY)               = 3
open("hello2.txt", O_WRONLY|O_CREAT, 0644) = 4
read(3, "Hello, world!\n", 4096)           = 14
write(4, "Hello, world!\n", 14)            = 14
close(3)                                   = 0
close(4)                                   = 0

This opens the source file, then creates a second file. It then reads the contents of the source file to memory and writes that memory to the destination file. This requires several context switches and some disk I/O which can be quite high for large files. If you move a file however, you see:

rename("hello1.txt", "hello2.txt")         = 0

It's important to remember that you will only see the file be renamed if it is on the same filesystem on the same physical disk. If you create a huge, multi-gigabyte file and then move it between two locations in your home, you will notice the action completes instantly. If, on the other hand, you move it to an external device, it will take as long to move as if you used cp instead. The syscall trace would be identical to the first one, but with unlink("hello1.txt") at the very end. This is because renaming can only be used to move a file if the source and destination are on the same filesystem.

Comments

[Captain Man]

Ah, the most useful of error messages, received failure with description 'Failure'.

[ctrl-alt-delor]

Go to a terminal, and type in the command mv /root/some_other_folder/big_folder /root/big_folder. What error message do you get?

[Nemo]

I would probably go with cp -al

[user1717828]

OP's mv vs cp question is addressed, but I would love to hear why he was able to move the folder in one direction instantly but not the other.

[Pat Gunn]

The key to quickly moving it back is to find a way to stop the server-side processes (if this is possible). It sounds like you're in an environment where moving files can start processes automatically, and that's a pretty specific setup; without more information we might not be able to help.

[KamKamKee]

@CaptainMan hmm, a failure that failed, does that mean that it started working?

[David Richerby]

For essentially the same reason that it's much faster to move a book from one room to another than to create a copy of the book.

[Tommy]

Oh ... is that why I see the word "renaming" rather than "moving" in the output?

[Kusalananda]

@AGamePlayer "Failure" is unfortunately not a good error description. I would have used mv ~/some_other_folder/big_folder ~/ after making sure there is no other big_folder in the home directory. I have never used FileZilla.

[Tommy]

Why would there be other big_folder existed if the first move is done?

[Kusalananda]

@AGamePlayer I don't know what you have tried to do in the meanwhile. You mentioning attempting to copy the folder, for example.

[Tommy]

It's the failure of the moving back. Because of that failure, I have to copy the files.

[Tommy]

btw, do you have any idea what is the speed of cp? like xxM/s?

[Kusalananda]

Let us continue this discussion in chat.

[Mark Stewart]

Another reason not to depend on Windows GUI tools to do some file maintenance on Unix.

[Captain Man]

@AGamePlayer To make a metaphor, think of moving a file like changing the page number beside a chapter in a table of contents and coping a file like adding extra pages.

[ctrl-alt-delor]

@MarkStewart why “on Unix” at the end of your comment?; Is there a time when it is a good idea?

[jamesqf]

@AGamePlayer: That's why the most trivial use of mv is to simply rename a file. As for the speed of cp, it depends on the speed of your hardware. Really fast on an SSD, slower on a disk drive, really slow on a filesystem mounted over a slow network connection.

[ygreaney]

@jamesqf Indeed, people new to *nix are surprised to find that there is no separate "rename" command. It's entirely logical, because as the parent says, a "move" within the same filesystem is the same thing as a rename.

[Kusalananda]

@MontyHarder The standard C library interface for moving a file within a single filesystem is still called rename() though.

[A.L]

The OP moved a directory, not a file.

[Damon]

In addition to making a copy of the data, copying files also has to read and access-check every inode (recursively) within the directory, and has to create inodes at the destination. Depending on implementation details, this can be much slower than actually copying the bulk of data. Moving ideally (when on the same volume) only changes one link.

[glace]

It does still apply tho, unless OP is moving empty folders, which would be the only case where no files are involved

[phemmer]

Note that on some filesystems, copy operations can be as fast, or nearly as fast as move operations. This is because they are COW (copy-on-write) based. Some such filesystems are XFS and Btrfs, and this COW behavior can be used by doing cp --reflink=always (or auto).

[Izkata]

@ctrl-alt-delor Probably a reference to this

[Thegs]

@A.L In Unix-like systems everything is a file.

[forest]

@A.L A text file was just an example. For a directory, the only difference is that you would have some getdents() and mkdir() calls sprinkled around.

[jamesqf]

@Monty Harder: Well, I've always (and by that, I mean since the days I used Unix on a PDP-11) thought that mv for "move" was an illogical name. Should have been rn for rename, but I suppose that would make it too easy to mistype rm instead.