Automatically moving files to a directory, one by one, and only when the target folder is empty

Question

Is it possible? And in reverse alphabetical order?

Essentially, this: How can I move files by type recursively from a directory and its sub-directories to another directory?

Except that each file is not moved to the destination directory unless a separate process has fetched the sole file in that destination directory and moved it elsewhere (thus the target folder is empty and 'ready' for the next file to be moved there).

Solution 1

Do you want something like this?

#!/usr/bin/env bash
## This is the target path, the directory
## you want to copy to.
target="some/path with/spaces";

## Find all files and folders in the current directory, sort
## them reverse alphabetically and iterate through them
find . -maxdepth 1 -type f | sort -r | while IFS= read -r file; do
    ## Set the counter back to 0 for each file
    counter=0;
    ## The counter will be 0 until the file is moved
    while [ $counter -eq 0 ]; do
      ## If the directory has no files
      if find "$target" -maxdepth 0 -empty | read; 
      then 
          ## Move the current file to $target and increment
          ## the counter.
          mv -v "$file" "$target" && counter=1; 
      else
          ## Uncomment the line below for debugging 
          # echo "Directory not empty: $(find "$target" -mindepth 1)"

          ## Wait for one second. This avoids spamming 
          ## the system with multiple requests.
          sleep 1; 
      fi;
    done;
done

This script will run until all files have been copied. It will only copy a file into $target if the target is empty so it will hang for ever unless another process is removing the files as they come in.

It will break if your files' or $target's name contain new lines (\n) but should be fine with spaces and other strange characters.

Solution 2

Straightforward solution using inotify-wait from inotify-tools:

#!/bin/bash
SOURCE=$1
DEST=$2
(
IFS=$'\n'
for FILE in $(find "$SOURCE" -type f | sort -r); do
  mv "$FILE" "$DEST"
  inotifywait -e moved_from "$DEST"
done
)

Explanation:

• (IFS=$'\n' ...)

  Runs the body of the script in a subshell where the internal field separator $IFS is set to a newline character, allowing the for loop to handle filenames with spaces properly. The $'string' syntax makes bash interpret escape sequences in string, so $'\n' is interpreted properly as the newline character.

• for FILE in $(find $SOURCE -type f | sort -r)

  Builds a reverse sorted list of files in $SOURCE and its subdirectories and iterates through the list one file at a time setting the value of $FILE to the next file to be moved.

• mv "$FILE" "$DEST"

  Moves the current $FILE to $DEST directory.

• inotifywait -e moved_from "$DEST"

  Establishes an inotify watch which is triggered when a file is moved out of the watched directory. This will cause the script to block while the the $DEST directory is emptied. It is assumed that $DEST is empty when the script is invoked.

Comments

[terdon]

Nice one for intotify! This will break on file names with spaces though.

[Thomas Nyman]

@terdon Fixed handling of file names with spaces.

[iceequal]

Thanks a million! Relieved to know there is a simple answer for people on OS X, which doesn't have inotify. Only thing was I couldn't seem to feed it a target path with spaces in it. (Tried escaping the path manually and using a proper backspaced path copied and pasted from Terminal, but could be making a silly mistake somehow I suppose). Anyway I was able to work around it with a symbolic link. Note for others: crucially, once the directory is gotten to with a symlink, the script happily accepts all files, including those with spaces and bizarre characters.

[terdon]

@iceequal there should be no problem with spaces. Was it the $target that contained a space? Did you put it in quotes as in my example?

[iceequal]

Thought your comment must just meant I was completely mistaken but I've tested it again... just putting it down to a 'quirk of OS X'. P.S., is there any way to 'debug' this script? It used to work beautifully but now it's just sitting on 'sleep' and I can't figure out why it won't try to move any files ... Lol. (I've triple checked that I'm cd'ed to the right directory, and that the external program is grabbing and moving the files when I put them in the target folder manually).

[terdon]

@iceequal if the target path is quoted, you don't need to escape the spaces (see updated script). Both target=foo/bar\ baz and target="foo/bar baz" should work, but target="foo/bar\ baz" won't. If it is hanging, that probably means the target is not empty, perhaps there is a hidden file? Try ls -a "$target" to check. To debug, you can change the script so it prints the files it finds in target, just uncomment the line I've added under the else, it will print any files found in the target.

[iceequal]

I somehow missed notification of your comment. Boom, that works! Thanks!