Clone directory tree structure and copy files to the corresponding directories modified after a specific date

Question

I have a folder with more than 30 sub directories and I want to get the list of the files which was modified after a specified date(say sep 8 which is the real case) and to be copied with the same tree structure with only the modified files in that folder

I have say 30 dir from that I have the list of the files I need found using last modified date Find command output

a/a.txt
a/b/b.txt
a/www.txt
etc..

For example I want the folder "a" created and only the a.txt in it...like wise for the other also "a/b" to be created and b.txt inside it...

Solution 1

Assuming you have your desired files in a text file, you could do something like

while IFS= read -r file; do 
    echo mkdir -p ${file%/*}; 
    cp /source/"$file" /target/${file%/*}/${file##*/}; 
done < files.txt 

That will read each line of your list, extract the directory and the file name, create the directory and copy the file. You will need to change source and target to the actual parent directories you are using. For example, to copy /foo/a/a.txt to /bar/a/a.txt, change source to foo and target to bar.

---

I can't tell from your question whether you want to copy all directories and then only specific files or if you just want the directories that will contain files. The solution above will only create the necessary directories. If you want to create all of them, use

find /source -type d -exec mkdir -p {} /target

That will create the directories. Once those are there, just copy the files:

while IFS= read -r file; do 
    cp /source/"$file" /target/"$file"
done

---

Update

This little script will move all the files modified after September 8. It assumes the GNU versions of find and touch. Assuming you're using Linux, that's what you will have.

#!/usr/bin/env bash    

## Create a file to compare against.
tmp=$(mktemp)
touch -d "September 8" "$tmp"

## Define the source and target parent directories
source=/path/to/source
target=/path/to/target

## move to the source directory
cd "$source"

## Find the files that were modified more recently than $tmp and copy them
find ./ -type f -newer "$tmp" -printf "%h %p\0" | 
    while IFS= read -rd '' path file; do
        mkdir -p "$target"/"$path"
        cp "$file" "$target"/"$path"
    done

Strictly speaking, you don't need the tmp file. However, this way, the same script will work tomorrow. Otherwise, if you use find's -mtime, you would have to calculate the right date every day.

---

Another approach would be to first find the directories, create them in the target and then copy the files:

1. Create all directories

    find ./ -type d -exec mkdir -p ../bar/{} \;
   

2. Find and copy the relevant files

    find ./ -type f -newer "$tmp" -exec cp {} /path/to/target/bar/{} \;
   

3. Remove any empty directories

    find ../bar/ -type d -exec rmdir {} \;
   

Solution 2

pax

Pax is the best tool for this task. It's POSIX's replacement for cpio and tar (and unlike tar it includes a pass-through mode, not just archive creation and extraction). Sadly, it's omitted from the default installation in some Linux distributions, but it's only an apt-get/yum/emerge/… invocation away.

Pax has a limitation: recently modified directories will be copied even if they contain no file to be copied. You can remove empty directory afterwards (unless there are pre-existing empty directories in the destination that you want to preserve).

cd /path/to/source
pax -rw -pp -T 201409080000 . /path/to/destination/
find /path/to/destination/ -depth -type d -exec rmdir {} +

I'll mention other ways that can be adapted to similar but not identical use cases where pax doesn't have a filter.

Zsh

Use the glob qualifier m to match files by their modification time (e.g. m-10 for files modified in the past 10 days; you can use other units instead), and . to match regular files. The history modifier h retains the directory part of the file name.

cd /path/to/source
for x in **/*(.m-10); do
  mkdir -p -- $x:h
  cp -p -- $x /path/to/destination/$x
done

Alternatively, you can use the zmv function to copy files matched by a wildcard expression. There's no built-in way to create the destination directory, so I provide a function to do it.

autoload -U zmv
mkdir_cp () {
  mkdir -p -- ${(P)#}:h
  cp -- $@
}
cd /path/to/source
zmv -p mkdir_cp -Q '**/*(.m-10)' '/path/to/destination/$f'

POSIX find

With find, you can use -mtime -10 to match files modified in the past 10 days, or -newer reference_time to match files modified after reference_files.

touch -t 201409080000 /tmp/reference_time
cd /path/to/source
find . -type f -newer /tmp/reference_time -exec sh -c '
  mkdir -p /path/to/destination/"${0%/*}"
  cp "$0" "/path/to/destination/$0"
' {} \;

Solution 3

You could use find with rsync to do this.

find ./source/ -newer /tmp/foo -print0 | rsync -av --files-from=- --from0 ./ ./destination/ 

Testing

1. I created 2 directories named source and destination.

2. For testing purposes, I created a temporary file with a timestamp as below.

   touch --date "2014-09-08" /tmp/foo
   

3. Now, inside source directory, I created bunch of sub-directories that need to be copied to the destination directory (some sub-directories with spaces as well, for testing purposes). I also created some sample files. The destination directory is currently empty.

4. I have changed the creation date for one of the files inside the sub directory.

   touch -c -t 1110111730 oldfile
   

5. Now, I execute the find command as I had specified. The find command finds all the files based on the condition that we specify and the rsync after the pipe gets the list of files that our find command has produced.

Now, inside the destination directory, the file named oldfile shouldn't be there as it has a time previous than the one that we desire.

So, I go inside the destination directory to check and I have all the sub directory structure as I desire and all the files that matched our find condition and the file named oldfile is not present.

References

http://www.commandlinefu.com/commands/view/1481/rsync-find

Solution 4

This is exactly how cpio was designed to be used.

find . -whatever -print | cpio -pvmd /path/to/copy/to/.

Done.

View more solutions

Comments

[terdon]

Do you have the list of files in a file or are you running a find command? If you run find, can we see the command? It would be easy to do with find's -exec option for example.

[Danny]

@ terdon this is the command find -printf "%TY-%Tm-%Td %TT %p\n" | sort -n which gives me the list of files from that I ll get the neccessary files copied manually to a txt file(Mostly last 50 files something like that)

[terdon]

Manually? Manually? No my son, do not sin in this way against the *nix gods! Why should anything be done manually? Why don't you explain the entire thing you are trying to do, it should be possible with a single find. Edit your question and explain exactly what you're attempting.

[Danny]

I am editing the post with what I want to accomplish it thanks for the guidance

[terdon]

OK, done. It should work now. Let me know if there are any problems.

[Danny]

the source will always differ it can be "a" or "b"...I have a txt file which has the path to the files as shown on the OP got from the Find command

[Gilles 'SO- stop being evil']

Why are you doing such complex parsing? find … -exec!

[Gilles 'SO- stop being evil']

You should say what -whatever is, that's part of the question. pax is the new cpio. Use -print0 | cpio -0 if your system supports it to cope with all file names (even the ones that contain newlines).

[terdon]

@Gilles that was my first thought as well but for some reason, using -exec sh -c '' did not occur to me so I couldn't figure out how to create the target dirs. Facepalm.

[terdon]

@Gilles I know dammit! I even read it and upvoted it yesterday. Hence the facepalm.

[Danny]

@terdon Creating the empty directory and copying the needed files Worked like a charm simple and well wrote