Linux: Continuously synchronize files, one way
Solution 1
You can also use inotifywait
from the inotify-tools package.
inotifywait -r -m -e close_write --format '%w%f' /tmp | while read MODFILE
do
echo need to rsync $MODFILE ...
done
Solution 2
Lsyncd would be a good solution for this.
Lsyncd watches a local directory trees event monitor interface (inotify or fsevents). It aggregates and combines events for a few seconds and then spawns one (or more) process(es) to synchronize the changes. By default this is rsync. Lsyncd is thus a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or blockdevices and does not hamper local filesystem performance.
Bottom-line, it uses the same kind of tools to do the job (inotify and rsync) as suggested by other answers, but it's easier to set up for someone not familiar with shell scripting.
Solution 3
I need this a lot since my code needs to run on remote boxes and I write code on local machine. I found a nice tool which you can use to continuously monitor your local folders and sync them to remote or local folder: https://github.com/axkibe/lsyncd
A simple command to continuously sync a local dir with remote machine over ssh will be:
lsyncd -log all -nodaemon -rsyncssh <local_path> <user>@<ip> <remote_path>
Just like with any other rsync command, make sure you give the folder path right and check before you run the command. I almost had killed one of my remote machine because I missed to give a correct destination directory. Make sure yo don't miss out the remote path and don't use '/' unless you know what you are doing.
Solution 4
If you need to observe filesystem, then inotify is the way to do it. I would write a simple python script using pyinotify to perform sync when filesystem get changed. See documentation. You might also checkout the autosync.py for some inspiration. Have fun.
Solution 5
What I did once is have a bash script running ls -l
in a loop (with some sleep) and comparing to the previous output. If it changed, do your synchronization.
#!/bin/bash
listcommand="ls -l $*"
newfilelist=$( $listcommand )
while true
do
if [[ $oldfilelist != $newfilelist ]]
then
oldfilelist=$newfilelist
# run your synchronization tool
fi
sleep 10 || exit 2
newfilelist=$( $listcommand )
done
Start this script in a new terminal with the file names as arguments (after putting in your synchronization tool).
(I used this to start a compilation, not to synchronize, but this would work a similar way.)
Related videos on Youtube
stolsvik
Updated on September 18, 2022Comments
-
stolsvik over 1 year
I've been fooling around with generic methods lately and came across methods that look like this:
public static void Requires<TException>(bool condition, string message) where TException : Exception
To my understanding, when using the above method you provide a
Type
that inherits fromException
and if thecondition
isfalse
the providedException
type is thrown.How does this work under the hood?
Is theTException
instantiated like sothrow new TException();
?
And how can you pass in themessage
parameter if theType
is unknown to the method (all it knows is that it inherits typeException
)?-
Aniruddh Chandratre almost 13 yearsconsider using the rsync tool, or sharing a folder in the webserver's document root so you could operate on the files directly under windows
-
stolsvik almost 13 yearsrsync is "one go". I need continous updates, that is the entire point here - I edit a file, save it, and the product/system/idea I request would pick this save-action up and upload the new version immediately. NB: Both sides are Linux. NB2: I want to edit on local files, or else sshfs itself would cut it.
-
Karl Bielefeldt almost 13 yearsOff-topic voters, this is a boundary case, but I think it falls fairly clearly under "tools commonly used by programmers" in the FAQ.
-
Panagiotis Kanavos almost 9 yearsIs this Code Contracts code? In this case
Requires
means more than simply checking a condition and throwing an exception -
Admin almost 9 years@PanagiotisKanavos I saw it in Code Contracts yes. I'm not so much interested in what it does, I'm more interested in how one would take the generic type, create a new instance of it, and then use it.
-
Panagiotis Kanavos almost 9 yearsIf the method has the
where T:new
constraint, you simply callnew T()
. Code Contracts methods probably have this constraint as well, or use specialized versions for eg. argument exceptions. The source code is available in Github -
phs almost 8 yearsWhile it's not a perfect match, you might look at overlayfs
-
-
Karl Bielefeldt almost 13 yearsOn the inotify-tools website, there's a pretty good example of using inotify-wait to trigger an rsync.
-
CenterOrbit almost 13 yearsI was looking for this exact thing last night! ahhh I love superuser
-
stolsvik almost 13 yearsThanks for answering. However, I specifically excluded this very approach in the question.
-
Ciantic over 10 yearsnot only easier, it also tries to handle situations like moving directory properly, without rsyncing.
-
Admin almost 9 yearsSure, I'm with you so far. But what if I needed to make a new instance of that
Type
in a method I make myself? -
Panagiotis Kanavos almost 9 yearsIf the type has the
new
constraint, you can simply callnew TException()
-
NeddySpaghetti almost 9 yearsYou can use
default(Type)
or if you have anew()
constraint you can usenew Type()
. -
Flash almost 7 yearsIs there a way to do this but keep the socket open?
rsync
/scp
has an annoying delay while establishing the ssh connection. -
Friedrich over 6 yearsBeware: This program has some problematic features. As of writing this: 1. It deletes remote files not present at the source per default. 2. "remote:" refers to "remote:/" instead of the home folder. 3. It daemonizes, so you do not know what is going on. 4. It does not immediately respect the TERM signal.
-
Will Angley about 6 years@Flash yes, you can use SSH multiplexing to keep the socket open.
-
hanshenrik about 2 yearsunfortunately this gets me auto firewall IP banned xD
-
Admin almost 2 yearsThis gets my ip banned under a firewall after sometime.