[mdlug] rsync scenarios

[Aaron Kulkis]

Michael ORourke wrote:
> Lug Nuts,
> 
> I have a couple of scenarios that I was trying to figure out.
> 
> First scenario...
> What happens if you execute multiple rsync processes against the same directory at about the same time.  Will it cause problems?
> 


What experiments have you conducted, and what were the results?

> Suppose you have an automated process, that checks a directory on Srv-1/dir1, then fires off a rsync process to Srv-2/dir1, what happens if the first rsync doesn't finish before another rsync process is fired off?  Ideally, I would like to have it wait, then re-execute or throw an error (rsync in process or something).  Of course I could write a wrapper script that sets a lockfile when the rsync process starts, and clears it when it finishes.  So I could guarantee that only one rsync process is running at a given time.
> 
> Another scenario...  what if you have a process that is writing to a file on Srv-1, but hasn't completed yet when the first rsync process fires off?  Does that file get skipped, or does it get partially rsync'd?
> 
> Anyone know how rsync handles these 2 different scenarios?

The best way to find out is to create some large files, run a
script which starts one rsync process asynchronously (put
an '&' at the end of the command -- the script will continue
on to the next command WITHOUT waiting for the &'ed command
to finish -- for more information, look in the sh, ksh, csh,
bash or tcsh manual), and then look at the results you get.


But in general, the rule is most likely this:

If two rsync processes are READING from the same location(s),
but WRITING to different locations, then everything should
be safe.  In any other scenario (both writing to the same
paths and/or  one writing to the same place another is reading
from), then you risk a race condition which is destructive.

> 
> Thanks,
> Mike
> 
> _______________________________________________
> mdlug mailing list
> mdlug at mdlug.org
> http://mdlug.org/mailman/listinfo/mdlug