Please integrate --drop-cache option in your upstream.
It should allow to avoid filling up the io cache with the copied data.
The current behavior pushes the cached data of other process out of the memory
and slows down the system as the ram gets filled up.
For exemple, I use rsync without this patch, rsync use 7GB of ram and when it's finished, cache ram is always used.
When I use rsync with this patch, rsync use 7GB of ram but when it's finished, cache ram is removed and avoid an unnecessary swap.
Website of the patch : http://insights.oetiker.ch/linux/fadvise/
Patch file : http://tobi.oetiker.ch/patches/rsync-3.0.9-2-fadvise.patch
I hate to say “me too”, but I would really like to see this. Is there anything I can do to help make this happen? As you can see at http://tobi.oetiker.ch/patches/, the patch is maintained since 2007!
yes , this would be usefull. Filling up memory with cache is not always usefull.
Since the patch is very linux-specific and overly klugy, it is better handled by a general-purpose (linux-specific) wrapper that can be used with any copy utility.
I'd suggest checking into this utility:
I've compiled/installed it and done some basic testing, and it seems to do a good job of issuing the fadvise64() calls on the files rsync opens.
nocache rsync -aiv --rsync-patch='nocache rsync' some-host:/src/ /dest/
nocache rsync --daemon
> nocache rsync -aiv --rsync-patch='nocache rsync' some-host:/src/ /dest/
That was supposed to be --rsync-path=...
Created attachment 11169 [details]
fadvice without effect on remote
Hi, I'm testing the patch on 3.1.1 on an embedded device with swapless linux.
The patch works perfect on my application but to avoid to apply the patch also on the remote side (where I have no control) I have saparated the new command line option --drop-cache in 2:
--drop-cache that works local
--remote-drop-cache that works on remote
I think that the patch is more or less mandatory on embedded linux machine to avoid oomkill from not o well tuned kernel and to increase general performance.
regards, and many thanks