I've got a server that gets backed up during normal operation. While the rsync sender is building up the file list, it saturates the disk subsystem with the directory traversal which drives load due to other operations to silly heights.
The patch to follow implements a --filelimit=n option that limits the number of files to be looked up per second, very similar to the --bwlimit option.
Created attachment 2096 [details]
implementation of --filelimit
Rsync already has a slow-down.diff that does something similar.
Yes, it would be nice to be able to get rsync to not saturate the file I/O in some way. Perhaps something more like the --slow-down option would be better than this, but I haven't looked at it enough yet to decide, so I'll leave this open for now.
This is really asking for the same thing as bug 3491.
*** This bug has been marked as a duplicate of 3491 ***