Bug 3784 - Showing Progress without being Verbose
Summary: Showing Progress without being Verbose
Alias: None
Product: rsync
Classification: Unclassified
Component: core (show other bugs)
Version: 3.0.0
Hardware: All Linux
: P3 enhancement (vote)
Target Milestone: ---
Assignee: Wayne Davison
QA Contact: Rsync QA Contact
Depends on:
Reported: 2006-05-17 20:10 UTC by Alan Siu-Lung Tam
Modified: 2014-12-01 19:30 UTC (History)
6 users (show)

See Also:

patch to give total run progress indication (4.80 KB, patch)
2008-07-04 15:54 UTC, Graeme Humphries (mail address dead)
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alan Siu-Lung Tam 2006-05-17 20:10:29 UTC
I'd like to rsync 100+ million files. I want to show the overall progress (e.g. 20% done) instead of generating one line per file like --progress.
Comment 1 P. K. CHAN, Esq. [TnC] 2006-12-21 09:46:35 UTC
How's the progress here?  I'd say this is an important enhancement as I routinely transfer tens of thousands (small) files from one system to another and --progress would be an overkill for the purpose of monitoring progress -- all I want is just a neat and tidy summary.
Comment 2 Graeme Humphries (mail address dead) 2008-07-04 12:17:25 UTC
I second this, it's about the only feature I consistently wish for in rsync. Also, can Wayne or Tam change this to Hardware: All please?
Comment 3 Graeme Humphries (mail address dead) 2008-07-04 15:54:53 UTC
Created attachment 3390 [details]
patch to give total run progress indication

Here's a little patch that creates a new flag "--total-progress", which causes the progress indication to be based on data-copied/total-data, rather than on a per-file basis. It may be used with or without -v(erbose).

Is this useful, or would people prefer to just use the existing --progress/-P flag, and have the global totals only used in non-verbose mode?

Also, when used with -r(ecursive) mode, you can see some interesting oddities, as it scans subdirectories as it goes. This means it's possible to have the total number of files (and the % completed) *decrease* as it runs as it scans more subdirectories. I'm not sure if there's any real way around this.
Comment 4 Graeme Humphries (mail address dead) 2008-07-04 15:55:43 UTC
Oh, forgot to mention, but that patch should apply to v3.0.3.
Comment 5 Wayne Davison 2008-07-13 23:29:15 UTC
Thanks for the patch.  I'm incorporating this idea into the latest rsync (3.1.0dev) using the new general-purpose --info=FLAGS option to turn this on:  the --progress option will infer --info=progress (along with the other output flags that it traditionally affects) and gives you the normal per-file progress.  You can instead specify --info=progress2 to get overall transfer progress, which will output on a single line (if rsync is not outputting any names).  I'm still working on what I want the various fields to display, but it is working in the latest git repository version.
Comment 6 Graeme Humphries (mail address dead) 2008-07-14 00:18:59 UTC
Great!! I wasn't sure what output format, options, and behavior would end up being the most useful or user friendly, so I'll defer to your judgment on that. In any case, I know quite a few people who'll be very happy when this feature makes it into the major distributions they use, so thanks again. :)
Comment 7 Bassu Khan 2009-12-25 07:30:34 UTC
There seems to an error with speed calculation with --info=progress in version 3.1dev. It's showing a continues speed of 26.90MB/s which is just impossible to attain under my internet downstream. 

# rsync3 -arth --info=progress2 --no-motd --exclude=debug/ rsync://mirrors.kernel.org/centos/5.4/updates/i386 /Raid/centos/5.4/updates

        135.08M   8%   26.90MB/s    0:00:54  Quit68, to-chk=275/346)
Comment 8 João M. S. Silva 2014-12-01 19:30:57 UTC
I'd also like this enhancement very much, especially because when I use -v the process stalls somehow during the transfer (simple home directory backup) and I have to "killall rsync".

Why does --progress imply --verbose? I was expecting -v to be one thing and --progress to be another, completely independent.

Thanks for this great program.