Bug 5299 - 2.6.9 client cannot receive files from 3.0.0 server
2.6.9 client cannot receive files from 3.0.0 server
Status: CLOSED FIXED
Product: rsync
Classification: Unclassified
Component: core
3.0.3
x86 Windows XP
: P3 major
: ---
Assigned To: Wayne Davison
Rsync QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2008-03-02 23:06 UTC by Robert Epps
Modified: 2008-07-28 20:33 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Epps 2008-03-02 23:06:54 UTC
My client is a Windows machine running rsync 2.6.9 under Cygwin. The server is a Debian system that, until today, was also running rsync 2.6.9. Every day on the Windows machine I would issue a command something like the following:

rsync -tzvr --delete rob@xx.xx.xx.xx:sourcepath destpath

to sync some files from the Linux machine to the Windows machine. This worked fine until today, when I upgraded rsync on the server to 3.0.0. (The newer version is not available for Cygwin yet). Now when I try to do the above operation, I get:

Invalid block length 146536 [sender]
rsync error: protocol incompatibility (code 2) at io.c(1324) [sender=3.0.0]
rsync: connection unexpectedly closed (639 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packaging/tmp/rsync-2.6.9/io.c(453) [receiver=2.6.9]
rsync: connection unexpectedly closed (639 bytes received so far) [generator]
rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packaging/tmp/rsync-2.6.9/io.c(453) [generator=2.6.9]

Shouldn't a 2.6.9 client still be able to copy files from a 3.0.0 server (with the server falling back to the older protocol)?
Comment 1 Wayne Davison 2008-03-03 01:03:39 UTC
It works fine in all the testing I've done on Linux systems, so you should do some more investigation to see if a particular option is causing Cygwin problems (perhaps -z?) or perhaps a certain file?
Comment 2 Robert Epps 2008-03-03 15:37:38 UTC
I booted my Sidux LiveCD on the Windows system, and ran rsync 2.6.9 to fetch the same set of files, and got the same error messages as mentioned above. So I don't think it has anything to do with Cygwin.

The files I'm copying are many gigabytes each; I tried some smaller files (in the 2-5Mb range) and they went across fine. So this seems to only happen with really big files. I'll keep experimenting and see if I can find a definite cutoff size. (Something tells me it's gonna be the infamous 2Gb or 4Gb.)

Also, while still booted into Sidux, I upgraded to rsync 3.0.0 and tried my original file transfer again. This fixed the problem. So hurry up with the Cygwin port of 3.0.0!  :-D
Comment 3 Wayne Davison 2008-03-16 22:15:16 UTC
So far I haven't had any luck trying to reproduce this.  If you have a set of files that makes this reproducible and you can point me at them for me to download and test, please email me.  I will also go over the protocol-29 code to see if I made any unintended changes.
Comment 4 Robert Epps 2008-03-17 11:15:10 UTC
I'm still working on coming up with a set of files that I can share with you that will reproduce the problem. So far I have only seen this happen with my VMware files for my work development environment. The company I work for would be most unhappy with me letting that out in the wild.
Comment 5 Chad Wallace 2008-05-04 21:29:15 UTC
I'm experiencing the same problem when copying from a 3.0.0 rsync client to a 2.6.9 server.

---
janus:/var/log/asterisk# rsync --bwlimit=1000 -v --progress --protocol=29 full neptune::janus/var/log/asterisk/
Invalid block length 132328 [sender]
rsync error: protocol incompatibility (code 2) at io.c(1324) [sender=3.0.0]
---

I've narrowed it down to that one file (/var/log/asterisk/full), which is only 23MB but was once (and still is on the destination) 17GB.  I suspect if I delete or truncate the 17GB copy on the destination, the problem will not occur. 

(...trying it out...brb...)

Bingo!

---
janus:/var/log/asterisk# rsync --bwlimit=1000 -v --progress --protocol=29 full neptune::janus/var/log/asterisk/
full
    24391536 100% 1001.09kB/s    0:00:23 (xfer#1, to-check=0/1)

sent 24394593 bytes  received 38 bytes  995699.22 bytes/sec
total size is 24391536  speedup is 1.00
---

Well, now that I'm compressing my log files, I won't have any more that are 17GB... but I thought I'd share my experience with you guys anyway...  Unfortunately, that file (/var/log/asterisk/full) contains a lot of sensitive information, so I can't share it.  I'll try to reproduce it later by creating large files from /dev/zero on the destination.
Comment 6 Remy Blank 2008-06-23 02:40:00 UTC
I have been bitten by this issue as well, when transferring a single big file:

  -rw-r-----  1 root backup 21474836480 2008-03-24 16:59 morpheus.athome.vol

That is, it's 20GB big. I am copying from a 3.0.2 client to a 2.6.6 server. The command line I use is the following:

  rsync -av --ignore-times --inplace -e "ssh -i /root/.ssh/id_backup" --stats /home/mirror/morpheus.athome.vol rsync:backup/morpheus.athome.vol

And this gives the following error:

  Invalid block length 146536 [sender]
  rsync error: protocol incompatibility (code 2) at io.c(1367) [sender=3.0.2]

If I fix the block size to 128kB by adding "-B 131072", the transfer succeeds. So it seems that older rsync versions don't cope with block sizes over 128kB, which means a file size limit of 16GB.
Comment 7 Wayne Davison 2008-06-24 01:21:42 UTC
This is now fixed in the latest code -- rsync 3 will use/allow a block size that is compatible with an older rsync when using protocol 29 or lower.  (The 3.0.3pre3 release contains this fix.)

Thanks for all the help getting the cause sorted out.