Bug 6183 - rsync error file too large (27)
rsync error file too large (27)
Status: NEEDINFO
Product: rsync
Classification: Unclassified
Component: core
3.0.4
Other AIX
: P3 normal
: ---
Assigned To: Wayne Davison
Rsync QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2009-03-11 10:25 UTC by BrunoB
Modified: 2011-06-01 07:58 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description BrunoB 2009-03-11 10:25:13 UTC
I am on AIX 5.3 TL8, 

I get this error when trying to transfer files, the file size is 2729126908 bytes and stops at 2G, (see output at bottom)

i tried to copy it using scp and it works fine,


rsync: writefd_unbuffered failed to write 4092 bytes [sender]: Broken pipe (32)
rsync: write failed on "/export/data/c2dev/c2/backup/mksysb/daily/mksysb_0310_abcdef51.gz": File too large (27)
rsync error: error in file IO (code 11) at receiver.c(298) [receiver=3.0.4]
rsync: connection unexpectedly closed (14240 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(632) [sender=3.0.4]

My version of rsync are identical on both sides:
# rsync --version
rsync  version 3.0.4  protocol version 30
Copyright (C) 1996-2008 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 32-bit inums, 32-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, no IPv6, batchfiles, inplace,
    append, ACLs, no xattrs, no iconv, no symtimes


here is the output of rsync using multiple -v

data recv 32768 at 2147221504
data recv 32768 at 2147254272
data recv 32768 at 2147287040
data recv 32768 at 2147319808
data recv 32768 at 2147352576
data recv 32768 at 2147385344
data recv 32768 at 2147418112
data recv 32768 at 2147450880
rsync: write failed on "/export/data/c2dev/c2/backup/mksysb/daily/mksysb_0311_abcdep51.gz": File too large (27)
_exit_cleanup(code=11, file=receiver.c, line=298): entered
rsync error: error in file IO (code 11) at receiver.c(298) [receiver=3.0.4]
_exit_cleanup(code=11, file=receiver.c, line=298): about to call exit(11)
rsync: connection unexpectedly closed (49 bytes received so far) [generator]
_exit_cleanup(code=12, file=io.c, line=632): entered
rsync error: error in rsync protocol data stream (code 12) at io.c(632) [generator=3.0.4]
_exit_cleanup(code=12, file=io.c, line=632): about to call exit(12)
Comment 1 BrunoB 2009-03-11 11:38:57 UTC
Justed tested using rsync 3.0.2, and it works fine (same file, same servers)

so the problem occursd using rsync 3.0.4
Comment 2 Wayne Davison 2009-03-13 12:11:51 UTC
"File too large" is an OS error, so you need to figure out why your OS is returning that.  I'd imagine that rsync was not compiled with large-file support.  If so, let me know if there are any configure changes that could be made to improve the compilation on your system.
Comment 3 Steffan Rhoads 2009-04-01 12:32:14 UTC
I am on AIX 6.1 TL02.  Been using older version and encountered the file too large error (27) so I downloaded and tried 3.0.5 but encountered the same error but ~10GB has been copied:

sending incremental file list
patch-depot/db/obi/
patch-depot/db/obi/B24442-01_1of3.zip
  1061715968  83%   12.78MB/s    0:00:15
rsync: writefd_unbuffered failed to write 4092 bytes [sender]: Broken pipe (32)
rsync: write failed on "/opsshare/patch-depot/patch-depot/db/obi/B24442-01_1of3.zip": File too large (27)
rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.5]
rsync: connection unexpectedly closed (10306 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.5]

real    3m30.71s
user    0m0.00s
sys     0m0.01s
tusobidbd1/home/root# ls -l "/opsshare/patch-depot/patch-depot/db/obi/B24442-01_1of3.zip"
-rw-r--r--    1 oraupk   dba      1073741312 Apr 01 10:00 /opsshare/patch-depot/patch-depot/db/obi/B24442-01_1of3.zip
Comment 4 Wayne Davison 2011-05-28 05:13:33 UTC
As mentioned, this is an issue with how rsync was compiled and/or linked.  If it is compiled with large-file support in the OS libraries, then it supports large files.  If the libraries don't support large files, or it was not compiled knowing that it should support large files, then large files aren't supported.

So, someone needs to figure out what AIX requires for proper compilation (e.g. what support libraries are not installed, what options are not used, or whatever).
Comment 5 Dal 2011-06-01 07:58:01 UTC
The issue happens intermittently. Most of the time large files are sync'd fine.
I have run rsync on the failed file several times now and it hasn't failed. Is there any debugging I can put in place ?

----- Original Message -----
From: samba-bugs@samba.org
To: rsync-qa@samba.org
At:  5/28  6:14:11

https://bugzilla.samba.org/show_bug.cgi?id=6183

Wayne Davison <wayned@samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |NEEDINFO

--- Comment #4 from Wayne Davison <wayned@samba.org> 2011-05-28 05:13:33 UTC ---
As mentioned, this is an issue with how rsync was compiled and/or linked.  If
it is compiled with large-file support in the OS libraries, then it supports
large files.  If the libraries don't support large files, or it was not
compiled knowing that it should support large files, then large files aren't
supported.

So, someone needs to figure out what AIX requires for proper compilation (e.g.
what support libraries are not installed, what options are not used, or
whatever).