Hello, server ubuntu 64bits (3.0.4 compiled) client mac os x leopard (3.0.4 compiled) command on client : /usr/local/bin/rsync -avz -i --iconv=UTF8-MAC,ISO8859-15 --delete -e ssh /Users/kalypso/Documents kalypso@IP_SERVER:/home/MACUSERS/kalypso/ got the error on client : rsync error: error allocating core memory buffers (code 22) at io.c(635) regards.
I do not see how you could get that message with an rsync 3.0.4 client because io.c line 635 is in a comment. It would make sense if the client were 3.0.3, because in rsync 3.0.3, io.c line 635 contains the exit_cleanup(RERR_STREAMIO) that indicates a broken network connection. That still leaves us to explain why the printed message corresponds to RERR_MALLOC instead of RERR_STREAMIO. The way I can see this happening is if the client hit an RERR_MALLOC and tried to communicate with the server during cleanup but failed because the connection had broken. A recursive _exit_cleanup call should probably preserve the original file and line just like the original exit code in order to give consistent output. To troubleshoot this problem, we'll need to see the last part of rsync's output at -vvvv verbosity level, which will include the file and line of the initial call to _exit_cleanup.
Hello, Sorry the error message indeed came from the 3.0.3, i actually faced the problem with the 3.0.4. It seems that the problem is solved with using "--delete-after" instead of "--delete". Before doing that rsync seemed to be stuck on a particular file (tcpdump showed that there was no more network activity on the server), i CTRL+C the client and i went on the folder where the file concerned was, i noticed that i have file that starts with ~, deleting those files changed rsync behaviour (rsync was stuck on an other file on the next run), but as i do not know how rsync processes directories and files and cannot tell if it is relevant or not. I did switch to --delete-after and now it seems to be ok. I have many files with accentuated caracters on the client side. Regards.
Actually it does not work. I think it is because of some characters in my folder/file list : pc-imac-mj:~ ug$ /usr/local/bin/rsync -avz -i --iconv=UTF8-MAC,ISO8859-15 -elete-after -e ssh /Users/mario/Documents/Data kalypso@IP_SERVER:/home/MACUSERS/kalypso/ [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/PL-CULT-009 Procédure d'entreposage des cellules de CHO dans l’azote liquide.doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/Données expérimentales/Contrôle de pH par CO2/Contrôle du pH et de l’oxygène dissous par aération de su(2).doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/Données expérimentales/Contrôle de pH par CO2/Contrôle du pH et de l’oxygène dissous par aération de surf.doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/Données expérimentales/Contrôle de pH par CO2/Nom de variable change - Contrôle de l’oxygène dissous par.doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/Données expérimentales/Contrôle de pH par CO2/Nom de variable non change - Contrôle du pH et de l’oxygène.doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/Données expérimentales/Contrôle de pH par CO2/~$ntrôle du pH et de l’oxygène dissous par aération de surf.doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/Données expérimentales/Extraction PCA et acetonitrile/Préparation d’extraits - tests I-III.doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/Données expérimentales/Extraction PCA et acetonitrile/Préparation d’extraits -test IV.doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/PON/CHO/Extraction/ANA- Préparation d’extraits de cellules mammifères pour d(2).doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/PON/CHO/Extraction/ANA- Préparation d’extraits de cellules mammifères pour des.doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/PON/CHO/Extraction/Préparation d’extraits de cellules mammifères pour des me(2).doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/PON/CHO/Extraction/Préparation d’extraits de cellules mammifères pour des mesu.doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/PON/CHO/Extraction/~$éparation d’extraits de cellules mammifères pour des mesu.doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/PON/CHO/Extraction/Trash/ANA- Préparation d’extraits de cellules mammifères pour des.doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/PON/CHO/Extraction/Trash/Préparation d’extraits de cellules mammifères pour des me(2).doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Au rapport!/ze Gradués/McMurray, Virginie/Cellules de mammiferes/PON/CHO/Extraction/Trash/Préparation d’extraits de cellules mammifères pour des mesu.doc (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/De vive voix !/Publications/Brevets/esp@cenet — WO2007071072 (A1).pdf (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/De vive voix !/Publications/revisions/Biotechnol Bioeng/Aperçu de “s1-ln41846841427307575-1939656818Hwf210329021IdV-9506663404184684PDF_HI0001.pdf”.pdf (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Du porte à porte/Recherche subventionnée/CRSNG/AIM/2004/CRSNG - Subventions d'accès aux installations majeures (AIM) – Foire aux questions_fichiers (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Téléchargements/suivi ATP/Identin¼ücation of new leishmanicidal peptide.pdf (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Une fois, c't'un gars/EPFL/Cours gén biol/Intro/Sources dessins/Biotechn-Sanduhr — copie (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Une fois, c't'un gars/EPFL/Cours gén biol/Intro/Sources dessins/Biotechn-Sanduhr — copie/Biot - Sanduhr (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Une fois, c't'un gars/EPFL/Cours gén biol/Intro/Sources dessins/Biotechn-Sanduhr — copie/Biotech - Sanduhr (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Une fois, c't'un gars/EPFL/Cours gén biol/Intro/Sources dessins/Biotechn-Sanduhr — copie/FINDER.DAT (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Une fois, c't'un gars/GCH8650/ÉPFL-Urs Von Stockar/Intro/Sources dessins/Biotechn-Sanduhr — copie (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Une fois, c't'un gars/GCH8650/ÉPFL-Urs Von Stockar/Intro/Sources dessins/Biotechn-Sanduhr — copie/Biot - Sanduhr (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Une fois, c't'un gars/GCH8650/ÉPFL-Urs Von Stockar/Intro/Sources dessins/Biotechn-Sanduhr — copie/Biotech - Sanduhr (Invalid or incomplete multibyte or wide character) [receiver] cannot convert filename: Data/Une fois, c't'un gars/GCH8650/ÉPFL-Urs Von Stockar/Intro/Sources dessins/Biotechn-Sanduhr — copie/FINDER.DAT (Invalid or incomplete multibyte or wide character) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1040) [sender=3.0.4] pc-imac-mj:~ ug$ cat error.log building file list ... done IO error encountered -- skipping file deletion sent 632985 bytes received 6670 bytes 426436.67 bytes/sec total size is 19890777297 speedup is 31096.10 pc-imac-mj:~ ug$ Do have i to rename folders and files ? Regards.
Hello, I think i found the problem i faced with. rsync command is successful on an english locale Mac os X using : --iconv=UTF8-MAC,ISO-8859-15 The problem came from the fact that the Mac os X used for that bug report is in french, so i had to use : --iconv=ISO-8859-15,ISO-8859-15 and it is ok now. Regards.
Hello, Sorry for that long investigation, but maybe it will help someone else. --iconv=ISO-8859-15,ISO-8859-15 did not really solved the problem. The definitive solution was to rename all files that where tagged with the rsync error message "Invalid or incomplete multibyte or wide character". As data came from windows i assume that there was some characters that rsync was unable to recognize. Regards.
some more observations on this error. I actually got sync error: error allocating core memory buffers (code 22) at io.c(632) [sender=3.0.4] with 3.0.4 on debian and 3.0.4 on cygwin as receiver, with iconv=. it detects UTF-8,CP1252 now the fun about it .. it definitely boils down to incomplete multibyte characters in filename (copying with samba obv. under certain cirumstances destroys umlauts etc.).. if I rename them, all is fine more fun .. if i switch -vv or more .. it reduces to a warning in the end rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1058) [sender=3.0.4] and I can see all the problematic files with entries like [receiver] cannot convert filename: /CD1/(13) Kâ as incomplete multibyte chars are defective, it makes sense to repair those names. But still: a) no reason for rsync to complain about memory errors b) obviously not because it doesn't crash when executed more verbose does that help?.. bud
I agree that this is a bug; reopening. Bud, please provide your full command line and, if possible, run the sender under gdb with a breakpoint on the "_exit_cleanup" function and give us the stack trace when that function is reached.
btw. the first initial run was done with rsync 3.0.4 on that debian from ext2 fs to ntfs-3g fs, which is now residing in a windows 2003 r2 and accessed with cygwin 1.7 cygdrive I don't think this is essential, but also gives one clue .. a local/local transfer even with damaged multibyte file names does not have this issue.. bud
(In reply to comment #7) > I agree that this is a bug; reopening. As an additional note, the same thing happens with the Netware port of Rsync 2.6.3. We get the same sort of error when RSync tries to process anything with an extended character set. TM symbols, apostrophes, anything aside from standard Alphabet, by the look of it. 2009/05/06 19:33:11 [1] readlink "DATA:\/USERS/JSAWERS/Favorites/Morrell r in Christchurch - White PagesT New Zealand.url" failed: No such file or directory The symbol after "Pages" is a Trademark symbol. The other issue seems to be that 2.6.3 is the latest version of RSync for Netware and there doesn't appear to be any plan to update this. Cheers, Mark
Not sure if this is related, but I get more or less the same error message out of rsync 3.0.9 on AMD64 Linux rsyncing from an internal XFS-formatted HDD to an external EXT3-formatted eSATA HDD (i.e. sender and receiver are the same host communicating via pipes). rsync: writefd_unbuffered failed to write 5 bytes to socket [sender]: Broken pipe (32) rsync: connection unexpectedly closed (209441 bytes received so far) [sender] rsync error: error allocating core memory buffers (code 22) at io.c(605) [sender=3.0.9] Failed: incremental rsync fails (22) rsync command (I can post the backup script if interested): rsync -aHA \ --link-dest="${LAST_OFFSITE_BACKUP_DIR}" \ ${LAST_ONSITE_BACKUP_DIR}/ ${THIS_OFFSITE_BACKUP_DIR}/ \ 2>&1 || die 8 "incremental rsync fails ($?)" An almost identical command that copies from the live FS (some EXT4, some XFS) to the internal backup HDD (XFS) works fine.