From xftroxgpx on mailing list, see https://lists.samba.org/archive/rsync/2018-March/031507.html script to reproduce: #!/bin/bash #tested to fail as below: ArchLinux's rsync-3.1.3-1-x86_64.pkg.tar.xz #tested to fail as below: ArchLinux's rsync-3.1.3pre1-1-x86_64.pkg.tar.xz #tested to work ok : ArchLinux's rsync-3.1.2-8-x86_64.pkg.tar.xz if test "$1" == "clean"; then rm -vrf destdir sourcedir sourcedir2 sourcedir3 exit 0 fi echo '!! test 1:' mkdir -p destdir mkdir -p sourcedir/a #one \0 followed by a non-\0 (so, using a space) required: echo -ne '\0 ' > sourcedir/a/b #non zero size file required: echo -ne 'c' > sourcedir/c echo 'sourcedir' >/tmp/filesfrom.lst.tmp rsync --recursive --perms --checksum --delay-updates --numeric-ids --preallocate --sparse --files-from=/tmp/filesfrom.lst.tmp -- ./ ./destdir/ #rsync: write failed on "/home/xftroxgpx/sandbox/rsync/nsfod_issues/try2/destdir/sourcedir/a/b": No such file or directory (2) #rsync error: error in file IO (code 11) at receiver.c(374) [receiver=3.1.3] # ^ this happens first and any subsequent times! echo '!! test 2:' mkdir -p sourcedir2/a #one \0 followed by a non-\0 (so, using an M) required: echo -ne '\0M' > sourcedir2/a/b #one \0 followed by a non-\0 (so, using an M) required: echo -ne '\0M' > sourcedir2/c #in order to see this error: (for file 'c') #non-zero file size required and it must be prefixed by '.' aka dot #echo -ne '1' > sourcedir2/.d #XXX: ^ (un)comment, don't forget to ./go clean afterwards, then ./go #otherwise, the error is for file "a/b" echo 'sourcedir2' >/tmp/filesfrom.lst.tmp rsync --recursive --perms --checksum --delay-updates --numeric-ids --preallocate --sparse --files-from=/tmp/filesfrom.lst.tmp -- ./ ./destdir/ #rsync: write failed on "/home/xftroxgpx/sandbox/rsync/nsfod_issues/try2/destdir/sourcedir2/a/b": No such file or directory (2) #rsync error: error in file IO (code 11) at receiver.c(374) [receiver=3.1.3pre1] echo '!! test 3:' #same as 2 but an extra file '.d' exists! mkdir -p sourcedir3/a #one \0 followed by a non-\0 (so, using an M) required: echo -ne '\0M' > sourcedir3/a/b #one \0 followed by a non-\0 (so, using an M) required: echo -ne '\0M' > sourcedir3/c #non-zero file size required and it must be prefixed by '.' aka dot echo -ne '1' > sourcedir3/.d echo 'sourcedir3' >/tmp/filesfrom.lst.tmp rsync --recursive --perms --checksum --delay-updates --numeric-ids --preallocate --sparse --files-from=/tmp/filesfrom.lst.tmp -- ./ ./destdir/ #rsync: write failed on "/home/xftroxgpx/sandbox/rsync/nsfod_issues/try2/destdir/sourcedir2/c": No such file or directory (2) #rsync error: error in file IO (code 11) at receiver.c(374) [receiver=3.1.3] A pristine copy of this script can be found here: https://github.com/xftroxgpx/a3/blob/37ebff3e0fe9d294aeec899a082dc2c51c486eb4/system/Z575/OSes/3archlinux/on_baremetal/filesystem_now/archlinux/home/xftroxgpx/sandbox/rsync/nsfod_issues/try2/go Just run ./go to start all 3 tests. All 3 tests should fail with: ArchLinux's rsync-3.1.3-1-x86_64.pkg.tar.xz ArchLinux's rsync-3.1.3pre1-1-x86_64.pkg.tar.xz All 3 tests will succeed with ArchLinux's rsync-3.1.2-8-x86_64.pkg.tar.xz (aka rsync version 3.1.2 protocol version 31)
Bug will be triggered if: options include both --sparse and --preallocate, AND the second (or subsequent) file to be copied begins with one or more zero bytes. .Dave.
Created attachment 14018 [details] Simple although probably suboptimal fix
Problem introduced by commit f3873b3d88b61167b106e7b9227a20147f8f6197 Author: Wayne Davison <wayned@samba.org> Date: Mon Oct 10 11:49:50 2016 -0700 Support --sparse combined with --preallocate or --inplace. The new code tries to punch holes in the destination file using newer Linux fallocate features. It also supports a --whole-file + --sparse + --inplace copy on any filesystem by truncating the destination file. Root cause is that the new variable 'sparse_past_write' is not reset when switching files, thus leading to miscalculation of the current seek position in do_punch_hole(). Quick fix is to reset this variable to the current seek position on each call to write_sparse() - probably not the most efficient fix, but it seems to work. .Dave.
Thanks for this fix Dave, worksforme!
This was fixed in commit 888ce058.