i found that for our osx host "gonzo" , vanished files (not the warning message itself) appear in stdout - for linux hosts they _both_ appear in stderr. no information about vanished files appears in stdout (in my backup script, i´m redirecting stdout to rsync.log and stderr to rsync.err) what explains this difference in behaviour? is this a bug ? or is it related to rsynv version ? backupvm1->spacewalk is 3.0.9->3.0.9 and backupvm1->gonzo is 3.0.9->3.1.2 (installed from brew repo) [root@backupvm1 backup]# grep -i vanished spacewalk/rsync.* spacewalk/rsync.err:file has vanished: "/var/lib/rhn/search/indexes/server/_80a2.cfs" spacewalk/rsync.err:file has vanished: "/var/lib/rhn/search/indexes/server/_80a2_6.del" spacewalk/rsync.err:file has vanished: "/var/lib/rhn/search/indexes/server/_80a3.cfs" spacewalk/rsync.err:file has vanished: "/var/lib/rhn/search/indexes/server/_80a4.cfs" spacewalk/rsync.err:file has vanished: "/var/lib/rhn/search/indexes/server/_80a5.cfs" spacewalk/rsync.err:file has vanished: "/var/lib/rhn/search/indexes/server/segments_n801" spacewalk/rsync.err:rsync warning: some files vanished before they could be transferred (code 24) at main.c(1518) [generator=3.0.9] [root@backupvm1 backup]# grep -i vanished gonzo/rsync.* gonzo/rsync.err:rsync warning: some files vanished before they could be transferred (code 24) at main.c(1518) [generator=3.0.9] gonzo/rsync.log:file has vanished: "/Library/Server/Mail/Data/spool/maildrop/AFD811E2D07C" gonzo/rsync.log:file has vanished: "/Library/Server/Wiki/Database.xpg/Cluster.pg/pg_xlog/000000010000004F00000025" gonzo/rsync.log:file has vanished: "/Library/Server/Wiki/Database.xpg/Cluster.pg/pg_xlog/archive_status/000000010000004F00000025.done" gonzo/rsync.log:file has vanished: "/Library/Server/Wiki/Database.xpg/backup/000000010000004F00000026.partial"
i searched into this a little bit and it`s not a problem with differing rsync versions, it also happens with rsync 3.1.2 (linux) -> rsync 3.1.2 (osx) what i found is, that rsync 3.1.2 behaves correctly when run on osx locally, i.e. if i rsync -av /tmp/localdir/rapidly-appearing+disappearing-files-inside /tmp/localdir2 on osx locally, i`m getting all vanished files messages including which files are missing on stdout. if i do the same from linux/remote, then the information which files are vanished goes to stdout instead of sterr. the message "rsync warning: some files vanished before they could be transferred (code 24) at main.c(1650) [generator=3.1.2]" still goes to sterr (as expected)
it`s also no linux->osx interacting issue, the problem also appears when doing rsync on osx via ssh/localhost. but why ? ssh & osx does handle stdout and stdout correctly, as shown here: gonzo:tmp root# ssh root@localhost "echo stdout; echo stderr >&2" >/dev/null stderr gonzo:tmp root# ssh root@localhost "echo stdout; echo stderr >&2" 2>/dev/null stdout
apparently, ssh as a transport is the culprit, as it doesn`t happen with rsync in daemon mode. i found some osx related ticket on "stderr/stdout merge" ( https://github.com/ansible/ansible/issues/14960 ) after further searching, i found some infos on pty/tty allocation: https://stackoverflow.com/questions/7114990/pseudo-terminal-will-not-be-allocated-because-stdin-is-not-a-terminal (see comment "Optional background information:") and... # ssh root@localhost "echo stdout; echo stderr >&2" >/dev/null stderr # ssh root@localhost "echo stdout; echo stderr >&2" 2>/dev/null stdout # ssh -t root@localhost "echo stdout; echo stderr >&2" 2>/dev/null stdout stderr gotcha ! now the question is how to work around the problem...
mhh, if the error is not a warning but an error, the offending file is shown (for the warning of vanished files, the warning itself goes to stderr but the vanished filename goes to stdout). but only on OSX why that difference ? rsync: readlink "/private/var/folders/7p/mcsl60wn0djct6d9_kpzy1nc0000gp/0/com.apple.LaunchServices.dv" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1518) [generator=3.0.9]