Bug 6896 - Copy to nfs mounted partitions fails - RHEL 5.4
Copy to nfs mounted partitions fails - RHEL 5.4
Status: CLOSED WORKSFORME
Product: rsync
Classification: Unclassified
Component: core
2.6.8
x64 Linux
: P3 major
: ---
Assigned To: Wayne Davison
Rsync QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2009-11-15 21:08 UTC by Simon Lyall
Modified: 2010-03-17 03:07 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Simon Lyall 2009-11-15 21:08:31 UTC
I am running RHEL 5.4 as both client and server. The server is exporting
a partition to client. I am attempting to rsync local files on the client
to the NFS mounted partition. This appears to be failing with the RHEL 5.4
upgrade ( from RHEL 5.3, was working ok then). I strongly suspect this is might be related to O_EXEL (see http://lwn.net/Articles/251012/ ) as we were getting similar problems with another application.

$ rsync --version
rsync  version 2.6.8  protocol version 29

The local (source) directory

$ ls -l crontab 
-rw-rw-r-- 1 webmastr webmastr 2681 Nov 16 16:00 crontab


The destination:

$ mount
nfs_server:/wwwroot on /export/www/wwwroot type nfs4 (rw,noatime,bg,retrans=8,addr=172.22.0.12)

$ ls -la /export/www/wwwroot/live/data/test/
total 16
drwxr-xr-x 2 webmastr webmastr 4096 Nov 16 15:58 ./
drwxr-xr-x 6 webmastr webmastr 4096 Nov 16 15:44 ../


Works okay with normal copy.

$ cp -v crontab /export/www/wwwroot/live/data/test/
`crontab' -> `/export/www/wwwroot/live/data/test/crontab'

$ ls -al /export/www/wwwroot/live/data/test/
total 24
drwxr-xr-x 2 webmastr webmastr 4096 Nov 16 16:01 ./
drwxr-xr-x 6 webmastr webmastr 4096 Nov 16 15:59 ../
-rw-rw-r-- 1 webmastr webmastr 2681 Nov 16 16:01 crontab

$ rm /export/www/wwwroot/live/data/test/crontab 

Doesn't when I use rsync.

$ rsync crontab /export/www/wwwroot/live/data/test/
rsync: mkstemp "/export/www/wwwroot/live/data/test/.crontab.Tb0UDW" failed: Permission denied (13)
rsync error: some files could not be transferred (code 23) at main.c(892) [sender=2.6.8]

$ ls -al /export/www/wwwroot/live/data/test/
total 20
drwxr-xr-x 2 webmastr webmastr 4096 Nov 16 16:02 ./
drwxr-xr-x 6 webmastr webmastr 4096 Nov 16 15:59 ../
-r-x--xr-T 1 webmastr webmastr    0 Dec 26  1970 .crontab.Tb0UDW*

NOTE the actual data of the file seems to change but is almost always several years into the past.

Running with more debugging:

$ rsync -vvvvv crontab /export/www/wwwroot/live/data/test/
cmd= machine= user= path=/export/www/wwwroot/live/data/test/
cmd[0]=. cmd[1]=/export/www/wwwroot/live/data/test/ 
(Client) Protocol versions: remote=29, negotiated=29
(Server) Protocol versions: remote=29, negotiated=29
note: iconv_open("UTF-8", "UTF-8") succeeded.
[sender] make_file(crontab,*,2)
server_recv(2) starting pid=29535
[sender] i=0 <NULL> crontab mode=0100664 len=2681 flags=0
send_file_list done
file list sent
send_files starting
recv_file_name(crontab)
received 1 names
[receiver] i=0 1 crontab mode=0100664 len=2681 flags=0
recv_file_list done
get_local_name count=1 /export/www/wwwroot/live/data/test/
recv_files(1) starting
generator starting pid=29535 count=1
delta-transmission disabled for local transfer or --whole-file
recv_generator(crontab,0)
send_files(0, crontab)
count=0 n=0 rem=0
send_files mapped crontab of size 2681
calling match_sums crontab
crontab
sending file_sum
false_alarms=0 hash_hits=0 matches=0
sender finished crontab
generate_files phase=1
send_files phase=1
recv_files(crontab)
rsync: mkstemp "/export/www/wwwroot/live/data/test/.crontab.LrB2Oc" failed: Permission denied (13)
data recv 2681 at 0
got file_sum
recv_files phase=1
generate_files phase=2
send_files phase=2
send files finished
total: matches=0  hash_hits=0  false_alarms=0 data=2681
recv_files phase=2
generate_files phase=3
recv_files finished
generate_files finished
client_run waiting on 29535

sent 2769 bytes  received 42 bytes  5622.00 bytes/sec
total size is 2681  speedup is 0.95
_exit_cleanup(code=0, file=main.c, line=892): entered
rsync error: some files could not be transferred (code 23) at main.c(892) [sender=2.6.8]
_exit_cleanup(code=0, file=main.c, line=892): about to call exit(23)


$ ls -al /export/www/wwwroot/live/data/test/
total 24
drwxr-xr-x 2 webmastr webmastr 4096 Nov 16 16:03 ./
drwxr-xr-x 6 webmastr webmastr 4096 Nov 16 15:59 ../
-r-xr----T 1 webmastr webmastr    0 Dec 27  1970 .crontab.LrB2Oc*
-r-x--xr-T 1 webmastr webmastr    0 Dec 26  1970 .crontab.Tb0UDW*
Comment 1 Matt McCutchen 2009-11-15 21:26:00 UTC
IIUC, the upstream rsync project does not support 2.6.8 any more (i.e., the 3.0.0 daemon security fixes were officially backported only to 2.6.9), so unless you can reproduce the problem with a newer rsync, you may be better advised to bug Red Hat.

That aside, to understand the issue, we would need to see strace output showing the mkstemp-related system calls leading up to the EACCES.  The temporary file is somehow getting really weird permissions, and since NFS isn't so crisp about checking permissions only at file-open time, that could certainly explain the EACCES.
Comment 2 Matt McCutchen 2010-03-17 02:30:47 UTC
Closing due to lack of response.  Simon, feel free to reopen this if you can provide strace output.
Comment 3 Simon Lyall 2010-03-17 03:07:58 UTC
(In reply to comment #2)
> Closing due to lack of response.  Simon, feel free to reopen this if you can
> provide strace output.
> 

Apologies for not updating this earlier. The problem turned out to be with the NFSV4 in the Redhat kernel:

https://www.redhat.com/security/data/cve/CVE-2009-3286.html
https://bugzilla.redhat.com/show_bug.cgi?id=524520
https://rhn.redhat.com/errata/RHSA-2009-1548.html

Not sure why I was still seeing the bug after the patch was released. Either my system wasn't fully uptodate or dates are wrong somewhere.