Bug 7928 - Samba problems with kernel oplocks option set to "no"
Samba problems with kernel oplocks option set to "no"
Status: RESOLVED FIXED
Product: Samba 3.6
Classification: Unclassified
Component: Build environment
3.6.0pre1
Other Linux
: P3 normal
: ---
Assigned To: Jeremy Allison
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-01-21 07:26 UTC by Pavel Shilovsky
Modified: 2011-05-30 07:19 UTC (History)
4 users (show)

See Also:


Attachments
Capture from the description (5.59 KB, application/octet-stream)
2011-01-21 07:27 UTC, Pavel Shilovsky
no flags Details
Script that reproduces the problem (526 bytes, text/x-python)
2011-01-21 07:28 UTC, Pavel Shilovsky
no flags Details
Samba-3.6.0pre1 capture with both lock/oplock problems (5.57 KB, application/octet-stream)
2011-01-21 12:33 UTC, Pavel Shilovsky
no flags Details
Script-reproducer of both lock/oplock problems (525 bytes, text/x-python)
2011-01-21 12:34 UTC, Pavel Shilovsky
no flags Details
Samba-3.6.0pre1 level 10 log (48.83 KB, text/x-log)
2011-01-21 13:00 UTC, Pavel Shilovsky
no flags Details
Patch for 3.5.x - part #1 (2.52 KB, patch)
2011-01-31 17:26 UTC, Jeremy Allison
vl: review+
Details
Patch for 3.5.x - part #2 (1.06 KB, patch)
2011-01-31 17:27 UTC, Jeremy Allison
vl: review+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Pavel Shilovsky 2011-01-21 07:26:59 UTC
I have two clients that mounted Samba server, that have 'kernel oplocks = no' in smb.conf.

1) client1 opens file;
2) client1 locks file from pos=0 to pos=1(excluded) with windows mandatory locking style;
3) client1 write 'a' to file from pos=0;
4) client2 opens file;
5) client2 reads 1 byte from pos=0 successfully! (ERROR)

testparm:
[global]
	server string = %h server (Samba 3.6.0pre1, Ubuntu)
	map to guest = Bad User
	obey pam restrictions = Yes
	pam password change = Yes
	passwd program = /usr/bin/passwd %u
	passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
	unix password sync = Yes
	syslog = 0
	log file = /var/log/samba/log.%m
	max log size = 1000
	dns proxy = No
	kernel oplocks = No
	usershare allow guests = Yes
	panic action = /usr/share/samba/panic-action %d

[printers]
	comment = All Printers
	path = /var/spool/samba
	create mask = 0700
	printable = Yes
	browseable = No

[print$]
	comment = Printer Drivers
	path = /var/lib/samba/printers

[test]
	path = /mnt/samba_share
	read only = No

-------
If I switch "kernel oplocks" to "yes" the problem dissapears. It seems to me that it's a not correct behaviour of this option.

Linux kernel version: v2.6.37.
cifs mount options: noperm,direct,forcemand,rw,iocharset=utf8.

Also, the problem dissapears if I use 'kernel oplocks = no' and 'strict locking = yes'.
Comment 1 Pavel Shilovsky 2011-01-21 07:27:59 UTC
Created attachment 6220 [details]
Capture from the description
Comment 2 Pavel Shilovsky 2011-01-21 07:28:29 UTC
Created attachment 6221 [details]
Script that reproduces the problem
Comment 3 Volker Lendecke 2011-01-21 09:56:27 UTC
This looks like expected behaviour to me. With oplocked files, Samba assumes that locking takes place completely in the client when having "strict locking" set to the default "Auto". Your script seems to indicate that you are using the "lockf" system call under Linux. I don't know what the python interpreter calls when you call the python function lockf(f1, LOCK_EX, 1, 0), but if I assume it is the lockf(2) syscall, then it is entirely expected that the read call down below allows the read. Unix lockf is an advisory lock, not a mandatory one.

Without a debug level 10 log I can't really say why Samba allows the read to go through, sorry.

Jeff, assigning to you to comment on the expected behaviour of a read covered by a lockf.

Volker
Comment 4 Pavel Shilovsky 2011-01-21 10:11:17 UTC
(In reply to comment #3)
> This looks like expected behaviour to me. With oplocked files, Samba assumes
> that locking takes place completely in the client when having "strict locking"
> set to the default "Auto". Your script seems to indicate that you are using the
> "lockf" system call under Linux. I don't know what the python interpreter calls
> when you call the python function lockf(f1, LOCK_EX, 1, 0), but if I assume it
> is the lockf(2) syscall, then it is entirely expected that the read call down
> below allows the read. Unix lockf is an advisory lock, not a mandatory one.
> 
> Without a debug level 10 log I can't really say why Samba allows the read to go
> through, sorry.
> 
> Jeff, assigning to you to comment on the expected behaviour of a read covered
> by a lockf.
> 
> Volker
> 

As I wrote before I am using these mount options: noperm,direct,forcemand,rw,iocharset=utf8. 'forcemand' - is switch on mandatory locking behaviour for the client. So, in this case the behavior is incorrect. I will provide debug level log as well soon.

Comment 5 Jeff Layton 2011-01-21 10:20:59 UTC
(In reply to comment #3)
> Jeff, assigning to you to comment on the expected behaviour of a read covered
> by a lockf.

As you said...Unix locking is generally advisory and in Pavel's reproducer the
second client didn't issue a lock. So issuing a read to the server seems like
correct behavior. (Nit: lockf however is not a syscall, it's just a C wrapper
around fcntl, IIUC).

Do we know whether the server actually granted an oplock to client2 in this
reproducer? It seems a little odd that it would if the other client has it open
and locked...

Actually, as I look now...it looks like there's only a single client involved,
not two. Pavel could you amend your description of the reproducer? Does this
actually happen when you use two separate hosts as clients?


Comment 6 Jeff Layton 2011-01-21 10:42:43 UTC
Another note... In frame 25, it looks like the server granted a level 2 oplock for FID 0x26a5. FID 0x26a3 however already had a byte range lock on the first byte of the file (frames 16 and 17).

Is that correct behavior? Should samba grant an oplock on a file when there's already a BRL on it by another FID?

Comment 7 Jeff Layton 2011-01-21 10:44:16 UTC
Reassigned back to Volker for him to comment on samba's behavior wrt to granting oplocks on files that have byte range locks on them already.

Comment 8 Pavel Shilovsky 2011-01-21 11:19:18 UTC
The description of the script is on Descriptoin of the bug. I tested it only on one host.
Comment 9 Pavel Shilovsky 2011-01-21 11:27:57 UTC
(In reply to comment #6)
> Another note... In frame 25, it looks like the server granted a level 2 oplock
> for FID 0x26a5. FID 0x26a3 however already had a byte range lock on the first
> byte of the file (frames 16 and 17).

It's exactly the problem I am reporting about. The client1 set mandatory byte-range lock and in this case the server should not grant level2 oplocks for any other clients.

> 
> Is that correct behavior? Should samba grant an oplock on a file when there's
> already a BRL on it by another FID?
> 

It's correct for POSIX, because in POSIX we use advisory locking style and in this case nothing prevents Samba to grant level 2 oplock. But it's wrong for mandatory locking style (that uses by Windows clients and Linux cifs client with forcemand or nounix mount options) - that can break all read-ahead operations that client can produce if it has level2 oplock. And it gives client to read from the cache information but was locked by another client - but many Windows network application uses this mechanism for synchronization. 
Comment 10 Pavel Shilovsky 2011-01-21 11:33:49 UTC
my previous comment should be for https://bugzilla.samba.org/show_bug.cgi?id=7923.

Jeff, in the capture from the attachment the server doesn't grant level2 oplock. it's non oplock. But in the bug, I pointed above, it is. So, I think that problems may have the same reason, because both reproduced in 'kernel oplocks = no' case.
Comment 11 Pavel Shilovsky 2011-01-21 11:42:44 UTC
(In reply to comment #5)
> (In reply to comment #3)
> > Jeff, assigning to you to comment on the expected behaviour of a read covered
> > by a lockf.
> 
> As you said...Unix locking is generally advisory and in Pavel's reproducer the
> second client didn't issue a lock. So issuing a read to the server seems like
> correct behavior. (Nit: lockf however is not a syscall, it's just a C wrapper
> around fcntl, IIUC).

Why issuing a read is a correct behavior? With forcemand mount option all locks are mandatory and the server should drop such a read.

> 
> Do we know whether the server actually granted an oplock to client2 in this
> reproducer? It seems a little odd that it would if the other client has it open
> and locked...
> 

Oh, you are right, sorry for previous message - I was wrong - the server grants level2 oplocks in this case too. And it's should be be. Good catch!

Comment 12 Pavel Shilovsky 2011-01-21 12:21:01 UTC
*** Bug 7923 has been marked as a duplicate of this bug. ***
Comment 13 Pavel Shilovsky 2011-01-21 12:33:33 UTC
Created attachment 6222 [details]
Samba-3.6.0pre1 capture with both lock/oplock problems
Comment 14 Pavel Shilovsky 2011-01-21 12:34:27 UTC
Created attachment 6223 [details]
Script-reproducer of both lock/oplock problems
Comment 15 Pavel Shilovsky 2011-01-21 12:43:08 UTC
I investigated in more and found out the situation that show both problems (new capture and script from the attachments).

Script is rather simple:
1) client1 locks file with mandatory lock from 0 to 1,
2) client2 reads from the same file from 0 position 1 byte.

Capture shows us the following problems:
1) the server grants Oplock level 2 for both clients (frame #23, #25) while the first keeps mandatory byte-range lock on it (frame #18, #19).
2) the server successfully returns read data for the second client (frame #27) from the previously locked region by the first (frame #18, #19).

I think both are incorrect and looks like serious bugs.
Linux kernel version: v2.6.37
cifs mount options: noperm,iocharset=utf8,rw,nounix
Comment 16 Pavel Shilovsky 2011-01-21 13:00:37 UTC
Created attachment 6224 [details]
Samba-3.6.0pre1 level 10 log
Comment 17 Jeremy Allison 2011-01-21 16:06:48 UTC
jlayton wrote:

"Is that correct behavior? Should samba grant an oplock on a file when there's
already a BRL on it by another FID?"

Byte range locks and oplocks are completely orthogonal. There is no relation between BRL and oplocks - so yes, Samba can grant a level II oplock on a file so long as the first open allows it.

Jeremy.
Comment 18 Pavel Shilovsky 2011-01-21 23:41:00 UTC
(In reply to comment #17)
> jlayton wrote:
> 
> "Is that correct behavior? Should samba grant an oplock on a file when there's
> already a BRL on it by another FID?"
> 
> Byte range locks and oplocks are completely orthogonal. There is no relation
> between BRL and oplocks - so yes, Samba can grant a level II oplock on a file
> so long as the first open allows it.
> 
> Jeremy.
> 

But from another hand, Oplocks level2 are contended by a write or a byte-range lock requests and some other requests. So, in this case we can see the relation.

Please look at these materials:
http://msdn.microsoft.com/en-us/library/cc308443.aspx
http://msdn.microsoft.com/en-us/library/ff549259(v=vs.85).aspx

Some re-phrases:
"Level2 oplocks can be granted if there are no current Byte Range Locks on the stream".
"Level2 oplocks are always broken to None on every byte range lock operation on the given stream."

I think Samba should follow the spec in this case if you want it work correctly with Windows clients.
Comment 19 Jeremy Allison 2011-01-31 17:26:50 UTC
Created attachment 6237 [details]
Patch for 3.5.x - part #1
Comment 20 Jeremy Allison 2011-01-31 17:27:12 UTC
Created attachment 6238 [details]
Patch for 3.5.x - part #2
Comment 21 Jeremy Allison 2011-02-04 19:34:17 UTC
Comment on attachment 6237 [details]
Patch for 3.5.x - part #1

Patch is incorrect. It's making the change to fsp->oplock_type in the wrong place. I'll update soon.

Jeremy.
Comment 22 Karolin Seeger 2011-02-09 13:55:03 UTC
Re-assigning to Jeremy.
Comment 23 Jeremy Allison 2011-02-09 14:25:53 UTC
Don't delay 3.5.7 for this one. I'm tracking down a possible regression in the oplock code introduced by this - and it's not a blocker bug (it's one we've had a while, we can live with it for one more release).

Jeremy.
Comment 24 Pavel Shilovsky 2011-02-17 04:32:08 UTC
(In reply to comment #23)
> Don't delay 3.5.7 for this one. I'm tracking down a possible regression in the
> oplock code introduced by this - and it's not a blocker bug (it's one we've had
> a while, we can live with it for one more release).
> 
> Jeremy.
> 

Any luck with this one? As the next 3.5.7 release will be at February 22 (as it's announced in samba-technical), we have a chance to get it.

fyi: I rebased my git v3-5-test branch up to date. The links to commits:
http://git.etersoft.ru/people/piastry/packages?p=samba4.git;a=commitdiff;h=a478d87f14b3f11104d57f7e34cab2e1d926ac04
http://git.etersoft.ru/people/piastry/packages?p=samba4.git;a=commitdiff;h=6e7e77d3218d912c6504d152b9f857b014537f19
Comment 25 Pavel Shilovsky 2011-05-30 07:19:35 UTC
This bug was fixed science 3.6.0pre2 only. Let's consider as resolved.