Bug 12001 - samba-4.3/4.4: unable to create new files if the SMB share's path starts off a read-only mounted NFS export
samba-4.3/4.4: unable to create new files if the SMB share's path starts off ...
Product: Samba 4.1 and newer
Classification: Unclassified
Component: File services
x64 Linux
: P5 major
: ---
Assigned To: Jeremy Allison
Samba QA Contact
Depends on:
  Show dependency treegraph
Reported: 2016-07-01 00:17 UTC by robert-sz
Modified: 2016-07-14 15:28 UTC (History)
1 user (show)

See Also:

logs and pcaps of a failed Notepad 'save as' and a successful console redirection (1.52 MB, application/octet-stream)
2016-07-04 15:56 UTC, robert-sz
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description robert-sz 2016-07-01 00:17:30 UTC
Apologies for the long post, I try to give as much information and context as possible:

We have a Linux server (toffler) which Read-Only exports over NFS (to Linux workstations) a "symlink farm", a tree of directories containing symlinks to other, automounted and writeable(!) NFS exports.

We have been sharing this "symlink farm" to Windows computers through an AD-bound samba 3.6.4 as well but due to recent discoveries of vulnerabilities in earlier samba versions, we would like to update to samba 4.

On samba-4.3.8/4.3.9/4.3.10 and 4.4.4 however windows clients (Terminal server sessions, Windows Server 2008R2 and Windows Server 2012) cannot create new files at a depth which has branched off to another writeable NFS export. Editing/saving existing files is possible, but not creating new ones or directories.

/jobs -> /mpc/vanh-jobs                               (autofs mounted Read-Only NFS share containing the job directories)
/jobs/rnd_dev                                         (just a directory for the 'rnd_dev' job)
/jobs/rnd_dev/config -> /mpc/toffler2/rnd_dev/config  (autofs mounted Read-Write NFS share on a different server)

So, although I cannot write to /jobs or /jobs/rnd_dev, I expect to be able to write to /jobs/rnd_dev/config as on the samba server that path is now actually a read-write mounted directory. This was the behaviour with samba 3.6.x but not with samba 4.3.x and 4.4.x (all versions are vanilla builds)

The Windows Security tabs on the directories (of any depth) show the correct owners and groups, and all 3 ACEs (Everyone, Owner, Group) have 'Special permissions' which seem to contain the right permissions - they follow the POSIX permissions, but every attempt to create a new file results in a 'You can't save here' in Notepad for example.

The _odd_ thing is this is _only_ using Windows Explorer and other graphical tools like Notepad. Strangely I can create new files using command prompt:

echo Hello World > Y:\rnd_dev\config\robtest.txt

That works. I also have no problem creating files and directories using smbclient running on the samba server connecting to itself.

Looking at the debug (10) log I discovered some lines which *may* be a red-herring:
[2016/06/30 15:27:08.128333, 10, pid=6530, effective(13349, 20), real(13349, 0)] ../source3/smbd/open.c:4808(create_file_unixpath)
   create_file_unixpath: NT_STATUS_OBJECT_NAME_NOT_FOUND
[2016/06/30 15:27:08.128351, 10, pid=6530, effective(13349, 20), real(13349, 0)] ../source3/smbd/open.c:5086(create_file_default)
[2016/06/30 15:27:08.128377,  3, pid=6530, effective(13349, 20), real(13349, 0)] ../source3/smbd/smb2_server.c:3098(smbd_smb2_request_error_ex)
   smbd_smb2_request_error_ex: smbd_smb2_request_error_ex: idx[1] status[NT_STATUS_OBJECT_NAME_NOT_FOUND] || at ../source3/smbd/smb2_create.c:293
[2016/06/30 15:27:08.128400, 10, pid=6530, effective(13349, 20), real(13349, 0)] ../source3/smbd/smb2_server.c:2989(smbd_smb2_request_done_ex)
   smbd_smb2_request_done_ex: idx[1] status[NT_STATUS_OBJECT_NAME_NOT_FOUND] body[8] dyn[yes:1] at ../source3/smbd/smb2_server.c:3146

And an entry which seems to correspond to the ACCESS DENIED message in Notepad:

[2016/06/30 15:27:08.179909, 10, pid=6530, effective(13349, 20), real(13349, 0), class=acls] ../source3/smbd/posix_acls.c:848(print_canon_ace_list)
  print_canon_ace_list: canonicalise_acl: ace entries after arrange
  canon_ace index 0. Type = allow SID = S-1-5-21-2167877535-115312784-728001843-25778 uid 1202 (shotman) SMB_ACL_USER_OBJ ace_flags = 0x0 perms rwx
  canon_ace index 1. Type = allow SID = S-1-22-2-0 gid 0 (root) SMB_ACL_GROUP_OBJ ace_flags = 0x0 perms rwx
  canon_ace index 2. Type = allow SID = S-1-1-0 other SMB_ACL_OTHER ace_flags = 0x0 perms r-x
[2016/06/30 15:27:08.180283, 10, pid=6530, effective(13349, 20), real(13349, 0), class=acls] ../source3/smbd/posix_acls.c:1110(map_canon_ace_perms)
  map_canon_ace_perms: Mapped (UNIX) 1c0 to (NT) 1f01ff
[2016/06/30 15:27:08.180308, 10, pid=6530, effective(13349, 20), real(13349, 0), class=acls] ../source3/smbd/posix_acls.c:1110(map_canon_ace_perms)
  map_canon_ace_perms: Mapped (UNIX) 1c0 to (NT) 1f01ff
[2016/06/30 15:27:08.180327, 10, pid=6530, effective(13349, 20), real(13349, 0), class=acls] ../source3/smbd/posix_acls.c:1110(map_canon_ace_perms)
  map_canon_ace_perms: Mapped (UNIX) 140 to (NT) 1200a9
[2016/06/30 15:27:08.180362, 10, pid=6530, effective(13349, 20), real(13349, 0)] ../source3/smbd/open.c:174(smbd_check_access_rights)
  smbd_check_access_rights: file . requesting 0x40 returning 0x40 (NT_STATUS_ACCESS_DENIED)
[2016/06/30 15:27:08.180387, 10, pid=6530, effective(13349, 20), real(13349, 0)] ../source3/smbd/open.c:179(smbd_check_access_rights)
  smbd_check_access_rights: acl for . is:

The ACEs in the above excerpt refer to the permissions of /jobs:
[root@vanhsamba-a /] # ls -lad /jobs/
drwxrwxr-x 144 shotman root 4096 Jun 17 07:09 /jobs/

Here is my smb.conf:
workgroup = MPC
realm = AD.MPC.LOCAL
netbios name = vanhsamba-a
server string = VANHSAMBA-A %v
security = ADS
encrypt passwords = yes
preferred master = no
local master = no
os level = 33
#log level = 2
log level = 10
#syslog = 1
log file = /var/log/samba/%m
max log size = 500
ldap ssl = no
#idmap uid = 950-20000
#idmap gid = 600-20000
template shell = /bin/bash
winbind separator = +
winbind use default domain = yes
logon home = \\vanhsamba-a\%U

unix extensions = no
follow symlinks = yes
wide links = yes
allow insecure wide links = yes

printcap name = /dev/null
load printers = no
show add printer wizard = no
disable spoolss = yes
printing = bsd
allow trusted domains = no
winbind:ignore domains = TPNMEDIA AM EU
kernel oplocks = no
deadtime = 30

        # Important: The ranges of the default (*) idmap config
        # and the domain(s) must not overlap!

        # Default idmap config used for BUILTIN and local accounts/groups
        idmap config * : backend = tdb
        idmap config * : range = 510000000-519999999

        # idmap config for domain MPC (it has to include Linux GID and UIDs from the lowest to the highest (GID 20 [user] onward) )
        idmap config MPC : backend = ad
        idmap config MPC : schema_mode = rfc2307
        idmap config MPC : default = yes
        idmap config MPC : range = 20-509999999

        # Use settings from AD for login shell and home directory
        winbind nss info = rfc2307

        #winbind enum users = yes
        #winbind enum groups = yes
        winbind expand groups = 10
        winbind nested groups = yes
        winbind refresh tickets = yes
        idmap_ldb:use rfc2307 = yes

and the share definition:

        comment = MPC Job Hierarchy
        path = /jobs
        dfree command = /disk1/samba/4.3.10-mpc01/bin/dfree
        read only = no
        create mask = 0664
        directory mask = 0775
        blocking locks = no
        locking = no
        oplocks = no
        level2 oplocks = no
        posix locking = no
        strict locking = no
        case sensitive = yes
        force group = user
        hosts allow = <list of subnets - removed>

Can you please advise?

Comment 1 Jeremy Allison 2016-07-01 23:02:40 UTC
It looks like the client is testing for share writability by trying to open the "." directory with the SEC_DIR_DELETE_CHILD     = 0x00000040 access mask.

As the root directory is read only then this (correctly) is being denied.

Try getting a wireshark trace along with a debug level 10 log showing one clients attempt to save and upload here. Samba 3.x didn't check the existing security descriptors on open for delete, Samba 4.x does - that would explain the difference.

Let's find the exact issue first then we can see if anything can be done.
Comment 2 robert-sz 2016-07-04 15:56:18 UTC
Created attachment 12259 [details]
logs and pcaps of a failed Notepad 'save as' and a successful console redirection
Comment 3 robert-sz 2016-07-04 16:05:22 UTC
Thanks Jeremy for the reply.

Please find attached detailed logs of the 'save as' attempt in Notepad.

I also included a successful console redirection:

echo "Hello World!" > Y:\rnd_dev\config\helloworld_console.txt

worked without a hitch.

I am somewhat confused by your comment. In Windows world:

'.' is Y:\rnd_dev\config which is a writeable directory on the samba server.
Only Y:\ and Y:\rnd_dev are read-only.

Thanks again for taking a look!

Comment 4 robert-sz 2016-07-13 15:45:10 UTC
Hi Jeremy

Can I provide any more info to help troubleshoot this issue?

Comment 5 Jeremy Allison 2016-07-14 00:22:09 UTC
(In reply to robert-sz from comment #4)

The failed_saveas_new_file.pcap is strange. Look at packets 422 onwards. The client looks for file rnd_dev\config\helloworld.txt and doesn't find it (gets STATUS_NO_SUCH_FILE). Then it tries to open it using a file disposition of 0x1:

NTCREATEX_DISP_OPEN 1          /* if file exists open it, else fail */

which *must* fail, as it already checked the file doesn't exist.

It then repeats the same pattern twice more and then the trace stops.

Given the trace you sent me, the client isn't even trying to create the file, so there's no surprise it fails.
Comment 6 robert-sz 2016-07-14 15:28:41 UTC
Thanks for going through the logs. I don't know why notepad would not even try to create the file before opening it, but that is the behaviour that we are experiencing. Likely the "client" (in the loosest sense of the word, I am meaning Windows Explorer/Notepad) has already determined that the path is read-only and as a result won't even try to create a file.

Even when the path is open in Windows Explorer, the context menu (Shift-F10) won't give me the "New" option. I simply chose Notepad because it showed the error, the problem was reproducible and was easy to capture the logs when trying to do a "Save As".

I will send in a fuller tcpdump and samba log.

I appreciate your time troubleshooting this!