Bug 13175 - Accessing ZFS snapshot directories over SMB
Summary: Accessing ZFS snapshot directories over SMB
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: VFS Modules (show other bugs)
Version: unspecified
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Karolin Seeger
QA Contact: Samba QA Contact
: 11658 (view as bug list)
Depends on:
Reported: 2017-12-05 09:24 UTC by Ralph Böhme
Modified: 2018-12-04 12:04 UTC (History)
7 users (show)

See Also:

WIP patch for master (1.73 KB, patch)
2017-12-05 09:24 UTC, Ralph Böhme
no flags Details
Patch for master (2.82 KB, patch)
2018-05-09 11:53 UTC, Ralph Böhme
no flags Details
Possible patch for master (4.44 KB, patch)
2018-05-18 11:35 UTC, Ralph Böhme
no flags Details
Patch for 4.8 cherry-picked from master (4.79 KB, patch)
2018-10-31 10:20 UTC, Ralph Böhme
jra: review+

Note You need to log in before you can comment on or make changes to this bug.
Description Ralph Böhme 2017-12-05 09:24:24 UTC
Created attachment 13845 [details]
WIP patch for master

Currently trying to access ZFS snapshot directories over SMB fails:

$ bin/smbclient -U slow%x //localhost/test
Try "help" to get a list of possible commands.
smb: \> ls
  .                                   D        0  Tue Dec  5 09:05:27 2017
  ..                                  D        0  Tue Dec  5 08:49:27 2017
  .zfs                               DH        0  Tue Dec  5 08:49:27 2017
  test                                D        0  Tue Dec  5 09:06:06 2017

                1949611 blocks of size 1024. 1949578 blocks available
smb: \> ls .zfs/*

This happens because the acl() library call fails with ENOTSUP on the snapshot directories. This could be easily fixed by synthesizing an ACL based on the POSIX mode, much like the getfacl() command on FreeBSD does. The attached WIP patch implements this.

Question do we want this?
Comment 1 Jeremy Allison 2017-12-05 17:10:32 UTC
Yes we do (so long as it doesn't open up security). ZFS-based filers are very popular, and so long as Linux doesn't have RichACLs this is the only game in town. So +1 from me ! (Jeremy).
Comment 2 Timur Bakeyev 2017-12-07 05:18:44 UTC
Thanks, Ralph, we are going to look. We have related problem with the snapshots and extended attributes, could be that it lays here in fact.
Comment 3 Ralph Böhme 2017-12-07 13:28:11 UTC
(In reply to Timur Bakeyev from comment #2)
Thanks Timur! Btw, I actually built and tested the patch this time. :)
Comment 4 Ralph Böhme 2018-02-10 08:03:08 UTC
Timur, any chance for an update from your side? Afaict my patch is correct and Jeremy ACKed as well so we could push this upstream.
Comment 5 Timur Bakeyev 2018-02-17 04:36:06 UTC
(In reply to Ralph Böhme from comment #4)

Hi, Ralph!

Sorry for the delay with the answer, seems I mis-configured my Gmail filters, so updates on the Bugzilla silently ended up in the folder.

It seems this issue attracted quite an attention 

There is a patch for ZFS at least for FreeBSD, that should eliminate the said error in the future versions of the OS:


My colleague Andrew Walker also made his own patch, similar in principal to yours, to address the same issue:


His patch somewhat similar to the call to make_default_filesystem_acl(DEFAULT_ACL_EVERYONE) with the exception that also owner and group ACEs are set. Also it's handy that there is a configuration option that allows to enable and disable this feature.

Originally this patch was even more sophisticated as it tried to convert mode bits into corresponding NFSv4 ACEs:

Comment 6 Timur Bakeyev 2018-02-17 05:09:30 UTC
(In reply to Timur Bakeyev from comment #5)

I would suggest to take ZFS in-kernel implementation as a sample for emulation on zfsacl level. In the perfect world we wouldn't need any emulation, as ZFS(at least on FreeBSD) would perfectly handle the situation by itself, but there still be old versions of FreeBSD, ZFS on Linux and Illumos.

So, we have:

/* Common access mode for all virtual directories under the ctldir */
const u_short zfsctl_ctldir_mode = S_IRUSR | S_IXUSR | S_IRGRP | S_IXGRP |

aka 0555.

This is translated to NFSv4 ACL with some cleanups:

777 	int
778 	zfsctl_common_getacl(ap)
779 	        struct vop_getacl_args /* {
780 	                struct vnode *vp;
781 	                acl_type_t a_type;
782 	                struct acl *a_aclp;
783 	                struct ucred *cred;
784 	                struct thread *td;
785 	        } */ *ap;
786 	{
787 	        int i;
789 	        if (ap->a_type != ACL_TYPE_NFS4)
790 	                return (EINVAL);
792 	        acl_nfs4_sync_acl_from_mode(ap->a_aclp, zfsctl_ctldir_mode, 0);
793 	        /*
794 	         * acl_nfs4_sync_acl_from_mode assumes that the owner can always modify
795 	         * attributes.  That is not the case for the ctldir, so we must clear
796 	         * those bits.  We also must clear ACL_READ_NAMED_ATTRS, because xattrs
797 	         * aren't supported by the ctldir.
798 	         */
799 	        for (i = 0; i < ap->a_aclp->acl_cnt; i++) {
800 	                struct acl_entry *entry;
801 	                entry = &(ap->a_aclp->acl_entry[i]);
802 	                uint32_t old_perm = entry->ae_perm;
803 	                entry->ae_perm &= ~(ACL_WRITE_ACL | ACL_WRITE_OWNER |
805 	                    ACL_READ_NAMED_ATTRS );
806 	        }
808 	        return (0);
809 	}

I can imagine a separate ACL type option for the `make_default_filesystem_acl()` that would generate such a special NFSv4 ACL or, which is possibly cleaner, take Andrew's approach and implement creation of such an ACL in the `zfsacl` module itself, as it's quite specific to it.

What I miss in both implementations is the check that we are actually dealing with `.zfs/` or `.zfs/snapshot` or `.zfs/shares` directories. Not sure how critical is this. I can imagine some weird mounts of NFS or UFS file systems into ZFS share that possibly can give similar ENOSYS error.
Comment 8 Jeremy Allison 2018-02-20 19:14:03 UTC
What is the state of this one ? Seems to have stalled, and I want to make sure we fully enable ZFS ACLs as a 'first class' citizen in Samba ?
Comment 9 Timur Bakeyev 2018-02-22 02:32:22 UTC
(In reply to Jeremy Allison from comment #8)
We'd love to see ZFSACL as a first class citizen as well. Or, maybe, NFSv4 in general next to POSIX, like FreeBSD does.

I've left in the comments how we'd like to see this feature to be implemented in the ZFSACL module, so next question who and how is going to implement it. I'd again suggest to join the approaches and implement emulation of the FreeBSD ZFS behavior regarding .zfs directories in the ZFSACL itself, using mocked up NFSv4 ACLs.

Not sure now about the need of the configuration switch here, as in some versions of FreeBSD we'll get those NVSv4 ACLs naturally from the OS itself without any switch, so we can only try to transparently emulate OS in the case we are on an older system.

That shouldn't affect ZFS on Linux as so far they still don't have NFSv4 ACLs, hence ZFSACL module isn't applicable.

So, the question is who is going to write the code? We can do this, but also Ralph seems interested in it. What would be your advise?
Comment 10 Ralph Böhme 2018-02-22 09:08:14 UTC
(In reply to Timur Bakeyev from comment #9)
Timur, if you have the resources, go for it. :)
Comment 11 Timur Bakeyev 2018-02-27 02:42:57 UTC
(In reply to Ralph Böhme from comment #10)
Not much, but I'll prepare yet another WIP, so we can discuss it farther :)
Comment 12 Ted Cabeen 2018-02-28 22:49:02 UTC
It would also be useful to ensure this works when using vfs_acl_xattr with acl_xattr:ignore system acls = yes.  In this case, Samba has full control of the ACLs, but ZFS snapshot directories are still inaccessible (getfattr -n security.NTACL .zfs returns "Operation not supported"), and mapping the ZFS snapshots into Windows Volume Shadow Copies via vfs_shadow_copy2 also fails.
Comment 13 Timur Bakeyev 2018-02-28 23:03:26 UTC
(In reply to Ted Cabeen from comment #12)

Ted, can you specify what system/OS are you referring to and what is your setup then, that uses vfs_acl_xattr?

So far we were talking about vfs_zfsacl, mostly applicable to FreeBSD(as Solaris is half-way dead and has it's own SMB implementation AFAIK).
Comment 14 Ted Cabeen 2018-02-28 23:22:10 UTC
This is on CentOS 7.4 with the ZFS on Linux modules.  In my case, I'm not using ZFS or Unix ACLs at all.  The filesystems in question are only accessed by Samba, and the POSIX mode and owner/group are largely unused.  I was concerned that a solution would be developed that mocked up ACLs from POSIX data that's unused in this scenario.

I did open an issue with the ZoL group, as I think it might make more sense to just fix this at the ZFS layer, by returning the ACL/xattrs of the ZFS filesystem  mountpoint when accessing ACLs or xattrs on the .zfs and .zfs/snapshot directories: https://github.com/zfsonlinux/zfs/issues/7253
Comment 15 Ralph Böhme 2018-05-09 11:19:27 UTC
(In reply to Timur Bakeyev from comment #5)
Picking this up again...

Timur, afaict the only difference between the patch I proposed (attachment 13845 [details]) and the patch from Andrew that you ended up using, is that Andrew's patch sets SEC_DESC_DACL_PROTECTED which is a very good idea.

Other then that, both versions should set a similar ACL with ACEs for the owner, group and everyone. Note that my patch already uses DEFAULT_ACL_POSIX not DEFAULT_ACL_EVERYONE.

I'm therefor proposing to move on with a version of my patch, that sets SEC_DESC_DACL_PROTECTED.

Any cons?
Comment 16 Ralph Böhme 2018-05-09 11:53:15 UTC
Created attachment 14186 [details]
Patch for master

Something like this. It also has the parametic option, but note that I'm defaulting to enable the behaviour.

It's an undocumented parametric option and I don't want to make it unnecessarily hard for our users to get a well working Samba on FreeBSD. :)
Comment 17 Ralph Böhme 2018-05-18 11:35:50 UTC
Created attachment 14204 [details]
Possible patch for master
Comment 18 Timur Bakeyev 2018-05-20 15:23:09 UTC
(In reply to Ralph Böhme from comment #17)

Hi, Ralph!

I tried to test your patch on the recent FreeBSD system, seems it works, but there are few comments I got. 

One thing is that it would be handy to have DBG_DEBUG() statement there that indicates that we are actually synthesizing the ACLs for the given path as now it's a bit difficult to say which route was taken. With newer versions of FreeBSD OS actually produces the necessary ACLs itself, so you'd like to know.

Saying that here is difference in behavior I now observe:

# smbcacls //localhost/books .zfs -U timur
Enter CAVE\timur's password:
Failed to open \.zfs: NT_STATUS_NOT_SUPPORTED

Old behavior, deprecated.

# smbcacls //localhost/test1 .zfs -U timur
Enter CAVE\timur's password:
OWNER:Unix User\root
GROUP:Unix Group\wheel
ACL:Unix User\root:ALLOWED/0x0/READ
ACL:Unix Group\wheel:ALLOWED/0x0/READ

This is the old FreeBSD system with the Ralph's patch applied.

# smbcacls //localhost/test1 .zfs -U timur
Enter CAVE\timur's password:
OWNER:Unix User\root
GROUP:Unix Group\wheel
ACL:Unix User\root:ALLOWED/0x0/0x001200a1
ACL:Unix Group\wheel:ALLOWED/0x0/0x001200a1

This is a newer FreeBSD 11.2, that synthesizes ACLs for ".zfs/" itself.

For the last two cases we see some difference in the produced ACLs. Most significant differences are PD control flag and extra ACE:


The rest flags are almost, but not exactly the same:




I.e. difference is the SEC_FILE_READ_EA in your patch and SEC_FILE_EXECUTE is set by OS.

In general it would be great those code paths to match exactly, but not sure, how critical is that.
Comment 19 Ralph Böhme 2018-05-20 19:08:39 UTC
(In reply to Timur Bakeyev from comment #18)
Commit f93cc232377d4c686ac35ee5e14e798974bc0700 is now in master.

I don't think there are any critical differences, we're at least making progress. :)

Older FreeBSD versions will use the proven make_default_filesystem_acl() which is known to generate useful ACLs in vfs_acl_xattr. With newer FreeBSD versions you get a slightly different ACL with little functional difference.

We can certainly add a follow-up patch with a DEBUG statement. Do you want to provide one?
Comment 20 Jeremy Allison 2018-05-21 22:32:48 UTC
Ralph's flags are the correct ones IMHO. The native ones missing READ_EA permissions will cause problems as Samba uses EA's a lot these days.
Comment 21 Timur Bakeyev 2018-05-22 00:33:23 UTC
(In reply to Jeremy Allison from comment #20)

Please note that ctldirs(.zfs/, .zfs/snapshot, .zfs/share) are purely virtual and don't have real representation in the FS hierarchy. Here is the quote from the ZFS patch that exposes .zfs/* directory:

         * acl_nfs4_sync_acl_from_mode assumes that the owner can always modify
         * attributes.  That is not the case for the ctldir, so we must clear
         * those bits.  We also must clear ACL_READ_NAMED_ATTRS, because xattrs
         * aren't supported by the ctldir.

Also see https://svnweb.freebsd.org/base/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c?view=markup

So, in case some piece of code will try to read xattrs for that directories it'll fail anyhow, being mislead by the SEC_FILE_READ_EA.

In general, variant of the DEFAULT_ACL_EVERYONE could be a better option here, as there is no much use to examine stat info for.zfs/ as it's ALWAYS will be 0555(and doesn't reflect reality in any form).
Comment 22 Jeremy Allison 2018-05-22 20:20:19 UTC
What error message will a getxattr() call return on a snapshot file ?
Comment 23 Timur Bakeyev 2018-05-22 20:50:46 UTC
(In reply to Jeremy Allison from comment #22)

Not sure what you called "snapshot file".

If we are talking about actual snapshot under `.zfs/snapshot/`, for example, `.zfs/snapshot/snap1/` directory then you'll get normal xattrs and everything, as actual snapshots are automounted copies of the dataset, taken in some moment of the time and which do have all the normal ZFS dataset attributes, including xattrs and NFSv4 ACLs.

But if we are talking about `.zfs/` or `.zfs/snapshot/` or `.zfs/share/` control directories - those are purely virtual, not backed by any real FS and which exist only on the VFS level. And, being purely virtual they are crippled in certain aspects, like they don't have(and support) xattrs, you can't really change their permissions from 0555 and, possibly, some other limitations.

So, if we do:

# lsextattr user snapshot
lsextattr: snapshot: failed: Operation not supported

# getextattr user NTacl snapshot
getextattr: snapshot: failed: Operation not supported

And so on. That's behavior is the same for the old and new FreeBSD versions.
Comment 24 Ralph Böhme 2018-10-31 10:20:58 UTC
Created attachment 14552 [details]
Patch for 4.8 cherry-picked from master

The patch is already in 4.9, just 4.8 was missing the backport.
Comment 25 Ralph Böhme 2018-11-02 10:33:18 UTC
*** Bug 11658 has been marked as a duplicate of this bug. ***
Comment 26 Jeremy Allison 2018-11-21 19:54:52 UTC
Re-assigning to Karolin for inclusion in 4.8.next.
Comment 27 Karolin Seeger 2018-11-27 13:18:17 UTC
Pushed to autobuild-v4-8-test.
Comment 28 Karolin Seeger 2018-12-04 12:04:33 UTC
(In reply to Karolin Seeger from comment #27)
Pushed to v4-8-test.
Closing out bug report.