the problem of unordered NFS4 ACLs was partly discussed in bug 5653 before. Quick summary of the status quo: Generally NFS4 ACL filesystems allow the ACEs to be in any alow/deny order. Windows explorer (not the Windows kernel though) enforces ACLs to be ordered, so that deny ACLs take precedence over allow ACEs. Using Windows explorer's ACL editor will only be usable after the user accepts it to "order the permissions correctly". This "correct ordering" is changing the effective permissions for users and groups in most cases, so doing this is a very bad idea. Brainstorming on this: Calculating the effective permissions for all trustees of an ACL to match an ordered ACL looks close to impossible to do. One idea for Samba to behave better in cases like this and save users from screwing permissions up would be another generic nfs4 option like nfs4:protect_unordered_acls, that returns the unordered ACL as-is but with an explicit "deny modify permissions for trustee everybody" on top. This way the permissions on those file would still apply but messing up the permissions of files with unordered ACLs would be prevented. Side note: The ACL ordering is also something that we might want to take into account for the SMB3 POSIX extensions.
As far as I can see the NFS ACL model and Windows one are close but just not compatible. I'd be wary of trying to encode anything into a protocol-level (SMB3+POSIX) change here. I don't think we should mess with the server to save users from "order the permissions correctly" changes made by the Windows ACL editor. That code is doing what Microsoft designed it to do. If server admins don't want users to be able to do that to ACLs they own, then it's the admins responsibility to prevent it, not ours.
> If server admins don't want users to be able to do that to ACLs they own, > then it's the admins responsibility to prevent it, not ours. I fear nfs4:protect_unordered_acls would be the only way for them to prevent it.