Bug 8773 - smbd fails to run as root because it belongs to more than 16 groups on MacOS X
Summary: smbd fails to run as root because it belongs to more than 16 groups on MacOS X
Status: RESOLVED FIXED
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: File services (show other bugs)
Version: 4.15.0rc5
Hardware: All Mac OS X
: P5 normal (vote)
Target Milestone: ---
Assignee: Jule Anger
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-02-23 01:40 UTC by LaSombra
Modified: 2022-05-05 06:24 UTC (History)
6 users (show)

See Also:


Attachments
Hack to workaround the NGROUPS_MAX issue (612 bytes, patch)
2012-02-23 01:41 UTC, LaSombra
no flags Details
patch for get_groups_unix() in lib/system_samba.c (351 bytes, patch)
2012-07-30 05:03 UTC, Adam Yearout
no flags Details
patch from master backported to 4.15 (6.12 KB, patch)
2021-09-12 21:46 UTC, Andrew Bartlett
jra: review+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description LaSombra 2012-02-23 01:40:03 UTC
When running nmbd & smbd as root under OS X, 10.7 at least, smbd fails to run with the following error:

get_current_groups: user is in 16 groups: 0, 404, 1, 2, 3, 4, 5, 8, 9, 12, 20, 29, 33, 61, 80, 98
Primary group is 0 and contains 0 supplementary groups
get_user_groups: failed to get the unix group list
Failed to get groups from sam account.

and running as myself I have various permissions errors, the latest being:

bind failed on port 445 socket_addr = ::.
Error = Permission denied

It looks like the problem shows up during a call to getgrouplist, where OS X returns its NGROUPS_MAX which is set to 16. More details at https://github.com/mxcl/homebrew/issues/5954#issuecomment-3353375

I could workaround this by changing the hack from angerman, shown in the above URL, and cleaning it up a bit in the attached patch.

I'm no developer, but I can try an test any suggested patches.

Thank you
Comment 1 LaSombra 2012-02-23 01:41:10 UTC
Created attachment 7340 [details]
Hack to workaround the NGROUPS_MAX issue
Comment 2 Volker Lendecke 2012-02-24 12:01:59 UTC
This patch won't help. According to https://developer.apple.com/library/mac/#documentation/Darwin/Reference/Manpages/man2/setgroups.2.html we can't set more than NGROUPS_MAX groups using the setgroups(2) call that we need. If you have information how to increase the limit for the setgroups(2) call, please re-open this bug.

If that's a limitation of OS/X, please contact Apple to remove this limitation. Linux and OpenSolaris have now fixed it.

Volker
Comment 3 LaSombra 2012-02-24 12:25:16 UTC
According to this post, http://lists.apple.com/archives/darwin-kernel/2009/Sep/msg00035.html, it seems that you can prevent this behavior.

"...For one thing, it will have an incredibly tough time of it unless it calls initgroups instead of setgroups. Even then, it's still going to have a tough time of it because there is a very specific order of operations required to opt a newly created credential into external group membership resolution by directory services, part of which require that both the client and server are bound into the same security association. This is necessary so that the vended group list is identical in both places so directory services gives the same answer to the "is cr_gmuid a member of group X?" question."

Maybe OS X needs initgroups(3) before setgroups(2)?

(In reply to comment #2)
> This patch won't help. According to
> https://developer.apple.com/library/mac/#documentation/Darwin/Reference/Manpages/man2/setgroups.2.html
> we can't set more than NGROUPS_MAX groups using the setgroups(2) call that we
> need. If you have information how to increase the limit for the setgroups(2)
> call, please re-open this bug.
> 
> If that's a limitation of OS/X, please contact Apple to remove this limitation.
> Linux and OpenSolaris have now fixed it.
> 
> Volker
Comment 4 Volker Lendecke 2012-02-24 12:35:01 UTC
The problem here is that we have to switch credentials potentially very often. Anonymous and authenticated access is very frequently done on the same TCP connection. If I understand initgroups right, it reaches out to opendirectory. You just can not do that for every SMB request. Moreover, if the kernel does not support more than 16 groups per user, there is just no way to support that. It is a decision Apple needs to make, Samba can not do anything about it.
Comment 5 LaSombra 2012-02-24 12:36:40 UTC
An this is, apparently, Dovecot's fix to this same issue:
http://hg.dovecot.org/dovecot-2.1/rev/bffc8788eb94
and the discussion
http://markmail.org/message/64s5oo5tk4wfa5jt#query:+page:1+mid:64s5oo5tk4wfa5jt+state:results

(In reply to comment #3)
> According to this post,
> http://lists.apple.com/archives/darwin-kernel/2009/Sep/msg00035.html, it seems
> that you can prevent this behavior.
> 
> "...For one thing, it will have an incredibly tough time of it unless it calls
> initgroups instead of setgroups. Even then, it's still going to have a tough
> time of it because there is a very specific order of operations required to opt
> a newly created credential into external group membership resolution by
> directory services, part of which require that both the client and server are
> bound into the same security association. This is necessary so that the vended
> group list is identical in both places so directory services gives the same
> answer to the "is cr_gmuid a member of group X?" question."
> 
> Maybe OS X needs initgroups(3) before setgroups(2)?
> 
> (In reply to comment #2)
> > This patch won't help. According to
> > https://developer.apple.com/library/mac/#documentation/Darwin/Reference/Manpages/man2/setgroups.2.html
> > we can't set more than NGROUPS_MAX groups using the setgroups(2) call that we
> > need. If you have information how to increase the limit for the setgroups(2)
> > call, please re-open this bug.
> > 
> > If that's a limitation of OS/X, please contact Apple to remove this limitation.
> > Linux and OpenSolaris have now fixed it.
> > 
> > Volker
Comment 6 LaSombra 2012-02-24 12:39:02 UTC
I see.
From what I read, it is not that they do not support more than 16 groups per user, but more like the UNIX function calls are hindered or force limited due to reasons of their own.

I read through this Python bug, http://bugs.python.org/issue7900, where again this happens.

(In reply to comment #4)
> The problem here is that we have to switch credentials potentially very often.
> Anonymous and authenticated access is very frequently done on the same TCP
> connection. If I understand initgroups right, it reaches out to opendirectory.
> You just can not do that for every SMB request. Moreover, if the kernel does
> not support more than 16 groups per user, there is just no way to support that.
> It is a decision Apple needs to make, Samba can not do anything about it.
Comment 8 eduo 2012-02-26 22:31:06 UTC
(In reply to comment #4)
> The problem here is that we have to switch credentials potentially very often.
> Anonymous and authenticated access is very frequently done on the same TCP
> connection. If I understand initgroups right, it reaches out to opendirectory.
> You just can not do that for every SMB request. Moreover, if the kernel does
> not support more than 16 groups per user, there is just no way to support that.
> It is a decision Apple needs to make, Samba can not do anything about it.

The weird part is that this was already fixed. 3.2.15 works perfectly in Lion. The necessary changes were proposed by James Peach in 2007 and even today are included in the configure.in and sec_ctx.c files for 3.6.3.

The problem seems to be that they don't work any more. Something else seems to have changed in the source that rendered this working code (which contains the proper sequence of events for group selection in Darwin) moot.

I may be misunderstanding the reason for the WONTFIX but it doesn't seem to check out with maintaining the code that originally fixed this issue 5 years ago in the current source. The fact that it's there seems to point at a bug in 3.6.0 that hasn't been fixed yet (since a lot of people compile Samba through macports, which has an older version in it, this may have gone unnoticed until now).

The reason I'm also trying to look for a fix to this is that I want to upgrade the MacPorts port to it solves to a recent version. Anything beyond 3.6.0 (inclusive) gives me the same error. But the code to "fix" this is there (verbatim, too, since 2007).
Comment 9 Jeremy Allison 2012-02-28 02:15:29 UTC
So, can you reproduce this problem with a standalone program using the James Peach Samba "solution" on OSX 10.7 ? Does a standalone program successfully using the Samba set_unix_security_ctx() method work ?
Comment 10 eduo 2012-02-28 06:56:57 UTC
(In reply to comment #9)
> So, can you reproduce this problem with a standalone program using the James
> Peach Samba "solution" on OSX 10.7 ? Does a standalone program successfully
> using the Samba set_unix_security_ctx() method work ?

Ah. I'm sorry. I wasn't that clear in my description. I meant to say that in 3.2.15 Samba never crashes due to this error but >3.6 crashes hard as soon as authentication is successful. My assumption was that James's code, while not enabling authentication, at least allowed smbd to fail gracefully, whereas now we get a coredump.

Sorry I wasn't clear. I haven't tried James's code standalone.
Comment 11 Matt Koss 2012-05-30 22:58:50 UTC
Why is this marked as Resolved ?

The issue seems to be still there in version 3.6.5.
Comment 12 Adam Yearout 2012-07-30 05:02:19 UTC
The attached patch, as far as I can tell, gets around the group limit problem on OS X platforms.

As mentioned, you can't modify NGROUPS_MAX, as the correct value is required to perform the SYS_initgroups() system call that was originally submitted by James Peach.  The submitted patch instead bypasses the groups_max() call in getgroups_unix_user() by specifying a static value.

This appears to let SYS_initgroups() do its thing, and allows getgroups() to return more than NGROUPS_MAX groups.

I've tested the patch on my system using Samba 3.6.6 with no issues. A user in 20 groups correctly resolves all groups. I also configured a test share with 20 individual folders, each with different group permissions, and Samba correctly resolved the permissions for all folders.
Comment 13 Adam Yearout 2012-07-30 05:03:33 UTC
Created attachment 7727 [details]
patch for get_groups_unix() in lib/system_samba.c
Comment 14 Akihiro Suda 2021-09-04 20:04:51 UTC
This patch seems still required to run samba on macOS with a user that belongs to more 16 groups.

Note that even non-root user may belong to more than 16 groups on some environments.
https://github.com/Homebrew/homebrew-core/pull/84673

Any chance to get the patch merged into the upstream?

This issue seems marked as "RESOLVED WONTFIX", but 9 years have passed since then, so I think this is good time to reconsider.
Comment 15 Alex Richardson 2021-09-07 15:31:15 UTC
I've submitted the patch that we are using to enable smb shares on macOS QEMU here: https://gitlab.com/samba-team/samba/-/merge_requests/2156
Comment 16 Samba QA Contact 2021-09-09 17:44:09 UTC
This bug was referenced in samba master:

2c18a982537ea1a62e4d802c9ae0ef06b36158dc
Comment 17 Andrew Bartlett 2021-09-12 21:46:23 UTC
Created attachment 16805 [details]
patch from master backported to 4.15

Backport of https://gitlab.com/samba-team/samba/-/merge_requests/2156 to 4.15.

Just 4.15 for now, I think it is reasonable that the few users that need this just run a current Samba.
Comment 18 Jeremy Allison 2021-09-13 00:55:21 UTC
Comment on attachment 7727 [details]
patch for get_groups_unix() in lib/system_samba.c

>--- lib/system_smbd.c.orig	2012-07-29 19:48:57.000000000 -0700
>+++ lib/system_smbd.c	2012-07-29 19:49:09.000000000 -0700
>@@ -210,7 +210,11 @@
> 	gid_t *groups;
> 	int i;
> 
>+#if defined(DARWINOS)
>+	max_grp = 128;
>+#else
> 	max_grp = MIN(128, groups_max());
>+#endif
> 	temp_groups = SMB_MALLOC_ARRAY(gid_t, max_grp);
> 	if (! temp_groups) {
> 		return False;
Comment 19 Jeremy Allison 2021-09-13 00:56:32 UTC
Re-assigning to Jule for inclusion in 4.15.0.
Comment 20 Jule Anger 2021-09-13 07:41:22 UTC
Pushed to autobuild-v4-15-test.
Comment 21 Samba QA Contact 2021-09-13 08:55:12 UTC
This bug was referenced in samba v4-15-test:

bd7302091099d6b5052f59ea0f5dca4539954327
Comment 22 Jule Anger 2021-09-13 11:41:52 UTC
Closing out bug report.

Thanks!
Comment 23 Samba QA Contact 2021-09-13 13:55:36 UTC
This bug was referenced in samba v4-15-stable (Release samba-4.15.0rc7):

bd7302091099d6b5052f59ea0f5dca4539954327