Bug 7588 - 'Permission denied' if AD user is member in more than 16 groups on OpenIndiana/Illumos/OpenSolaris/Solaris
Summary: 'Permission denied' if AD user is member in more than 16 groups on OpenIndian...
Status: RESOLVED WORKSFORME
Alias: None
Product: Samba 3.5
Classification: Unclassified
Component: File services (show other bugs)
Version: 3.5.4
Hardware: x64 Solaris
: P3 major
Target Milestone: ---
Assignee: Volker Lendecke
QA Contact: Samba QA Contact
URL: https://www.illumos.org/issues/3691
Keywords:
Depends on:
Blocks:
 
Reported: 2010-07-26 12:06 UTC by Marcis
Modified: 2013-06-25 15:19 UTC (History)
4 users (show)

See Also:


Attachments
log for machine when connecting user is member in 16 unix groups (92.99 KB, text/plain)
2010-07-26 12:08 UTC, Marcis
no flags Details
log for machine when connecting user is member in 17 unix groups (84.65 KB, text/plain)
2010-07-26 12:09 UTC, Marcis
no flags Details
truss -f -o /tmp/smbd.truss.17groups smbd -D (819.77 KB, text/plain)
2010-07-26 17:22 UTC, Marcis
no flags Details
"truss -f -o /tmp/smbd.truss.16groups smbd -D" (845.35 KB, text/plain)
2010-07-26 17:23 UTC, Marcis
no flags Details
nss_winbind patch for master / 3.6 (should apply to other versions as well) (1.04 KB, patch)
2013-06-08 03:37 UTC, Andrew Bartlett
no flags Details
smbd patch for setgroups.c for 3.6 / master (should apply to other versions too) (1.91 KB, patch)
2013-06-08 03:40 UTC, Andrew Bartlett
no flags Details
nss_winbind patch for Samba 4 (1.35 KB, patch)
2013-06-25 15:17 UTC, Ralph Böhme
no flags Details
smbd patch for setgroups.c for Samba 4 (2.22 KB, patch)
2013-06-25 15:19 UTC, Ralph Böhme
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Marcis 2010-07-26 12:06:01 UTC
I had problems with NGROUPS_MAX limit being 16 on OpenSolaris snv_134 - AD users could not access shares if they were members of more than 16 groups unless permissions granted access to user or users primary group. I increased this limit to 1024 in OS, and recompiled samba, but the limit still seems to apply to samba. I tested and it is possible to create unix user who is member of 1024 groups, but samba still throws 'Permission denied' for user with more than 16 groups.

I even have tried modifying /usr/include/limits.h (NGROUPS_MAX) and /usr/include/sys/param.h (NGROUPS_MAX_DEFAULT) values to reflect new group limit.

At some point I was able to rise this limit to 35, but never to 1024.

Running samba 3.5.4 on OpenSolaris snv_134
Comment 1 Marcis 2010-07-26 12:08:40 UTC
Created attachment 5866 [details]
log for machine when connecting user is member in 16 unix groups
Comment 2 Marcis 2010-07-26 12:09:00 UTC
Created attachment 5867 [details]
log for machine when connecting user is member in 17 unix groups
Comment 3 Marcis 2010-07-26 14:43:10 UTC
Thread with information how I eventualy got the Samba 3.5.4 compiling & working  on OpenSolaris snv_134 http://opensolaris.org/jive/thread.jspa?threadID=132223&tstart=0

And threads with some more info regarding the 16 user limitation and how I came to point where I'm now with samba on OpenSolaris
http://marc.info/?t=127910771600002&r=1&w=2
http://marc.info/?t=127952183800001&r=1&w=2
Comment 4 Volker Lendecke 2010-07-26 15:47:42 UTC
Please truss smbd of the failure and upload the output.

Thanks,

Volker
Comment 5 Marcis 2010-07-26 17:22:36 UTC
Created attachment 5868 [details]
truss -f -o /tmp/smbd.truss.17groups smbd -D

Gathered with "truss -f -o /tmp/smbd.truss.17groups smbd -D" when user is member in 17 groups
Comment 6 Marcis 2010-07-26 17:23:22 UTC
Created attachment 5869 [details]
"truss -f -o /tmp/smbd.truss.16groups smbd -D"

Gathered with "truss -f -o /tmp/smbd.truss.16groups smbd -D" when user is member in 16 groups
Comment 7 Volker Lendecke 2010-07-28 03:27:03 UTC
OpenSolaris denies access. I don't see any difference in what Samba would do differently, except just to add that group.

I know that it is blasphemy to point at the possibility that OpenSolaris might behave in an unexpected way, but if I was you I would file that as a bug with OpenSolaris.

Volker
Comment 8 Marcis 2010-07-29 03:01:06 UTC
Strange, if user is member in "domain admins" then everything works. Does membership in "domain admins" makes samba to behave differently (not drop privileges or something)?
Comment 9 Marcis 2010-08-05 01:58:06 UTC
Hey! Is it possible to agree on the following, we leave bug open, but we note, that it's waiting for more input from me. I have some ideas how to debug this better, but I'll be able to do that only in my spare time, so it might take long.
Comment 10 Allen Belletti 2013-02-15 23:24:13 UTC
I have run into what appears to be the identical problem several years later.  I've tried the opencsw.org package for 3.6.11.  I've also compiled (using gcc, not Sun cc) 3.6.12.

The operating system is OpenIndiana 151a7 (latest updates) based on the Illumos kernel.  I've got "set ngroups_max = 128" in /etc/system and verified that this is working.  I'm able to manipulate users with up to 128 groups.  At least, I can do so as long as they're defined locally in /etc/passwd and /etc/group.

My problem is that AD users obtained via Winbind are running into a groups limitation which appears to be at 16.  This is confusing because it works most places.  If I "su GGC\\abelletti" and then run "id -a", I'm able to see every assigned group.  I've added a large number (but less than 128) for testing.

However, if I then attempt to "cd" into a directory which I should be able to access due to group permissions, I fail.  This same operation works when those groups are assigned locally.  If running Samba, I get basically the same problem:

[2013/02/15 18:13:29.427975,  4] smbd/vfs.c:780(vfs_ChDir)
  vfs_ChDir to /shares/oet-networking
[2013/02/15 18:13:29.428073,  3] smbd/service.c:190(set_current_service)
  chdir (/shares/oet-networking) failed, reason: Permission denied

I've run "pcred" on the smbd that belongs to GGC\abelletti and verified that all group ids are present.  That would have me assume that this is an OpenIndiana problem except that it works fine with locally assigned users/groups as above.

I've tried modifying NGROUPS_MAX in limits.h and NGROUPS_MAX_DEFAULT in sys/param.h and recompiling, also without success.

I'm happy to provide any logs and debugging information which might help to debug this.  OpenIndiana is still the best ZFS platform, at least until ZFS on Linux is a bit more accepted.

Thanks,
Allen
Comment 11 Björn Jacke 2013-02-18 13:22:06 UTC
as Volker already mentioned, this is clearly a OpenSolaris issue. If the access denied from the operating systems for /tmp/ comes when you put the user in the 17th group, then there is nothing Samba can do about it.

If someone from the OpenSolaris users already diled a bug for OpenSolaris (I hope someone did) then please add the link to that bug report here.

For Samba this is a worksforme bug.
Comment 12 Allen Belletti 2013-02-18 17:42:41 UTC
That would be my conclusion too, if it always failed.  However, it works for large numbers of groups if they are defined locally in /etc/group.  It's only when the groups are obtained via Winbind that the problem appears.  Hence it could be an OpenSolaris issue, but it could just as easily be a Winbind issue, no?
Comment 13 Björn Jacke 2013-02-19 12:24:44 UTC
if you want to have an idea of what sick things have been implemented in that OS, read for example http://lists.samba.org/archive/samba/2006-March/118463.html .
And the log and truss here is pretty clear also. If nobody file a bug against OpenSolaris since it was pointed out that this is obviously a OS bug back in 2010 then, well ...
Comment 14 Allen Belletti 2013-02-19 21:07:32 UTC
(In reply to comment #13)
> if you want to have an idea of what sick things have been implemented in that
> OS, read for example
> http://lists.samba.org/archive/samba/2006-March/118463.html .
> And the log and truss here is pretty clear also. If nobody file a bug against
> OpenSolaris since it was pointed out that this is obviously a OS bug back in
> 2010 then, well ...

FYI, bug #3577 (https://www.illumos.org/issues/3577) opened with Illumos.  Perhaps we can get this solved.  Samba is the best CIFS package, Illumos is the best ZFS platform, and Active Directory is... entrenched.  The combination needs to work together :-)
Comment 15 Andrew Bartlett 2013-06-08 03:37:18 UTC
Created attachment 8950 [details]
nss_winbind patch for master / 3.6 (should apply to other versions as well)

For those so unfortunate to be unable to fix their OS, this patch may help for non-Samba applications (samba patch to follow). 

Thanks to Allen Belletti for testing.
Comment 16 Andrew Bartlett 2013-06-08 03:40:40 UTC
Created attachment 8951 [details]
smbd patch for setgroups.c for 3.6 / master (should apply to other versions too)

This patch will make smbd sort the group list before it calls setgroups(). 

This fix belongs in the kernel or libc, but if someone else is so unfortunate as to be suffering this bug, and can't fix those, this patch may save some of their remaining grey hair ;-)

Thanks to NETGEAR for testing.
Comment 17 James Relph 2013-06-23 00:03:47 UTC
(In reply to comment #16)
> Created attachment 8951 [details]
> smbd patch for setgroups.c for 3.6 / master (should apply to other versions
> too)
> 
> This patch will make smbd sort the group list before it calls setgroups(). 
> 
> This fix belongs in the kernel or libc, but if someone else is so unfortunate
> as to be suffering this bug, and can't fix those, this patch may save some of
> their remaining grey hair ;-)
> 
> Thanks to NETGEAR for testing.

Thanks for these patches, I'm able to apply these to Samba3 successfully, but with Samba4 the winbind patch succeeds, but the setgroups patch fails with:

Hunk #1 succeeded at 879 (offset -295 lines).
Hunk #2 succeeded at 929 (offset -295 lines).
Hunk #3 FAILED at 955.
Hunk #4 succeeded at 964 (offset -297 lines).
Hunk #5 FAILED at 1056.
2 out of 5 hunks FAILED -- saving rejects to file source3/lib/system.c.rej

Is there an easy change that can be made to that patch that will allow it to work with Samba4?

Thanks.
Comment 18 Andrew Bartlett 2013-06-23 11:52:25 UTC
Yes, the patch should not be hard to alter to make it work in any version.  It really is just ensuring we go via the 'BROKEN' codepath that creates a malloc()ed group list, and then running qsort().  Look at the rejects and I'm sure you can fix it up.
Comment 19 Ralph Böhme 2013-06-25 15:17:18 UTC
Created attachment 8996 [details]
nss_winbind patch for Samba 4
Comment 20 Ralph Böhme 2013-06-25 15:19:28 UTC
Created attachment 8997 [details]
smbd patch for setgroups.c for Samba 4