Bug 8635 - Samba 4 denies Linux/Samba single sign on
Summary: Samba 4 denies Linux/Samba single sign on
Status: RESOLVED FIXED
Alias: None
Product: Samba 4.0
Classification: Unclassified
Component: AD: LDB/DSDB/SAMDB (show other bugs)
Version: unspecified
Hardware: All Linux
: P5 regression (vote)
Target Milestone: ---
Assignee: Andrew Bartlett
QA Contact: samba4-qa@samba.org
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-11-29 20:03 UTC by steve (retry timeout exceeded; no DNS MX or A record)
Modified: 2012-04-19 23:26 UTC (History)
5 users (show)

See Also:


Attachments
add posix to a s4 group (439 bytes, application/octet-stream)
2012-02-17 09:08 UTC, steve (retry timeout exceeded; no DNS MX or A record)
no flags Details
create a posix s4 user and add to s4 posix group (1.17 KB, application/octet-stream)
2012-02-17 09:09 UTC, steve (retry timeout exceeded; no DNS MX or A record)
no flags Details
Samba4 group add - msilveira version (681 bytes, text/plain)
2012-03-15 18:37 UTC, Mauricio Silveira
no flags Details
Samba4 user add - msilveira version (1.40 KB, text/plain)
2012-03-15 18:38 UTC, Mauricio Silveira
no flags Details
s4user with windows attributes (1.81 KB, application/octet-stream)
2012-03-15 19:54 UTC, steve (retry timeout exceeded; no DNS MX or A record)
no flags Details
posixGroup before and after fix (1.27 KB, application/octet-stream)
2012-03-27 15:46 UTC, steve (retry timeout exceeded; no DNS MX or A record)
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description steve (retry timeout exceeded; no DNS MX or A record) 2011-11-29 20:03:19 UTC
Scenario:
LAN of linux and win7 clients currently with Samba 3.6 and
LDAP. Linux users authenticate against LDAP and are placed in their
nfs'd /home folder. They can also logon to windows. Their roaming profile is stored in their /home folder.

Samba 4 will not allow this. There is no way of having a single sign on LAN with Samba 4.

There is no way of migrating  Samba 3/LDAP/Linux users over to Samba 4.

Please: This should be fixed before the stable release goes out.

Thanks
Steve
Comment 1 Matthias Dieter Wallnöfer 2011-11-29 21:07:31 UTC
Do you mean the missing Domain Services for UNIX support? Or that s4 winbind does not honor the special UID/GID attributes of our user/group accounts in the directory?
Or do you mean interoperability with an existing directory (LDAP) service? Given that Samba 4 consists in an AD implementation we will only be working together with AD or AD-compatible synchronisation. There have been attempts to directly operate against Fedora/389DS and OpenLDAP but these were stopped after lack of maintenance.
Comment 2 steve (retry timeout exceeded; no DNS MX or A record) 2011-11-30 00:26:47 UTC
>
Simply:
With Samba 3.6/LDAP/windows I can sit at either a windows client or a linux client and authenticate using the same username and password. With Samba 4 I have to have 2 sets of files. One for windows and another for linux. With Samba 4, I have lost single sign on.

The Samba 4 LDAP server seems to be hidden. Linux clients can't authenticate against it.

With Linux becoming increasingly popular in networks (not just on servers) I would have expected the Samba 4 LDAP service to be made available for linux clients too. With Samba 4, only the windows clients are served. On a heterogeneous LAN, this is back to the bad old days of having to have more that one password.

Samba 4 seems to cater only for windows clients. There seems no way to migrate from a heterogeneous LAN with linux and windows clients under 3.6/LDAP to Samba 4.

I'm sure that there is a solution to this migration. I feel certain that Samba 4 can be altered to make the LDAP server available to everyone. Otherwise it's a show stopper for those of us who need single sign on. And that's a lot of folk!

More simply:

How do those of us with Samba 3.x/LDAP with win and linux clients move to Samba 4 with win and linux clents?

Do you plan to document this and make the transition possible?

Thanks so much for your patience.
Comment 3 Geza Gemes (dead mail address) 2011-11-30 05:46:59 UTC
Recommended reading (although for Solaris):
http://phaedrus77.blogspot.com/2010/04/samba4-ad-domain-controller-to-serve.html
I think it describes your situation.
Comment 4 steve (retry timeout exceeded; no DNS MX or A record) 2011-11-30 06:48:39 UTC
@Geza comment 3

Yea. What he's done is what we need integrated into Samba 4. In Samba 3, openSUSE's Yast 'Create User' module does this for you. It creates an LDAP user with Samba attributes, a Linux /home folder and gives you therefore single sign on for linux and windows clients. Superb. It stores roaming profiles in the users /home folder. OK, if Samba 4 wants to store them somewhere else, fine. Just put a link in an equivalent script if the user is being created for Linux and Windows logons.

I feel Samba 4 development should include a mechanism for this. Perhaps as an added option to be able to create either a windows only user, or a user who will logon to both Linux and windows clients. this should be supplied as a script on the Linux side to be able to create the AD entry AND supply users with a Linux logon capability.

With the increasing number of Linux boxes as Clients on a LAN, I feel Samba 4 should address this issue before rolling out beyond it's alpha phase.

I will work with you and will test anything you like but please don't leave it as it is. Please make provision for Linux users too.

I think the key to understanding this issue lies in the openSUSE Yast module I mention.

Thanks
Comment 5 Matthieu Patou 2011-11-30 17:19:55 UTC
In reply to comment 2
> >
> Simply:
> With Samba 3.6/LDAP/windows I can sit at either a windows client or a linux
> client and authenticate using the same username and password. With Samba 4 I
> have to have 2 sets of files. One for windows and another for linux. With Samba
> 4, I have lost single sign on.

You are mixing files and logon, that's not the same.
I suppose also what you call Samba 4 is in fact the Active Directory domain controller implementation of Samba.

The upcomming Samba 4.0 release will support other modes: the NT domain controler, the domain file/print server member and the standalone file/print server.

It's possible that the Samba AD part won't allow you all the things that you were able to do with the NT domain controller. It might be added to subsequent release but for the 4.0 release we still focus on simple AD setup.
It's up to you to weight what is the most important to you.

> 
> The Samba 4 LDAP server seems to be hidden. Linux clients can't authenticate
> against it.
That's not exactly true, you need most probably to modify your schema to be able to do it.
And that's like this because by default Samba AD ships with a AD compatible schema, pretty much like if you install Windows server and run DC promo, the difference is that for the moment we don't offer a simple way to add SFU.

That being said, I've been running a Samba AD network for almost 4 years now, and I've able to acheive single Windows and Linux login for my users.

I used the libnss_winbind for the user/group listing and pam_winbind for the authentification, the constraint here is that the path for the home directory must follow a fixed scheme and is not related to the profile attribute in the user object.

Most probably you can use the libnss_ldap, check http://en.gentoo-wiki.com/wiki/Active_Directory_Authentication_using_LDAP. Although you have to pay attention that for the moment we don't have a SFU schema.

The best way to help on resolution of this bug is:

* read how roaming profile works in Active directory
* try with libnss_winbind + pam_winbind, reports what is not working. I think one of the problem is that winbind on the AD DC server (Samba4) won't be able to allocate uid/gid the same way winbind on the linux workstations
* try with ldap, do tracing between the client and the server and see what are the request the client is doing.


> 
> With Linux becoming increasingly popular in networks (not just on servers) I
> would have expected the Samba 4 LDAP service to be made available for linux
> clients too. With Samba 4, only the windows clients are served. On a
> heterogeneous LAN, this is back to the bad old days of having to have more that
> one password.
No that's not true see before.> More simply:
> 
> How do those of us with Samba 3.x/LDAP with win and linux clients move to Samba
> 4 with win and linux clents?
> 
If you don't need the AD feature there is nothing to do mostly once Samba 4.0 will be released we will continue to support NT domains and your 3.x setup will work in 4.x but without support for AD.
Comment 6 Matthieu Patou 2011-11-30 18:08:54 UTC
(In reply to comment #4)
> @Geza comment 3
> 
> Yea. What he's done is what we need integrated into Samba 4. In Samba 3,
> openSUSE's Yast 'Create User' module does this for you. It creates an LDAP user
> with Samba attributes, a Linux /home folder and gives you therefore single sign
> on for linux and windows clients. Superb. It stores roaming profiles in the
> users /home folder. OK, if Samba 4 wants to store them somewhere else, fine.

Would be good that you explain clearly what do you mean by Samba 4 ?, is it Samba 4.0 AD or Samba 4.0 File and NT domain, for the later it should work as for 3.X series, for the AD I'm pretty sure distro will have to update their tools so that they can create a user in the AD database with the needed class and attributes.


> Just put a link in an equivalent script if the user is being created for Linux
> and Windows logons.
> 
> I feel Samba 4 development should include a mechanism for this. Perhaps as an
> added option to be able to create either a windows only user, or a user who
> will logon to both Linux and windows clients.
A good idea but definitly not showstopper.
> this should be supplied as a
> script on the Linux side to be able to create the AD entry AND supply users
> with a Linux logon capability.
> 
> With the increasing number of Linux boxes as Clients on a LAN, I feel Samba 4
> should address this issue before rolling out beyond it's alpha phase.
> 
> I will work with you and will test anything you like but please don't leave it
> as it is. Please make provision for Linux users too.
Don't expect samba 4.0 to do everything for you out of the box, user creation script might be needed 
> 
> I think the key to understanding this issue lies in the openSUSE Yast module I
> mention.
> 
> Thanks
Comment 7 steve (retry timeout exceeded; no DNS MX or A record) 2011-12-02 05:50:45 UTC
Basically all I want to do is to be able to create an AD user who is also a Linux user at the same time. I attach an LDIF of the attributes I use with Samba 3 and LDAP to do this. It seems that in Samba 4 all that is missing is the users /home directory and the posixGroup attribute. Would it not be a possibility to add these attributes so that the linux user are one and the same as the AD user when the user is first created? Or at least give an option for this to happen when the user is first created. Or are you saying it is probably going to be supplied by a tool created in the distros once Samba 4 is released? It's the 'probably' that concerns me here!

When a Samba 4 user creates a file, where is it stored?


Thanks.

Linux/samba 3 ldif for a user called don:

version: 1

# Entry 1: uid=don,ou=visitors,dc=site
dn: uid=don,ou=visitors,dc=site
cn: d
gidnumber: 1000
homedirectory: /home/don
loginshell: /bin/bash
objectclass: top
objectclass: posixAccount
objectclass: inetOrgPerson
objectclass: sambaSamAccount
ou: visitors
sambaacctflags: [U          ]
sambalmpassword: 5978DB2BF043C13DAAD3B435B51404EE
sambantpassword: 97F6342013263BE4CD8675B045CA2136
sambaprimarygroupsid: S-1-5-21-2903347571-4210458020-258296108-1201
sambapwdcanchange: 1321818576
sambapwdlastset: 1321818576
sambapwdmustchange: 2147483647
sambasid: S-1-5-21-2903347571-4210458020-258296108-3008
sn: d
uid: don
uidnumber: 1004
userpassword: {ssha}Qj/c9zKzAo18Di+qf6WHVRYSnqNOSENFTA==

And the posixGroup the user is in:

version: 1

# Entry 1: cn=ldapusers,ou=group,dc=site
dn: cn=ldapusers,ou=group,dc=site
cn: ldapusers
gidnumber: 1000
objectclass: posixGroup
objectclass: top
objectclass: namedObject
Comment 8 steve (retry timeout exceeded; no DNS MX or A record) 2011-12-02 05:57:05 UTC
(In reply to comment #3)
> Recommended reading (although for Solaris):
> http://phaedrus77.blogspot.com/2010/04/samba4-ad-domain-controller-to-serve.html
> I think it describes your situation.

Thanks Geza. Be assured that yes, I have read it. Thanks for the reference. What I'd like is for the script he mentions to be written and included by the devs (who know the Samba 4 ldap attributes needed better than anyone else) for when the Samba 4 relese goes public and out of alpha.

Cheers.
Comment 9 Adam Tauno Williams 2011-12-02 15:28:27 UTC
Steve, what you are looking for is for it to store RFC2307 style attributes.  The attributes you really need are -

User:

gidnumber: 1000
homedirectory: /home/don
loginshell: /bin/bash
objectclass: posixAccount
uid: don
uidnumber: 1004

Group:

cn: ldapusers
gidnumber: 1000
objectclass: posixGroup

[all the other attributes you listed are just Samba3-SAM cruft that isn't relevant anymore to S4 - the upgrade script already considers those attributes and migrates them to their AD equivalents]

*IF* the source object when doing a domain upgrade contains those objectclasses it would be *VERY USEFUL* if the upgrade could provision those into the migrated AD object.  This would immensely ease transition.  Of course, that is, if the S4 winbind supported RFC2307 style information.
Comment 10 steve (retry timeout exceeded; no DNS MX or A record) 2011-12-02 22:12:35 UTC
(In reply to comment #9)
> Steve, what you are looking for is for it to store RFC2307 style attributes. 
> The attributes you really need are -
> 
> User:
> 
> gidnumber: 1000
> homedirectory: /home/don
> loginshell: /bin/bash
> objectclass: posixAccount
> uid: don
> uidnumber: 1004
> 
> Group:
> 
> cn: ldapusers
> gidnumber: 1000
> objectclass: posixGroup
> 
> [all the other attributes you listed are just Samba3-SAM cruft that isn't
> relevant anymore to S4 - the upgrade script already considers those attributes
> and migrates them to their AD equivalents]
> 
> *IF* the source object when doing a domain upgrade contains those objectclasses
> it would be *VERY USEFUL* if the upgrade could provision those into the
> migrated AD object.  This would immensely ease transition.  Of course, that is,
> if the S4 winbind supported RFC2307 style information.

'VERY USEFUL' +1

Thanks for extracting the objects for me. I'm getting closer to understanding this now.

Adam, would the script you mention migrate user don from Samba 3/LDAP to Samba 4/AD thus maintaining his ability to logon to both  Linux and windows clients using the same password?

Where is the script?

Thanks so much to everyone for your patience.
Comment 11 steve (retry timeout exceeded; no DNS MX or A record) 2011-12-02 22:25:31 UTC
(In reply to comment #5)
> In reply to comment 2
> > >
> > Simply:
> > With Samba 3.6/LDAP/windows I can sit at either a windows client or a linux
> > client and authenticate using the same username and password. With Samba 4 I
> > have to have 2 sets of files. One for windows and another for linux. With Samba
> > 4, I have lost single sign on.
> 
> You are mixing files and logon, that's not the same.
> I suppose also what you call Samba 4 is in fact the Active Directory domain
> controller implementation of Samba.
> 
> The upcomming Samba 4.0 release will support other modes: the NT domain
> controler, the domain file/print server member and the standalone file/print
> server.
> 
> It's possible that the Samba AD part won't allow you all the things that you
> were able to do with the NT domain controller. It might be added to subsequent
> release but for the 4.0 release we still focus on simple AD setup.
> It's up to you to weight what is the most important to you.
> 
> > 
> > The Samba 4 LDAP server seems to be hidden. Linux clients can't authenticate
> > against it.
> That's not exactly true, you need most probably to modify your schema to be
> able to do it.
> And that's like this because by default Samba AD ships with a AD compatible
> schema, pretty much like if you install Windows server and run DC promo, the
> difference is that for the moment we don't offer a simple way to add SFU.
> 
> That being said, I've been running a Samba AD network for almost 4 years now,
> and I've able to acheive single Windows and Linux login for my users.
> 
> I used the libnss_winbind for the user/group listing and pam_winbind for the
> authentification, the constraint here is that the path for the home directory
> must follow a fixed scheme and is not related to the profile attribute in the
> user object.
> 
> Most probably you can use the libnss_ldap, check
> http://en.gentoo-wiki.com/wiki/Active_Directory_Authentication_using_LDAP.
> Although you have to pay attention that for the moment we don't have a SFU
> schema.
> 
> The best way to help on resolution of this bug is:
> 
> * read how roaming profile works in Active directory
> * try with libnss_winbind + pam_winbind, reports what is not working. I think
> one of the problem is that winbind on the AD DC server (Samba4) won't be able
> to allocate uid/gid the same way winbind on the linux workstations
> * try with ldap, do tracing between the client and the server and see what are
> the request the client is doing.
> 
> 
> > 
> > With Linux becoming increasingly popular in networks (not just on servers) I
> > would have expected the Samba 4 LDAP service to be made available for linux
> > clients too. With Samba 4, only the windows clients are served. On a
> > heterogeneous LAN, this is back to the bad old days of having to have more that
> > one password.
> No that's not true see before.> More simply:
> > 
> > How do those of us with Samba 3.x/LDAP with win and linux clients move to Samba
> > 4 with win and linux clents?
> > 
> If you don't need the AD feature there is nothing to do mostly once Samba 4.0
> will be released we will continue to support NT domains and your 3.x setup will
> work in 4.x but without support for AD.

Sorry to go on but this doesn't seem to be true. Here is my [homes] share in Samba 3.6:

[homes]
	comment = Home Directories
	valid users = %S, %D%w%S
	browseable = No
	read only = No
	inherit acls = Yes

There are no attributes in place to support this. This is not recognised in Samba 4.

Steve.
Comment 12 steve (retry timeout exceeded; no DNS MX or A record) 2011-12-18 13:16:26 UTC
Hi

The main issue I see at the moment is what will be in included in the final release.

The things that I, and I think others, are having trouble with are:

1. dns
2. joining a linux box to a samba domain
3. administering the Samba 4 AD

Notes:
1. This is frightening. You have gone some way to helping with your named.conf include file. On openSUSE it mainly works but here are issues with named in a chroot. It doesn't 'just work'. 

On Ubuntu I had no option but to compile bind 9.8.1-SP from source. You need a spare weekend for that and even with decent installation instructions, some major twaking is necessary for it to fire up with Samba 4.

2. Whilst I can get Win 7 boxes to join an logon to the Samba 4 AD, I can join neither openSUSE 12.1 nor Ubuntu 11.10 machines. This is the key to my SSO requirement.

As both bind and likewise are opensource, would it be a good idea to integrate them into the final samba 4 release? 

3. Having to use a Win 7 ultimate box with the remote AD tools loaded (another nightmareish procedure!) is a pain. It also ties up a second computer.

These guys have written a Samba 4 AD gui for Linux, like the Windows remote AD administration:
 
http://www.resara.org/index.php?option=com_content&view=article&id=49

Would it be also be possible to include this in the Samba 4 release?

Summary:
It would give the distros a head start with Samba 4 integration if these tools could be provided at the release date.

Thanks for your time and I hope you will be able to consider my suggestions.
Comment 13 steve (retry timeout exceeded; no DNS MX or A record) 2011-12-24 15:36:21 UTC
Is it possible to include a fix for this?

's4 winbind does not honor the special UID/GID attributes of our user/group accounts in the directory.'

As above, we need this: RFC2307

It would make a great piece of software much more usable.

Thanks so much.
Comment 14 steve (retry timeout exceeded; no DNS MX or A record) 2012-01-04 10:41:20 UTC
I've added the rfc2307 attributes to Samba 4 LDAP and got SSO working with Kerberos.

Here is my simple 29 step fix/workaround;)

http://linuxcostablanca.blogspot.com/2011/12/samba-4-linux-integration-first-i-want.html

It's not very elegant and I'm sure that you guys could do a lot better.

There are two magic lines I use in Samba 3 (also without winbind):

passdb backend = ldapsam:ldap://myserver
idmap backend = ldap:ldap://myserver

I would like to present my ideas on samba-technical, but have not yet dared join for fear of being a newbie and because of the possibility of getting in your way.
Comment 15 Adam Tauno Williams 2012-01-12 22:00:38 UTC
Holy awesome; it got better.  I just tested an upgrade of our production domain and it appears that Samba4 took [and kept] the UID number from the existing account.

Production
-------------
[root@littleboy ~]# id adam
uid=437(adam) gid=230(cis) groups=230(cis)

Test Server
------------
barbel:~ # wbinfo -i adam
BACKBONE\adam:*:437:100:Adam Williams:/home/BACKBONE/adam:/bin/false

Home directory is a bit wierd, and the gidNumber didn't stick.  But at least I have the uidNumber.

4.0.0alpha18-GIT-103c1cb [openSUSE 12.1 x86_64] transitioned via "samba-tool domain samba3upgrade" from Samba S3w/LDAPSAM.
Comment 16 steve (retry timeout exceeded; no DNS MX or A record) 2012-01-13 04:03:16 UTC
Well, something has changed. My method of adding the rfc2307 attributes no longer works.

http://linuxcostablanca.blogspot.com/2011/12/samba-4-linux-integration-first-i-want.html

Could we please have it back as it was before, which allows my method?

Or, of course fix the attributes.
Thanks
Comment 17 steve (retry timeout exceeded; no DNS MX or A record) 2012-01-26 17:53:03 UTC
It's working fine now. My dns blew up, that's all.

I've updated my howto. Adding rfc2307 attributes is straightforward. I've also added kerberized nfs4 as the fileserver and idmapd and nss-pam-ldapd for id mappings by binding to the LDAP via gssapi. S4 now serves w7 and Linux with correct uid:gid mapping, roaming profiles, home folders... 

At last a heterogeneous lan under s4. Am currently working on scripts to automate adding/modifying users without having to use dsa.msc

Any advice most welcome. There's some security stuff too, like where to put the ticket cache and whether to put everything in one keytab or not.
Comment 18 steve (retry timeout exceeded; no DNS MX or A record) 2012-02-17 09:08:32 UTC
Created attachment 7329 [details]
add posix to a s4 group
Comment 19 steve (retry timeout exceeded; no DNS MX or A record) 2012-02-17 09:09:51 UTC
Created attachment 7330 [details]
create a posix s4 user and add to s4 posix group
Comment 20 steve (retry timeout exceeded; no DNS MX or A record) 2012-02-17 09:27:47 UTC
Attached 2 scripts:

1. s4group
synopsis: s4group <groupname>
description: Creates a windows group with rfc2307 attributes

2. s4user
synopsis: s4user <user> <groupname>
description: Creates a windows user with rfc2307 attributes and his home folder. Adds <user> to <groupname>. Sets his windows primaryGroup to groupname.

Bare metal scripts. No error checking, but not dangerous. Tested from openSUSE  PDC build 4.0.0alpha18-GIT-567f05e on openSUSE, Ubuntu and w7 clients.

Can idiot-proof/edit/add-to them if anyone thinks they may be useful.
Comment 21 Matthias Dieter Wallnöfer 2012-02-17 20:05:25 UTC
Andrew, Matthieu,

should we integrate these scripts?
Comment 22 steve (retry timeout exceeded; no DNS MX or A record) 2012-02-27 07:40:38 UTC
(In reply to comment #21)
> Andrew, Matthieu,
> 
> should we integrate these scripts?

I've added to the scripts to make them more flexible and give more options.

There are now 4 scripts to deal with most of the posix attributes defined in posixAccount and posixGroup classes in the AD schema. Still very version 0.1-ish but usable:

http://linuxcostablanca.blogspot.com/2012/02/samba-4-posix-domain-user.html

This just about solves it for us. The rather sorry state of nfs4 acl's is another story:-(

Could I use this opportunity to make a request? samba-too user and samba-tool group work perfectly as they are for adding users and groups to the domain. Would it be possible to ask that the devs don't change this functionality? IOW, leave the basic functionality as it is now? Of course develop it further but do so by means of adding options to it. Dare I suggest a -p option? e.g.

samba-tool user add steve -p suseusers

This is a link to my s4user script which runs the s4user script, creating steve as a domain user and setting his default group to suseusers on both the windows and Linux sides.

This effectively gives user steve SSO to both Windows and Linux clients.
classes.
Comment 23 Mauricio Silveira 2012-03-01 00:35:08 UTC
I'll test this "schema" in a few days.
I really wonder why samba4 got the posix attributes out ( I read somewhere that in some past version, S4 schema included posix attributes ).

I'm assigned on a mission to get S4, Openchange and SOGo up and running... and I want to prevent other problems just in case I come to use some "trick" using gid/uid/home attribute data.

Thanks steve.
Comment 24 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-01 07:07:45 UTC
(In reply to comment #23)
> I'll test this "schema" in a few days.
> I really wonder why samba4 got the posix attributes out ( I read somewhere that
> in some past version, S4 schema included posix attributes ).
> 
> I'm assigned on a mission to get S4, Openchange and SOGo up and running... and
> I want to prevent other problems just in case I come to use some "trick" using
> gid/uid/home attribute data.
> 
> Thanks steve.

I took the posix attributes directly from the 2008-R2 schema which I believe was given to us by microsoft. It comes with the s4 source anyway. The scripts include nothing which is not specified in the schema. The relevant posix stuff I used is here:
http://linuxcostablanca.blogspot.com/2012/02/microsoft-2k8r2-ad-schema.html
which follows rfc-2307 to the letter:-)

I just hope and prey our s4 devs do not decide to change this bit of the schema. (confirmation guys?). Maybe this bugzilla will answer your other point about why posix was taken out in the first place. I looked at an old a17 dpkg recently and they had been taken out by then. 

Just to confirm, we have seen no adverse effects on any of the accounts to which the posix attrs have been added and even under heavy lan load, access to e.g. Linux logon info is fine although admittedly not as instantaneous as a local /etc/passwd logon. If you do have a go with the scripts maybe we could have a go at idiot-proofing them?
Good luck with your assignment.
Comment 25 Mauricio Silveira 2012-03-13 17:39:27 UTC
I used to think Samba4 needs to have its schema extended ( as of alpha17 )...
but after trying
´´ldapsearch -H ldap://localhost -b CN=Schema,CN=Configuration,DC=MYDOMAIN,DC=LOCAL "(&(objectclass=attributeSchema)(|(cn=uidNumber)(cn=gidNumber)(cn=gecos)(cn=loginShell)(cn=unixHomeDirectory)))" -LLL´´

revealed that everything is already there ( AFAIK ) except for homeDirectory, renamed to unixHomeDirectory by MS ( OID is the same as RFC2307 NIS OID 1.3.6.1.1.1.1.3 ), probably to avoid collision with "Home-Directory" attribute, where lDAPDisplayName = homeDirectory .

If rID is enough to be mapped to uid/gid ( on small environments, I bet this is OK ), then, using cn -> "RID Set"'s rIDNextRID value AFTER adding a user or group to the domain, is enough.

"CN=RID Set,CN=DCHOST,OU=Domain Controllers,DC=MYDOMAIN,DC=LOCAL" is, as far as my research has reached, is what holds the rID of the last created object ( those that holds an rID, of course ).

I'll add more info later. Just want to keep track of what I'm doing, and help others understand the path of my concepts.

PS: Openchange 1.0 is almost there, so uid/gid/home mapping for S4 might be necessary or mandatory for getting postfix,dovecot, etc up and running. That's why I'm here :)
Comment 26 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-14 17:25:02 UTC
(In reply to comment #25)
> I used to think Samba4 needs to have its schema extended ( as of alpha17 )...
> but after trying
> ´´ldapsearch -H ldap://localhost -b
> CN=Schema,CN=Configuration,DC=MYDOMAIN,DC=LOCAL
> "(&(objectclass=attributeSchema)(|(cn=uidNumber)(cn=gidNumber)(cn=gecos)(cn=loginShell)(cn=unixHomeDirectory)))"
> -LLL´´
> 
> revealed that everything is already there ( AFAIK ) except for homeDirectory,
> renamed to unixHomeDirectory by MS ( OID is the same as RFC2307 NIS OID
> 1.3.6.1.1.1.1.3 ), probably to avoid collision with "Home-Directory" attribute,
> where lDAPDisplayName = homeDirectory .
> 
> If rID is enough to be mapped to uid/gid ( on small environments, I bet this is
> OK ), then, using cn -> "RID Set"'s rIDNextRID value AFTER adding a user or
> group to the domain, is enough.
> 
> "CN=RID Set,CN=DCHOST,OU=Domain Controllers,DC=MYDOMAIN,DC=LOCAL" is, as far as
> my research has reached, is what holds the rID of the last created object (
> those that holds an rID, of course ).
> 

All the rfc2307 stuff is already in the schema. All I've done is add the attributes under the dn of the user in sam.ldb. I extract them on Linux clients using nss-ldapd. It works fine. What I'm afraid of is that something will change and tell me that this is forbidden.

So please, don't let's change this part of it!
> I'll add more info later. Just want to keep track of what I'm doing, and help
> others understand the path of my concepts.
> 
> PS: Openchange 1.0 is almost there, so uid/gid/home mapping for S4 might be
> necessary or mandatory for getting postfix,dovecot, etc up and running. That's
> why I'm here :)
Comment 27 Matthieu Patou 2012-03-14 19:59:53 UTC
Comment on attachment 7329 [details]
add posix to a s4 group

Script is not generic, it's just ok for dc=hh3,dc=site
Comment 28 Matthieu Patou 2012-03-14 20:01:00 UTC
Comment on attachment 7330 [details]
create a posix s4 user and add to s4 posix group

script is specific to a domain
Comment 29 Matthieu Patou 2012-03-14 20:03:51 UTC
(In reply to comment #21)
> Andrew, Matthieu,
> 
> should we integrate these scripts?

Not like that.

There is not enough error checks and if we want to have this I think it's better to have it integrated in samba-tool.

Steve can you rework the stuff so that it can be options/arguments of samba-tool ? I'm thinking at something like --add-posix-uid --add-posix-gid.
Comment 30 Matthieu Patou 2012-03-14 20:13:40 UTC
And before doing anything I think we need a clear understanding of how windows is behaving so please someone has to test it.

I will disregard/revert any patch in this area that hasn't a clear explanation on why we are doing like that.
Comment 31 Mauricio Silveira 2012-03-14 22:46:34 UTC
(In reply to comment #30)
> And before doing anything I think we need a clear understanding of how windows
> is behaving so please someone has to test it.
> 
> I will disregard/revert any patch in this area that hasn't a clear explanation
> on why we are doing like that.

The real question is "why did this "bug" was reported?" at first place. I personally don't consider this as a bug itself if winbind isn't involved in this matter.

The problem is this: In samba3, it was possible to even "loopback" winbind to export users to the "DC" and use many idmapping options ( uid/gid ) to map LDAP users ( using samba3 with ldap backend  ).

If you have a native Windows200x server, you still have the option to use samba3/winbind to do the auto uid/gid mapping to the UNIX world and do the other configs necessary to get pam/NIS working. In either case, you have a fragile idmapping ( winbind idmap auto-generated cache is a security concern IMHO ).

The truth is, even with native windows AD servers, even if you install SFU, you have to deal with uid, gid, home, and other data mapping to get pam/ldap working and achieve a central user database for both Windows and UNIX worlds; that is, if you really want to get rid of winbind.

Well, as I stated before, I'm trying to get S4/Openchange/SOGo working, so, I need to usa a central user db. Actually we're using S3. Our plan is to move to the S4/OC/SOGo "combo".

The problem is: I haven't managed to get S4-alpha17 to run winbind yet ( it simply terminates without errors ).

Since Samba4 aims to be an exact replica for an AD implementation, S4 devel team approach ATM is reasonable. The point is: even though Samba4 is an AD implementation replica, it is *nix based and its config+tools should have options/triggers to allow us to either seamlessly, via smb.conf option defaults and/or declared as a samba-tool option - to populate the *nix specific attributes.

All of my Samba deploys are small-sized ( up to 200 users max ); I consider automatic rID->gidNumber/uidNumber mapping the best approach. "It deduplicates *ID data"

Let's close this post with a summary: this *nix attributes issue is directly related to the winbind and its idmapping features.

I may provide my test environment details if you like.
PS: You'll probably find me at efnet's #Samba as VenomX

Regards,
Mauricio
Comment 32 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-15 07:22:31 UTC
The bug is solved for me by bypassing winbind. Please remember the original point of the bug report: we have a lan of win 7 and Linux clients and we want single sign on. Well, we've got that now. Here is a posix group:

 ldbsearch --url=/usr/local/samba/private/sam.ldb 'cn=suseusers'
# record 1
dn: CN=suseusers,CN=Users,DC=hh3,DC=site
cn: suseusers
instanceType: 4
whenCreated: 20120308073234.0Z
uSNCreated: 3852
name: suseusers
objectGUID: 03b7ebe8-9a3e-40ae-820e-71e0f79cc624
objectSid: S-1-5-21-2871321456-443247610-264051687-1123
sAMAccountName: suseusers
sAMAccountType: 268435456
groupType: -2147483646
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=hh3,DC=site
objectClass: top
objectClass: group
objectClass: posixGroup
gidNumber: 6006
whenChanged: 20120308073628.0Z
uSNChanged: 3864
distinguishedName: CN=suseusers,CN=Users,DC=hh3,DC=site

# Referral
ref: ldap://hh3.site/CN=Configuration,DC=hh3,DC=site

# Referral
ref: ldap://hh3.site/DC=DomainDnsZones,DC=hh3,DC=site

# Referral
ref: ldap://hh3.site/DC=ForestDnsZones,DC=hh3,DC=site

# returned 4 records
# 1 entries
# 3 referrals

And here is a posix user in the group:

ldbsearch --url=/usr/local/samba/private/sam.ldb 'cn=steve2'
# record 1
dn: CN=steve2,CN=Users,DC=hh3,DC=site
cn: steve2
instanceType: 4
whenCreated: 20120308073617.0Z
uSNCreated: 3858
name: steve2
objectGUID: 182ec165-bda5-4dbd-bc8a-b323f228239f
badPwdCount: 0
codePage: 0
countryCode: 0
badPasswordTime: 0
lastLogoff: 0
lastLogon: 0
objectSid: S-1-5-21-2871321456-443247610-264051687-1124
logonCount: 0
sAMAccountName: steve2
sAMAccountType: 805306368
userPrincipalName: steve2@hh3.site
objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=hh3,DC=site
pwdLastSet: 129756657770000000
uidNumber: 3000027
gidNumber: 6006
unixHomeDirectory: /home/CACTUS/steve2
loginShell: /bin/bash
objectClass: top
objectClass: person
objectClass: posixAccount
objectClass: organizationalPerson
objectClass: user
memberOf: CN=Domain Users,CN=Users,DC=hh3,DC=site
primaryGroupID: 1123
profilePath: \\hh3\profiles\steve2
homeDrive: Z:
homeDirectory: \\hh3\home\steve2
userAccountControl: 66048
accountExpires: 0
whenChanged: 20120314191228.0Z
uSNChanged: 3964
distinguishedName: CN=steve2,CN=Users,DC=hh3,DC=site

# Referral
ref: ldap://hh3.site/CN=Configuration,DC=hh3,DC=site

# Referral
ref: ldap://hh3.site/DC=DomainDnsZones,DC=hh3,DC=site

# Referral
ref: ldap://hh3.site/DC=ForestDnsZones,DC=hh3,DC=site

# returned 4 records
# 1 entries
# 3 referrals

Nothing is changed on the AD-windows side and the (rather nice) nss-ldapd does the mapping on the Linux side via GSSAPI. We use kerberized NFS for the fileserver. It works perfectly.

The reason I have not closed the bug as solved is that I'd like to know what I'm doing wrong by doing this. IOW, by bypassing winbind am I missing out on something? 
1. Will the schema withdraw the rfc2307 attributes at some stage?
2. Will s4 include SFU and if s will this interfere with my implementation?
3. Any other advice, particularly with the scripts we've written.
Thanks
Comment 33 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-15 12:50:29 UTC
(In reply to comment #29)
> (In reply to comment #21)
> > Andrew, Matthieu,
> > 
> > should we integrate these scripts?
> 
> Not like that.
> 
> There is not enough error checks and if we want to have this I think it's
> better to have it integrated in samba-tool.
> 
> Steve can you rework the stuff so that it can be options/arguments of
> samba-tool ? I'm thinking at something like --add-posix-uid --add-posix-gid.

I can only do bash, and even then very badly as you can see:-(  There's some windows stuff in there too like setting the primaryGroupID, home drive letter and profile path.

Could we get the samba-tool/python guys onto it?
Comment 34 Mauricio Silveira 2012-03-15 18:33:34 UTC
(In reply to comment #32)
> The bug is solved for me by bypassing winbind. Please remember the original
> point of the bug report: we have a lan of win 7 and Linux clients and we want
> single sign on.

Let's get things cleared: There are 2 POVs in this matter:
1.) From an AD POV, there's no problem. ( This looks like the current devel POV )
2.) From the SAMBA ( as a *nix software ) POV, this is a huge problem.

> The reason I have not closed the bug as solved is that I'd like to know what
> I'm doing wrong by doing this. IOW, by bypassing winbind am I missing out on
> something? 

AFAIK, winbind is only required for NTLM, not KRB/GSSAPI. If you don't nedd NTLM, you can live without winbind. But what if you need NTLM?

> 1. Will the schema withdraw the rfc2307 attributes at some stage?
I don't think so. Every LDAP implementation is providing rfc2307 extension, even MS AD (2003 and 2008), which is the ground of the S4 goal . Why should SAMBA drop it?

> 2. Will s4 include SFU and if s will this interfere with my implementation?
It would be better to have ADCU include the editable rfc2307 attributes rather than supporting SFU in Samba. SFU does more than modify LDAP.

> 3. Any other advice, particularly with the scripts we've written.
I'm attaching my version of the scripts. They rely fully on ldapmodify, not ldbmodify, so it should work from any workstation or other domain member with the correct krb5 setup.
Comment 35 Mauricio Silveira 2012-03-15 18:37:45 UTC
Created attachment 7394 [details]
Samba4 group add - msilveira version

This is my version of Steve's s4group. Uses ldapmodify, enbales rid->uid/gid mapping
Comment 36 Mauricio Silveira 2012-03-15 18:38:42 UTC
Created attachment 7395 [details]
Samba4 user add - msilveira version

This is my version of Steve's s4user. Uses ldapmodify, enbales rid->uid/gid mapping
Comment 37 Mauricio Silveira 2012-03-15 19:25:44 UTC
About adding "objectClass: PosixGroup" to a group:

I've tried this group modification with a W2k8R2 box and the S4 test box today, using MS AD Explorer on both cases plus phpldapadmin with the S4 BOX.

If you pay attention, you'll see that, in S4, a "posixified" group will not behave as an AD security group correctly. It will show up with a different ("missing") icon under ADCU and if you try to edit it, the only thing you'll see is the General tab with its name and a "Description box" instead of all the General, members, member Of, etc Tabs.

This is S4-alpha17, I'll be updating and re-provisioning this test domain later on.

In the windows 2008R2 box, I've managed to create a test group, add the posixGroup objectClass and gidNumber without errors using AD Explorer. And the group behaved as it should: Both a Security group, with all the tabs it should have when editing under ADCU.

I tried creating a group under ADCU and adding posixGroup to objectClass and 1550 to gidNumber. Both AD Explorer and phpldapadmin returned errors. phpldapadmin provided better error data:
---------------------------------------------------
LDAP said:      Object class violation
Error number:	0x41 (LDAP_OBJECT_CLASS_VIOLATION)
Description:	You tried to perform an operation that would cause an undefined attribute to exist or that would remove a required attribute, given the current list of ObjectClasses. This can also occur if you do not specify a structural objectClass when creating an entry, or if you specify more than one structural objectClass.
---------------------------------------------------
I managed to get the S4 group posixified by using ldapmodify. but its type  under ADCU became simply posixgroup instead of "Security Group". I guess it might make it useless for windows machines.

Our goal includes sharing the groups with both Windows and *nix, right?

So, I got AD Explorer and dumped both groups attributes and compared the posixified group from both Windows 2008R2 and Samba4: the only "difference" in attributes was the order of the classes (objectClass):
2008R2 -> top;posixGroup;group
Samba4+ldapmodify -> top;group;posixGroup

I was not satisfied, so I compared the dump of the posixGroup and posixAccount schemas from both S4 and 2008R2 :: there's no difference at all!

Summary: S4 group posixified under S4 ldapmodify becomes of type posixGroup, while groups posixified using AD Explorer under Windows2008R2 are still of type "Security Group - $SCOPE$".

I'm updating samba4 to alpha18 right now.... more info on tests results later.
Comment 38 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-15 19:53:21 UTC
Yes, you lose the 'Member of' and other tabs by adding posixGroup class. I've a thread open on the samba list about this and they suggested there that I contact Microsoft as to why. I did not have a 2008 server to test against. Maybe now we have the info we need? 

I can confirm however, that the posixified group behaves as a normal group for both win and linux in as much as you can set acl's, permissions etc.

Here is the script we use to add some other useful windows stuff to a user.
Comment 39 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-15 19:54:27 UTC
Created attachment 7396 [details]
s4user with windows attributes
Comment 40 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-15 20:00:33 UTC
(In reply to comment #37)
> About adding "objectClass: PosixGroup" to a group:
> 
> I've tried this group modification with a W2k8R2 box and the S4 test box today,
> using MS AD Explorer on both cases plus phpldapadmin with the S4 BOX.
> 
> If you pay attention, you'll see that, in S4, a "posixified" group will not
> behave as an AD security group correctly. It will show up with a different
> ("missing") icon under ADCU and if you try to edit it, the only thing you'll
> see is the General tab with its name and a "Description box" instead of all the
> General, members, member Of, etc Tabs.
> 
> This is S4-alpha17, I'll be updating and re-provisioning this test domain later
> on.
> 
> In the windows 2008R2 box, I've managed to create a test group, add the
> posixGroup objectClass and gidNumber without errors using AD Explorer. And the
> group behaved as it should: Both a Security group, with all the tabs it should
> have when editing under ADCU.
> 
> I tried creating a group under ADCU and adding posixGroup to objectClass and
> 1550 to gidNumber. Both AD Explorer and phpldapadmin returned errors.
> phpldapadmin provided better error data:
> ---------------------------------------------------
> LDAP said:      Object class violation
> Error number:    0x41 (LDAP_OBJECT_CLASS_VIOLATION)
> Description:    You tried to perform an operation that would cause an undefined
> attribute to exist or that would remove a required attribute, given the current
> list of ObjectClasses. This can also occur if you do not specify a structural
> objectClass when creating an entry, or if you specify more than one structural
> objectClass.
> ---------------------------------------------------
> I managed to get the S4 group posixified by using ldapmodify. but its type 
> under ADCU became simply posixgroup instead of "Security Group". I guess it
> might make it useless for windows machines.

No, it still works fine. Groups can be manipulated both under Linux and Windows equally well.

> 
> Our goal includes sharing the groups with both Windows and *nix, right?
> 
Certainly for us at least. But I think that's what we have here no? The guys on the Samba list suggested it was a bug in ADCU. The posixified group works fine in both Lin and win.

> So, I got AD Explorer and dumped both groups attributes and compared the
> posixified group from both Windows 2008R2 and Samba4: the only "difference" in
> attributes was the order of the classes (objectClass):
> 2008R2 -> top;posixGroup;group
> Samba4+ldapmodify -> top;group;posixGroup
> 
> I was not satisfied, so I compared the dump of the posixGroup and posixAccount
> schemas from both S4 and 2008R2 :: there's no difference at all!
> 
> Summary: S4 group posixified under S4 ldapmodify becomes of type posixGroup,
> while groups posixified using AD Explorer under Windows2008R2 are still of type
> "Security Group - $SCOPE$".
> 
> I'm updating samba4 to alpha18 right now.... more info on tests results later.
Comment 41 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-15 20:04:27 UTC
from: MS-AD_Schema_2K8_R2_Classes, lines 2879 to 2930

cn: PosixAccount
ldapDisplayName: posixAccount
governsId: 1.3.6.1.1.1.2.0
objectClassCategory: 3
rdnAttId: uid
subClassOf: top
mayContain: uid, cn, uidNumber, gidNumber, unixHomeDirectory,homeDirectory, userPassword, unixUserPassword, loginShell, gecos,description
schemaIdGuid:ad44bb41-67d5-4d88-b575-7b20674e76d8
defaultSecurityDescriptor: D:(A;;RPWPCRCCDCLCLORCWOWDSDDTSW;;;DA)(A;;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)(A;;RPLCLORC;;;AU)
defaultHidingValue: TRUE
systemOnly: FALSE
defaultObjectCategory: CN=PosixAccount,CN=Schema,CN=Configuration,<RootDomainDN>

cn: PosixGroup
ldapDisplayName: posixGroup
governsId: 1.3.6.1.1.1.2.2
objectClassCategory: 3
rdnAttId: cn
subClassOf: top
mayContain: cn, userPassword, unixUserPassword, description,gidNumber, memberUid
schemaIdGuid:2a9350b8-062c-4ed0-9903-dde10d06deba
defaultSecurityDescriptor: D:(A;;RPWPCRCCDCLCLORCWOWDSDDTSW;;;DA)(A;;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)(A;;RPLCLORC;;;AU)
defaultHidingValue: TRUE
systemOnly: FALSE
defaultObjectCategory: CN=PosixGroup,CN=Schema,CN=Configuration,<RootDomainDN>
Comment 42 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-15 22:21:46 UTC
Group members tab:

The members of a posixified group appear in the same format as for security groups in
 /usr/local/samba/private/sam.ldb.d/DC=xxx,DC=yyy.ldb
along with their acl's and ace's.

We don't have the code to fix ADUC but we should be able to script a list-posix-group-members script from our side.
Comment 43 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-15 22:26:25 UTC
(In reply to comment #40)
> (In reply to comment #37)
> > About adding "objectClass: PosixGroup" to a group:
> > 
> > I've tried this group modification with a W2k8R2 box and the S4 test box today,
> > using MS AD Explorer on both cases plus phpldapadmin with the S4 BOX.
> > 
> > If you pay attention, you'll see that, in S4, a "posixified" group will not
> > behave as an AD security group correctly. It will show up with a different
> > ("missing") icon under ADCU and if you try to edit it, the only thing you'll
> > see is the General tab with its name and a "Description box" instead of all the
> > General, members, member Of, etc Tabs.
> > 
> > This is S4-alpha17, I'll be updating and re-provisioning this test domain later
> > on.
> > 
> > In the windows 2008R2 box, I've managed to create a test group, add the
> > posixGroup objectClass and gidNumber without errors using AD Explorer. And the
> > group behaved as it should: Both a Security group, with all the tabs it should
> > have when editing under ADCU.
> > 
> > I tried creating a group under ADCU and adding posixGroup to objectClass and
> > 1550 to gidNumber. Both AD Explorer and phpldapadmin returned errors.
> > phpldapadmin provided better error data:
> > ---------------------------------------------------
> > LDAP said:      Object class violation
> > Error number:    0x41 (LDAP_OBJECT_CLASS_VIOLATION)
> > Description:    You tried to perform an operation that would cause an undefined
> > attribute to exist or that would remove a required attribute, given the current
> > list of ObjectClasses. This can also occur if you do not specify a structural
> > objectClass when creating an entry, or if you specify more than one structural
> > objectClass.
> > ---------------------------------------------------
> > I managed to get the S4 group posixified by using ldapmodify. but its type 
> > under ADCU became simply posixgroup instead of "Security Group". I guess it
> > might make it useless for windows machines.
> 
> No, it still works fine. Groups can be manipulated both under Linux and Windows
> equally well.
> 
> > 
> > Our goal includes sharing the groups with both Windows and *nix, right?
> > 
> Certainly for us at least. But I think that's what we have here no? The guys on
> the Samba list suggested it was a bug in ADCU. The posixified group works fine
> in both Lin and win.
> 
> > So, I got AD Explorer and dumped both groups attributes and compared the
> > posixified group from both Windows 2008R2 and Samba4: the only "difference" in
> > attributes was the order of the classes (objectClass):
> > 2008R2 -> top;posixGroup;group
> > Samba4+ldapmodify -> top;group;posixGroup
> > 
Could we use ldbedit to reorder?

> > I was not satisfied, so I compared the dump of the posixGroup and posixAccount
> > schemas from both S4 and 2008R2 :: there's no difference at all!
> > 
> > Summary: S4 group posixified under S4 ldapmodify becomes of type posixGroup,
> > while groups posixified using AD Explorer under Windows2008R2 are still of type
> > "Security Group - $SCOPE$".
> > 
> > I'm updating samba4 to alpha18 right now.... more info on tests results later.
Comment 44 Mauricio Silveira 2012-03-16 00:27:41 UTC
(In reply to comment #43)
> > > So, I got AD Explorer and dumped both groups attributes and compared the
> > > posixified group from both Windows 2008R2 and Samba4: the only "difference" in
> > > attributes was the order of the classes (objectClass):
> > > 2008R2 -> top;posixGroup;group
> > > Samba4+ldapmodify -> top;group;posixGroup
> > > 
> Could we use ldbedit to reorder?

I bet this is not the issue, I tried, using ADExplorer, but an error happened again. There is probably some schema inconsistency, that's why AD Explorer and phpldapadmin returned an error when adding posixGroup to objectClass.

0x41 (LDAP_OBJECT_CLASS_VIOLATION) -> phpldapadmin is clear

OpenLDAP FAQs might provide some insight: (not sure if it is related )
http://www.openldap.org/faq/data/cache/650.html
http://www.openldap.org/faq/data/cache/883.html
http://www.openldap.org/lists/openldap-software/200507/msg00055.html


I still wonder why ldapmodify / ldbmodify accept the changes without complains...
Comment 45 Mauricio Silveira 2012-03-16 00:43:58 UTC
ldbedit on S4 posixified group results: ( trying to remove the posixGroupobjectClass )
failed to modify CN=posixified,CN=Users,DC=sogo,DC=local - objectclass: the specified objectclasses are not exactly the same as on the entry!
A transaction is still active in ldb context [0x985970] on /var/lib/samba4/private/sam.ldb

Now, I'll try to posixify a fresh group entry:
failed to modify CN=posixldbedited,CN=Users,DC=sogo,DC=local - objectclass: the specified objectclasses are not exactly the same as on the entry!
A transaction is still active in ldb context [0xd7f970] on /var/lib/samba4/private/sam.ldb


If I try to add/remote another entry such as gidNumber, all is fine.

Definitely, there is something wrong with the samba schema or another bug preventing this posixGroup operation to succeed.

Just out of curiosity, trying to make a posixified user from ldbedit:Same error as before.

I'll avoid posting until I have alpha18 up and running with a new provision.

FYI: Openchange 1.0 is out. Now I really need this fixed :)
Comment 46 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-16 08:38:40 UTC
(In reply to comment #44)
> (In reply to comment #43)
> > > > So, I got AD Explorer and dumped both groups attributes and compared the
> > > > posixified group from both Windows 2008R2 and Samba4: the only "difference" in
> > > > attributes was the order of the classes (objectClass):
> > > > 2008R2 -> top;posixGroup;group
> > > > Samba4+ldapmodify -> top;group;posixGroup
> > > > 
> > Could we use ldbedit to reorder?
> 
> I bet this is not the issue, I tried, using ADExplorer, but an error happened
> again. There is probably some schema inconsistency, that's why AD Explorer and
> phpldapadmin returned an error when adding posixGroup to objectClass.
> 
> 0x41 (LDAP_OBJECT_CLASS_VIOLATION) -> phpldapadmin is clear
> 
> OpenLDAP FAQs might provide some insight: (not sure if it is related )
> http://www.openldap.org/faq/data/cache/650.html
> http://www.openldap.org/faq/data/cache/883.html
> http://www.openldap.org/lists/openldap-software/200507/msg00055.html
> 
> 
> I still wonder why ldapmodify / ldbmodify accept the changes without
> complains..
.
Well, when posixGroup has been added, the group behaves exactly as a Linux and windows group. It can be given security acl's from the security tab under windows and they are respected. samba-tool can be used to add and remove members to and from it and ADUC can be used to set it as primaryGroupID. It seems that the only thing we are missing is to be able to manipulate and view group membership from ADCU.

Is there anything I can do to help?
Comment 47 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-17 08:29:43 UTC
Would it be an idea to open another bugzilla asking if we can get the posixGroup class sorted out in the schema? Having said that, all the information needed to look into the problem is here in _this_ report.

It seems to be working fine. The frustration for me is that we have lost the 'Members' tab under Active Directory Computers and Users.
Comment 48 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-17 08:44:00 UTC
(In reply to comment #46)
> (In reply to comment #44)
> > (In reply to comment #43)
> > > > > So, I got AD Explorer and dumped both groups attributes and compared the
> > > > > posixified group from both Windows 2008R2 and Samba4: the only "difference" in
> > > > > attributes was the order of the classes (objectClass):
> > > > > 2008R2 -> top;posixGroup;group
> > > > > Samba4+ldapmodify -> top;group;posixGroup
> > > > > 
> > > Could we use ldbedit to reorder?
> > 
> > I bet this is not the issue, I tried, using ADExplorer, but an error happened
> > again. There is probably some schema inconsistency, that's why AD Explorer and
> > phpldapadmin returned an error when adding posixGroup to objectClass.

4.0.0alpha19-GIT-7c71520
I am able to add posixGroup under objectClass using phpldapadmin without the error. But as soon as I do that, we lose the Members tab in ADUC. It seems as though posixGroup takes priority over Security Group. Also under phpldapadmin, you can add members to the posixGroup but it doesn't list them. e.g. if you try to add a member, it will only let you if the member is not already there (correct) but it does not list the members which are already there. Annoying!

> > 
> > 0x41 (LDAP_OBJECT_CLASS_VIOLATION) -> phpldapadmin is clear
> > 
> > OpenLDAP FAQs might provide some insight: (not sure if it is related )
> > http://www.openldap.org/faq/data/cache/650.html
> > http://www.openldap.org/faq/data/cache/883.html
> > http://www.openldap.org/lists/openldap-software/200507/msg00055.html
> > 
> > 
> > I still wonder why ldapmodify / ldbmodify accept the changes without
> > complains..
> .
> Well, when posixGroup has been added, the group behaves exactly as a Linux and
> windows group. It can be given security acl's from the security tab under
> windows and they are respected. samba-tool can be used to add and remove
> members to and from it and ADUC can be used to set it as primaryGroupID. It
> seems that the only thing we are missing is to be able to manipulate and view
> group membership from ADCU.
> 
> Is there anything I can do to help?
Comment 49 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-17 09:15:42 UTC
This may help us trace what is wrong:
1. Manipulating the security acl's of a posixified group:

http://4.bp.blogspot.com/-eEJsd2TOny8/T05PbDh85zI/AAAAAAAAARI/Axx-76I4DEA/s1600/w712.png

Works as expected.

2. Setting the primary group for a user to a posixified group:

http://2.bp.blogspot.com/-oDBqT03MB78/Tzk2-FN9C6I/AAAAAAAAALU/4Ihs7VgK2Yk/s1600/s6-steve6user.png

Works as expected.

3. phpldapadmin representation of a posixified group:

http://4.bp.blogspot.com/-oeTty-Y6HFo/TzT49_mZe3I/AAAAAAAAALE/zGb00l_WMC4/s1600/ldapadmin.png

Does not list the members who are already in the group but works in that attempting to add a user who is already there gives an error. You can only add users who are not already members. This is better that ADUC in that it gives us the equivalent of the 'Members' tab.
Comment 50 Andrew Bartlett 2012-03-23 21:55:58 UTC
I'm still lost as to what the actual bug is here.  

The issue with ADUC is well known, but unless you can show that Microsoft servers behave any differently, I'm not sure we can do anything.
Comment 51 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-25 07:40:08 UTC
(In reply to comment #50)
> I'm still lost as to what the actual bug is here.  
> 
> The issue with ADUC is well known, but unless you can show that Microsoft
> servers behave any differently, I'm not sure we can do anything.

Copmment (In reply to comment #50)
> I'm still lost as to what the actual bug is here.  
> 
> The issue with ADUC is well known, but unless you can show that Microsoft
> servers behave any differently, I'm not sure we can do anything.

It has already been shown (please see comment 43) that 2008 server behaves correctly when the posixGroup objectClass is added. s4 does not. Would it be possible to correct this please.
Comment 52 Matthias Dieter Wallnöfer 2012-03-26 08:17:07 UTC
This should have been fixed now. The problem was that the (last) structural or 88 object class has to be the last value (according to MS-ADTS) of the "objectClass" attribute.
Therefore it is our fault, not ADUC ones.
Comment 53 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-26 09:05:39 UTC
(In reply to comment #52)
> This should have been fixed now. The problem was that the (last) structural or
> 88 object class has to be the last value (according to MS-ADTS) of the
> "objectClass" attribute.
> Therefore it is our fault, not ADUC ones.

Excellent.  Where can I get the fix? Would a git pull get me there? Thank you so much for your time.
Comment 54 Matthias Dieter Wallnöfer 2012-03-26 09:23:07 UTC
You said it (a "git pull" from our "master" branch).
Comment 55 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-27 15:44:12 UTC
Sorry to reopen
Version 4.0.0alpha19-GIT-7290a62

Confirmation that the posixGroup now works perfectly. Thank you. BUT is there an easy way to make changes to posixGroup objects that were existing before the fix?

I've tried ldbedit to change the order of the objectClass classes but it will not accept the change. The order stays the same. Is there an easy way to fix this apart from recreating the posixGroups?
Comment 56 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-27 15:46:26 UTC
Created attachment 7407 [details]
posixGroup before and after fix

The only difference I see is the order of the objectClasses. Can we apply the fix to posixGroups created before the fix?
Comment 57 Matthias Dieter Wallnöfer 2012-03-27 16:30:04 UTC
Please try a modify request on each entry containing a replace action on attribute "objectClass" with all object classes as values embedded.

In "ldbmodify" it should work like this:

dn: <entry>
changetype: modify
replace: objectClass
objectClass: top
objectClass: posixGroup
objectClass: group
-

abartlet, should we integrate a fix into "dbcheck"?
Comment 58 Mauricio Silveira 2012-03-27 18:58:31 UTC
Glad to see this fixed!
After all, the observation of the objectClass order as in comment #37 was not in vain. Really, I never thought that would be the real issue. I thougt it could point to something else.

Commit 0f8ffa9ce1777d0b368eb765a7f69f93e68118bd, right?

I'll try this fix later!
Comment 59 Andrew Bartlett 2012-03-27 20:49:56 UTC
We need to patch dbcheck to find and fix this.

mdw:  Do you think you could take on that?  We would need to expose the sort function to python, or at least an incorrect-order detection function, and then dbcheck could automatically do as you suggest.
Comment 60 steve (retry timeout exceeded; no DNS MX or A record) 2012-03-28 15:16:24 UTC
(In reply to comment #57)
> Please try a modify request on each entry containing a replace action on
> attribute "objectClass" with all object classes as values embedded.
> 
> In "ldbmodify" it should work like this:
> 
> dn: <entry>
> changetype: modify
> replace: objectClass
> objectClass: top
> objectClass: posixGroup
> objectClass: group
> -
> 
> abartlet, should we integrate a fix into "dbcheck"?

It needs a bigger hammer:

dn: CN=yourOldGroup,CN=Users,DC=hh3,DC=site
changetype: modify
delete: objectClass
objectClass: posixGroup
-
add: objectClass
objectClass: posixGroup

Then the order is correct. Thanks.
Comment 61 Matthias Dieter Wallnöfer 2012-04-18 10:28:13 UTC
Finally fixed ("dbcheck" now handles wrong "objectClass"es).
Comment 62 steve (retry timeout exceeded; no DNS MX or A record) 2012-04-18 21:07:24 UTC
Thanks for fixing the group order.

The next problem concerns the memberUid attribute which assigns a user as a member of a posixGroup.

If, as the schema allows, the memberUid attribute has been assigned to a user, then deleting the user only deletes his domain group memberships. The memberUid which is part of 'may contain' for the posixGroup, remains set. This doesn't make sense. getent group shows a group member who is no longer part of the directory. 

Would it be possible for either
 samba-tool user delete
or
 samba-tool dbcheck
to clear the memberUid attribute too?

Currently we are using ldbedit to search and delete the user entries from the groups the user may have been a member of:-(

I think with this, we are close to posixPerfection in Samba4.
Comment 63 steve (retry timeout exceeded; no DNS MX or A record) 2012-04-19 12:38:21 UTC
e.g. cheap and cheerful perhaps:

db="/usr/local/samba/private/sam.ldb"
. . .
#Show them the groups before they go ahead
echo "$1 is currently a member of these Posix groups:"
getent group | grep $1 | cut -d ":" -f1 > usergroups
cat usergroups
#Ask are you sure and stuff. . .

# Delete the Posix memberships
while read group
do echo removing from $group;
echo "dn: cn=$group,cn=Users,dc=your,dc=domain
changetype: modify
delete: memberUid
memberUid: $1" > /tmp/$1
echo "Deleting $1 from $group"
sleep 2
ldbmodify --url=$db /tmp/$1 
rm /tmp/$1
done < "usergroups"

# Then, finally delete the Windows memberships:
# get his SID first
samba-tool user delete $1
# Tidy up idmap.ldb
# Tidy up home folder and profile stuff
Comment 64 Matthias Dieter Wallnöfer 2012-04-19 20:32:19 UTC
What you are asking is not as simple as it might seems.

As you can read here the approach of using the "memberUid" attribute is not the only possibility to depict membership: http://www.openldap.org/lists/openldap-software/200304/msg00740.html
The other way consists in the use of "groupOfUniqueNames" objects with "uniqueMember" attributes instead.

At the moment I am not seeing this issue related to the s4 server code (Windows Server from my point of view does not behave differently). As a consequence I would also not be changing "dbcheck" since there is no automatism connected to these attributes (differently to "member"/"memberOf").

"samba-tool user delete" could be extended though I think. Andrew, ekacnet, do you have an idea where to handle this best?
Comment 65 steve (retry timeout exceeded; no DNS MX or A record) 2012-04-19 23:09:00 UTC
(In reply to comment #64)
> What you are asking is not as simple as it might seems.
> 
> As you can read here the approach of using the "memberUid" attribute is not the
> only possibility to depict membership:
> http://www.openldap.org/lists/openldap-software/200304/msg00740.html
> The other way consists in the use of "groupOfUniqueNames" objects with
> "uniqueMember" attributes instead.
> 

Neither of the those two alternatives are available for either posixAccount nor posixGroup objectclasses in the AD schema. 

> At the moment I am not seeing this issue related to the s4 server code (Windows
> Server from my point of view does not behave differently). As a consequence I
> would also not be changing "dbcheck" since there is no automatism connected to
> these attributes (differently to "member"/"memberOf").
> 
> "samba-tool user delete" could be extended though I think. Andrew, ekacnet, do
> you have an idea where to handle this best?

This is a bit 'us' specific, but we have a working model here:
#!/bin/bash
# general user delete tool for Win and Linux users.
if [ -z $1 ]
then
echo "USAGE: s4userdel <username>"              
exit
fi

db="/usr/local/samba/private/sam.ldb"

echo "Deleting user $1"
echo "Are you sure? (Y or N)"
read D
if [ "$D" != "Y" ]
then
exit
fi

echo "$1 is currently a member of these Posix groups:"
getent group | grep $1 | cut -d ":" -f1 > usergroups
cat usergroups

while read group
do echo removing from $group;
echo "dn: cn=$group,cn=Users,dc=hh3,dc=site
changetype: modify
delete: memberUid
memberUid: $1" > /tmp/$1
echo "Deleting $1 from $group"
sleep 6
ldbmodify --url=$db /tmp/$1 
rm /tmp/$1
done < "usergroups"

echo "Tidying up. . ."
sid=$(wbinfo --name-to-sid="$1")
usersid=$(echo "$sid" | cut -d " " -f1)
echo "found $1 sid= $usersid" 
ldbdel --url=/usr/local/samba/private/idmap.ldb CN=$usersid

#finally delete him from the directory altogether
samba-tool user delete $1
echo "Remove the user data? (n Y)"
read D
if [ "$D" == "Y" ]
then
echo deleting data
rm -r /home2/MARINA/$1
rm -r /home2/MARINA/profiles/"$1"*
else
echo not deleting data
fi
echo "$1 deleted"
Comment 66 Andrew Bartlett 2012-04-19 23:26:52 UTC
memberUid group handling is not AD-style group handling.

This bug is rapidly getting off track.  We still recommend using only winbindd on clients to gain posix access to AD, be it Samba4 or Microsoft's AD.

The work to store the uidNumber and gidNumber (such that winbindd can use idmap_ad) is valuable, but we need bugs in our bug database to reflect realistic depictions of issues that we can fix, not wishlists for site-specific hacks. 

Additionally, if you must use only nss_ldap, it can handle member/memberOf, but we still do not recommend nss_ldap.  Even if we supported memberUid, it is specifically written to allow it to point at usernames not in the directory (that is why it isn't a DN), and so tight integration is specifically not the intention. 

As far as I can tell, the original and subsequent issues in this bug have been dealt with.  Please open a new specific bug if you can find issues with our behaviour that doesn't match Windows 2008 (of which trial versions are available).

In order to get to a release, we need to stay focused on the many existing issues, before we start trying to support things going well beyond that.  (Particularly because we must inter-operate with Microsoft DCs, which will not have any of the extra behaviours)

Thanks,