Bug 3375 - creation of a new pdb_*sql team
Summary: creation of a new pdb_*sql team
Status: RESOLVED FIXED
Alias: None
Product: Samba 3.0
Classification: Unclassified
Component: pdb_sql (show other bugs)
Version: 3.0.21a
Hardware: All All
: P3 normal
Target Milestone: none
Assignee: pdb_sql maintainers mail alias
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-01-04 14:18 UTC by Florian Effenberger
Modified: 2009-03-29 06:48 UTC (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Effenberger 2006-01-04 14:18:25 UTC
This task issue is meant to coordinate the further development of the pdb_*sql modules.

DO NOT REMOVE THIS MODULE FROM THE CODEBASE. We have found volunteers to contribute and will coordinate through the lists and this issue.

Volunteers so far are:
Filip Jirsak - filip@jirsak.org
dmcguire - dmcguire@pegasys.cc

Hope to get feedback from other volunteers soon.

I will act as a coordinator for this module and can help in testing and debugging. AGAIN, PLEASE DO NOT REMOVE THIS MODULE FROM THE CODEBASE.


TODO:

Please let us know which changes will occur in 3.0.22 (and further versions), so we can adapt the module.

Currently known is a bug that after a restart of MySQL (e.g. through logrotate), Samba does not reconnect. This needs to be fixed.
Comment 1 Florian Effenberger 2006-01-04 14:25:00 UTC
For MySQL bug see also
https://bugzilla.samba.org/show_bug.cgi?id=3369
Comment 2 Filip Jirsák 2006-01-05 08:22:57 UTC
OK, if nobody is unwilling, I can maintain pdb*sql code.

From Bugzilla search it seems there is nowadays one blocker BUG 3351, which has proposed patch.

There is another bug marked as blocker - BUG 3191, but it seems it is problem with example configuration - yes, it is problem, but it doesn't crash existing installation.

Please fill another bugs as another bugzilla bugs and keep this bug for discussion about future od pdb_sql module.
Comment 3 Filip Jirsák 2006-01-05 08:25:23 UTC
I'll copy&paste here list created by Florian Effenberger:

Here is a compiled list out of the release notes on what might (!) be an
important change to pdb causing the error:


* Allow pdbedit to set the domain for a user account.

* Port some of the non-critical changes from HEAD to 3_0. 
  The main one is the change in pdb_enum_alias_memberships 
  to match samr.idl a bit closer.

* Implement a new caching API for enumerating the pdb elements.

* Convert the RAP user and group enumeration functions to the 
  utilized the pdb_search API. 

* Fix up example pdb modules after prototype change for 
  setsampwent.

* Add the capability to set account description using pdbedit.

* BUG 892: Default unknown_6 field to 1260 in mySQL pdb module.

* BUG 1957: Implement minimal update of fields in mySQL pdb 
  module.

* Cache the result of a pdb_getsampwnam for later SID lookup 
  queries.

* BUG 2080: Fix duplicate call to pdb_get_acct_desc().

* Backport pdbedit changes from trunk.

Comment 4 Florian Effenberger 2006-01-05 10:50:49 UTC
Filip: You maintaining the code would be a great idea. dmcguire (Darrell is the right name, I guess?) also wanted to contribute. So we have at least two maintainers on the development "area" and me as a tester and contact person and the guy who can work in Bugzilla. :-)

The two open bugs you've just mentioned already have fixes attached. If you can test them and verify that they are working, they could be included in the next release. Please test them on a blank system, so we can assure these fixes work (for me they did with MySQL).

A new bug is Bug 3369, have not checked on this so far.

Jerry & Co:
Can we keep the module in the main Samba tree, if we have new maintainers now, or would you still like to remove it from the main tree?
Comment 5 Volker Lendecke 2006-01-09 06:06:05 UTC
Just had a phone call with Florian. We agreed that the sql modules will be removed for 3.2 and that when I remove them I will attach a patch to this bug report.

3.0 will keep it at the current state, although we don't expect major 3.0 releases anymore.

Volker
Comment 6 Florian Effenberger 2006-01-09 06:13:27 UTC
I agree, as Volker told me that it is not just some little patches missing, but a lot of functionality in the SQL module as well. I myself will try to switch to LDAP.

Regarding the pdbsql team: If you are willing to provide this patch for 3.2 as well I will help with testing and debugging, so we can at least keep the state of 3.0 for the upcoming 3.2 release.

I think we should wait until a 3.2 release is available and then go on discussing on this bug whether we want to continue the SQL module as an inofficial patch for some time to allow smooth switching to LDAP, or whether we will keep with 3.0.
Comment 7 Filip Jirsák 2006-01-09 06:42:56 UTC
Okay, we should wait for 3.2 release. Than we can work on unofficial patch quietly if there'll be asking for it.
Comment 8 Darrell McGuire 2006-01-09 06:49:20 UTC
I can think of a lot of function that I lose when switching to LDAP (easy integration into the security databases of a lot of third party apps in education and government that use MySQL, simple authoring of web based user management tools, etc...) ; bud what do I gain if I begin to switch our clients to LDAP?
Comment 9 Filip Jirsák 2006-01-09 07:02:45 UTC
LDAP is dedicated for storing data related with people and organization, for example MS Active Directory is sort of LDAP. In theory there should be lot of tools for administration of it. But in practice I don't know - I switched from LDAP to pgsql backend year ago :-) I can only give you link to pam_ldap module, if it can help: http://www.padl.com/OSS/pam_ldap.html
Comment 10 Florian Effenberger 2006-01-09 09:52:00 UTC
I also was strictly against that removal (probably that's the reason I called Volker ;-), however, what Volker said sounded reasonable.

The LDAP module is four times larger than the MySQL module which at least partially shows the more functionality the LDAP module has.

In his opinion, it IS possible to get the MySQL up to the level of the LDAP module supporting more fields, entries and all that stuff, however, it has to be done.

If anyone is willing to do, I'd be happy to test and debug. But I can't code, and the Samba team does not have the resources for it.
Comment 11 Mario Lipinski 2006-02-03 03:29:12 UTC
please take me on the list.

i am also interested in the samba mysql modules. but i dont have the time to care for actually. but soon or later i will come to this.

Moving SQL Authentication to something else forces me to keep redundant data on user accounts and just generating ldap from MySQL or something like this. I think the MySQL module has much advantages compared to the LDAP stuff in some envieronments.
Comment 12 Florian Effenberger 2006-02-04 02:33:39 UTC
Hi Mario.

Simply said: If no one is *actively* contributing to the module, it will die. It definitely will be removed from Samba 3.2, the next major revision.

Right now, *no one* of the new volunteers is actively working on the code. Possibilites would include adding functionality as it is found in the LDAP module, so we are head to head.

If that does not happen, I fear, the module simply will die, no matter if anyone uses it or not.

So, please, if you are a coder and have some spare time: Have a look at the module, compare its functionality to the LDAP module and try to add. Or try to fix the bugs.

I myself will switch to LDAP, as I can see no "real" progress in this matter...

Thanks
Florian
Comment 13 Volker Lendecke 2006-02-04 02:37:54 UTC
Hey, that's a coincidence. Florian commented right as 13338 came in...

Just to inform you: I just removed the experimental sql and xml modules.

I'll send further info mail to samba-pdbsql@samba.org.

Volker
Comment 14 Filip Jirsák 2006-02-04 03:49:54 UTC
(In reply to comment #11)
> Moving SQL Authentication to something else forces me to keep redundant data on
> user accounts and just generating ldap from MySQL or something like this. I
> think the MySQL module has much advantages compared to the LDAP stuff in some
> envieronments.
> 


Next version of Samba - Samba 4 - has integrated LDAP server and has AD-like behaviours (whereas AD is sort of LDAP). I think non-LDAP password backends than would grow to be more complicated. Instead of wasting work to keep other pdb modules up-to-date with LDAP pdb (in fact to simulate LDAP over SQL - by the way OpenLDAP can work with SQL backend) it would be better to create and support some tools and utilities, which can keep synchronized necessary data in SQL an LDAP.

In practice there is no need for have every Samba's data in SQL - everyone of us need SQL only for authentication from other apps, so we don't need pdb_sql module, we only need to have synchronized username and password (sometimes fullname and uid), don't we? Some simpler things than pdb_sql module can do this.

By the way my experience is there is more authentication modules for LDAP than for SQL :-)
Comment 15 Darrell McGuire 2006-02-04 07:21:22 UTC
Speaking for myself, my team is at least *trying* to contribute.  We are however making the transition from Samba users to Samba programmers.  This is not a trivial thing.  Energy is being spent; I regret that it is not bearing fruit at the apparently required rate.  It is frustrating from our standpoint to have the carpet yanked from under our feet so quickly.  

I am concerned that if we shift over to technology X-  say LDAP- that we may find ourselves in this position again in a couple years.  What  is to say that if we use OpenLDAP that we may find a huge incompatibility between that and the schemas used in the upcoming Samba 4?  

I am willing to stop trying to figure this out and move toward whatever technology the Samba team is going to standardize on as long as it is a standard which they pledge to support for the foreseeable future.

Respectfully,
   Darrell McGuire
Comment 16 Gerald (Jerry) Carter (dead mail address) 2006-02-04 08:57:21 UTC
Darrell,  Here is the problem. If code is included in 
our tree, people expect it to work in the release.  Even 
marking code as experimental does not seem to work.  The 
best solution is to maintain this code out of the tree 
if you still desire to do so.  I can help you work out 
the build system and the release process management.  That 
way people can get the code from you and you can certify 
it for a given Samba release.

We have officially supported an LDAP backend for over 
4 years now and will continue to do so.  We are very aware 
of any upgrade issues when any of the LDAP code changes.
I wouldn't worry so much about Samba 4 right now since it
really is a long way out.  And even then, it will have to 
provide an upgrade path from a Samba 3 installation.  
Comment 17 Florian Effenberger 2006-02-04 09:20:42 UTC
I think we should wait and see how Samba 4 does LDAP. There will be a more issues when switching over to Samba 4 I guess, so when doing the migration from Samba 3 to Samba 4, it is a good time to switch over to LDAP.

My personal suggestion is as follows:
- All of us will run Samba 3 for some time, I guess, and it will take some months until Samba 4 comes out and until anyone will switch its production machines over.

- The pdb_sql modules work for us now, otherwise we wouldn't use it. :-)

- I don't have the necessity of having all pdb_ldap features available in pdb_sql. It is okay to have the current functionality working.

- Samba 3.0.x will work fine with pdb_ldap, but Samba 3.2 won't have the module anymore.

So, actually, there two issues/tasks I would like to address:

- FIXING BUGS: Let's fix the current bugs in the pdb_sql module. There was one bug with re-starting SQL server and losing connection, IIRC. Maybe even more.

- KEEPING UP THE MODULE FOR SAMBA 3.X CODELINE: Maybe the module (the diff from svn) will work with 3.2 flawlessly, maybe we experience the same issues we had with 3.0.21a - corrupted password fields. We then could try to fix it our own. Volker told me it is not so hard to do, we just have to check what changes have been in the pdb module and adapt it.

With these two things, we could have a SQL module working (with no new functionality) in Samba 3.x and then do the LDAP migration for Samba 4.

What do you guys think, would you be interested in something like that? Or do you switch over to Samba3+LDAP instead of waiting for Samba4?
Comment 18 Mario Lipinski 2006-02-04 09:46:58 UTC
Actually, the user/authentication stuff is module based, that means for me that there is a interface that is well defined and where can i write modules for. Will there be some kind of module structure in Samba 4?

Why don't just take out the pdb_*sql stuff out of the Samba codebase and develop it somewhere else? Is there a -dev package that contains the necessary header files so the modules can be built without the whole samba source?
So the Samba team does not have to care about the maintainance of this stuff and it can be developed further. I am looking forward to do some things with the mysql backend soon.
Comment 19 Collen Blijenberg 2006-02-14 04:27:31 UTC
Hmm i must say, we've been using the mysql backend from the start
and it was working great for us (till version 3.0.14)
we have several sub system's attatched to the mysql database (account printing, intranet, ftp ect ect)
so you'll see, that just a simple backend switch isn't simple..!!

we find it a big bummer that the mysql backend is going to be removed.
just because the lack of programmers...!??? (lame argument/excuse)
several people and organisations rely on these backends.!
i mean, samba3 started with all these backend, users made there choice, and now they kinda forced to switch to ldap...!

waiting for samba 4 is not the isue here, since this discussion is about samba 3

most of us will be ok, with a stable mysql backend, ok so there are not as manny functions as with the ldap... we have to take that for granted.
it still will be cheaper to stick with mysql, that migrating to ldap!

I'm just one of the users, not the maintainer. withou anny free spare time.. 
(sorry)

my guesses are, that the best thing is to remove the mysql_backend and keep it as a seperate download/module (as well as the other passwd modules!)
so that you have 1) samba and you need 2) a passwd backend..
developing it some were else, isn't a good idea, just leave it with samba, but as a differend tree/project ??

Greets, Collen
just giving my opinion..
Comment 20 Florian Effenberger 2006-02-14 04:34:20 UTC
I will create a SourceForge project for this module and upload the latest version available, so everyone can start developing on it. I'll let you know the URL as soon as the project has been approved by SF.net.

However, what we need is programmers. That a lot of people (including me ;-) are saying, yes, we would continue using this module and yes, we would help
Comment 21 Mario Lipinski 2006-02-14 06:15:04 UTC
I am also willing to help. However actually the limiting factor is time. But soon or later i will have come to this, since i decided to stick with MySQL.
Comment 22 Florian Effenberger 2006-02-14 06:25:55 UTC
Sounds good! I'll let you know as soon as the sf.net project is up and running. :-)
Comment 23 Gerald (Jerry) Carter (dead mail address) 2006-02-14 07:10:50 UTC
Florian,  Just to give you a heads that we will be changing 
a lot for 3.0.22.  

Initially, there are probably 3 things to focus on:

(a) getting the build working outside of the Samba source tree,
(b) marking which versions of the pdb_sql code is compatibile 
    with what versions of Samba, and
(c) fix the pdb_sql code to work with the current SAMBA_3_0 tree.

Let me know if you need help with (a).  I can probably point you 
in the right direction.

I would recommend that all discussion be moved to the samba-technical 
mailing list and possibly CC'd to the samba-pdbsql@samba.org address as 
well in case anyone is not subscribed to the tech list.
Comment 24 Koobs 2006-02-15 08:18:30 UTC
Just to get a bearing on where this is, what version of the official samba source contained a working pdb_mysql implementation?

I was under the impression since I first contributed my bug confirmation in bug 2531 that the last working version was 3.0.11.
Comment 25 Florian Effenberger 2006-02-15 13:16:30 UTC
Hi Jerry,

thanks for that information. Does that mean that the module won't work with 3.0.22 anymore? I initially thought that 3.0 line will be supported, and that beginning with 3.2 the module will be dropped, so that at least security fixes will be available for the pdb_mysql users as well?

Support for (a) sounds good! Shall we move that discussion to samba-technical, or will you attach information to this issue? Or shall we meet in IRC?

Thanks!
Florian
Comment 26 Florian Effenberger 2006-02-15 13:20:40 UTC
I have set-up a SourceForge project at

https://sourceforge.net/projects/pdbsql

including three mailing lists. I ask everyone interested in further development of the pdb_sql module to subscribe to samba-technical@samba and pdbsql-discuss@sf
Comment 27 Gerald (Jerry) Carter (dead mail address) 2006-02-15 13:24:14 UTC
Florian,   3.0.22 will be what was propsoed as 3.2.
And yes we should move to the samba-technical ml.  
Comment 28 Florian Effenberger 2006-02-15 16:28:10 UTC
Okay, I will try to sum up some things and get on to samba-technical.

Jerry: Do you already know what in case of security issues with 3.0.21 will happen? Will there be a backport for the .21 series, so pdbsql users can at least run without any security issues? I guess not?
Comment 29 Peter Rindfuss 2009-03-28 10:34:25 UTC
done
Comment 30 pdb_sql maintainers mail alias 2009-03-29 06:48:10 UTC
fixed