Bug 7530 - Provisioning fails with fedora-ds backend
Summary: Provisioning fails with fedora-ds backend
Status: RESOLVED FIXED
Alias: None
Product: Samba 4.0
Classification: Unclassified
Component: AD: LDB/DSDB/SAMDB (show other bugs)
Version: unspecified
Hardware: x86 Linux
: P3 normal (vote)
Target Milestone: ---
Assignee: Andrew Bartlett
QA Contact: samba4-qa@samba.org
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-06-21 16:59 UTC by Attila Bogar
Modified: 2010-10-31 12:51 UTC (History)
2 users (show)

See Also:


Attachments
0001-s4-ldb-ldb_msg_el_map_remote-should-rename-the-remot.patch (1.61 KB, patch)
2010-06-23 17:01 UTC, Endi Sukma Dewata
no flags Details
0001-s4-auth-Fixed-authsam_expand_nested_groups-to-find-e.patch (1.33 KB, patch)
2010-06-28 11:18 UTC, Endi Sukma Dewata
no flags Details
0001-s4-dsdb-Fixed-partition_search-not-to-pass-special-D.patch (1.24 KB, patch)
2010-06-28 11:26 UTC, Endi Sukma Dewata
no flags Details
0001-s4-spnupdate-Fixed-spnupdate-to-use-secrets-credenti.patch (2.74 KB, patch)
2010-06-28 11:28 UTC, Endi Sukma Dewata
no flags Details
0001-s4-libcli-Register-LDB_CONTROL_REVEAL_INTERNALS-and-.patch (1.47 KB, patch)
2010-06-28 11:30 UTC, Endi Sukma Dewata
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Attila Bogar 2010-06-21 16:59:48 UTC
Using Fedora 13 (i686) with updates applied and samba4 compiled from the git master.

[root@fed samba]# rpm -qa | grep 389
389-ds-base-1.2.6-0.1.a1.fc13.i686

Samba's smb.conf a'la http://freeipa.org/page/Samba_4_Configuration

[root@fed samba]# ./sbin/provision --server-role="domain controller" \
        --domain="TEST" --realm=AD.TEST.LOCAL \
        --ldap-backend-type=fedora-ds \
        --root=root --adminpass=Secret123 --ldapadminpass=Secret123 \
        --host-name=ad --host-ip=127.0.0.1 \
        --slapd-path=/usr/sbin/ns-slapd --setup-ds-path=/usr/sbin/setup-ds.pl 

Converted 58 records with 0 failures
Warning: using root as the server user id.  You are strongly encouraged to use a non-root user.
Your new DS instance 'ad' was successfully created.
Exiting . . .
Log file is '/tmp/setupges4Nz.log'

importing data ...
[21/Jun/2010:22:41:17 +0100] - Information: Non-Secure Port Disabled
[21/Jun/2010:22:41:17 +0100] - Backend Instance: sambaData
[21/Jun/2010:22:41:18 +0100] - dblayer_instance_start: pagesize: 4096, pages: 305246, procpages: 9720
[21/Jun/2010:22:41:18 +0100] - cache autosizing: import cache: 204800k 
[21/Jun/2010:22:41:18 +0100] - li_import_cache_autosize: 50, import_pages: 51200, pagesize: 4096
[21/Jun/2010:22:41:18 +0100] - WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database
[21/Jun/2010:22:41:18 +0100] - dblayer_instance_start: pagesize: 4096, pages: 305246, procpages: 9720
[21/Jun/2010:22:41:18 +0100] - cache autosizing: import cache: 204800k 
[21/Jun/2010:22:41:18 +0100] - li_import_cache_autosize: 50, import_pages: 51200, pagesize: 4096
[21/Jun/2010:22:41:18 +0100] - import sambaData: Beginning import job...
[21/Jun/2010:22:41:18 +0100] - import sambaData: Index buffering enabled with bucket size 100
[21/Jun/2010:22:41:18 +0100] - import sambaData: Processing file "/usr/local/samba/private/ldap/fedorads-samba.ldif"
[21/Jun/2010:22:41:18 +0100] - import sambaData: Finished scanning file "/usr/local/samba/private/ldap/fedorads-samba.ldif" (4 entries)
[21/Jun/2010:22:41:19 +0100] - import sambaData: Workers finished; cleaning up...
[21/Jun/2010:22:41:19 +0100] - import sambaData: Workers cleaned up.
[21/Jun/2010:22:41:19 +0100] - import sambaData: Cleaning up producer thread...
[21/Jun/2010:22:41:19 +0100] - import sambaData: Indexing complete.  Post-processing...
[21/Jun/2010:22:41:19 +0100] - import sambaData: Flushing caches...
[21/Jun/2010:22:41:19 +0100] - import sambaData: Closing files...
[21/Jun/2010:22:41:20 +0100] - All database threads now stopped
[21/Jun/2010:22:41:20 +0100] - import sambaData: Import complete.  Processed 4 entries in 2 seconds. (2.00 entries/sec)
Failed to bind - LDAP client internal error: NT_STATUS_UNEXPECTED_NETWORK_ERROR
Failed to connect to 'ldapi://%2Fusr%2Flocal%2Fsamba%2Fprivate%2Fldap%2Fldapi'
[21/Jun/2010:22:41:20 +0100] - Information: Non-Secure Port Disabled
[21/Jun/2010:22:41:20 +0100] - 389-Directory/1.2.6.a1 B2010.025.182 starting up
Failed to bind - LDAP client internal error: NT_STATUS_UNEXPECTED_NETWORK_ERROR
Failed to connect to 'ldapi://%2Fusr%2Flocal%2Fsamba%2Fprivate%2Fldap%2Fldapi'
[21/Jun/2010:22:41:21 +0100] - I'm resizing my cache now...cache was 209715200 and is now 8000000
Failed to bind - LDAP client internal error: NT_STATUS_UNEXPECTED_NETWORK_ERROR
Failed to connect to 'ldapi://%2Fusr%2Flocal%2Fsamba%2Fprivate%2Fldap%2Fldapi'
Failed to bind - LDAP client internal error: NT_STATUS_UNEXPECTED_NETWORK_ERROR
Failed to connect to 'ldapi://%2Fusr%2Flocal%2Fsamba%2Fprivate%2Fldap%2Fldapi'
[21/Jun/2010:22:41:23 +0100] - Listening on /usr/local/samba/private/ldap/ldapi for LDAPI requests
Setting up secrets.ldb
Setting up the registry
Setting up the privileges database
Setting up idmap db
Setting up SAM db
Setting up sam.ldb partitions and settings
Setting up sam.ldb rootDSE
Pre-loading the Samba 4 and AD schema
Adding DomainDN: DC=ad,DC=test,DC=local
pdc_fsmo_init: no domain object present: (skip loading of domain details)

Traceback (most recent call last):
  File "./sbin/provision", line 256, in <module>
    useeadb=eadb)
  File "/usr/local/samba/lib/python2.6/site-packages/samba/provision.py", line 1433, in provision
    dom_for_fun_level=dom_for_fun_level, am_rodc=am_rodc)
  File "/usr/local/samba/lib/python2.6/site-packages/samba/provision.py", line 1025, in setup_samdb
    "DESCRIPTOR": descr
  File "/usr/local/samba/lib/python2.6/site-packages/samba/provision.py", line 345, in setup_add_ldif
    ldb.add_ldif(data, controls)
  File "/usr/local/samba/lib/python2.6/site-packages/samba/__init__.py", line 217, in add_ldif
    self.add(msg,controls)
_ldb.LdbError: (1, 'Operations error (1)')

It seems to me, that it is trying to bind before 389-ds actually started listening on the socket.

Although this issue shows similarities to bug #7040 - the patches for bug #7040 have been incorporated into the git tree.
Comment 1 Endi Sukma Dewata 2010-06-23 16:45:16 UTC
Hi Attila,

I've been investigating this issue. The bind failure that you see is not a problem, it's the provisioning tool trying to see if the backend server is running. The real problem is with the operations error you see at the bottom. It is caused by several things. This is what I found so far:

1. The ldb_msg_el_map_remote() does not rename the remote attribute names into local names as defined in simple_ldap_map.c. I will submit a patch for this.

2. There is an attribute type definition missing from the FDS's core schema. See https://bugzilla.redhat.com/show_bug.cgi?id=606545. A patch is already provided.

With the above problems fixed, the provisioning works fine, but there is still a problem running Samba with FDS backend which I'm still investigating.
Comment 2 Endi Sukma Dewata 2010-06-23 17:01:59 UTC
Created attachment 5804 [details]
0001-s4-ldb-ldb_msg_el_map_remote-should-rename-the-remot.patch

This patch fixes problem #1 described earlier. The ldb_msg_el_merge_wildcard() is using ldb_msg_el_map_remote() to copy all remote attributes and hoping that they will be overwritten by more specific rules later on. The problem is ldb_msg_el_map_remote() will use the remote attribute name. The function has been fixed so it will search and use the local attribute name as defined in simple_ldap_map.c.
Comment 3 Matthias Dieter Wallnöfer 2010-06-24 04:34:27 UTC
Comment on attachment 5804 [details]
0001-s4-ldb-ldb_msg_el_map_remote-should-rename-the-remot.patch

Endi, thanks for the patch, it is applied.

Please tell us when also the other issue has been fixed - so I can close the bug
Comment 4 Attila Bogar 2010-06-24 04:58:50 UTC
(In reply to comment #3)
> 
> Please tell us when also the other issue has been fixed - so I can close the
> bug

BTW, I've also tested with OpenLDAP backend on Fedora13.  The quicktest is failing.  Will test with the patch included.
Comment 5 Endi Sukma Dewata 2010-06-28 11:18:48 UTC
Created attachment 5809 [details]
0001-s4-auth-Fixed-authsam_expand_nested_groups-to-find-e.patch

When an LDAP server is used as a backend, the DN in the memberOf attribute will not contain SID, so authsam_expand_nested_groups() will fail. The function has been fixed to perform dsdb_search to find the SID if it's not present in the DN.
Comment 6 Endi Sukma Dewata 2010-06-28 11:26:37 UTC
Created attachment 5810 [details]
0001-s4-dsdb-Fixed-partition_search-not-to-pass-special-D.patch

Some special DN's such as @REPLCHANGED are being passed to the LDAP backend creating some problems. These DN's should only be handled by the main partition.
Comment 7 Endi Sukma Dewata 2010-06-28 11:28:37 UTC
Created attachment 5811 [details]
0001-s4-spnupdate-Fixed-spnupdate-to-use-secrets-credenti.patch

The samba_spnupdate needs to use the credentials from the secrets database to access SamDB.
Comment 8 Endi Sukma Dewata 2010-06-28 11:30:17 UTC
Created attachment 5812 [details]
0001-s4-libcli-Register-LDB_CONTROL_REVEAL_INTERNALS-and-.patch

The LDB_CONTROL_REVEAL_INTERNALS and DSDB_CONTROL_PASSWORD_CHANGE_STATUS_OID controls need to be registered.
Comment 9 Matthias Dieter Wallnöfer 2010-06-28 12:21:23 UTC
Endi, can I mark this as "FIXED"?
Comment 10 Andrew Bartlett 2010-06-28 17:43:04 UTC
As mentioned on the list, this will need to be reworked.  The auth and dsdb patches are wrong:

The partitions patch will, as I read it, totally break replication, as
it will remove the search for @REPLCHANGED from being propagated down to
each backend database. (so we know if a particular database needs
replication)

The expand_nested_groups patch will work, but I do not wish us to take
this approach.  The LDAP backend needs to provide, one way or another,
this information - if we start to have fallbacks in the code, we will
duplicate the whole extended DN infrastructure in each caller.  The
OpenLDAP backend provides this by a server-side module, and either
Fedora DS must do the same, or fake it up in a Samba module at the
bottom of the stack. 
Comment 11 Matthias Dieter Wallnöfer 2010-09-11 12:34:24 UTC
How to proceed here?
Comment 12 Matthias Dieter Wallnöfer 2010-10-31 12:51:36 UTC
I think this has been fixed. If not, please reopen!