Bug 6969 - upgradeprovision from alpha9 not able to upgrade from alpha8
Summary: upgradeprovision from alpha9 not able to upgrade from alpha8
Status: RESOLVED FIXED
Alias: None
Product: Samba 4.0
Classification: Unclassified
Component: Tools (show other bugs)
Version: unspecified
Hardware: x64 Linux
: P3 normal (vote)
Target Milestone: ---
Assignee: Andrew Bartlett
QA Contact: samba4-qa@samba.org
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-12-04 21:50 UTC by Cioris
Modified: 2009-12-17 09:01 UTC (History)
2 users (show)

See Also:


Attachments
proposal fix (1.09 KB, patch)
2009-12-12 09:37 UTC, Matthieu Patou
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Cioris 2009-12-04 21:50:40 UTC
per instructions in upgrading-samba4.txt I run upgradeprovision to upgrade my previos alpha8 installation. The script fails at line 480 because the att 'name' cannot be modified. It should be renamed.

i looked over the code and it seems that we try to rename the att name (lines 460-462), but my understanding is that we rename it twice. First we add a "foo" suffix, but after that we change everything back (second rename).

So finally nothing is changed and after that when we need to modify atts, we include the 'name' too. This one cannot be modified so line 480 fails.
Comment 1 Matthias Dieter Wallnöfer 2009-12-05 03:19:20 UTC
Ekacnet, would you like to comment on this?
Comment 2 Matthieu Patou 2009-12-05 07:15:20 UTC
The thing is that the name attribute can not be change directly. So we loop twice on the change in update_secrets. On the first loop if the script detect that there is a need to update the value of name attribute it will instead rename the object to currentnamefoo and then back to currentname. Normally it should succeed ! That's why when we loop on the second time (line 467-480) the attribute name should have the good value and so it shouldn't be elected for modification.

Comment 3 Matthieu Patou 2009-12-05 07:42:20 UTC
Cioris,

I've just reprovisionned an alpha8  and upgrade it and it worked like a charm.
In order to go further can you do the following:

* ldbedit -H path_to_your_provision/private/secrets.ldb, it should spawn something like vi or whatever is defined in $EDITOR, save the temporary file somewhere (ie. /tmp/secret.dump) 
* You should have a folder called referenceprovisionxxxxxx in  path_to_your_provision/private/, can you do the same as above but for the secrets.ldb in referenceprovision (ie. ldbedit -H path_to_your_provision/private/referenceprovisionxxxxx/private/secrets.ldb)
* can you reexcute upgradeprovision but adding --debugchange 

Can you post the two secret.dump and post the output of upgradeprovision (with --debugchange) in this bug report ?
Comment 4 Andrew Bartlett 2009-12-06 23:00:15 UTC
Do you think we should wait to solve this before we push alpha10?
Comment 5 Matthieu Patou 2009-12-07 01:42:59 UTC
Well, for me it seems that it didn't impact all the alpha8 provisions.
I'm pretty eager to understand what in his configuration makes this error.
I can issue a small fix but it will just mask the problem (if general problem there is) rather than solving it.

Let's wait until the end of the day. 
Comment 6 Cioris 2009-12-08 20:28:25 UTC
Sorry guys,

I was out of town for few days w/ no e-mail access. You make me feel bad. You were so responsive to my problem I was not able to keep up. Soory again.

Here are the results you asked for: (btw, I installed everything in default path /usr/local/samba)

* ldbedit -H /usr/local/samba/private/secrets.ldb returned the following error:
module operational initialization failed
module update_keytab initialization failed
Unable to load modules for /usr/local/samba/private/secrets.ldb: Could not build aggregate schema DN
Failed to connect to /usr/local/samba/private/secrets.ldb - Could not build aggregate schema DN

* ldbedit -H /usr/local/samba/private/referenceprovision5A5M_A/private/secrets.ldb

# editing 12 records
# record 1
dn: @MODULES
@LIST: samba_secrets
distinguishedName: @MODULES

# record 2
dn: @INDEX:REALM:XXX.COM
@IDXVERSION: 2
@IDX: flatname=XXX,cn=Primary Domains
@IDX: servicePrincipalName=DNS/XXX.com,CN=Principals
distinguishedName: @INDEX:REALM:XXX.COM

# record 3
dn: flatname=XXX,cn=Primary Domains
flatname: XXX
objectClass: top
objectClass: primaryDomain
objectClass: kerberosSecret
realm: XXX.COM
saltPrincipal: host/XXXserver.XXX.com@XXX.COM
msDS-KeyVersionNumber: 1
privateKeytab: secrets.keytab
secret: wTfRzUF7o6PO
samAccountName: XXXserver$
secureChannelType: 6
objectSid: S-1-5-21-3314698270-2113363784-1804276535
objectGUID: 3b08e417-eb4b-4b75-b118-74fda54bfc93
whenCreated: 20091209021131.0Z
whenChanged: 20091209021131.0Z
uSNCreated: 7
uSNChanged: 7
name: XXX
distinguishedName: flatname=XXX,cn=Primary Domains

# record 4
dn: @ATTRIBUTES
cn: CASE_INSENSITIVE
realm: CASE_INSENSITIVE
flatname: CASE_INSENSITIVE
sAMAccountName: CASE_INSENSITIVE
distinguishedName: @ATTRIBUTES

# record 5
dn: CN=LSA Secrets
objectClass: top
objectClass: container
cn: LSA Secrets
objectGUID: 7637dfa1-4214-43db-843f-3788e7336870
whenCreated: 20091209021106.0Z
whenChanged: 20091209021106.0Z
uSNCreated: 5
uSNChanged: 5
name: LSA Secrets
distinguishedName: CN=LSA Secrets

# record 6
dn: @BASEINFO
whenChanged: 20091209021131.0Z
sequenceNumber: 8
distinguishedName: @BASEINFO

# record 7
dn: CN=Primary Domains
objectClass: top
objectClass: container
cn: Primary Domains
objectGUID: 11b825a6-1876-452a-99a3-460747fe673b
whenCreated: 20091209021106.0Z
whenChanged: 20091209021106.0Z
uSNCreated: 6
uSNChanged: 6
name: Primary Domains
distinguishedName: CN=Primary Domains

# record 8
dn: @INDEX:FLATNAME:XXX
@IDXVERSION: 2
@IDX: flatname=XXX,cn=Primary Domains
distinguishedName: @INDEX:FLATNAME:XXX

# record 9
dn: servicePrincipalName=DNS/XXX.com,CN=Principals
objectClass: top
objectClass: secret
objectClass: kerberosSecret
realm: XXX.COM
servicePrincipalName: DNS/XXX.com
msDS-KeyVersionNumber: 1
privateKeytab: dns.keytab
secret: YCcleNCQ+3Fx
objectGUID: b7bae3bf-17aa-48c2-bd14-85ade2e4f113
whenCreated: 20091209021131.0Z
whenChanged: 20091209021131.0Z
uSNCreated: 8
uSNChanged: 8
name: DNS/XXX.com
distinguishedName: servicePrincipalName=DNS/XXX.com,CN=Principals

# record 10
dn: @INDEX:CN:LSA SECRETS
@IDXVERSION: 2
@IDX: CN=LSA Secrets
distinguishedName: @INDEX:CN:LSA SECRETS

# record 11
dn: @INDEXLIST
@IDXATTR: cn
@IDXATTR: flatname
@IDXATTR: realm
distinguishedName: @INDEXLIST

# record 12
dn: @INDEX:CN:PRIMARY DOMAINS
@IDXVERSION: 2
@IDX: CN=Primary Domains
distinguishedName: @INDEX:CN:PRIMARY DOMAINS



* debug messages for upgradeprovision:
# ./scripting/bin/upgradeprovision -s /usr/local/samba/etc/smb.conf --debugchange
Creating a reference provision
Copy samdb
Update partitions filename if needed
update secrets.ldb
Found attribute name on  flatname=XXX,CN=Primary Domains, must rename the DN
 Adding/Changing attribute flatname to flatname=XXX,CN=Primary Domains
 Adding/Changing attribute name to flatname=XXX,CN=Primary Domains
 Adding/Changing attribute saltPrincipal to flatname=XXX,CN=Primary Domains
Traceback (most recent call last):
  File "./scripting/bin/upgradeprovision", line 765, in <module>
    update_secrets(newpaths,paths,creds,session)
  File "./scripting/bin/upgradeprovision", line 480, in update_secrets
    secrets_ldb.modify(delta);
_ldb.LdbError: (67, "Modify of 'name' on flatname=XXX,CN=Primary Domains not permitted, must use 'rename' operation instead")

I replaced the actual names with XXX/XXXserver
Comment 7 Matthieu Patou 2009-12-09 02:08:09 UTC
Well 
that's a bit weird.
Ok can you retry for the current secrets.ldb with:

ldbedit -H /usr/local/samba/private/secrets.ldb -o modules:samba_secrets

And publish the result here ?
When you replace the names by XXX please pay attention at the case.
I'm guessing that the renaming is not working correctly or completely for you.
When did you provisioned your s4 ? (is it an alpha or a git changeset)?

I tried with an alpha3 and the upgrade worked (with the rename trick).

Are you in hurry to upgrade it ?

Comment 8 Cioris 2009-12-09 21:18:35 UTC
Here's the result:

# editing 13 records
# record 1
dn: @MODULES
@LIST: update_keytab,operational,objectguid,rdn_name
distinguishedName: @MODULES

# record 2
dn: @INDEX:REALM:3XXX.COM
@IDX: servicePrincipalName=DNS/3xxx.com,CN=Principals
@IDX: flatname=3xxx,CN=Primary Domains
@IDX: samAccountName=krbtgt,flatname=3xxx,CN=Principals
distinguishedName: @INDEX:REALM:3XXX.COM

# record 3
dn: flatname=3xxx,CN=Primary Domains
objectClass: top
objectClass: primaryDomain
objectClass: kerberosSecret
flatname: 3xxx
realm: 3XXX.COM
secret: 5RCU6HwFMPWY
secureChannelType: 6
sAMAccountName: XXXSERVERXXX$
msDS-KeyVersionNumber: 1
objectSid: S-1-5-21-3314698270-2113363784-1804276535
privateKeytab: secrets.keytab
objectGUID: 4c561b6b-a98b-4fb6-96d1-1c264c270e36
whenCreated: 20091026013711.0Z
whenChanged: 20091026013711.0Z
uSNCreated: 7
uSNChanged: 7
name: 3xxx
distinguishedName: flatname=3xxx,CN=Primary Domains

# record 4
dn: @ATTRIBUTES
realm: CASE_INSENSITIVE
flatname: CASE_INSENSITIVE
sAMAccountName: CASE_INSENSITIVE
distinguishedName: @ATTRIBUTES

# record 5
dn: samAccountName=krbtgt,flatname=3xxx,CN=Principals
objectClass: top
objectClass: secret
objectClass: kerberosSecret
flatname: 3xxx
realm: 3XXX.COM
sAMAccountName: krbtgt
objectSid: S-1-5-21-3314698270-2113363784-1804276535
servicePrincipalName: kadmin/changepw
krb5Keytab: HDB:samba4:/usr/local/samba/private/sam.ldb:
objectGUID: bd249e05-44bd-470a-8ebb-884a4fc64469
whenCreated: 20091026013711.0Z
whenChanged: 20091026013711.0Z
uSNCreated: 8
uSNChanged: 8
name: krbtgt
distinguishedName: samAccountName=krbtgt,flatname=3xxx,CN=Principals

# record 6
dn: CN=LSA Secrets
objectClass: top
objectClass: container
cn: LSA Secrets
objectGUID: 8a456f52-73ab-4fc5-87f5-1adaae0be4ab
whenCreated: 20091026013602.0Z
whenChanged: 20091026013602.0Z
uSNCreated: 5
uSNChanged: 5
name: LSA Secrets
distinguishedName: CN=LSA Secrets

# record 7
dn: @BASEINFO
sequenceNumber: 9
whenChanged: 20091026013711.0Z
distinguishedName: @BASEINFO

# record 8
dn: CN=Primary Domains
objectClass: top
objectClass: container
cn: Primary Domains
objectGUID: 09aba986-86e9-44f2-8945-be4491f415b9
whenCreated: 20091026013602.0Z
whenChanged: 20091026013602.0Z
uSNCreated: 6
uSNChanged: 6
name: Primary Domains
distinguishedName: CN=Primary Domains

# record 9
dn: @INDEX:FLATNAME:3XXX
@IDX: samAccountName=krbtgt,flatname=3xxx,CN=Principals
@IDX: flatname=3xxx,CN=Primary Domains
distinguishedName: @INDEX:FLATNAME:3XXX

# record 10
dn: servicePrincipalName=DNS/3xxx.com,CN=Principals
objectClass: top
objectClass: secret
objectClass: kerberosSecret
realm: 3XXX.COM
servicePrincipalName: DNS/3xxx.com
msDS-KeyVersionNumber: 1
privateKeytab: dns.keytab
secret: JUQb7ixVgOUa
objectGUID: ee1d6fba-55c9-4fb7-aa14-c448cbc0cbf2
whenCreated: 20091026013711.0Z
whenChanged: 20091026013711.0Z
uSNCreated: 9
uSNChanged: 9
name: DNS/3xxx.com
distinguishedName: servicePrincipalName=DNS/3xxx.com,CN=Principals

# record 11
dn: @INDEX:CN:LSA SECRETS
@IDX: CN=LSA Secrets
distinguishedName: @INDEX:CN:LSA SECRETS

# record 12
dn: @INDEXLIST
@IDXATTR: cn
@IDXATTR: flatname
@IDXATTR: realm
distinguishedName: @INDEXLIST

# record 13
dn: @INDEX:CN:PRIMARY DOMAINS
@IDX: CN=Primary Domains
distinguishedName: @INDEX:CN:PRIMARY DOMAINS

I took care of case. Please be aware that my domain name starts with digit 3 - maybe it is important. I left the digit just to be more clear. Where the name appeared uppercase, it was replaced with XXX; where it was small case it was replaced w/ xxx. At the same time, the servername was replaced with XXXSERVERXXX (the name was big case).

The installation/provisioning was made with a clean alpha8 downloaded from ftp.samba.org. I did not use git.

It would be nice to upgrade to alpha9. What I would like to do is to establish a trust between a samba4 AD and a Win2003 AD. This feature was not working on alpha8 and I understood that it is supposed to work under alpha9.

Do you know who can help me w/ ntlm_auth? I created a bug report but nothing happened. The problem I have is that ntlm_auth doen't work on aplha8. I want squid to authnticate against samba AD, but ntlm_auth returns no results all the time. I know this is a different topic, but maybe you can help... or point me to somebody else able/willing to help..:))).

Thanks,
Comment 9 Matthieu Patou 2009-12-10 16:41:02 UTC
So i get it we have a mismatch because one is in lower case and the other is in uppercase.
dn: flatname=3xxx,CN=Primary Domains

dn: flatname=3xxx,CN=Primary Domains
<snip>
name: 3xxx
distinguishedName: flatname=3xxx,CN=Primary Domains


dn: flatname=XXX,cn=Primary Domains
flatname: XXX
<snip>
name: XXX
distinguishedName: flatname=XXX,cn=Primary Domains

I'll post shortly a patch for you that take care of this problem (that in fact I already met on some other part) hope to do it in a couple of hours
Comment 10 Cioris 2009-12-11 16:05:46 UTC
Please let me kow hen ready.

Thanks.
Comment 11 Matthieu Patou 2009-12-12 09:37:15 UTC
Created attachment 5085 [details]
proposal fix
Comment 12 Matthieu Patou 2009-12-12 09:38:39 UTC
cioris, 
can you try this patch it should fix the problem as you have just a case problem.

You must apply this patch, I guess you'll figure out !

Let us know.
Comment 13 Matthias Dieter Wallnöfer 2009-12-15 08:42:27 UTC
Should be fixed in "master".
Comment 14 Cioris 2009-12-16 20:07:26 UTC
Hi Matthieu,

I tried your fix and it was ALMOST correct..:)))
the new procedure you defined "identic_rename" has a small problem. ldb is received as parameter so it created a conflict. Here is the change I did and everything worked fine:

def identic_rename(ldb_arg,dn):
	(before,sep,after)=str(dn).partition('=')
	ldb_arg.rename(dn,ldb.Dn(ldb_arg,"%s=foo%s"%(before,after)))
	ldb_arg.rename(ldb.Dn(ldb_arg,"%s=foo%s"%(before,after)),dn)

Please review it and let me know.
Thanks

P.S. Any idea how can I fix/debug the other problem I have w/ ntlm_auth?
Comment 15 Matthieu Patou 2009-12-17 02:09:12 UTC
Cioris,

This patch was wrong, I posted it a bit too quickly (read without doing all the tests). You need to take the thoses threes patches: 
http://gitweb.samba.org/samba.git/?p=samba.git;a=commit;h=fb85977d7c62d623997160d0c90fcb0ae83953aa

http://gitweb.samba.org/samba.git/?p=samba.git;a=commit;h=
d2ec67eb53b93c6e70c411ea9e892b4d4dca9ccc

http://gitweb.samba.org/samba.git/?p=samba.git;a=commit;h=

To cleanly fix the problem.
Please try by upgrading to changeset ef5508bc7864dcd7128cf91017407efede7572d2 (or higher) or by cherry picking this 3 patches !


 
Comment 16 Cioris 2009-12-17 09:01:16 UTC
I agree that the 3 patches are ok. In fact my proposal is included in the 3rd patch. It looks more like a "cosmetic" change, but it has something important: makes the difference between the ldb object and the class.

Thanks