> I'm trying to upgrade an old installation of Samba 4 from > 4.0.0alpha12-GIT-77b9b97 to 4.0.0alpha18-GIT-35605fa. > > The build works fine. > > samba-tool dbcheck --fix finds and fixes a bunch of things. The vast majority of the fixes are like this: ERROR: missing GUID component for objectCategory in object CN=System,DC=example,DC=com - CN=Container,CN=Schema,CN=Configuration,DC=example,DC=com Change DN to <GUID=47288117-6896-4151-b2af-4921180a3ccf>;CN=Container,CN=Schema,CN=Configuration,DC=example,DC=com? [YES] Fixed missing GUID on attribute objectCategory and it ends like this: ERROR: dsServiceName not in GUID form in @ROOTDSE Change dsServiceName to GUID form? [y/N] y Changed dsServiceName to GUID form Checked 870 objects (226 errors) If I run it again immediately I get this: # samba-tool dbcheck --fix Checking 870 objects Fix isDeleted originating_change_time on 'CN=Deleted Objects,DC=example,DC=com' [y/N/all/none] y Checked 870 objects (1 errors) # samba-tool dbcheck --fix Checking 870 objects Fix isDeleted originating_change_time on 'CN=Deleted Objects,DC=example,DC=com' [y/N/all/none] y Checked 870 objects (1 errors) So although it asks if I want to fix that error, it can't seem to do it. And with -d10: # samba-tool dbcheck --fix --yes -d10 INFO: Current debug levels: [...] lpcfg_load: refreshing parameters from /usr/local/samba/etc/smb.conf params.c:pm_process() - Processing configuration file "/usr/local/samba/etc/smb.conf" Processing section "[globals]" Processing section "[netlogon]" Processing section "[sysvol]" pm_process() returned Yes Security token SIDs (1): SID[ 0]: S-1-5-18 Privileges (0xFFFFFFFFFFFFFFFF): [...] Rights (0x 0): lpcfg_servicenumber: couldn't find ldb schema_fsmo_init: we are master[yes] updates allowed[no] Checking 870 objects Fix isDeleted originating_change_time on 'CN=Deleted Objects,DC=example,DC=com' [YES] ldb:acl_modify: isDeleted Sorting rpmd with attid exception 3 rDN=CN DN=CN=Deleted Objects,DC=example,DC=com Checked 870 objects (1 errors) > upgradeprovision completes without complaint. This is what it actually says: # upgradeprovision Creating a reference provision No IPv6 address will be assigned Copy privilege Update base samdb by searching difference with reference one You still have the old DNS object for managing dynamic DNS, but you didn't supply --full so a correct update can't be done so no errors, but I'm not sure if it did anything. - Show quoted text - With the following workaround upgradeprovision --full completes, but obviously hasn't fixed everything it should have: --- a/sbin/upgradeprovision +++ b/sbin/upgradeprovision @@ -886,7 +886,7 @@ def checkKeepAttributeWithMetadata(delta, att, message, refe dn = current[0].dn for att in list(delta): - if att in ["dn", "objectSid"]: + if att in ["dn", "objectSid", "msDS-hasMasterNCs"]: delta.remove(att) continue # upgradeprovision --full Creating a reference provision No IPv6 address will be assigned Copy privilege Update base samdb by searching difference with reference one Starting update of samdb There are 76 missing objects Reloading a merged schema, which might trigger reindexing so please be patient Schema reloaded! There are 7 changed objects Update of samdb finished Update of secrets.ldb IMPORTANT!!! If you were using Dynamic DNS before you need to update your configuration, so that the tkey-gssapi-credential has the following value: DNS/samba.example.com Update machine account Some defaultSecurityDescriptors and/orsecurityDescriptor have changed, recalculating SD Unable to set ACLs on policies related objects: an integer is required Upgrade finished! Reopenning samdb to trigger reindexing if needed after modification Reindexing finished Also, the message about not being able to set ACLs on policies related to objects looks like it might be a problem. Running dbcheck again after the upgradeprovision finds a bunch more GUID errors: # samba-tool dbcheck --fix --yes Checking 944 objects ERROR: incorrect GUID component for objectCategory in object DC=l.root-servers.net,DC=RootDNSServers,CN=MicrosoftDNS,DC=DomainDnsZones,DC=example,DC=com - <GUID=aae56a27-1de9-4188-afff-3a05b57aef73>;CN=Dns-Node,CN=Schema,CN=Configuration,DC=example,DC=com Change DN to <GUID=cad206ef-f29f-4d88-8822-4b30cf4aef1b>;CN=Dns-Node,CN=Schema,CN=Configuration,DC=example,DC=com? [YES] Fixed incorrect GUID on attribute objectCategory [...] Checked 944 objects (83 errors) And re-running dbcheck immediately finds another 2 errors (in addition to the one from before) that it doesn't fix: # samba-tool dbcheck --fix --yes Checking 944 objects ERROR: missing GUID component for wellKnownObjects in object DC=DomainDnsZones,DC=example,DC=com - B:32:18E2EA80684F11D2B9AA00C04F79F805:CN=Deleted Objects,DC=DomainDnsZones,DC=example,DC=com unable to find object for DN CN=Deleted Objects,DC=DomainDnsZones,DC=example,DC=com - (No such Base DN: CN=Deleted Objects,DC=DomainDnsZones,DC=example,DC=com) Not removing dangling forward link ERROR: missing GUID component for wellKnownObjects in object DC=ForestDnsZones,DC=example,DC=com - B:32:18E2EA80684F11D2B9AA00C04F79F805:CN=Deleted Objects,DC=ForestDnsZones,DC=example,DC=com unable to find object for DN CN=Deleted Objects,DC=ForestDnsZones,DC=example,DC=com - (No such Base DN: CN=Deleted Objects,DC=ForestDnsZones,DC=example,DC=com) Not removing dangling forward link Fix isDeleted originating_change_time on 'CN=Deleted Objects,DC=example,DC=com' [YES] Checked 944 objects (3 errors)
Reassigning as desired on the technical mailing list.
Where I pasted "- Show quoted text -" in the description of the problem, it should have had the following: > But upgradeprovision --full fails as follows (with some debug print > statements added just before the failure): > > Creating a reference provision > No IPv6 address will be assigned > Copy privilege > Update base samdb by searching difference with reference one > Starting update of samdb > There are 76 missing objects > Reloading a merged schema, which might trigger reindexing so please be patient > Schema reloaded! > dn= CN=NTDS Settings,CN=SAMBA,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=example,DC=com > basedn= DC=example,DC=com > attrs= msDS-hasMasterNCs > res.count == 0 > Exception during upgrade of samdb: > Traceback (most recent call last): > File "/usr/local/samba/sbin/upgradeprovision", line 1205, in update_partition > provisionUSNs) > File "/usr/local/samba/sbin/upgradeprovision", line 1080, in update_present > basedn, usns, samdb) > File "/usr/local/samba/sbin/upgradeprovision", line 904, in > checkKeepAttributeWithMetadata > curval, refval, delta) > File "/usr/local/samba/sbin/upgradeprovision", line 788, in handle_links > for e in res[0][att]: > IndexError: list index out of range > Update failed > Rolling back all changes. Check the cause of the problem > Your system is as it was before the upgrade > > So it seems that this search for CN=NTDS Settings returns nothing: > > res = samdb.search(expression="dn=%s" % dn, base=basedn, > controls=["search_options:1:2", "reveal:1"], > attrs=[att]) > > The strange thing is that if I do the same search from the command > line it works fine: > > # ldbsearch -H private/sam.ldb --controls="search_options:1:2 > reveal:1" "dn=CN=NTDS > Settings,CN=SAMBA,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=example,DC=com" > msDS-hasMasterNCs > # record 1 > dn: CN=NTDS Settings,CN=SAMBA,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=example,DC=com > msDS-hasMasterNCs: CN=Configuration,DC=example,DC=com > msDS-hasMasterNCs: CN=Schema,CN=Configuration,DC=example,DC=com > msDS-hasMasterNCs: DC=example,DC=com
Hello Michael, Can you try to apply the following patch, it should help me to understand what are the difference. Can you also do a ldbsearch with --show-deleted --reveal on the CN=NTDS ... object for msDS-hasMasterNCs. Can you also open another bug report for the dbcheck not being able to fix problems in the DB ? Thanks.
Created attachment 7218 [details] Add debug to the upgrade provision
(In reply to comment #3) > Hello Michael, > > Can you try to apply the following patch, it should help me to understand what > are the difference. This is what it prints: [...] Reloading a merged schema, which might trigger reindexing so please be patient Schema reloaded! Current provision: CN=Configuration,DC=example,DC=com Current provision: CN=Schema,CN=Configuration,DC=example,DC=com Current provision: DC=example,DC=com Reference provision: CN=Configuration,DC=example,DC=com Reference provision: CN=Schema,CN=Configuration,DC=example,DC=com Reference provision: DC=example,DC=com Reference provision: DC=ForestDnsZones,DC=example,DC=com Reference provision: DC=DomainDnsZones,DC=example,DC=com Exception during upgrade of samdb: [...] > Can you also do a ldbsearch with --show-deleted --reveal on the CN=NTDS ... > object for msDS-hasMasterNCs. Perhaps what you're looking for here is in comment 3? This returns nothing: # ldbsearch -H private/sam.ldb --show-deleted --reveal "CN=NTDS Settings" msDS-hasMasterNCs # Referral ref: ldap://example.com/CN=Configuration,DC=example,DC=com # returned 1 records # 0 entries # 1 referrals but using --controls="search_options:1:2" shows it as in comment 3: dn: CN=NTDS Settings,CN=SAMBA,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=example,DC=com msDS-hasMasterNCs: CN=Configuration,DC=example,DC=com msDS-hasMasterNCs: CN=Schema,CN=Configuration,DC=example,DC=com msDS-hasMasterNCs: DC=example,DC=com > Can you also open another bug report for the dbcheck not being able to fix > problems in the DB ? OK thanks.
A few days ago I updated to e39df67669f61056692736db9c8dc16fbf2c3624 and the problem was still there (of course). I have now added the debugging patch (attachment 7218 [details]) and also these three commits: f66ef5cfbc932dc03a5bea61e9cb10dd8d948128 f05edc0ecb9da2cb00a83b38d0be5812cc4ccf77 3213d1e0b770690b1a964f38fb57ebbcd8ce0746 The upgradeprovision no longer gets stuck on the NTDS Settings. It also did not print the debugging info like it did in comment 5. Does this mean that this issue is likely resolved now? I notice that it still complains about not being able to set ACLs on policy-related objects. Should I open another bug report about that? Creating a reference provision No IPv6 address will be assigned Copy privilege Update base samdb by searching difference with reference one Starting update of samdb There are 76 missing objects Reloading a merged schema, which might trigger reindexing so please be patient Schema reloaded! There are 3 changed objects Update of samdb finished Update of secrets.ldb IMPORTANT!!! If you were using Dynamic DNS before you need to update your configuration, so that the tkey-gssapi-credential has the following value: DNS/samba.example.com Update machine account Some defaultSecurityDescriptors and/orsecurityDescriptor have changed, recalculating SD Unable to set ACLs on policies related objects: an integer is required Upgrade finished! Reopenning samdb to trigger reindexing if needed after modification Reindexing finished Incidentally, if I run dbcheck after this it again fixes a large number of issues, but now has 3 issues it is apparently unable to fix instead of just 1.
(In reply to comment #6) > A few days ago I updated to e39df67669f61056692736db9c8dc16fbf2c3624 and the > problem was still there (of course). I have now added the debugging patch > (attachment 7218 [details]) and also these three commits: > > f66ef5cfbc932dc03a5bea61e9cb10dd8d948128 > f05edc0ecb9da2cb00a83b38d0be5812cc4ccf77 > 3213d1e0b770690b1a964f38fb57ebbcd8ce0746 > > The upgradeprovision no longer gets stuck on the NTDS Settings. It also did > not print the debugging info like it did in comment 5. Does this mean that > this issue is likely resolved now? > Yes and no, There was a 2 bugs, one easy and one not so easy. The thing is that there is an issue with DNS container not being created. So I would not recommend using it for the moment. > I notice that it still complains about not being able to set ACLs on > policy-related objects. Should I open another bug report about that? > Well I'll try to have a quick look it seems that you provided me some info so I could slip the change pretty quickly in master tree. > Incidentally, if I run dbcheck after this it again fixes a large number of > issues, but now has 3 issues it is apparently unable to fix instead of just 1. What are those issues did you open a bug for it ?
(In reply to comment #7) > > The upgradeprovision no longer gets stuck on the NTDS Settings. It also did > > not print the debugging info like it did in comment 5. Does this mean that > > this issue is likely resolved now? > > > Yes and no, There was a 2 bugs, one easy and one not so easy. > The thing is that there is an issue with DNS container not being created. > > So I would not recommend using it for the moment. OK. > > I notice that it still complains about not being able to set ACLs on > > policy-related objects. Should I open another bug report about that? > > > Well I'll try to have a quick look it seems that you provided me some info so I > could slip the change pretty quickly in master tree. Thanks. > > Incidentally, if I run dbcheck after this it again fixes a large number of > > issues, but now has 3 issues it is apparently unable to fix instead of just 1. > What are those issues did you open a bug for it ? I added them to the existing bug report for dbcheck not being able to fix the other issue, although in this case it doesn't seem to try to fix it. (bug 8683) Here's what I get: # samba-tool dbcheck --fix --yes Checking 944 objects ERROR: missing GUID component for wellKnownObjects in object DC=DomainDnsZones,DC=example,DC=com - B:32:18E2EA80684F11D2B9AA00C04F79F805:CN=Deleted Objects,DC=DomainDnsZones,DC=example,DC=com unable to find object for DN CN=Deleted Objects,DC=DomainDnsZones,DC=example,DC=com - (No such Base DN: CN=Deleted Objects,DC=DomainDnsZones,DC=example,DC=com) Not removing dangling forward link ERROR: missing GUID component for wellKnownObjects in object DC=ForestDnsZones,DC=example,DC=com - B:32:18E2EA80684F11D2B9AA00C04F79F805:CN=Deleted Objects,DC=ForestDnsZones,DC=example,DC=com unable to find object for DN CN=Deleted Objects,DC=ForestDnsZones,DC=example,DC=com - (No such Base DN: CN=Deleted Objects,DC=ForestDnsZones,DC=example,DC=com) Not removing dangling forward link Fix isDeleted originating_change_time on 'CN=Deleted Objects,DC=example,DC=com' [YES] Checked 944 objects (3 errors) so perhaps this is related to not creating the DNS containers you mentioned above.
Created attachment 7230 [details] upgradeprovision --full --debugall
The original provision is of course very old and uses bind9 flat files instead of DLZ or the built-in name server. I suppose upgradeprovision should just upgrade and keep the flat file DNS scheme. I would like to migrate to DLZ, but would understand if that was a separate process or if you needed to use something like "upgradeprovision --migrate-dns-to-dlz". Would it be simplest now to get upgradeprovision to check if the old provision is using flat files/DLZ and upgrade to the same method?
Still an issue?
(In reply to comment #11) > Still an issue? Well, I haven't used upgradeprovision again because as far as I know it's still broken. But it seems that the issue with NTDS Settings was resolved as per comment 6, so maybe this bug should be closed.
I mark this as fixed, feel free to reopen if needed.