When a DC was provisioned/joined with a Samba version older than 4.7 is upgraded to 4.7 (or later), it can happen that the garbage collection (dsdb_garbage_collect_tombstones()), triggered periodically by the 'kcc' task of 'samba' or my 'samba-tool domain tombstones expunge' corrupt the linked attributes. This is similar to Bug #13095 - Broken linked attribute handling, but it's not triggered by an originating change. The bug happens in replmd_modify_la_delete() were get_parsed_dns_trusted() generates a sorted array of struct parsed_dn based on the values in old_el->values. If the database doesn't support the sortedLinks compatibleFeatures in the @SAMBA_DSDB record, it's very likely that the array of old_dns is sorted differently than the values in old_el->values. The problem is that struct parsed_dn has just a pointer 'struct ldb_val *v' that points to the corresponding value in old_el->values. Now if vanish_links is true the damage happens here: if (vanish_links) { unsigned j = 0; for (i = 0; i < old_el->num_values; i++) { if (old_dns[i].v != NULL) { old_el->values[j] = *old_dns[i].v; j++; } } old_el->num_values = j; } old_el->values[0] = *old_dns[0].v; can change the value old_dns[1].v is pointing at! That means that some values can get lost while others are stored twice, because the LDB_FLAG_INTERNAL_DISABLE_SINGLE_VALUE_CHECK allows it to be stored.
Created attachment 13914 [details] Simple fix for the problem using a tmp talloc array Here's a fix for the corruption bug, but I may propose a nicer fix for inclusion in master and the effected release branches.
As a temporary solution admins can add "server services = -kcc" to the global section of smb.conf.
Oh, good catch! Thanks for tracking this down!
There is a tombstones-expunge.sh which perhaps should have detected this problem, e.g. samba4.blackbox.tombstones-expunge.release-4-5-0-pre1. It looks like it checks links were deleted but doesn't verify that the DB is in a good/correct state afterwards. (I haven't looked into the test too closely, it was just something Garming and I noticed).
Created attachment 13928 [details] Patches for v4-7-test (part1)
Created attachment 13929 [details] Patches for v4-8-test (part1)
Pushed to autobuild-v4-{7,8}-test.
Pushed to v4-{7,8}-test. Waiting for the dbcheck improvements
Created attachment 13943 [details] Patches for v4-8-test (part2)
Created attachment 13944 [details] Patches for v4-7-test (part2)
Reassigning to Karolin for inclusion in 4.7 and 4.8.
(In reply to Ralph Böhme from comment #11) Pushed to autobuild-v4-{7,8}-test.
(In reply to Karolin Seeger from comment #12) Pushed to both branches. Closing out bug report. Thanks!