Attachment 12700 Details for Bug 12453 – patches for 4.5

[patch] patches for 4.5

patches (text/plain), 161.54 KB, created by Garming Sam on 2016-11-29 02:07:17 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Garming Sam

Created: 2016-11-29 02:07:17 UTC

Size: 161.54 KB

patch

obsolete

>From 0441c14685334f8494346393354807c76c893db8 Mon Sep 17 00:00:00 2001
>From: Garming Sam <garming@catalyst.net.nz>
>Date: Fri, 21 Oct 2016 11:40:51 +1300
>Subject: [PATCH 01/24] tombstones-expunge: Add a test for deleting links to
> recycled objects
>
>Currently this fails because we rely on a GUID DN, which fails to
>resolve in the case that the GUID no longer exists in the database (i.e.
>when that object has been purged after 6 months).
>
>The tests use a made up extended DN built from fred where the GUID has
>been tweaked.
>
>Signed-off-by: Garming Sam <garming@catalyst.net.nz>
>Reviewed-by: Andrew Bartlett <abartlet@samba.org>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12385
>(cherry picked from commit dba624364cde1c885640386c4e2bca17e9d5297c)
>---
> selftest/knownfail                                     |  1 +
> .../release-4-5-0-pre1/add-dangling-link.ldif          |  5 +++++
> .../release-4-5-0-pre1/expected-expunge-output.txt     |  2 +-
> .../release-4-5-0-pre1/expected-match-rule-links.ldif  | 18 +++++++++++-------
> testprogs/blackbox/tombstones-expunge.sh               |  9 +++++++++
> 5 files changed, 27 insertions(+), 8 deletions(-)
> create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/add-dangling-link.ldif
>
>diff --git a/selftest/knownfail b/selftest/knownfail
>index c1899da..1b543b9 100644
>--- a/selftest/knownfail
>+++ b/selftest/knownfail
>@@ -292,3 +292,4 @@
> #ntvfs server blocks copychunk with execute access on read handle
> ^samba4.smb2.ioctl.copy_chunk_bad_access
> ^samba4.drs.getnc_exop.python.*getnc_exop.DrsReplicaPrefixMapTestCase.test_regular_prefix_map_ex_attid.*
>+^samba4.blackbox.tombstones-expunge.release-4-5-0-pre1.tombstones_expunge
>diff --git a/source4/selftest/provisions/release-4-5-0-pre1/add-dangling-link.ldif b/source4/selftest/provisions/release-4-5-0-pre1/add-dangling-link.ldif
>new file mode 100644
>index 0000000..67a294d
>--- /dev/null
>+++ b/source4/selftest/provisions/release-4-5-0-pre1/add-dangling-link.ldif
>@@ -0,0 +1,5 @@
>+# fred-clone is a duplication of CN=fred,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp with the GUID slightly modified and a different DN
>+dn: CN=Domain Users,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+changetype: modify
>+add: member
>+member: <GUID=2302a65c-5b43-4ca9-850e-12d4a712cfb5>;<RMD_ADDTIME=131116485990000000>;<RMD_CHANGETIME=131116485990000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3745>;<RMD_ORIGINATING_USN=3745>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1111>;CN=fred-clone,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>diff --git a/source4/selftest/provisions/release-4-5-0-pre1/expected-expunge-output.txt b/source4/selftest/provisions/release-4-5-0-pre1/expected-expunge-output.txt
>index bcc5955..6826257 100644
>--- a/source4/selftest/provisions/release-4-5-0-pre1/expected-expunge-output.txt
>+++ b/source4/selftest/provisions/release-4-5-0-pre1/expected-expunge-output.txt
>@@ -1 +1 @@
>-Removed 7 objects and 1 links successfully
>+Removed 7 objects and 2 links successfully
>diff --git a/source4/selftest/provisions/release-4-5-0-pre1/expected-match-rule-links.ldif b/source4/selftest/provisions/release-4-5-0-pre1/expected-match-rule-links.ldif
>index 2b2f021..1553c1b 100644
>--- a/source4/selftest/provisions/release-4-5-0-pre1/expected-match-rule-links.ldif
>+++ b/source4/selftest/provisions/release-4-5-0-pre1/expected-match-rule-links.ldif
>@@ -4,31 +4,35 @@ member: CN=fred,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=user1,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> 
> # record 2
>+dn: CN=Domain Users,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+member: CN=fred-clone,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# record 3
> dn: CN=ddg\0ADEL:fb8c2fe3-5448-43de-99f9-e1d3b9357cfc,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=User UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=User1 UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> 
>-# record 3
>+# record 4
> dn: CN=dsg\0ADEL:6d66d0ef-cad7-4e5d-b1b6-4a233a21c269,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=User UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=User1 UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> 
>-# record 4
>+# record 5
> dn: CN=gdg\0ADEL:e0f581e7-14ee-4fc2-839c-8f46f581c72a,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=User UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=User1 UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> 
>-# record 5
>+# record 6
> dn: CN=gsg\0ADEL:91aa85cc-fc19-4b8c-9fc7-aaba425439c7,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=User UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=User1 UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> 
>-# record 6
>+# record 7
> dn: CN=udg\0ADEL:7cff5537-51b1-4d26-a295-0225dbea8525,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=User UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=User1 UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> 
>-# record 7
>+# record 8
> dn: CN=usg\0ADEL:d012e8f5-a4bd-40ea-a2a1-68ff2508847d,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=User UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
> member: CN=User1 UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>@@ -42,6 +46,6 @@ ref: ldap:///DC=DomainDnsZones,DC=release-4-5-0-pre1,DC=samba,DC=corp
> # Referral
> ref: ldap:///DC=ForestDnsZones,DC=release-4-5-0-pre1,DC=samba,DC=corp
> 
>-# returned 10 records
>-# 7 entries
>+# returned 11 records
>+# 8 entries
> # 3 referrals
>diff --git a/testprogs/blackbox/tombstones-expunge.sh b/testprogs/blackbox/tombstones-expunge.sh
>index 49a5073..33cb0b1 100755
>--- a/testprogs/blackbox/tombstones-expunge.sh
>+++ b/testprogs/blackbox/tombstones-expunge.sh
>@@ -68,6 +68,14 @@ tombstones_expunge() {
>     fi
> }
> 
>+add_dangling_link() {
>+    ldif=$release_dir/add-dangling-link.ldif
>+    TZ=UTC $ldbmodify -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb.d/DC%3DRELEASE-4-5-0-PRE1,DC%3DSAMBA,DC%3DCORP.ldb $ldif
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+}
>+
> add_two_more_users() {
>     ldif=$release_dir/add-two-more-users.ldif
>     TZ=UTC $ldbadd -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb $ldif
>@@ -172,6 +180,7 @@ if [ -d $release_dir ]; then
>     testit $RELEASE undump
>     testit "add_two_more_users" add_two_more_users
>     testit "add_four_more_links" add_four_more_links
>+    testit "add_dangling_link" add_dangling_link
>     testit "remove_one_link" remove_one_link
>     testit "remove_one_user" remove_one_user
>     testit "check_match_rule_links" check_match_rule_links
>-- 
>1.9.1
>
>
>From 7506086073f40acb17af7e98f225ee9354bbf0fa Mon Sep 17 00:00:00 2001
>From: Garming Sam <garming@catalyst.net.nz>
>Date: Fri, 21 Oct 2016 15:50:09 +1300
>Subject: [PATCH 02/24] collect_tombstones: Allow links to recycled objects to
> be deleted
>
>The reason we choose to provide the string DN is because extended_dn_in
>will try to correct the <GUID=...> by searching on it (despite the fact
>it does not exist and then failing on a ldb_dn_validate in
>objectclass_attrs).
>
>We can now also remove the dangling link test from the knownfail.
>
>Signed-off-by: Garming Sam <garming@catalyst.net.nz>
>Reviewed-by: Andrew Bartlett <abartlet@samba.org>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12385
>
>Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
>Autobuild-Date(master): Thu Nov  3 01:46:43 CET 2016 on sn-devel-144
>
>(cherry picked from commit ef7e46d68a6596be6e904caaa04e917c576dd9d3)
>---
> selftest/knownfail                            | 1 -
> source4/dsdb/kcc/garbage_collect_tombstones.c | 5 +++--
> 2 files changed, 3 insertions(+), 3 deletions(-)
>
>diff --git a/selftest/knownfail b/selftest/knownfail
>index 1b543b9..c1899da 100644
>--- a/selftest/knownfail
>+++ b/selftest/knownfail
>@@ -292,4 +292,3 @@
> #ntvfs server blocks copychunk with execute access on read handle
> ^samba4.smb2.ioctl.copy_chunk_bad_access
> ^samba4.drs.getnc_exop.python.*getnc_exop.DrsReplicaPrefixMapTestCase.test_regular_prefix_map_ex_attid.*
>-^samba4.blackbox.tombstones-expunge.release-4-5-0-pre1.tombstones_expunge
>diff --git a/source4/dsdb/kcc/garbage_collect_tombstones.c b/source4/dsdb/kcc/garbage_collect_tombstones.c
>index ad14d5e..1909cfe 100644
>--- a/source4/dsdb/kcc/garbage_collect_tombstones.c
>+++ b/source4/dsdb/kcc/garbage_collect_tombstones.c
>@@ -193,8 +193,9 @@ static NTSTATUS garbage_collect_tombstones_part(TALLOC_CTX *mem_ctx,
> 
> 				guid_buf_str = GUID_buf_string(&guid, &buf_guid);
> 				guid_search_str = talloc_asprintf(mem_ctx,
>-								  "<GUID=%s>",
>-								  guid_buf_str);
>+								  "<GUID=%s>;%s",
>+								  guid_buf_str,
>+								  dsdb_dn_get_linearized(mem_ctx, dn));
> 				cleanup_val = data_blob_string_const(guid_search_str);
> 
> 				talloc_free(dn);
>-- 
>1.9.1
>
>
>From 7298a5efb21a67734fc429aea4bb2af1c8177dbc Mon Sep 17 00:00:00 2001
>From: Garming Sam <garming@catalyst.net.nz>
>Date: Mon, 21 Nov 2016 15:06:22 +1300
>Subject: [PATCH 03/24] upgradeprovision: Remove objectCategory from
> constructed attrs
>
>The new dbcheck rules identify an error where the GUID of the
>objectCategory does not exist (pointing to a non-existent schema
>object). As objectClass was not copied over either, it makes sense not
>to copy over the objectCategory.
>
>Signed-off-by: Garming Sam <garming@catalyst.net.nz>
>Reviewed-by: Andrew Bartlett <abartlet@samba.org>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12297
>(cherry picked from commit 5889f399daad54124e0bb2be1fe81da1df67c84e)
>---
> source4/scripting/bin/samba_upgradeprovision | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/source4/scripting/bin/samba_upgradeprovision b/source4/scripting/bin/samba_upgradeprovision
>index bc6e36a..ccfc578 100755
>--- a/source4/scripting/bin/samba_upgradeprovision
>+++ b/source4/scripting/bin/samba_upgradeprovision
>@@ -94,7 +94,7 @@ __docformat__ = "restructuredText"
> # created
> # This also apply to imported object from reference provision
> replAttrNotCopied = [   "dn", "whenCreated", "whenChanged", "objectGUID",
>-                        "parentGUID", "objectCategory", "distinguishedName",
>+                        "parentGUID", "distinguishedName",
>                         "instanceType", "cn",
>                         "lmPwdHistory", "pwdLastSet", "ntPwdHistory",
>                         "unicodePwd", "dBCSPwd", "supplementalCredentials",
>-- 
>1.9.1
>
>
>From 8c254c69b65ea4afda14a9ffd2b81b3c6ac6052b Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Mon, 7 Nov 2016 11:39:53 +1300
>Subject: [PATCH 04/24] selftest: Add test for link and deleted link behaviour
> in dbcheck
>
>The other dbcheck tests were getting over-complex, so we start a new test
>here based on tombestone-expunge.sh, as we are looking at very similar
>problems
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12297
>(cherry picked from commit 8315d4d03ac77f1727ff01e87392f6e49ba40def)
>---
> selftest/knownfail                                 |   3 +
> selftest/tests.py                                  |   5 +
> .../expected-dbcheck-link-output.txt               |  42 +++++
> .../expected-deleted-links-after-link-dbcheck.ldif |  23 +++
> .../expected-links-after-link-dbcheck.ldif         |  22 +++
> .../expected-objects-after-link-dbcheck.ldif       |   5 +
> testprogs/blackbox/dbcheck-links.sh                | 188 +++++++++++++++++++++
> 7 files changed, 288 insertions(+)
> create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output.txt
> create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-deleted-links-after-link-dbcheck.ldif
> create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-links-after-link-dbcheck.ldif
> create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-objects-after-link-dbcheck.ldif
> create mode 100755 testprogs/blackbox/dbcheck-links.sh
>
>diff --git a/selftest/knownfail b/selftest/knownfail
>index c1899da..ccddab8 100644
>--- a/selftest/knownfail
>+++ b/selftest/knownfail
>@@ -292,3 +292,6 @@
> #ntvfs server blocks copychunk with execute access on read handle
> ^samba4.smb2.ioctl.copy_chunk_bad_access
> ^samba4.drs.getnc_exop.python.*getnc_exop.DrsReplicaPrefixMapTestCase.test_regular_prefix_map_ex_attid.*
>+^samba4.blackbox.dbcheck-links.release-4-5-0-pre1.dbcheck
>+^samba4.blackbox.dbcheck-links.release-4-5-0-pre1.dbcheck_clean
>+^samba4.blackbox.dbcheck-links.release-4-5-0-pre1.check_expected_after_deleted_links
>diff --git a/selftest/tests.py b/selftest/tests.py
>index 019784c..46ca88c 100644
>--- a/selftest/tests.py
>+++ b/selftest/tests.py
>@@ -109,6 +109,11 @@ plantestsuite(
>     ["PYTHON=%s" % python,
>      os.path.join(bbdir, "tombstones-expunge.sh"),
>      '$PREFIX_ABS/provision', 'release-4-5-0-pre1', configuration])
>+plantestsuite(
>+    "samba4.blackbox.dbcheck-links.release-4-5-0-pre1", "none",
>+    ["PYTHON=%s" % python,
>+     os.path.join(bbdir, "dbcheck-links.sh"),
>+     '$PREFIX_ABS/provision', 'release-4-5-0-pre1', configuration])
> planpythontestsuite("none", "samba.tests.upgradeprovision")
> planpythontestsuite("none", "samba.tests.xattr")
> planpythontestsuite("none", "samba.tests.ntacls")
>diff --git a/source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output.txt b/source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output.txt
>new file mode 100644
>index 0000000..ccbe0e2
>--- /dev/null
>+++ b/source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output.txt
>@@ -0,0 +1,42 @@
>+Checking 221 objects
>+ERROR: linked attribute 'member' to '<GUID=118943ce-41c2-48cb-a511-b68c6feaa8aa>;<RMD_ADDTIME=131116484670000000>;<RMD_CHANGETIME=131116484700000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3729>;<RMD_ORIGINATING_USN=3729>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1103>;CN=User UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp' is present on deleted object CN=gsg\0ADEL:91aa85cc-fc19-4b8c-9fc7-aaba425439c7,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Remove linked attribute member [YES]
>+Fixed undead forward link member
>+ERROR: linked attribute 'member' to '<GUID=50d78122-17c8-4352-acf0-8f549b5b5b3c>;<RMD_ADDTIME=131116484670000000>;<RMD_CHANGETIME=131116484700000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3729>;<RMD_ORIGINATING_USN=3729>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1104>;CN=User1 UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp' is present on deleted object CN=gsg\0ADEL:91aa85cc-fc19-4b8c-9fc7-aaba425439c7,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Remove linked attribute member [YES]
>+Fixed undead forward link member
>+ERROR: linked attribute 'member' to '<GUID=118943ce-41c2-48cb-a511-b68c6feaa8aa>;<RMD_ADDTIME=131116484690000000>;<RMD_CHANGETIME=131116484720000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3733>;<RMD_ORIGINATING_USN=3733>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1103>;CN=User UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp' is present on deleted object CN=udg\0ADEL:7cff5537-51b1-4d26-a295-0225dbea8525,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Remove linked attribute member [YES]
>+Fixed undead forward link member
>+ERROR: linked attribute 'member' to '<GUID=50d78122-17c8-4352-acf0-8f549b5b5b3c>;<RMD_ADDTIME=131116484690000000>;<RMD_CHANGETIME=131116484720000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3733>;<RMD_ORIGINATING_USN=3733>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1104>;CN=User1 UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp' is present on deleted object CN=udg\0ADEL:7cff5537-51b1-4d26-a295-0225dbea8525,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Remove linked attribute member [YES]
>+Fixed undead forward link member
>+ERROR: target DN is deleted for member in object CN=swimmers,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp - <GUID=2301a64c-5b42-4ca8-851e-12d4a711cfb4>;<RMD_ADDTIME=131116485990000000>;<RMD_CHANGETIME=131116485990000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3745>;<RMD_ORIGINATING_USN=3745>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1111>;CN=fred,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Target GUID points at deleted DN 'CN=fred\\0ADEL:2301a64c-5b42-4ca8-851e-12d4a711cfb4,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp'
>+Remove stale DN link? [YES]
>+Removed deleted DN on attribute member
>+ERROR: linked attribute 'member' to '<GUID=118943ce-41c2-48cb-a511-b68c6feaa8aa>;<RMD_ADDTIME=131116484690000000>;<RMD_CHANGETIME=131116484710000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3732>;<RMD_ORIGINATING_USN=3732>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1103>;CN=User UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp' is present on deleted object CN=gdg\0ADEL:e0f581e7-14ee-4fc2-839c-8f46f581c72a,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Remove linked attribute member [YES]
>+Fixed undead forward link member
>+ERROR: linked attribute 'member' to '<GUID=50d78122-17c8-4352-acf0-8f549b5b5b3c>;<RMD_ADDTIME=131116484690000000>;<RMD_CHANGETIME=131116484710000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3732>;<RMD_ORIGINATING_USN=3732>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1104>;CN=User1 UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp' is present on deleted object CN=gdg\0ADEL:e0f581e7-14ee-4fc2-839c-8f46f581c72a,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Remove linked attribute member [YES]
>+Fixed undead forward link member
>+ERROR: linked attribute 'member' to '<GUID=118943ce-41c2-48cb-a511-b68c6feaa8aa>;<RMD_ADDTIME=131116484670000000>;<RMD_CHANGETIME=131116484700000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3728>;<RMD_ORIGINATING_USN=3728>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1103>;CN=User UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp' is present on deleted object CN=dsg\0ADEL:6d66d0ef-cad7-4e5d-b1b6-4a233a21c269,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Remove linked attribute member [YES]
>+Fixed undead forward link member
>+ERROR: linked attribute 'member' to '<GUID=50d78122-17c8-4352-acf0-8f549b5b5b3c>;<RMD_ADDTIME=131116484670000000>;<RMD_CHANGETIME=131116484700000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3728>;<RMD_ORIGINATING_USN=3728>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1104>;CN=User1 UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp' is present on deleted object CN=dsg\0ADEL:6d66d0ef-cad7-4e5d-b1b6-4a233a21c269,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Remove linked attribute member [YES]
>+Fixed undead forward link member
>+ERROR: linked attribute 'member' to '<GUID=118943ce-41c2-48cb-a511-b68c6feaa8aa>;<RMD_ADDTIME=131116484680000000>;<RMD_CHANGETIME=131116484700000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3730>;<RMD_ORIGINATING_USN=3730>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1103>;CN=User UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp' is present on deleted object CN=usg\0ADEL:d012e8f5-a4bd-40ea-a2a1-68ff2508847d,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Remove linked attribute member [YES]
>+Fixed undead forward link member
>+ERROR: linked attribute 'member' to '<GUID=50d78122-17c8-4352-acf0-8f549b5b5b3c>;<RMD_ADDTIME=131116484680000000>;<RMD_CHANGETIME=131116484700000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3730>;<RMD_ORIGINATING_USN=3730>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1104>;CN=User1 UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp' is present on deleted object CN=usg\0ADEL:d012e8f5-a4bd-40ea-a2a1-68ff2508847d,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Remove linked attribute member [YES]
>+Fixed undead forward link member
>+ERROR: linked attribute 'member' to '<GUID=118943ce-41c2-48cb-a511-b68c6feaa8aa>;<RMD_ADDTIME=131116484680000000>;<RMD_CHANGETIME=131116484710000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3731>;<RMD_ORIGINATING_USN=3731>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1103>;CN=User UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp' is present on deleted object CN=ddg\0ADEL:fb8c2fe3-5448-43de-99f9-e1d3b9357cfc,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Remove linked attribute member [YES]
>+Fixed undead forward link member
>+ERROR: linked attribute 'member' to '<GUID=50d78122-17c8-4352-acf0-8f549b5b5b3c>;<RMD_ADDTIME=131116484680000000>;<RMD_CHANGETIME=131116484710000000>;<RMD_FLAGS=1>;<RMD_INVOCID=4e4496a3-7fb8-4f97-8a33-d238db8b5e2d>;<RMD_LOCAL_USN=3731>;<RMD_ORIGINATING_USN=3731>;<RMD_VERSION=1>;<SID=S-1-5-21-4177067393-1453636373-93818738-1104>;CN=User1 UT. Tester,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp' is present on deleted object CN=ddg\0ADEL:fb8c2fe3-5448-43de-99f9-e1d3b9357cfc,CN=Deleted Objects,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+Remove linked attribute member [YES]
>+Fixed undead forward link member
>+Checked 221 objects (13 errors)
>diff --git a/source4/selftest/provisions/release-4-5-0-pre1/expected-deleted-links-after-link-dbcheck.ldif b/source4/selftest/provisions/release-4-5-0-pre1/expected-deleted-links-after-link-dbcheck.ldif
>new file mode 100644
>index 0000000..af09a4f
>--- /dev/null
>+++ b/source4/selftest/provisions/release-4-5-0-pre1/expected-deleted-links-after-link-dbcheck.ldif
>@@ -0,0 +1,23 @@
>+# record 1
>+dn: CN=helpers,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+member: CN=user1,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# record 2
>+dn: CN=leaders,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# record 3
>+dn: CN=swimmers,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+member: CN=user1x,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# Referral
>+ref: ldap:///CN=Configuration,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# Referral
>+ref: ldap:///DC=DomainDnsZones,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# Referral
>+ref: ldap:///DC=ForestDnsZones,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# returned 6 records
>+# 3 entries
>+# 3 referrals
>diff --git a/source4/selftest/provisions/release-4-5-0-pre1/expected-links-after-link-dbcheck.ldif b/source4/selftest/provisions/release-4-5-0-pre1/expected-links-after-link-dbcheck.ldif
>new file mode 100644
>index 0000000..0151acf
>--- /dev/null
>+++ b/source4/selftest/provisions/release-4-5-0-pre1/expected-links-after-link-dbcheck.ldif
>@@ -0,0 +1,22 @@
>+# record 1
>+dn: CN=helpers,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# record 2
>+dn: CN=leaders,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# record 3
>+dn: CN=swimmers,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+member: CN=user1x,CN=Users,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# Referral
>+ref: ldap:///CN=Configuration,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# Referral
>+ref: ldap:///DC=DomainDnsZones,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# Referral
>+ref: ldap:///DC=ForestDnsZones,DC=release-4-5-0-pre1,DC=samba,DC=corp
>+
>+# returned 6 records
>+# 3 entries
>+# 3 referrals
>diff --git a/source4/selftest/provisions/release-4-5-0-pre1/expected-objects-after-link-dbcheck.ldif b/source4/selftest/provisions/release-4-5-0-pre1/expected-objects-after-link-dbcheck.ldif
>new file mode 100644
>index 0000000..18ba914
>--- /dev/null
>+++ b/source4/selftest/provisions/release-4-5-0-pre1/expected-objects-after-link-dbcheck.ldif
>@@ -0,0 +1,5 @@
>+sAMAccountName: user1
>+sAMAccountName: ddg
>+sAMAccountName: usg
>+sAMAccountName: fred
>+sAMAccountName: user2
>diff --git a/testprogs/blackbox/dbcheck-links.sh b/testprogs/blackbox/dbcheck-links.sh
>new file mode 100755
>index 0000000..11592f0
>--- /dev/null
>+++ b/testprogs/blackbox/dbcheck-links.sh
>@@ -0,0 +1,188 @@
>+#!/bin/sh
>+
>+if [ $# -lt 1 ]; then
>+cat <<EOF
>+Usage: dbcheck-links.sh PREFIX RELEASE
>+EOF
>+exit 1;
>+fi
>+
>+PREFIX_ABS="$1"
>+RELEASE="$2"
>+shift 2
>+
>+. `dirname $0`/subunit.sh
>+
>+release_dir=`dirname $0`/../../source4/selftest/provisions/$RELEASE
>+
>+ldbadd="ldbadd"
>+if [ -x "$BINDIR/ldbadd" ]; then
>+    ldbadd="$BINDIR/ldbadd"
>+fi
>+
>+ldbmodify="ldbmodify"
>+if [ -x "$BINDIR/ldbmodify" ]; then
>+    ldbmodify="$BINDIR/ldbmodify"
>+fi
>+
>+ldbdel="ldbdel"
>+if [ -x "$BINDIR/ldbdel" ]; then
>+    ldbdel="$BINDIR/ldbdel"
>+fi
>+
>+ldbsearch="ldbsearch"
>+if [ -x "$BINDIR/ldbsearch" ]; then
>+    ldbsearch="$BINDIR/ldbsearch"
>+fi
>+
>+ldbrename="ldbrename"
>+if [ -x "$BINDIR/ldbrename" ]; then
>+    ldbrename="$BINDIR/ldbrename"
>+fi
>+
>+undump() {
>+       if test -x $BINDIR/tdbrestore;
>+       then
>+	`dirname $0`/../../source4/selftest/provisions/undump.sh $release_dir $PREFIX_ABS/$RELEASE $BINDIR/tdbrestore
>+       else
>+	`dirname $0`/../../source4/selftest/provisions/undump.sh $release_dir $PREFIX_ABS/$RELEASE
>+       fi
>+}
>+
>+dbcheck() {
>+    tmpfile=$PREFIX_ABS/$RELEASE/expected-dbcheck-link-output.txt.tmp
>+    tmpldif1=$PREFIX_ABS/$RELEASE/expected-dbcheck-output2.txt.tmp1
>+
>+    TZ=UTC $ldbsearch -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb -s base -b '' | grep highestCommittedUSN > $tmpldif1
>+
>+    $PYTHON $BINDIR/samba-tool dbcheck -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb --fix --yes > $tmpfile
>+    if [ "$?" != "1" ]; then
>+	return 1
>+    fi
>+    diff $tmpfile $release_dir/expected-dbcheck-link-output.txt
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+
>+    tmpldif2=$PREFIX_ABS/$RELEASE/expected-dbcheck-output2.txt.tmp2
>+    TZ=UTC $ldbsearch -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb -s base -b '' | grep highestCommittedUSN > $tmpldif2
>+
>+    diff $tmpldif1 $tmpldif2
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+}
>+
>+dbcheck_clean() {
>+    tmpldif1=$PREFIX_ABS/$RELEASE/expected-dbcheck-output2.txt.tmp1
>+
>+    TZ=UTC $ldbsearch -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb -s base -b '' | grep highestCommittedUSN > $tmpldif1
>+
>+    $PYTHON $BINDIR/samba-tool dbcheck -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+    tmpldif2=$PREFIX_ABS/$RELEASE/expected-dbcheck-output2.txt.tmp2
>+    TZ=UTC $ldbsearch -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb -s base -b '' | grep highestCommittedUSN > $tmpldif2
>+
>+    diff $tmpldif1 $tmpldif2
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+}
>+
>+add_two_more_users() {
>+    ldif=$release_dir/add-two-more-users.ldif
>+    TZ=UTC $ldbadd -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb $ldif
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+}
>+
>+add_four_more_links() {
>+    ldif=$release_dir/add-four-more-links.ldif
>+    TZ=UTC $ldbmodify -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb $ldif
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+}
>+
>+remove_one_link() {
>+    ldif=$release_dir/remove-one-more-link.ldif
>+    TZ=UTC $ldbmodify -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb $ldif
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+}
>+
>+remove_one_user() {
>+    ldif=$release_dir/remove-one-more-user.ldif
>+    TZ=UTC $ldbmodify -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb $ldif
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+}
>+
>+move_one_user() {
>+    TZ=UTC $ldbrename -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb 'cn=user1,cn=users,DC=release-4-5-0-pre1,DC=samba,DC=corp' 'cn=user1x,cn=users,DC=release-4-5-0-pre1,DC=samba,DC=corp'
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+}
>+
>+check_expected_after_links() {
>+    tmpldif=$PREFIX_ABS/$RELEASE/expected-links-after-link-dbcheck.ldif.tmp
>+    TZ=UTC $ldbsearch -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb '(|(cn=swimmers)(cn=leaders)(cn=helpers))' -s sub -b DC=release-4-5-0-pre1,DC=samba,DC=corp --show-deleted --sorted member > $tmpldif
>+    diff $tmpldif $release_dir/expected-links-after-link-dbcheck.ldif
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+}
>+
>+check_expected_after_deleted_links() {
>+    tmpldif=$PREFIX_ABS/$RELEASE/expected-deleted-links-after-link-dbcheck.ldif.tmp
>+    TZ=UTC $ldbsearch -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb '(|(cn=swimmers)(cn=leaders)(cn=helpers))' -s sub -b DC=release-4-5-0-pre1,DC=samba,DC=corp --show-deleted --reveal --sorted member > $tmpldif
>+    diff $tmpldif $release_dir/expected-deleted-links-after-link-dbcheck.ldif
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+}
>+
>+check_expected_after_objects() {
>+    tmpldif=$PREFIX_ABS/$RELEASE/expected-objects-after-link-dbcheck.ldif.tmp
>+    TZ=UTC $ldbsearch -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb '(|(samaccountname=fred)(samaccountname=ddg)(samaccountname=usg)(samaccountname=user1)(samaccountname=user1x)(samaccountname=user2))' -s sub -b DC=release-4-5-0-pre1,DC=samba,DC=corp --show-deleted --reveal --sorted samAccountName | grep sAMAccountName > $tmpldif
>+    diff $tmpldif $release_dir/expected-objects-after-link-dbcheck.ldif
>+    if [ "$?" != "0" ]; then
>+	return 1
>+    fi
>+}
>+
>+if [ -d $release_dir ]; then
>+    testit $RELEASE undump
>+    testit "add_two_more_users" add_two_more_users
>+    testit "add_four_more_links" add_four_more_links
>+    testit "remove_one_link" remove_one_link
>+    testit "remove_one_user" remove_one_user
>+    testit "move_one_user" move_one_user
>+    testit "dbcheck" dbcheck
>+    testit "dbcheck_clean" dbcheck_clean
>+    testit "check_expected_after_deleted_links" check_expected_after_deleted_links
>+    testit "check_expected_after_links" check_expected_after_links
>+    testit "check_expected_after_objects" check_expected_after_objects
>+else
>+    subunit_start_test $RELEASE
>+    subunit_skip_test $RELEASE <<EOF
>+no test provision
>+EOF
>+
>+    subunit_start_test "tombstones_expunge"
>+    subunit_skip_test "tombstones_expunge" <<EOF
>+no test provision
>+EOF
>+fi
>+
>+if [ -d $PREFIX_ABS/${RELEASE} ]; then
>+    rm -fr $PREFIX_ABS/${RELEASE}
>+fi
>+
>+exit $failed
>-- 
>1.9.1
>
>
>From e9f1dbd8baf565b4314d258eb2b27fcd89f505b7 Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Tue, 25 Oct 2016 10:10:34 +1300
>Subject: [PATCH 05/24] dbcheck: Be more careful with link checks
>
>Here we are more careful when checking links, flagging errors only
>when a non-deleted forward link appears incorrect.  In particular, we
>trust the GUID more than we trust the name, as otherwise we can get
>caught out if there is a swap of names, (the link should follow the
>swap, staying on the same target GUID).
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12297
>(cherry picked from commit f051e5bf00d6df70048dd0cf901dd7b37be09669)
>---
> python/samba/dbchecker.py | 30 +++++++++++++++++++++++-------
> selftest/knownfail        |  3 ---
> 2 files changed, 23 insertions(+), 10 deletions(-)
>
>diff --git a/python/samba/dbchecker.py b/python/samba/dbchecker.py
>index 9b0784b..972efbb 100644
>--- a/python/samba/dbchecker.py
>+++ b/python/samba/dbchecker.py
>@@ -492,8 +492,9 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
>                           "Failed to remove deleted DN attribute %s" % attrname):
>             self.report("Removed deleted DN on attribute %s" % attrname)
> 
>-    def err_missing_dn_GUID(self, dn, attrname, val, dsdb_dn):
>-        """handle a missing target DN (both GUID and DN string form are missing)"""
>+    def err_missing_target_dn_or_GUID(self, dn, attrname, val, dsdb_dn):
>+        """handle a missing target DN (if specified, GUID form can't be found,
>+        and otherwise DN string form can't be found)"""
>         # check if its a backlink
>         linkID, _ = self.get_attr_linkID_and_reverse_name(attrname)
>         if (linkID & 1 == 0) and str(dsdb_dn).find('\\0ADEL') == -1:
>@@ -501,7 +502,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
>             return
>         self.err_deleted_dn(dn, attrname, val, dsdb_dn, dsdb_dn, False)
> 
>-    def err_incorrect_dn_GUID(self, dn, attrname, val, dsdb_dn, errstr):
>+    def err_missing_dn_GUID_component(self, dn, attrname, val, dsdb_dn, errstr):
>         """handle a missing GUID extended DN component"""
>         self.report("ERROR: %s component for %s in object %s - %s" % (errstr, attrname, dn, val))
>         controls=["extended_dn:1:1", "show_recycled:1"]
>@@ -510,11 +511,13 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
>                                     attrs=[], controls=controls)
>         except ldb.LdbError, (enum, estr):
>             self.report("unable to find object for DN %s - (%s)" % (dsdb_dn.dn, estr))
>-            self.err_missing_dn_GUID(dn, attrname, val, dsdb_dn)
>+            if enum != ldb.ERR_NO_SUCH_OBJECT:
>+                raise
>+            self.err_missing_target_dn_or_GUID(dn, attrname, val, dsdb_dn)
>             return
>         if len(res) == 0:
>             self.report("unable to find object for DN %s" % dsdb_dn.dn)
>-            self.err_missing_dn_GUID(dn, attrname, val, dsdb_dn)
>+            self.err_missing_target_dn_or_GUID(dn, attrname, val, dsdb_dn)
>             return
>         dsdb_dn.dn = res[0].dn
> 
>@@ -797,7 +800,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
>             guid = dsdb_dn.dn.get_extended_component("GUID")
>             if guid is None:
>                 error_count += 1
>-                self.err_incorrect_dn_GUID(obj.dn, attrname, val, dsdb_dn,
>+                self.err_missing_dn_GUID_component(obj.dn, attrname, val, dsdb_dn,
>                     "missing GUID")
>                 continue
> 
>@@ -822,7 +825,11 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
>                                         ])
>             except ldb.LdbError, (enum, estr):
>                 error_count += 1
>-                self.err_incorrect_dn_GUID(obj.dn, attrname, val, dsdb_dn, "incorrect GUID")
>+                self.report("ERROR: no target object found for GUID component for %s in object %s - %s" % (attrname, obj.dn, val))
>+                if enum != ldb.ERR_NO_SUCH_OBJECT:
>+                    raise
>+
>+                self.err_missing_target_dn_or_GUID(obj.dn, attrname, val, dsdb_dn)
>                 continue
> 
>             if fixing_msDS_HasInstantiatedNCs:
>@@ -874,6 +881,15 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
>                 self.err_deleted_dn(obj.dn, attrname, val, dsdb_dn, res[0].dn, False)
>                 continue
> 
>+            # We should not check for incorrect
>+            # components on deleted links, as these are allowed to
>+            # go stale (we just need the GUID, not the name)
>+            rmd_blob = dsdb_dn.dn.get_extended_component("RMD_FLAGS")
>+            if rmd_blob is not None:
>+                rmd_flags = int(rmd_blob)
>+                if rmd_flags & 1:
>+                    continue
>+
>             # check the DN matches in string form
>             if str(res[0].dn) != str(dsdb_dn.dn):
>                 error_count += 1
>diff --git a/selftest/knownfail b/selftest/knownfail
>index ccddab8..c1899da 100644
>--- a/selftest/knownfail
>+++ b/selftest/knownfail
>@@ -292,6 +292,3 @@
> #ntvfs server blocks copychunk with execute access on read handle
> ^samba4.smb2.ioctl.copy_chunk_bad_access
> ^samba4.drs.getnc_exop.python.*getnc_exop.DrsReplicaPrefixMapTestCase.test_regular_prefix_map_ex_attid.*
>-^samba4.blackbox.dbcheck-links.release-4-5-0-pre1.dbcheck
>-^samba4.blackbox.dbcheck-links.release-4-5-0-pre1.dbcheck_clean
>-^samba4.blackbox.dbcheck-links.release-4-5-0-pre1.check_expected_after_deleted_links
>-- 
>1.9.1
>
>
>From a813a5fc670dcc4822aaa61d7ae98e8066bafc30 Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Mon, 7 Nov 2016 11:04:03 +1300
>Subject: [PATCH 06/24] dbcheck: Correct message for orphaned backlinks
>
>The backlink name is in attrname, not in link_name
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12297
>(cherry picked from commit 04eb95a46b069f0238dbd232528fd1fadb745066)
>---
> python/samba/dbchecker.py | 8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
>diff --git a/python/samba/dbchecker.py b/python/samba/dbchecker.py
>index 972efbb..ef54174 100644
>--- a/python/samba/dbchecker.py
>+++ b/python/samba/dbchecker.py
>@@ -627,15 +627,15 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
>     def err_orphaned_backlink(self, obj, attrname, val, link_name, target_dn):
>         '''handle a orphaned backlink value'''
>         self.report("ERROR: orphaned backlink attribute '%s' in %s for link %s in %s" % (attrname, obj.dn, link_name, target_dn))
>-        if not self.confirm_all('Remove orphaned backlink %s' % link_name, 'fix_all_orphaned_backlinks'):
>-            self.report("Not removing orphaned backlink %s" % link_name)
>+        if not self.confirm_all('Remove orphaned backlink %s' % attrname, 'fix_all_orphaned_backlinks'):
>+            self.report("Not removing orphaned backlink %s" % attrname)
>             return
>         m = ldb.Message()
>         m.dn = obj.dn
>         m['value'] = ldb.MessageElement(val, ldb.FLAG_MOD_DELETE, attrname)
>         if self.do_modify(m, ["show_recycled:1", "relax:0"],
>-                          "Failed to fix orphaned backlink %s" % link_name):
>-            self.report("Fixed orphaned backlink %s" % (link_name))
>+                          "Failed to fix orphaned backlink %s" % attrname):
>+            self.report("Fixed orphaned backlink %s" % (attrname))
> 
>     def err_no_fsmoRoleOwner(self, obj):
>         '''handle a missing fSMORoleOwner'''
>-- 
>1.9.1
>
>
>From 21f6c8c68aa2e049021a7299d02ee43284b4cc45 Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Mon, 7 Nov 2016 11:58:04 +1300
>Subject: [PATCH 07/24] selftest: Ensure we catch errors from samba-tool domain
> tombstones expunge
>
>The previous code would overwrite $? before the return, so always returned 0
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12297
>(cherry picked from commit 44d209c893d28030cb9928b974c8aa31348ac395)
>---
> testprogs/blackbox/tombstones-expunge.sh | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/testprogs/blackbox/tombstones-expunge.sh b/testprogs/blackbox/tombstones-expunge.sh
>index 33cb0b1..e7e861f 100755
>--- a/testprogs/blackbox/tombstones-expunge.sh
>+++ b/testprogs/blackbox/tombstones-expunge.sh
>@@ -52,7 +52,7 @@ tombstones_expunge() {
> 
>     $PYTHON $BINDIR/samba-tool domain tombstones expunge -H tdb://$PREFIX_ABS/${RELEASE}/private/sam.ldb --current-time=2016-07-30 --tombstone-lifetime=4 > $tmpfile
>     if [ "$?" != "0" ]; then
>-	return $?
>+	return 1
>     fi
>     diff $tmpfile $release_dir/expected-expunge-output.txt
>     if [ "$?" != "0" ]; then
>-- 
>1.9.1
>
>
>From f714c901630508969f24431dc348c942facf434f Mon Sep 17 00:00:00 2001
>From: Clive Ferreira <cliveferreira@catalyst.net.nz>
>Date: Tue, 11 Oct 2016 15:33:06 +1300
>Subject: [PATCH 08/24] objectclass_attrs: correctly indent a comment
>
>Signed-off-by: Clive Ferreira <cliveferreira@catalyst.net.nz>
>Reviewed-by: Andrew Bartlett <abartlet@samba.org>
>Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12394
>(cherry picked from commit a3baf4b8049d222b8be71dce3bc1cd46b8391f73)
>---
> source4/dsdb/samdb/ldb_modules/objectclass_attrs.c | 13 +++++++------
> 1 file changed, 7 insertions(+), 6 deletions(-)
>
>diff --git a/source4/dsdb/samdb/ldb_modules/objectclass_attrs.c b/source4/dsdb/samdb/ldb_modules/objectclass_attrs.c
>index c83c2e9..616cff8 100644
>--- a/source4/dsdb/samdb/ldb_modules/objectclass_attrs.c
>+++ b/source4/dsdb/samdb/ldb_modules/objectclass_attrs.c
>@@ -435,12 +435,13 @@ static int attr_handler2(struct oc_context *ac)
> 	}
> 
> 	if (isSchemaAttr) {
>-		/* Before really adding an attribute in the database,
>-			* let's check that we can translate it into a dbsd_attribute and
>-			* that we can find a valid syntax object.
>-			* If not it's better to reject this attribute than not be able
>-			* to start samba next time due to schema being unloadable.
>-			*/
>+		/*
>+		 * Before really adding an attribute in the database,
>+		 * let's check that we can translate it into a dsdb_attribute and
>+		 * that we can find a valid syntax object.
>+		 * If not it's better to reject this attribute than not be able
>+		 * to start samba next time due to schema being unloadable.
>+		 */
> 		struct dsdb_attribute *att = talloc(ac, struct dsdb_attribute);
> 		const struct dsdb_syntax *attrSyntax;
> 		WERROR status;
>-- 
>1.9.1
>
>
>From 32dff77dee6e509688ff12b1ffdab1e12b0ef891 Mon Sep 17 00:00:00 2001
>From: Bob Campbell <bobcampbell@catalyst.net.nz>
>Date: Mon, 10 Oct 2016 16:58:57 +1300
>Subject: [PATCH 09/24] tests/getnc_exop: Improve the ridalloc test by
> performing an alloc against a new master
>
>Currently we fail against ourselves due to rIDNextRid and
>rIDPreviousAllocationPool normally being unset, despite being mandatory
>attributes (being the only attributes in this situation).
>
>Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
>Pair-programmed-with: Clive Ferreira <cliveferreira@catalyst.net.nz>
>Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
>Reviewed-by: Andrew Bartlett <abartlet@samba.org>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12394
>(cherry picked from commit 37aa11ce5b2b91cd0d84f6c7370d64674fcf5479)
>---
> selftest/knownfail                       |  1 +
> source4/torture/drs/python/getnc_exop.py | 76 ++++++++++++++++++++++++++++++++
> 2 files changed, 77 insertions(+)
>
>diff --git a/selftest/knownfail b/selftest/knownfail
>index c1899da..c615120 100644
>--- a/selftest/knownfail
>+++ b/selftest/knownfail
>@@ -292,3 +292,4 @@
> #ntvfs server blocks copychunk with execute access on read handle
> ^samba4.smb2.ioctl.copy_chunk_bad_access
> ^samba4.drs.getnc_exop.python.*getnc_exop.DrsReplicaPrefixMapTestCase.test_regular_prefix_map_ex_attid.*
>+^samba4.drs.getnc_exop.python.*getnc_exop.DrsReplicaSyncTestCase.test_edit_rid_master.*
>diff --git a/source4/torture/drs/python/getnc_exop.py b/source4/torture/drs/python/getnc_exop.py
>index d058e66..94d1402 100644
>--- a/source4/torture/drs/python/getnc_exop.py
>+++ b/source4/torture/drs/python/getnc_exop.py
>@@ -289,6 +289,82 @@ class DrsReplicaSyncTestCase(drs_base.DrsBaseTestCase, ExopBaseTest):
>         # We don't check the linked_attributes_count as if the domain
>         # has an RODC, it can gain links on the server account object
> 
>+    def test_edit_rid_master(self):
>+        """Test doing a RID allocation after changing the RID master from the original one.
>+           This should set rIDNextRID to 0 on the new RID master."""
>+        # 1. a. Transfer role to non-RID master
>+        #    b. Check that it succeeds correctly
>+        #
>+        # 2. a. Call the RID alloc against the former master.
>+        #    b. Check that it succeeds.
>+        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>+
>+        # 1. Swap RID master role
>+        m = ldb.Message()
>+        m.dn = ldb.Dn(self.ldb_dc1, "")
>+        m["becomeRidMaster"] = ldb.MessageElement("1", ldb.FLAG_MOD_REPLACE,
>+                                                  "becomeRidMaster")
>+
>+        # Make sure that ldb_dc1 == RID Master
>+
>+        server_dn = str(ldb.Dn(self.ldb_dc1, self.ldb_dc1.get_dsServiceName()).parent())
>+
>+        # self.ldb_dc1 == LOCALDC
>+        if server_dn == fsmo_owner['server_dn']:
>+            # ldb_dc1 == VAMPIREDC
>+            ldb_dc1, ldb_dc2 = self.ldb_dc2, self.ldb_dc1
>+        else:
>+            # Otherwise switch the two
>+            ldb_dc1, ldb_dc2 = self.ldb_dc1, self.ldb_dc2
>+
>+        try:
>+            # ldb_dc1 is now RID MASTER (as VAMPIREDC)
>+            ldb_dc1.modify(m)
>+        except ldb.LdbError, (num, msg):
>+            self.fail("Failed to reassign RID Master " +  msg)
>+
>+        try:
>+            # 2. Perform a RID alloc
>+            req8 = self._exop_req8(dest_dsa=fsmo_owner["ntds_guid"],
>+                    invocation_id=fsmo_not_owner["invocation_id"],
>+                    nc_dn_str=fsmo_dn,
>+                    exop=drsuapi.DRSUAPI_EXOP_FSMO_RID_ALLOC)
>+
>+            (drs, drs_handle) = self._ds_bind(fsmo_not_owner["dns_name"])
>+            # 3. Make sure the allocation succeeds
>+            try:
>+                (level, ctr) = drs.DsGetNCChanges(drs_handle, 8, req8)
>+            except RuntimeError, e:
>+                self.fail("RID allocation failed: " + str(e))
>+
>+            fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+
>+            self.assertEqual(level, 6, "Expected level 6 response!")
>+            self.assertEqual(ctr.source_dsa_guid, misc.GUID(fsmo_not_owner["ntds_guid"]))
>+            self.assertEqual(ctr.source_dsa_invocation_id, misc.GUID(fsmo_not_owner["invocation_id"]))
>+            ctr6 = ctr
>+            self.assertEqual(ctr6.extended_ret, drsuapi.DRSUAPI_EXOP_ERR_SUCCESS)
>+            self.assertEqual(ctr6.object_count, 3)
>+            self.assertNotEqual(ctr6.first_object, None)
>+            self.assertEqual(ldb.Dn(ldb_dc2, ctr6.first_object.object.identifier.dn), fsmo_dn)
>+            self.assertNotEqual(ctr6.first_object.next_object, None)
>+            self.assertNotEqual(ctr6.first_object.next_object.next_object, None)
>+            second_object = ctr6.first_object.next_object.object
>+            self.assertEqual(ldb.Dn(self.ldb_dc1, second_object.identifier.dn), fsmo_owner["rid_set_dn"])
>+            third_object = ctr6.first_object.next_object.next_object.object
>+            self.assertEqual(ldb.Dn(self.ldb_dc1, third_object.identifier.dn), fsmo_owner["server_acct_dn"])
>+        finally:
>+            # Swap the RID master back for other tests
>+            m = ldb.Message()
>+            m.dn = ldb.Dn(ldb_dc2, "")
>+            m["becomeRidMaster"] = ldb.MessageElement("1", ldb.FLAG_MOD_REPLACE, "becomeRidMaster")
>+            try:
>+                ldb_dc2.modify(m)
>+            except ldb.LdbError, (num, msg):
>+                self.fail("Failed to restore RID Master " +  msg)
>+
>+
> class DrsReplicaPrefixMapTestCase(drs_base.DrsBaseTestCase, ExopBaseTest):
>     def setUp(self):
>         super(DrsReplicaPrefixMapTestCase, self).setUp()
>-- 
>1.9.1
>
>
>From e7ec58e043d2a931c8b9da79f10e0677c7a593f1 Mon Sep 17 00:00:00 2001
>From: Garming Sam <garming@catalyst.net.nz>
>Date: Thu, 20 Oct 2016 16:19:43 +1300
>Subject: [PATCH 10/24] tests/getnc_exop: Finish a comment in getnc_exop.py
>
>Signed-off-by: Garming Sam <garming@catalyst.net.nz>
>Reviewed-by: Andrew Bartlett <abartlet@samba.org>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12394
>(cherry picked from commit 4c567932165229e7fa9c33b071e9fabe79d9eef0)
>---
> source4/torture/drs/python/getnc_exop.py | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/source4/torture/drs/python/getnc_exop.py b/source4/torture/drs/python/getnc_exop.py
>index 94d1402..941d323 100644
>--- a/source4/torture/drs/python/getnc_exop.py
>+++ b/source4/torture/drs/python/getnc_exop.py
>@@ -256,7 +256,7 @@ class DrsReplicaSyncTestCase(drs_base.DrsBaseTestCase, ExopBaseTest):
>         # has an RODC, it can gain links on the server account object
> 
>     def test_do_ridalloc_get_anc(self):
>-        """Test doing a RID allocation with a valid destination DSA guid and """
>+        """Test doing a RID allocation with a valid destination DSA guid and GET_ANC flag"""
>         fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>         (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
> 
>-- 
>1.9.1
>
>
>From e7722b922e46b070f8acbe88a6874712ae20ae2c Mon Sep 17 00:00:00 2001
>From: Clive Ferreira <cliveferreira@catalyst.net.nz>
>Date: Thu, 20 Oct 2016 16:20:49 +1300
>Subject: [PATCH 11/24] typo: supprise -> surprise
>
>Signed-off-by: Clive Ferreira <cliveferreira@catalyst.net.nz>
>Reviewed-by: Andrew Bartlett <abartlet@samba.org>
>Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12394
>(cherry picked from commit e0aa05609556cf7bc93d585944542d630862ba0f)
>---
> source4/dsdb/samdb/ldb_modules/rootdse.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/source4/dsdb/samdb/ldb_modules/rootdse.c b/source4/dsdb/samdb/ldb_modules/rootdse.c
>index 6a1b8ef..86ca89f 100644
>--- a/source4/dsdb/samdb/ldb_modules/rootdse.c
>+++ b/source4/dsdb/samdb/ldb_modules/rootdse.c
>@@ -1515,7 +1515,7 @@ static int rootdse_become_master(struct ldb_module *module,
> 
> 	/*
> 	 * We always delete the transaction, not commit it, because
>-	 * this gives the least supprise to this supprising action (as
>+	 * this gives the least surprise to this surprising action (as
> 	 * we will never record anything done to this point
> 	 */
> 	rootdse_del_trans(module);
>-- 
>1.9.1
>
>
>From bd412ac7ff12306888b390f11d49db7ecc6641c9 Mon Sep 17 00:00:00 2001
>From: Clive Ferreira <cliveferreira@catalyst.net.nz>
>Date: Tue, 11 Oct 2016 15:32:54 +1300
>Subject: [PATCH 12/24] objectclass_attrs: Only abort on a missing attribute
> when an attribute is both MUST and replicated
>
>If an attribute is not replicated or constructed, it is quite normal for
>it to be missing. This is the case with both rIDNextRid and
>rIDPreviousAllocationPool. This currently prevents us switching the RID
>master. On Windows, missing this attribute does not cause any problems
>for the RID manager.
>
>We may now remove the knownfail entry added earlier.
>
>Signed-off-by: Clive Ferreira <cliveferreira@catalyst.net.nz>
>Reviewed-by: Andrew Bartlett <abartlet@samba.org>
>
>Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
>Pair-programmed-with: Bob Campbell <bobcampbell@catalyst.net.nz>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12394
>
>Autobuild-User(master): Garming Sam <garming@samba.org>
>Autobuild-Date(master): Wed Nov  2 01:28:44 CET 2016 on sn-devel-144
>
>(cherry picked from commit 79dd22aacb4c12bd008d9ad354ec5ec088560748)
>---
> selftest/knownfail                                 |  1 -
> source4/dsdb/samdb/ldb_modules/objectclass_attrs.c | 23 ++++++++++++++++------
> 2 files changed, 17 insertions(+), 7 deletions(-)
>
>diff --git a/selftest/knownfail b/selftest/knownfail
>index c615120..c1899da 100644
>--- a/selftest/knownfail
>+++ b/selftest/knownfail
>@@ -292,4 +292,3 @@
> #ntvfs server blocks copychunk with execute access on read handle
> ^samba4.smb2.ioctl.copy_chunk_bad_access
> ^samba4.drs.getnc_exop.python.*getnc_exop.DrsReplicaPrefixMapTestCase.test_regular_prefix_map_ex_attid.*
>-^samba4.drs.getnc_exop.python.*getnc_exop.DrsReplicaSyncTestCase.test_edit_rid_master.*
>diff --git a/source4/dsdb/samdb/ldb_modules/objectclass_attrs.c b/source4/dsdb/samdb/ldb_modules/objectclass_attrs.c
>index 616cff8..e239fb9 100644
>--- a/source4/dsdb/samdb/ldb_modules/objectclass_attrs.c
>+++ b/source4/dsdb/samdb/ldb_modules/objectclass_attrs.c
>@@ -426,12 +426,23 @@ static int attr_handler2(struct oc_context *ac)
> 	 * replicated.
> 	 */
> 	if (found_must_contain[0] != NULL &&
>-	    ldb_msg_check_string_attribute(msg, "isDeleted", "TRUE") == 0 &&
>-	    ldb_request_get_control(ac->req, DSDB_CONTROL_DBCHECK) == NULL) {
>-		ldb_asprintf_errstring(ldb, "objectclass_attrs: at least one mandatory attribute ('%s') on entry '%s' wasn't specified!",
>-				       found_must_contain[0],
>-				       ldb_dn_get_linearized(msg->dn));
>-		return LDB_ERR_OBJECT_CLASS_VIOLATION;
>+	    ldb_msg_check_string_attribute(msg, "isDeleted", "TRUE") == 0) {
>+
>+		for (i = 0; found_must_contain[i] != NULL; i++) {
>+			const struct dsdb_attribute *broken_attr = dsdb_attribute_by_lDAPDisplayName(ac->schema,
>+												     found_must_contain[i]);
>+
>+			bool replicated = (broken_attr->systemFlags &
>+					   (DS_FLAG_ATTR_NOT_REPLICATED | DS_FLAG_ATTR_IS_CONSTRUCTED)) == 0;
>+
>+			if (replicated) {
>+				ldb_asprintf_errstring(ldb, "objectclass_attrs: at least one mandatory "
>+						       "attribute ('%s') on entry '%s' wasn't specified!",
>+						       found_must_contain[i],
>+						       ldb_dn_get_linearized(msg->dn));
>+				return LDB_ERR_OBJECT_CLASS_VIOLATION;
>+			}
>+		}
> 	}
> 
> 	if (isSchemaAttr) {
>-- 
>1.9.1
>
>
>From 335fe08ff35db3a8c4603f5202442322d02b6b8f Mon Sep 17 00:00:00 2001
>From: Garming Sam <garming@catalyst.net.nz>
>Date: Mon, 31 Oct 2016 15:24:49 +1300
>Subject: [PATCH 13/24] tests/ridalloc_exop: Add a new suite of tests for RID
> allocation
>
>This moves some tests from getnc_exop.py regarding RID sets as well as
>adding new tests for actions on join.
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=9954
>
>Pair-programmed-with: Clive Ferreira <cliveferreira@catalyst.net.nz>
>
>Signed-off-by: Andrew Bartlett <abartlet@samaba.org>
>Signed-off-by: Garming Sam <garming@catalyst.net.nz>
>Signed-off-by: Clive Ferreira <cliveferreira@catalyst.net.nz>
>
>Reviewed-by: Andrew Bartlett <abartlet@samba.org>
>(cherry picked from commit 1b40bb69d101b767ee453c96234cc6d573142ab3)
>---
> selftest/knownfail                          |   6 +
> source4/selftest/tests.py                   |  11 +-
> source4/torture/drs/python/getnc_exop.py    | 161 -------
> source4/torture/drs/python/ridalloc_exop.py | 714 ++++++++++++++++++++++++++++
> 4 files changed, 729 insertions(+), 163 deletions(-)
> create mode 100644 source4/torture/drs/python/ridalloc_exop.py
>
>diff --git a/selftest/knownfail b/selftest/knownfail
>index c1899da..26bbb6d 100644
>--- a/selftest/knownfail
>+++ b/selftest/knownfail
>@@ -292,3 +292,9 @@
> #ntvfs server blocks copychunk with execute access on read handle
> ^samba4.smb2.ioctl.copy_chunk_bad_access
> ^samba4.drs.getnc_exop.python.*getnc_exop.DrsReplicaPrefixMapTestCase.test_regular_prefix_map_ex_attid.*
>+^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_offline_manual_seized_ridalloc_with_dbcheck
>+^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_offline_ridalloc
>+^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_offline_samba_tool_seized_ridalloc
>+^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_join_time_ridalloc
>+^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_rid_set_dbcheck_after_seize
>+^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_rid_set_dbcheck
>diff --git a/source4/selftest/tests.py b/source4/selftest/tests.py
>index ef11a70..3ad94d5 100755
>--- a/source4/selftest/tests.py
>+++ b/source4/selftest/tests.py
>@@ -648,9 +648,16 @@ plantestsuite("samba4.blackbox.provision-backend", "none", ["PYTHON=%s" % python
> # Test renaming the DC
> plantestsuite("samba4.blackbox.renamedc.sh", "none", ["PYTHON=%s" % python, os.path.join(bbdir, "renamedc.sh"), '$PREFIX/provision'])
> 
>-for env in ['vampire_dc', 'promoted_dc']:
>+# DRS python tests
>+
>+env = 'vampire_dc'
>+planoldpythontestsuite(env, "ridalloc_exop",
>+                       extra_path=[os.path.join(samba4srcdir, 'torture/drs/python')],
>+                       name="samba4.drs.ridalloc_exop.python(%s)" % env,
>+                       environ={'DC1': "$DC_SERVER", 'DC2': '$%s_SERVER' % env.upper()},
>+                       extra_args=['-U$DOMAIN/$DC_USERNAME%$DC_PASSWORD'])
> 
>-    # DRS python tests
>+for env in ['vampire_dc', 'promoted_dc']:
>     planoldpythontestsuite("%s:local" % env, "samba.tests.blackbox.samba_tool_drs",
>                            environ={'DC1': '$DC_SERVER', 'DC2': '$%s_SERVER' % env.upper()},
>                            extra_args=['-U$DOMAIN/$DC_USERNAME%$DC_PASSWORD'])
>diff --git a/source4/torture/drs/python/getnc_exop.py b/source4/torture/drs/python/getnc_exop.py
>index 941d323..246d859 100644
>--- a/source4/torture/drs/python/getnc_exop.py
>+++ b/source4/torture/drs/python/getnc_exop.py
>@@ -204,167 +204,6 @@ class DrsReplicaSyncTestCase(drs_base.DrsBaseTestCase, ExopBaseTest):
>         self.assertEqual(ctr.source_dsa_guid, misc.GUID(fsmo_owner["ntds_guid"]))
>         self.assertEqual(ctr.source_dsa_invocation_id, misc.GUID(fsmo_owner["invocation_id"]))
> 
>-    def test_InvalidDestDSA_ridalloc(self):
>-        """Test RID allocation with invalid destination DSA guid"""
>-        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>-        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>-
>-        req8 = self._exop_req8(dest_dsa="9c637462-5b8c-4467-aef2-bdb1f57bc4ef",
>-                               invocation_id=fsmo_owner["invocation_id"],
>-                               nc_dn_str=fsmo_dn,
>-                               exop=drsuapi.DRSUAPI_EXOP_FSMO_RID_ALLOC)
>-
>-        (drs, drs_handle) = self._ds_bind(fsmo_owner["dns_name"])
>-        (level, ctr) = drs.DsGetNCChanges(drs_handle, 8, req8)
>-        self.assertEqual(level, 6, "Expected level 6 response!")
>-        self._check_exop_failed(ctr, drsuapi.DRSUAPI_EXOP_ERR_UNKNOWN_CALLER)
>-        self.assertEqual(ctr.source_dsa_guid, misc.GUID(fsmo_owner["ntds_guid"]))
>-        self.assertEqual(ctr.source_dsa_invocation_id, misc.GUID(fsmo_owner["invocation_id"]))
>-
>-    def test_do_ridalloc(self):
>-        """Test doing a RID allocation with a valid destination DSA guid"""
>-        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>-        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>-
>-        req8 = self._exop_req8(dest_dsa=fsmo_not_owner["ntds_guid"],
>-                               invocation_id=fsmo_owner["invocation_id"],
>-                               nc_dn_str=fsmo_dn,
>-                               exop=drsuapi.DRSUAPI_EXOP_FSMO_RID_ALLOC)
>-
>-        (drs, drs_handle) = self._ds_bind(fsmo_owner["dns_name"])
>-        (level, ctr) = drs.DsGetNCChanges(drs_handle, 8, req8)
>-        self.assertEqual(level, 6, "Expected level 6 response!")
>-        self.assertEqual(ctr.source_dsa_guid, misc.GUID(fsmo_owner["ntds_guid"]))
>-        self.assertEqual(ctr.source_dsa_invocation_id, misc.GUID(fsmo_owner["invocation_id"]))
>-        ctr6 = ctr
>-        self.assertEqual(ctr6.extended_ret, drsuapi.DRSUAPI_EXOP_ERR_SUCCESS)
>-        self.assertEqual(ctr6.object_count, 3)
>-        self.assertNotEqual(ctr6.first_object, None)
>-        self.assertEqual(ldb.Dn(self.ldb_dc1, ctr6.first_object.object.identifier.dn), fsmo_dn)
>-        self.assertNotEqual(ctr6.first_object.next_object, None)
>-        self.assertNotEqual(ctr6.first_object.next_object.next_object, None)
>-        second_object = ctr6.first_object.next_object.object
>-        self.assertEqual(ldb.Dn(self.ldb_dc1, second_object.identifier.dn), fsmo_not_owner["rid_set_dn"])
>-        third_object = ctr6.first_object.next_object.next_object.object
>-        self.assertEqual(ldb.Dn(self.ldb_dc1, third_object.identifier.dn), fsmo_not_owner["server_acct_dn"])
>-
>-        self.assertEqual(ctr6.more_data, False)
>-        self.assertEqual(ctr6.nc_object_count, 0)
>-        self.assertEqual(ctr6.nc_linked_attributes_count, 0)
>-        self.assertEqual(ctr6.drs_error[0], 0)
>-        # We don't check the linked_attributes_count as if the domain
>-        # has an RODC, it can gain links on the server account object
>-
>-    def test_do_ridalloc_get_anc(self):
>-        """Test doing a RID allocation with a valid destination DSA guid and GET_ANC flag"""
>-        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>-        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>-
>-        req8 = self._exop_req8(dest_dsa=fsmo_not_owner["ntds_guid"],
>-                               invocation_id=fsmo_owner["invocation_id"],
>-                               nc_dn_str=fsmo_dn,
>-                               exop=drsuapi.DRSUAPI_EXOP_FSMO_RID_ALLOC,
>-                               replica_flags=drsuapi.DRSUAPI_DRS_GET_ANC)
>-
>-        (drs, drs_handle) = self._ds_bind(fsmo_owner["dns_name"])
>-        (level, ctr) = drs.DsGetNCChanges(drs_handle, 8, req8)
>-        self.assertEqual(level, 6, "Expected level 6 response!")
>-        self.assertEqual(ctr.source_dsa_guid, misc.GUID(fsmo_owner["ntds_guid"]))
>-        self.assertEqual(ctr.source_dsa_invocation_id, misc.GUID(fsmo_owner["invocation_id"]))
>-        ctr6 = ctr
>-        self.assertEqual(ctr6.extended_ret, drsuapi.DRSUAPI_EXOP_ERR_SUCCESS)
>-        self.assertEqual(ctr6.object_count, 3)
>-        self.assertNotEqual(ctr6.first_object, None)
>-        self.assertEqual(ldb.Dn(self.ldb_dc1, ctr6.first_object.object.identifier.dn), fsmo_dn)
>-        self.assertNotEqual(ctr6.first_object.next_object, None)
>-        self.assertNotEqual(ctr6.first_object.next_object.next_object, None)
>-        second_object = ctr6.first_object.next_object.object
>-        self.assertEqual(ldb.Dn(self.ldb_dc1, second_object.identifier.dn), fsmo_not_owner["rid_set_dn"])
>-        third_object = ctr6.first_object.next_object.next_object.object
>-        self.assertEqual(ldb.Dn(self.ldb_dc1, third_object.identifier.dn), fsmo_not_owner["server_acct_dn"])
>-        self.assertEqual(ctr6.more_data, False)
>-        self.assertEqual(ctr6.nc_object_count, 0)
>-        self.assertEqual(ctr6.nc_linked_attributes_count, 0)
>-        self.assertEqual(ctr6.drs_error[0], 0)
>-        # We don't check the linked_attributes_count as if the domain
>-        # has an RODC, it can gain links on the server account object
>-
>-    def test_edit_rid_master(self):
>-        """Test doing a RID allocation after changing the RID master from the original one.
>-           This should set rIDNextRID to 0 on the new RID master."""
>-        # 1. a. Transfer role to non-RID master
>-        #    b. Check that it succeeds correctly
>-        #
>-        # 2. a. Call the RID alloc against the former master.
>-        #    b. Check that it succeeds.
>-        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>-        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>-
>-        # 1. Swap RID master role
>-        m = ldb.Message()
>-        m.dn = ldb.Dn(self.ldb_dc1, "")
>-        m["becomeRidMaster"] = ldb.MessageElement("1", ldb.FLAG_MOD_REPLACE,
>-                                                  "becomeRidMaster")
>-
>-        # Make sure that ldb_dc1 == RID Master
>-
>-        server_dn = str(ldb.Dn(self.ldb_dc1, self.ldb_dc1.get_dsServiceName()).parent())
>-
>-        # self.ldb_dc1 == LOCALDC
>-        if server_dn == fsmo_owner['server_dn']:
>-            # ldb_dc1 == VAMPIREDC
>-            ldb_dc1, ldb_dc2 = self.ldb_dc2, self.ldb_dc1
>-        else:
>-            # Otherwise switch the two
>-            ldb_dc1, ldb_dc2 = self.ldb_dc1, self.ldb_dc2
>-
>-        try:
>-            # ldb_dc1 is now RID MASTER (as VAMPIREDC)
>-            ldb_dc1.modify(m)
>-        except ldb.LdbError, (num, msg):
>-            self.fail("Failed to reassign RID Master " +  msg)
>-
>-        try:
>-            # 2. Perform a RID alloc
>-            req8 = self._exop_req8(dest_dsa=fsmo_owner["ntds_guid"],
>-                    invocation_id=fsmo_not_owner["invocation_id"],
>-                    nc_dn_str=fsmo_dn,
>-                    exop=drsuapi.DRSUAPI_EXOP_FSMO_RID_ALLOC)
>-
>-            (drs, drs_handle) = self._ds_bind(fsmo_not_owner["dns_name"])
>-            # 3. Make sure the allocation succeeds
>-            try:
>-                (level, ctr) = drs.DsGetNCChanges(drs_handle, 8, req8)
>-            except RuntimeError, e:
>-                self.fail("RID allocation failed: " + str(e))
>-
>-            fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>-
>-            self.assertEqual(level, 6, "Expected level 6 response!")
>-            self.assertEqual(ctr.source_dsa_guid, misc.GUID(fsmo_not_owner["ntds_guid"]))
>-            self.assertEqual(ctr.source_dsa_invocation_id, misc.GUID(fsmo_not_owner["invocation_id"]))
>-            ctr6 = ctr
>-            self.assertEqual(ctr6.extended_ret, drsuapi.DRSUAPI_EXOP_ERR_SUCCESS)
>-            self.assertEqual(ctr6.object_count, 3)
>-            self.assertNotEqual(ctr6.first_object, None)
>-            self.assertEqual(ldb.Dn(ldb_dc2, ctr6.first_object.object.identifier.dn), fsmo_dn)
>-            self.assertNotEqual(ctr6.first_object.next_object, None)
>-            self.assertNotEqual(ctr6.first_object.next_object.next_object, None)
>-            second_object = ctr6.first_object.next_object.object
>-            self.assertEqual(ldb.Dn(self.ldb_dc1, second_object.identifier.dn), fsmo_owner["rid_set_dn"])
>-            third_object = ctr6.first_object.next_object.next_object.object
>-            self.assertEqual(ldb.Dn(self.ldb_dc1, third_object.identifier.dn), fsmo_owner["server_acct_dn"])
>-        finally:
>-            # Swap the RID master back for other tests
>-            m = ldb.Message()
>-            m.dn = ldb.Dn(ldb_dc2, "")
>-            m["becomeRidMaster"] = ldb.MessageElement("1", ldb.FLAG_MOD_REPLACE, "becomeRidMaster")
>-            try:
>-                ldb_dc2.modify(m)
>-            except ldb.LdbError, (num, msg):
>-                self.fail("Failed to restore RID Master " +  msg)
>-
>-
> class DrsReplicaPrefixMapTestCase(drs_base.DrsBaseTestCase, ExopBaseTest):
>     def setUp(self):
>         super(DrsReplicaPrefixMapTestCase, self).setUp()
>diff --git a/source4/torture/drs/python/ridalloc_exop.py b/source4/torture/drs/python/ridalloc_exop.py
>new file mode 100644
>index 0000000..5a273fc
>--- /dev/null
>+++ b/source4/torture/drs/python/ridalloc_exop.py
>@@ -0,0 +1,714 @@
>+#!/usr/bin/env python
>+# -*- coding: utf-8 -*-
>+#
>+# Tests various RID allocation scenarios
>+#
>+# Copyright (C) Kamen Mazdrashki <kamenim@samba.org> 2011
>+# Copyright (C) Andrew Bartlett <abartlet@samba.org> 2016
>+# Copyright (C) Catalyst IT Ltd. 2016
>+#
>+# This program is free software; you can redistribute it and/or modify
>+# it under the terms of the GNU General Public License as published by
>+# the Free Software Foundation; either version 3 of the License, or
>+# (at your option) any later version.
>+#
>+# This program is distributed in the hope that it will be useful,
>+# but WITHOUT ANY WARRANTY; without even the implied warranty of
>+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>+# GNU General Public License for more details.
>+#
>+# You should have received a copy of the GNU General Public License
>+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
>+#
>+
>+#
>+# Usage:
>+#  export DC1=dc1_dns_name
>+#  export DC2=dc2_dns_name
>+#  export SUBUNITRUN=$samba4srcdir/scripting/bin/subunitrun
>+#  PYTHONPATH="$PYTHONPATH:$samba4srcdir/torture/drs/python" $SUBUNITRUN ridalloc_exop -U"$DOMAIN/$DC_USERNAME"%"$DC_PASSWORD"
>+#
>+
>+import drs_base
>+import samba.tests
>+
>+import ldb
>+from ldb import SCOPE_BASE
>+
>+from samba.dcerpc import drsuapi, misc
>+from samba.drs_utils import drs_DsBind
>+from samba.samdb import SamDB
>+
>+import shutil, tempfile, os
>+from samba.netcmd.main import cmd_sambatool
>+from samba.auth import system_session, admin_session
>+from samba.dbchecker import dbcheck
>+from samba.ndr import ndr_pack
>+from samba.dcerpc import security
>+
>+class ExopBaseTest:
>+    def _exop_req8(self, dest_dsa, invocation_id, nc_dn_str, exop,
>+                   replica_flags=0, max_objects=0, partial_attribute_set=None,
>+                   partial_attribute_set_ex=None, mapping_ctr=None):
>+        req8 = drsuapi.DsGetNCChangesRequest8()
>+
>+        req8.destination_dsa_guid = misc.GUID(dest_dsa) if dest_dsa else misc.GUID()
>+        req8.source_dsa_invocation_id = misc.GUID(invocation_id)
>+        req8.naming_context = drsuapi.DsReplicaObjectIdentifier()
>+        req8.naming_context.dn = unicode(nc_dn_str)
>+        req8.highwatermark = drsuapi.DsReplicaHighWaterMark()
>+        req8.highwatermark.tmp_highest_usn = 0
>+        req8.highwatermark.reserved_usn = 0
>+        req8.highwatermark.highest_usn = 0
>+        req8.uptodateness_vector = None
>+        req8.replica_flags = replica_flags
>+        req8.max_object_count = max_objects
>+        req8.max_ndr_size = 402116
>+        req8.extended_op = exop
>+        req8.fsmo_info = 0
>+        req8.partial_attribute_set = partial_attribute_set
>+        req8.partial_attribute_set_ex = partial_attribute_set_ex
>+        if mapping_ctr:
>+            req8.mapping_ctr = mapping_ctr
>+        else:
>+            req8.mapping_ctr.num_mappings = 0
>+            req8.mapping_ctr.mappings = None
>+
>+        return req8
>+
>+    def _ds_bind(self, server_name):
>+        binding_str = "ncacn_ip_tcp:%s[seal]" % server_name
>+
>+        drs = drsuapi.drsuapi(binding_str, self.get_loadparm(), self.get_credentials())
>+        (drs_handle, supported_extensions) = drs_DsBind(drs)
>+        return (drs, drs_handle)
>+
>+
>+class DrsReplicaSyncTestCase(drs_base.DrsBaseTestCase, ExopBaseTest):
>+    """Intended as a semi-black box test case for DsGetNCChanges
>+       implementation for extended operations. It should be testing
>+       how DsGetNCChanges handles different input params (mostly invalid).
>+       Final goal is to make DsGetNCChanges as binary compatible to
>+       Windows implementation as possible"""
>+
>+    def setUp(self):
>+        super(DrsReplicaSyncTestCase, self).setUp()
>+
>+    def tearDown(self):
>+        super(DrsReplicaSyncTestCase, self).tearDown()
>+
>+    def _determine_fSMORoleOwner(self, fsmo_obj_dn):
>+        """Returns (owner, not_owner) pair where:
>+             owner: dns name for FSMO owner
>+             not_owner: dns name for DC not owning the FSMO"""
>+        # collect info to return later
>+        fsmo_info_1 = {"dns_name": self.dnsname_dc1,
>+                       "invocation_id": self.ldb_dc1.get_invocation_id(),
>+                       "ntds_guid": self.ldb_dc1.get_ntds_GUID(),
>+                       "server_dn": self.ldb_dc1.get_serverName()}
>+        fsmo_info_2 = {"dns_name": self.dnsname_dc2,
>+                       "invocation_id": self.ldb_dc2.get_invocation_id(),
>+                       "ntds_guid": self.ldb_dc2.get_ntds_GUID(),
>+                       "server_dn": self.ldb_dc2.get_serverName()}
>+
>+        msgs = self.ldb_dc1.search(scope=ldb.SCOPE_BASE, base=fsmo_info_1["server_dn"], attrs=["serverReference"])
>+        fsmo_info_1["server_acct_dn"] = ldb.Dn(self.ldb_dc1, msgs[0]["serverReference"][0])
>+        fsmo_info_1["rid_set_dn"] = ldb.Dn(self.ldb_dc1, "CN=RID Set") + fsmo_info_1["server_acct_dn"]
>+
>+        msgs = self.ldb_dc2.search(scope=ldb.SCOPE_BASE, base=fsmo_info_2["server_dn"], attrs=["serverReference"])
>+        fsmo_info_2["server_acct_dn"] = ldb.Dn(self.ldb_dc2, msgs[0]["serverReference"][0])
>+        fsmo_info_2["rid_set_dn"] = ldb.Dn(self.ldb_dc2, "CN=RID Set") + fsmo_info_2["server_acct_dn"]
>+
>+        # determine the owner dc
>+        res = self.ldb_dc1.search(fsmo_obj_dn,
>+                                  scope=SCOPE_BASE, attrs=["fSMORoleOwner"])
>+        assert len(res) == 1, "Only one fSMORoleOwner value expected for %s!"%fsmo_obj_dn
>+        fsmo_owner = res[0]["fSMORoleOwner"][0]
>+        if fsmo_owner == self.info_dc1["dsServiceName"][0]:
>+            return (fsmo_info_1, fsmo_info_2)
>+        return (fsmo_info_2, fsmo_info_1)
>+
>+    def _check_exop_failed(self, ctr6, expected_failure):
>+        self.assertEqual(ctr6.extended_ret, expected_failure)
>+        #self.assertEqual(ctr6.object_count, 0)
>+        #self.assertEqual(ctr6.first_object, None)
>+        self.assertEqual(ctr6.more_data, False)
>+        self.assertEqual(ctr6.nc_object_count, 0)
>+        self.assertEqual(ctr6.nc_linked_attributes_count, 0)
>+        self.assertEqual(ctr6.linked_attributes_count, 0)
>+        self.assertEqual(ctr6.linked_attributes, [])
>+        self.assertEqual(ctr6.drs_error[0], 0)
>+
>+    def test_InvalidDestDSA_ridalloc(self):
>+        """Test RID allocation with invalid destination DSA guid"""
>+        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>+
>+        req8 = self._exop_req8(dest_dsa="9c637462-5b8c-4467-aef2-bdb1f57bc4ef",
>+                               invocation_id=fsmo_owner["invocation_id"],
>+                               nc_dn_str=fsmo_dn,
>+                               exop=drsuapi.DRSUAPI_EXOP_FSMO_RID_ALLOC)
>+
>+        (drs, drs_handle) = self._ds_bind(fsmo_owner["dns_name"])
>+        (level, ctr) = drs.DsGetNCChanges(drs_handle, 8, req8)
>+        self.assertEqual(level, 6, "Expected level 6 response!")
>+        self._check_exop_failed(ctr, drsuapi.DRSUAPI_EXOP_ERR_UNKNOWN_CALLER)
>+        self.assertEqual(ctr.source_dsa_guid, misc.GUID(fsmo_owner["ntds_guid"]))
>+        self.assertEqual(ctr.source_dsa_invocation_id, misc.GUID(fsmo_owner["invocation_id"]))
>+
>+    def test_do_ridalloc(self):
>+        """Test doing a RID allocation with a valid destination DSA guid"""
>+        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>+
>+        req8 = self._exop_req8(dest_dsa=fsmo_not_owner["ntds_guid"],
>+                               invocation_id=fsmo_owner["invocation_id"],
>+                               nc_dn_str=fsmo_dn,
>+                               exop=drsuapi.DRSUAPI_EXOP_FSMO_RID_ALLOC)
>+
>+        (drs, drs_handle) = self._ds_bind(fsmo_owner["dns_name"])
>+        (level, ctr) = drs.DsGetNCChanges(drs_handle, 8, req8)
>+        self.assertEqual(level, 6, "Expected level 6 response!")
>+        self.assertEqual(ctr.source_dsa_guid, misc.GUID(fsmo_owner["ntds_guid"]))
>+        self.assertEqual(ctr.source_dsa_invocation_id, misc.GUID(fsmo_owner["invocation_id"]))
>+        ctr6 = ctr
>+        self.assertEqual(ctr6.extended_ret, drsuapi.DRSUAPI_EXOP_ERR_SUCCESS)
>+        self.assertEqual(ctr6.object_count, 3)
>+        self.assertNotEqual(ctr6.first_object, None)
>+        self.assertEqual(ldb.Dn(self.ldb_dc1, ctr6.first_object.object.identifier.dn), fsmo_dn)
>+        self.assertNotEqual(ctr6.first_object.next_object, None)
>+        self.assertNotEqual(ctr6.first_object.next_object.next_object, None)
>+        second_object = ctr6.first_object.next_object.object
>+        self.assertEqual(ldb.Dn(self.ldb_dc1, second_object.identifier.dn), fsmo_not_owner["rid_set_dn"])
>+        third_object = ctr6.first_object.next_object.next_object.object
>+        self.assertEqual(ldb.Dn(self.ldb_dc1, third_object.identifier.dn), fsmo_not_owner["server_acct_dn"])
>+
>+        self.assertEqual(ctr6.more_data, False)
>+        self.assertEqual(ctr6.nc_object_count, 0)
>+        self.assertEqual(ctr6.nc_linked_attributes_count, 0)
>+        self.assertEqual(ctr6.drs_error[0], 0)
>+        # We don't check the linked_attributes_count as if the domain
>+        # has an RODC, it can gain links on the server account object
>+
>+    def test_do_ridalloc_get_anc(self):
>+        """Test doing a RID allocation with a valid destination DSA guid and GET_ANC flag"""
>+        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>+
>+        req8 = self._exop_req8(dest_dsa=fsmo_not_owner["ntds_guid"],
>+                               invocation_id=fsmo_owner["invocation_id"],
>+                               nc_dn_str=fsmo_dn,
>+                               exop=drsuapi.DRSUAPI_EXOP_FSMO_RID_ALLOC,
>+                               replica_flags=drsuapi.DRSUAPI_DRS_GET_ANC)
>+
>+        (drs, drs_handle) = self._ds_bind(fsmo_owner["dns_name"])
>+        (level, ctr) = drs.DsGetNCChanges(drs_handle, 8, req8)
>+        self.assertEqual(level, 6, "Expected level 6 response!")
>+        self.assertEqual(ctr.source_dsa_guid, misc.GUID(fsmo_owner["ntds_guid"]))
>+        self.assertEqual(ctr.source_dsa_invocation_id, misc.GUID(fsmo_owner["invocation_id"]))
>+        ctr6 = ctr
>+        self.assertEqual(ctr6.extended_ret, drsuapi.DRSUAPI_EXOP_ERR_SUCCESS)
>+        self.assertEqual(ctr6.object_count, 3)
>+        self.assertNotEqual(ctr6.first_object, None)
>+        self.assertEqual(ldb.Dn(self.ldb_dc1, ctr6.first_object.object.identifier.dn), fsmo_dn)
>+        self.assertNotEqual(ctr6.first_object.next_object, None)
>+        self.assertNotEqual(ctr6.first_object.next_object.next_object, None)
>+        second_object = ctr6.first_object.next_object.object
>+        self.assertEqual(ldb.Dn(self.ldb_dc1, second_object.identifier.dn), fsmo_not_owner["rid_set_dn"])
>+        third_object = ctr6.first_object.next_object.next_object.object
>+        self.assertEqual(ldb.Dn(self.ldb_dc1, third_object.identifier.dn), fsmo_not_owner["server_acct_dn"])
>+        self.assertEqual(ctr6.more_data, False)
>+        self.assertEqual(ctr6.nc_object_count, 0)
>+        self.assertEqual(ctr6.nc_linked_attributes_count, 0)
>+        self.assertEqual(ctr6.drs_error[0], 0)
>+        # We don't check the linked_attributes_count as if the domain
>+        # has an RODC, it can gain links on the server account object
>+
>+    def test_edit_rid_master(self):
>+        """Test doing a RID allocation after changing the RID master from the original one.
>+           This should set rIDNextRID to 0 on the new RID master."""
>+        # 1. a. Transfer role to non-RID master
>+        #    b. Check that it succeeds correctly
>+        #
>+        # 2. a. Call the RID alloc against the former master.
>+        #    b. Check that it succeeds.
>+        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>+
>+        # 1. Swap RID master role
>+        m = ldb.Message()
>+        m.dn = ldb.Dn(self.ldb_dc1, "")
>+        m["becomeRidMaster"] = ldb.MessageElement("1", ldb.FLAG_MOD_REPLACE,
>+                                                  "becomeRidMaster")
>+
>+        # Make sure that ldb_dc1 == RID Master
>+
>+        server_dn = str(ldb.Dn(self.ldb_dc1, self.ldb_dc1.get_dsServiceName()).parent())
>+
>+        # self.ldb_dc1 == LOCALDC
>+        if server_dn == fsmo_owner['server_dn']:
>+            # ldb_dc1 == VAMPIREDC
>+            ldb_dc1, ldb_dc2 = self.ldb_dc2, self.ldb_dc1
>+        else:
>+            # Otherwise switch the two
>+            ldb_dc1, ldb_dc2 = self.ldb_dc1, self.ldb_dc2
>+
>+        try:
>+            # ldb_dc1 is now RID MASTER (as VAMPIREDC)
>+            ldb_dc1.modify(m)
>+        except ldb.LdbError, (num, msg):
>+            self.fail("Failed to reassign RID Master " +  msg)
>+
>+        try:
>+            # 2. Perform a RID alloc
>+            req8 = self._exop_req8(dest_dsa=fsmo_owner["ntds_guid"],
>+                    invocation_id=fsmo_not_owner["invocation_id"],
>+                    nc_dn_str=fsmo_dn,
>+                    exop=drsuapi.DRSUAPI_EXOP_FSMO_RID_ALLOC)
>+
>+            (drs, drs_handle) = self._ds_bind(fsmo_not_owner["dns_name"])
>+            # 3. Make sure the allocation succeeds
>+            try:
>+                (level, ctr) = drs.DsGetNCChanges(drs_handle, 8, req8)
>+            except RuntimeError, e:
>+                self.fail("RID allocation failed: " + str(e))
>+
>+            fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+
>+            self.assertEqual(level, 6, "Expected level 6 response!")
>+            self.assertEqual(ctr.source_dsa_guid, misc.GUID(fsmo_not_owner["ntds_guid"]))
>+            self.assertEqual(ctr.source_dsa_invocation_id, misc.GUID(fsmo_not_owner["invocation_id"]))
>+            ctr6 = ctr
>+            self.assertEqual(ctr6.extended_ret, drsuapi.DRSUAPI_EXOP_ERR_SUCCESS)
>+            self.assertEqual(ctr6.object_count, 3)
>+            self.assertNotEqual(ctr6.first_object, None)
>+            self.assertEqual(ldb.Dn(ldb_dc2, ctr6.first_object.object.identifier.dn), fsmo_dn)
>+            self.assertNotEqual(ctr6.first_object.next_object, None)
>+            self.assertNotEqual(ctr6.first_object.next_object.next_object, None)
>+            second_object = ctr6.first_object.next_object.object
>+            self.assertEqual(ldb.Dn(self.ldb_dc1, second_object.identifier.dn), fsmo_owner["rid_set_dn"])
>+            third_object = ctr6.first_object.next_object.next_object.object
>+            self.assertEqual(ldb.Dn(self.ldb_dc1, third_object.identifier.dn), fsmo_owner["server_acct_dn"])
>+        finally:
>+            # Swap the RID master back for other tests
>+            m = ldb.Message()
>+            m.dn = ldb.Dn(ldb_dc2, "")
>+            m["becomeRidMaster"] = ldb.MessageElement("1", ldb.FLAG_MOD_REPLACE, "becomeRidMaster")
>+            try:
>+                ldb_dc2.modify(m)
>+            except ldb.LdbError, (num, msg):
>+                self.fail("Failed to restore RID Master " +  msg)
>+
>+    def test_offline_samba_tool_seized_ridalloc(self):
>+        """Perform a join against the non-RID manager and then seize the RID Manager role"""
>+
>+        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>+
>+        targetdir = self._test_join(fsmo_not_owner['dns_name'], "RIDALLOCTEST1")
>+        try:
>+            # Connect to the database
>+            ldb_url = "tdb://%s" % os.path.join(targetdir, "private/sam.ldb")
>+            smbconf = os.path.join(targetdir, "etc/smb.conf")
>+
>+            lp = self.get_loadparm()
>+            new_ldb = SamDB(ldb_url, credentials=self.get_credentials(),
>+                            session_info=system_session(lp), lp=lp)
>+
>+            # 1. Get server name
>+            res = new_ldb.search(base=ldb.Dn(new_ldb, new_ldb.get_serverName()),
>+                                 scope=ldb.SCOPE_BASE, attrs=["serverReference"])
>+            # 2. Get server reference
>+            server_ref_dn = ldb.Dn(new_ldb, res[0]['serverReference'][0])
>+
>+            # Assert that no RID Set has been set
>+            res = new_ldb.search(base=server_ref_dn,
>+                                 scope=ldb.SCOPE_BASE, attrs=['rIDSetReferences'])
>+
>+            self.assertFalse("rIDSetReferences" in res[0])
>+
>+            (result, out, err) = self.runsubcmd("fsmo", "seize", "--role", "rid", "-H", ldb_url, "-s", smbconf, "--force")
>+            self.assertCmdSuccess(result, out, err)
>+            self.assertEquals(err,"","Shouldn't be any error messages")
>+
>+            # 3. Assert we get the RID Set
>+            res = new_ldb.search(base=server_ref_dn,
>+                                 scope=ldb.SCOPE_BASE, attrs=['rIDSetReferences'])
>+
>+            self.assertTrue("rIDSetReferences" in res[0])
>+        finally:
>+            shutil.rmtree(targetdir, ignore_errors=True)
>+            self._test_force_demote(fsmo_not_owner['dns_name'], "RIDALLOCTEST1")
>+
>+    def _test_join(self, server, netbios_name):
>+        tmpdir = os.path.join(self.tempdir, "targetdir")
>+        creds = self.get_credentials()
>+        cmd = cmd_sambatool.subcommands['domain'].subcommands['join']
>+        result = cmd._run("samba-tool domain join",
>+                          creds.get_realm(),
>+                          "dc", "-U%s%%%s" % (creds.get_username(),
>+                                              creds.get_password()),
>+                          '--targetdir=%s' % tmpdir,
>+                          '--server=%s' % server,
>+                          "--option=netbios name = %s" % netbios_name)
>+        return tmpdir
>+
>+    def _test_force_demote(self, server, netbios_name):
>+        creds = self.get_credentials()
>+        cmd = cmd_sambatool.subcommands['domain'].subcommands['demote']
>+        result = cmd._run("samba-tool domain demote",
>+                          "-U%s%%%s" % (creds.get_username(),
>+                                              creds.get_password()),
>+                          '--server=%s' % server,
>+                          "--remove-other-dead-server=%s" % netbios_name)
>+
>+    def test_offline_manual_seized_ridalloc_with_dbcheck(self):
>+        """Peform the same actions as test_offline_samba_tool_seized_ridalloc,
>+        but do not create the RID set. Confirm that dbcheck correctly creates
>+        the RID Set.
>+
>+        Also check
>+        """
>+        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>+
>+        targetdir = self._test_join(fsmo_not_owner['dns_name'], "RIDALLOCTEST2")
>+        try:
>+            # Connect to the database
>+            ldb_url = "tdb://%s" % os.path.join(targetdir, "private/sam.ldb")
>+            lp = self.get_loadparm()
>+
>+            new_ldb = SamDB(ldb_url, credentials=self.get_credentials(),
>+                            session_info=system_session(lp), lp=lp)
>+
>+            serviceName = new_ldb.get_dsServiceName()
>+            m = ldb.Message()
>+            m.dn = fsmo_dn
>+            m["fSMORoleOwner"] = ldb.MessageElement(serviceName,
>+                                                   ldb.FLAG_MOD_REPLACE,
>+                                                   "fSMORoleOwner")
>+            new_ldb.modify(m)
>+
>+            # 1. Get server name
>+            res = new_ldb.search(base=ldb.Dn(new_ldb, new_ldb.get_serverName()),
>+                                 scope=ldb.SCOPE_BASE, attrs=["serverReference"])
>+            # 2. Get server reference
>+            server_ref_dn = ldb.Dn(new_ldb, res[0]['serverReference'][0])
>+
>+            # Assert that no RID Set has been set
>+            res = new_ldb.search(base=server_ref_dn,
>+                                 scope=ldb.SCOPE_BASE, attrs=['rIDSetReferences'])
>+
>+            self.assertFalse("rIDSetReferences" in res[0])
>+
>+            smbconf = os.path.join(targetdir, "etc/smb.conf")
>+
>+            chk = dbcheck(new_ldb, verbose=False, fix=True, yes=True, quiet=True)
>+
>+            self.assertEqual(chk.check_database(DN=server_ref_dn, scope=ldb.SCOPE_BASE), 1, "Should have fixed one error (missing RID Set)")
>+
>+            # 3. Assert we get the RID Set
>+            res = new_ldb.search(base=server_ref_dn,
>+                                 scope=ldb.SCOPE_BASE, attrs=['rIDSetReferences'])
>+
>+            self.assertTrue("rIDSetReferences" in res[0])
>+        finally:
>+            self._test_force_demote(fsmo_not_owner['dns_name'], "RIDALLOCTEST2")
>+            shutil.rmtree(targetdir, ignore_errors=True)
>+
>+    def test_offline_manual_seized_ridalloc_add_user(self):
>+        """Peform the same actions as test_offline_samba_tool_seized_ridalloc,
>+        but do not create the RID set. Confirm that user-add correctly creates
>+        the RID Set."""
>+        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>+
>+        targetdir = self._test_join(fsmo_not_owner['dns_name'], "RIDALLOCTEST3")
>+        try:
>+            # Connect to the database
>+            ldb_url = "tdb://%s" % os.path.join(targetdir, "private/sam.ldb")
>+            lp = self.get_loadparm()
>+
>+            new_ldb = SamDB(ldb_url, credentials=self.get_credentials(),
>+                            session_info=system_session(lp), lp=lp)
>+
>+            serviceName = new_ldb.get_dsServiceName()
>+            m = ldb.Message()
>+            m.dn = fsmo_dn
>+            m["fSMORoleOwner"] = ldb.MessageElement(serviceName,
>+                                                   ldb.FLAG_MOD_REPLACE,
>+                                                   "fSMORoleOwner")
>+            new_ldb.modify(m)
>+
>+            # 1. Get server name
>+            res = new_ldb.search(base=ldb.Dn(new_ldb, new_ldb.get_serverName()),
>+                                 scope=ldb.SCOPE_BASE, attrs=["serverReference"])
>+            # 2. Get server reference
>+            server_ref_dn = ldb.Dn(new_ldb, res[0]['serverReference'][0])
>+
>+            # Assert that no RID Set has been set
>+            res = new_ldb.search(base=server_ref_dn,
>+                                 scope=ldb.SCOPE_BASE, attrs=['rIDSetReferences'])
>+
>+            self.assertFalse("rIDSetReferences" in res[0])
>+
>+            smbconf = os.path.join(targetdir, "etc/smb.conf")
>+
>+            new_ldb.newuser("ridalloctestuser", "P@ssword!")
>+
>+            # 3. Assert we get the RID Set
>+            res = new_ldb.search(base=server_ref_dn,
>+                                 scope=ldb.SCOPE_BASE, attrs=['rIDSetReferences'])
>+
>+            self.assertTrue("rIDSetReferences" in res[0])
>+
>+        finally:
>+            self._test_force_demote(fsmo_not_owner['dns_name'], "RIDALLOCTEST3")
>+            shutil.rmtree(targetdir, ignore_errors=True)
>+
>+    def test_offline_manual_seized_ridalloc_add_user_as_admin(self):
>+        """Peform the same actions as test_offline_samba_tool_seized_ridalloc,
>+        but do not create the RID set. Confirm that user-add correctly creates
>+        the RID Set."""
>+        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>+
>+        targetdir = self._test_join(fsmo_not_owner['dns_name'], "RIDALLOCTEST4")
>+        try:
>+            # Connect to the database
>+            ldb_url = "tdb://%s" % os.path.join(targetdir, "private/sam.ldb")
>+            lp = self.get_loadparm()
>+
>+            new_ldb = SamDB(ldb_url, credentials=self.get_credentials(),
>+                            session_info=admin_session(lp, self.ldb_dc1.get_domain_sid()), lp=lp)
>+
>+            serviceName = new_ldb.get_dsServiceName()
>+            m = ldb.Message()
>+            m.dn = fsmo_dn
>+            m["fSMORoleOwner"] = ldb.MessageElement(serviceName,
>+                                                   ldb.FLAG_MOD_REPLACE,
>+                                                   "fSMORoleOwner")
>+            new_ldb.modify(m)
>+
>+            # 1. Get server name
>+            res = new_ldb.search(base=ldb.Dn(new_ldb, new_ldb.get_serverName()),
>+                                 scope=ldb.SCOPE_BASE, attrs=["serverReference"])
>+            # 2. Get server reference
>+            server_ref_dn = ldb.Dn(new_ldb, res[0]['serverReference'][0])
>+
>+            # Assert that no RID Set has been set
>+            res = new_ldb.search(base=server_ref_dn,
>+                                 scope=ldb.SCOPE_BASE, attrs=['rIDSetReferences'])
>+
>+            self.assertFalse("rIDSetReferences" in res[0])
>+
>+            smbconf = os.path.join(targetdir, "etc/smb.conf")
>+
>+            # Create a user to allocate a RID Set for itself (the RID master)
>+            new_ldb.newuser("ridalloctestuser", "P@ssword!")
>+
>+            # 3. Assert we get the RID Set
>+            res = new_ldb.search(base=server_ref_dn,
>+                                 scope=ldb.SCOPE_BASE, attrs=['rIDSetReferences'])
>+
>+            self.assertTrue("rIDSetReferences" in res[0])
>+
>+        finally:
>+            self._test_force_demote(fsmo_not_owner['dns_name'], "RIDALLOCTEST4")
>+            shutil.rmtree(targetdir, ignore_errors=True)
>+
>+    def test_join_time_ridalloc(self):
>+        """Perform a join against the RID manager and assert we have a RID Set"""
>+
>+        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>+
>+        targetdir = self._test_join(fsmo_owner['dns_name'], "RIDALLOCTEST5")
>+        try:
>+            # Connect to the database
>+            ldb_url = "tdb://%s" % os.path.join(targetdir, "private/sam.ldb")
>+            smbconf = os.path.join(targetdir, "etc/smb.conf")
>+
>+            lp = self.get_loadparm()
>+            new_ldb = SamDB(ldb_url, credentials=self.get_credentials(),
>+                            session_info=system_session(lp), lp=lp)
>+
>+            # 1. Get server name
>+            res = new_ldb.search(base=ldb.Dn(new_ldb, new_ldb.get_serverName()),
>+                                 scope=ldb.SCOPE_BASE, attrs=["serverReference"])
>+            # 2. Get server reference
>+            server_ref_dn = ldb.Dn(new_ldb, res[0]['serverReference'][0])
>+
>+            # 3. Assert we get the RID Set
>+            res = new_ldb.search(base=server_ref_dn,
>+                                 scope=ldb.SCOPE_BASE, attrs=['rIDSetReferences'])
>+
>+            self.assertTrue("rIDSetReferences" in res[0])
>+        finally:
>+            self._test_force_demote(fsmo_owner['dns_name'], "RIDALLOCTEST5")
>+            shutil.rmtree(targetdir, ignore_errors=True)
>+
>+    def test_rid_set_dbcheck(self):
>+        """Perform a join against the RID manager and assert we have a RID Set.
>+        Using dbcheck, we assert that we can detect out of range users."""
>+
>+        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>+
>+        targetdir = self._test_join(fsmo_owner['dns_name'], "RIDALLOCTEST6")
>+        try:
>+            # Connect to the database
>+            ldb_url = "tdb://%s" % os.path.join(targetdir, "private/sam.ldb")
>+            smbconf = os.path.join(targetdir, "etc/smb.conf")
>+
>+            lp = self.get_loadparm()
>+            new_ldb = SamDB(ldb_url, credentials=self.get_credentials(),
>+                            session_info=system_session(lp), lp=lp)
>+
>+            # 1. Get server name
>+            res = new_ldb.search(base=ldb.Dn(new_ldb, new_ldb.get_serverName()),
>+                                 scope=ldb.SCOPE_BASE, attrs=["serverReference"])
>+            # 2. Get server reference
>+            server_ref_dn = ldb.Dn(new_ldb, res[0]['serverReference'][0])
>+
>+            # 3. Assert we get the RID Set
>+            res = new_ldb.search(base=server_ref_dn,
>+                                 scope=ldb.SCOPE_BASE, attrs=['rIDSetReferences'])
>+
>+            self.assertTrue("rIDSetReferences" in res[0])
>+            rid_set_dn = ldb.Dn(new_ldb, res[0]["rIDSetReferences"][0])
>+
>+            # 4. Add a new user (triggers RID set work)
>+            new_ldb.newuser("ridalloctestuser", "P@ssword!")
>+
>+            # 5. Now fetch the RID SET
>+            rid_set_res = new_ldb.search(base=rid_set_dn,
>+                                         scope=ldb.SCOPE_BASE, attrs=['rIDNextRid',
>+                                                                      'rIDAllocationPool'])
>+            next_pool = int(rid_set_res[0]["rIDAllocationPool"][0])
>+            last_rid = (0xFFFFFFFF00000000 & next_pool) >> 32
>+
>+            # 6. Add user above the ridNextRid and at mid-range.
>+            #
>+            # We can do this with safety because this is an offline DB that will be
>+            # destroyed.
>+            m = ldb.Message()
>+            m.dn = ldb.Dn(new_ldb, "CN=ridsettestuser1,CN=Users")
>+            m.dn.add_base(new_ldb.get_default_basedn())
>+            m['objectClass'] = ldb.MessageElement('user', ldb.FLAG_MOD_ADD, 'objectClass')
>+            m['objectSid'] = ldb.MessageElement(ndr_pack(security.dom_sid(str(new_ldb.get_domain_sid()) + "-%d" % (last_rid - 10))),
>+                                                ldb.FLAG_MOD_ADD,
>+                                                'objectSid')
>+            new_ldb.add(m, controls=["relax:0"])
>+
>+            # 7. Check the RID Set
>+            chk = dbcheck(new_ldb, verbose=False, fix=True, yes=True, quiet=True)
>+
>+            # Should have one error (wrong rIDNextRID)
>+            self.assertEqual(chk.check_database(DN=rid_set_dn, scope=ldb.SCOPE_BASE), 1)
>+
>+            # 8. Assert we get didn't show any other errors
>+            chk = dbcheck(new_ldb, verbose=False, fix=False, quiet=True)
>+
>+            rid_set_res = new_ldb.search(base=rid_set_dn,
>+                                         scope=ldb.SCOPE_BASE, attrs=['rIDNextRid',
>+                                                                      'rIDAllocationPool'])
>+            last_allocated_rid = int(rid_set_res[0]["rIDNextRid"][0])
>+            self.assertEquals(last_allocated_rid, last_rid - 10)
>+
>+            # 9. Assert that the range wasn't thrown away
>+
>+            next_pool = int(rid_set_res[0]["rIDAllocationPool"][0])
>+            self.assertEqual(last_rid, (0xFFFFFFFF00000000 & next_pool) >> 32, "rid pool should have changed")
>+        finally:
>+            self._test_force_demote(fsmo_owner['dns_name'], "RIDALLOCTEST6")
>+            shutil.rmtree(targetdir, ignore_errors=True)
>+
>+
>+    def test_rid_set_dbcheck_after_seize(self):
>+        """Perform a join against the RID manager and assert we have a RID Set.
>+        We seize the RID master role, then using dbcheck, we assert that we can
>+        detect out of range users (and then bump the RID set as required)."""
>+
>+        fsmo_dn = ldb.Dn(self.ldb_dc1, "CN=RID Manager$,CN=System," + self.ldb_dc1.domain_dn())
>+        (fsmo_owner, fsmo_not_owner) = self._determine_fSMORoleOwner(fsmo_dn)
>+
>+        targetdir = self._test_join(fsmo_owner['dns_name'], "RIDALLOCTEST7")
>+        try:
>+            # Connect to the database
>+            ldb_url = "tdb://%s" % os.path.join(targetdir, "private/sam.ldb")
>+            smbconf = os.path.join(targetdir, "etc/smb.conf")
>+
>+            lp = self.get_loadparm()
>+            new_ldb = SamDB(ldb_url, credentials=self.get_credentials(),
>+                            session_info=system_session(lp), lp=lp)
>+
>+            # 1. Get server name
>+            res = new_ldb.search(base=ldb.Dn(new_ldb, new_ldb.get_serverName()),
>+                                 scope=ldb.SCOPE_BASE, attrs=["serverReference"])
>+            # 2. Get server reference
>+            server_ref_dn = ldb.Dn(new_ldb, res[0]['serverReference'][0])
>+
>+            # 3. Assert we get the RID Set
>+            res = new_ldb.search(base=server_ref_dn,
>+                                 scope=ldb.SCOPE_BASE, attrs=['rIDSetReferences'])
>+
>+            self.assertTrue("rIDSetReferences" in res[0])
>+            rid_set_dn = ldb.Dn(new_ldb, res[0]["rIDSetReferences"][0])
>+
>+            # 4. Seize the RID Manager role
>+            (result, out, err) = self.runsubcmd("fsmo", "seize", "--role", "rid", "-H", ldb_url, "-s", smbconf, "--force")
>+            self.assertCmdSuccess(result, out, err)
>+            self.assertEquals(err,"","Shouldn't be any error messages")
>+
>+            # 5. Add a new user (triggers RID set work)
>+            new_ldb.newuser("ridalloctestuser", "P@ssword!")
>+
>+            # 6. Now fetch the RID SET
>+            rid_set_res = new_ldb.search(base=rid_set_dn,
>+                                         scope=ldb.SCOPE_BASE, attrs=['rIDNextRid',
>+                                                                      'rIDAllocationPool'])
>+            next_pool = int(rid_set_res[0]["rIDAllocationPool"][0])
>+            last_rid = (0xFFFFFFFF00000000 & next_pool) >> 32
>+
>+            # 7. Add user above the ridNextRid and at almost the end of the range.
>+            #
>+            m = ldb.Message()
>+            m.dn = ldb.Dn(new_ldb, "CN=ridsettestuser2,CN=Users")
>+            m.dn.add_base(new_ldb.get_default_basedn())
>+            m['objectClass'] = ldb.MessageElement('user', ldb.FLAG_MOD_ADD, 'objectClass')
>+            m['objectSid'] = ldb.MessageElement(ndr_pack(security.dom_sid(str(new_ldb.get_domain_sid()) + "-%d" % (last_rid - 3))),
>+                                                ldb.FLAG_MOD_ADD,
>+                                                'objectSid')
>+            new_ldb.add(m, controls=["relax:0"])
>+
>+            # 8. Add user above the ridNextRid and at the end of the range
>+            m = ldb.Message()
>+            m.dn = ldb.Dn(new_ldb, "CN=ridsettestuser3,CN=Users")
>+            m.dn.add_base(new_ldb.get_default_basedn())
>+            m['objectClass'] = ldb.MessageElement('user', ldb.FLAG_MOD_ADD, 'objectClass')
>+            m['objectSid'] = ldb.MessageElement(ndr_pack(security.dom_sid(str(new_ldb.get_domain_sid()) + "-%d" % last_rid)),
>+                                                ldb.FLAG_MOD_ADD,
>+                                                'objectSid')
>+            new_ldb.add(m, controls=["relax:0"])
>+
>+            chk = dbcheck(new_ldb, verbose=False, fix=True, yes=True, quiet=True)
>+
>+            # Should have fixed two errors (wrong ridNextRid)
>+            self.assertEqual(chk.check_database(DN=rid_set_dn, scope=ldb.SCOPE_BASE), 2)
>+
>+            # 9. Assert we get didn't show any other errors
>+            chk = dbcheck(new_ldb, verbose=False, fix=False, quiet=True)
>+
>+            # 10. Add another user (checks RID rollover)
>+            # We have seized the role, so we can do that.
>+            new_ldb.newuser("ridalloctestuser3", "P@ssword!")
>+
>+            rid_set_res = new_ldb.search(base=rid_set_dn,
>+                                         scope=ldb.SCOPE_BASE, attrs=['rIDNextRid',
>+                                                                      'rIDAllocationPool'])
>+            next_pool = int(rid_set_res[0]["rIDAllocationPool"][0])
>+            self.assertNotEqual(last_rid, (0xFFFFFFFF00000000 & next_pool) >> 32, "rid pool should have changed")
>+        finally:
>+            self._test_force_demote(fsmo_owner['dns_name'], "RIDALLOCTEST7")
>+            shutil.rmtree(targetdir, ignore_errors=True)
>-- 
>1.9.1
>
>
>From 33e40dfae91d61be8a4ed377f078f201112359f5 Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Fri, 28 Oct 2016 16:08:57 +1300
>Subject: [PATCH 14/24] dsdb: Add python hooks to allocate a RID set and
> allocate a RID pool
>
>This will help us to correct errors during dbcheck
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=9954
>(cherry picked from commit 035df7adbe9cc119324275275c2605433f6c4292)
>---
> python/samba/samdb.py                     |  8 ++++
> source4/dsdb/pydsdb.c                     | 74 +++++++++++++++++++++++++++++++
> source4/dsdb/samdb/ldb_modules/ridalloc.c | 12 +++--
> source4/dsdb/samdb/ldb_modules/samldb.c   | 51 +++++++++++++++++++++
> source4/dsdb/samdb/samdb.h                | 10 +++++
> source4/setup/schema_samba4.ldif          |  2 +
> 6 files changed, 154 insertions(+), 3 deletions(-)
>
>diff --git a/python/samba/samdb.py b/python/samba/samdb.py
>index 3d7ea3e..eabe363 100644
>--- a/python/samba/samdb.py
>+++ b/python/samba/samdb.py
>@@ -963,3 +963,11 @@ accountExpires: %u
>             return dsdb._dsdb_garbage_collect_tombstones(self, dn,
>                                                          current_time,
>                                                          tombstone_lifetime)
>+
>+    def create_own_rid_set(self):
>+        '''create a RID set for this DSA'''
>+        return dsdb._dsdb_create_own_rid_set(self)
>+
>+    def allocate_rid(self):
>+        '''return a new RID from the RID Pool on this DSA'''
>+        return dsdb._dsdb_allocate_rid(self)
>diff --git a/source4/dsdb/pydsdb.c b/source4/dsdb/pydsdb.c
>index e53a245..ab1d0d2 100644
>--- a/source4/dsdb/pydsdb.c
>+++ b/source4/dsdb/pydsdb.c
>@@ -1078,6 +1078,74 @@ static PyObject *py_dsdb_am_pdc(PyObject *self, PyObject *args)
> 	return PyBool_FromLong(am_pdc);
> }
> 
>+/*
>+  call DSDB_EXTENDED_CREATE_OWN_RID_SET to get a new RID set for this server
>+ */
>+static PyObject *py_dsdb_create_own_rid_set(PyObject *self, PyObject *args)
>+{
>+	PyObject *py_ldb;
>+	struct ldb_context *ldb;
>+	int ret;
>+	struct ldb_result *ext_res;
>+
>+	if (!PyArg_ParseTuple(args, "O", &py_ldb))
>+		return NULL;
>+
>+	PyErr_LDB_OR_RAISE(py_ldb, ldb);
>+
>+	/*
>+	 * Run DSDB_EXTENDED_CREATE_OWN_RID_SET to get a RID set
>+	 */
>+
>+	ret = ldb_extended(ldb, DSDB_EXTENDED_CREATE_OWN_RID_SET, NULL, &ext_res);
>+
>+	PyErr_LDB_ERROR_IS_ERR_RAISE(py_ldb_get_exception(), ret, ldb);
>+
>+	TALLOC_FREE(ext_res);
>+
>+	Py_RETURN_NONE;
>+}
>+
>+/*
>+  call DSDB_EXTENDED_ALLOCATE_RID to get a new RID set for this server
>+ */
>+static PyObject *py_dsdb_allocate_rid(PyObject *self, PyObject *args)
>+{
>+	PyObject *py_ldb;
>+	struct ldb_context *ldb;
>+	int ret;
>+	uint32_t rid;
>+	struct ldb_result *ext_res = NULL;
>+	struct dsdb_extended_allocate_rid *rid_return = NULL;
>+	if (!PyArg_ParseTuple(args, "O", &py_ldb)) {
>+		return NULL;
>+	}
>+
>+	PyErr_LDB_OR_RAISE(py_ldb, ldb);
>+
>+	rid_return = talloc_zero(ldb, struct dsdb_extended_allocate_rid);
>+	if (rid_return == NULL) {
>+		return PyErr_NoMemory();
>+	}
>+
>+	/*
>+	 * Run DSDB_EXTENDED_ALLOCATE_RID to get a new RID
>+	 */
>+
>+	ret = ldb_extended(ldb, DSDB_EXTENDED_ALLOCATE_RID, rid_return, &ext_res);
>+	if (ret != LDB_SUCCESS) {
>+		TALLOC_FREE(rid_return);
>+		TALLOC_FREE(ext_res);
>+		PyErr_LDB_ERROR_IS_ERR_RAISE(py_ldb_get_exception(), ret, ldb);
>+	}
>+
>+	rid = rid_return->rid;
>+	TALLOC_FREE(rid_return);
>+	TALLOC_FREE(ext_res);
>+
>+	return PyInt_FromLong(rid);
>+}
>+
> static PyObject *py_dsdb_garbage_collect_tombstones(PyObject *self, PyObject *args)
> {
> 	PyObject *py_ldb, *py_list_dn;
>@@ -1245,6 +1313,12 @@ static PyMethodDef py_dsdb_methods[] = {
> 	{ "_dsdb_garbage_collect_tombstones", (PyCFunction)py_dsdb_garbage_collect_tombstones, METH_VARARGS,
> 		"_dsdb_kcc_check_deleted(samdb, [dn], current_time, tombstone_lifetime)"
> 		" -> (num_objects_expunged, num_links_expunged)" },
>+	{ "_dsdb_create_own_rid_set", (PyCFunction)py_dsdb_create_own_rid_set, METH_VARARGS,
>+		"_dsdb_create_own_rid_set(samdb)"
>+		" -> None" },
>+	{ "_dsdb_allocate_rid", (PyCFunction)py_dsdb_allocate_rid, METH_VARARGS,
>+		"_dsdb_allocate_rid(samdb)"
>+		" -> RID" },
> 	{ NULL }
> };
> 
>diff --git a/source4/dsdb/samdb/ldb_modules/ridalloc.c b/source4/dsdb/samdb/ldb_modules/ridalloc.c
>index 4c619b7..b5c7f52 100644
>--- a/source4/dsdb/samdb/ldb_modules/ridalloc.c
>+++ b/source4/dsdb/samdb/ldb_modules/ridalloc.c
>@@ -401,8 +401,8 @@ static int ridalloc_create_rid_set_ntds(struct ldb_module *module, TALLOC_CTX *m
> /*
>   create a RID Set object for this DC
>  */
>-static int ridalloc_create_own_rid_set(struct ldb_module *module, TALLOC_CTX *mem_ctx,
>-				       struct ldb_dn **dn, struct ldb_request *parent)
>+int ridalloc_create_own_rid_set(struct ldb_module *module, TALLOC_CTX *mem_ctx,
>+				struct ldb_dn **dn, struct ldb_request *parent)
> {
> 	TALLOC_CTX *tmp_ctx = talloc_new(mem_ctx);
> 	struct ldb_dn *rid_manager_dn, *fsmo_role_dn;
>@@ -466,7 +466,8 @@ static int ridalloc_create_own_rid_set(struct ldb_module *module, TALLOC_CTX *me
>   get a new RID pool for ourselves
>   also returns the first rid for the new pool
>  */
>-static int ridalloc_new_own_pool(struct ldb_module *module, uint64_t *new_pool, struct ldb_request *parent)
>+
>+int ridalloc_new_own_pool(struct ldb_module *module, uint64_t *new_pool, struct ldb_request *parent)
> {
> 	TALLOC_CTX *tmp_ctx = talloc_new(module);
> 	struct ldb_dn *rid_manager_dn, *fsmo_role_dn;
>@@ -685,6 +686,11 @@ int ridalloc_allocate_rid(struct ldb_module *module, uint32_t *rid, struct ldb_r
> 
> /*
>   called by DSDB_EXTENDED_ALLOCATE_RID_POOL extended operation in samldb
>+
>+  This is for the DRS server to allocate a RID Pool for another server.
>+
>+  Called by another server over DRS (which calls this extended
>+  operation), it runs on the RID Manager only.
>  */
> int ridalloc_allocate_rid_pool_fsmo(struct ldb_module *module, struct dsdb_fsmo_extended_op *exop,
> 				    struct ldb_request *parent)
>diff --git a/source4/dsdb/samdb/ldb_modules/samldb.c b/source4/dsdb/samdb/ldb_modules/samldb.c
>index cc101a6..b33cf24 100644
>--- a/source4/dsdb/samdb/ldb_modules/samldb.c
>+++ b/source4/dsdb/samdb/ldb_modules/samldb.c
>@@ -3869,12 +3869,63 @@ static int samldb_extended_allocate_rid_pool(struct ldb_module *module, struct l
> 	return ldb_module_done(req, NULL, NULL, LDB_SUCCESS);
> }
> 
>+static int samldb_extended_allocate_rid(struct ldb_module *module, struct ldb_request *req)
>+{
>+	struct ldb_context *ldb = ldb_module_get_ctx(module);
>+	struct dsdb_extended_allocate_rid *exop;
>+	int ret;
>+
>+	exop = talloc_get_type(req->op.extended.data,
>+			       struct dsdb_extended_allocate_rid);
>+	if (!exop) {
>+		ldb_set_errstring(ldb,
>+				  "samldb_extended_allocate_rid: invalid extended data");
>+		return LDB_ERR_PROTOCOL_ERROR;
>+	}
>+
>+	ret = ridalloc_allocate_rid(module, &exop->rid, req);
>+	if (ret != LDB_SUCCESS) {
>+		return ret;
>+	}
>+
>+	return ldb_module_done(req, NULL, NULL, LDB_SUCCESS);
>+}
>+
>+static int samldb_extended_create_own_rid_set(struct ldb_module *module, struct ldb_request *req)
>+{
>+	struct ldb_context *ldb = ldb_module_get_ctx(module);
>+	int ret;
>+	struct ldb_dn *dn;
>+
>+	if (req->op.extended.data != NULL) {
>+		ldb_set_errstring(ldb,
>+				  "samldb_extended_allocate_rid_pool_for_us: invalid extended data (should be NULL)");
>+		return LDB_ERR_PROTOCOL_ERROR;
>+	}
>+
>+	ret = ridalloc_create_own_rid_set(module, req,
>+					  &dn, req);
>+	if (ret != LDB_SUCCESS) {
>+		return ret;
>+	}
>+
>+	return ldb_module_done(req, NULL, NULL, LDB_SUCCESS);
>+}
>+
> static int samldb_extended(struct ldb_module *module, struct ldb_request *req)
> {
> 	if (strcmp(req->op.extended.oid, DSDB_EXTENDED_ALLOCATE_RID_POOL) == 0) {
> 		return samldb_extended_allocate_rid_pool(module, req);
> 	}
> 
>+	if (strcmp(req->op.extended.oid, DSDB_EXTENDED_ALLOCATE_RID) == 0) {
>+		return samldb_extended_allocate_rid(module, req);
>+	}
>+
>+	if (strcmp(req->op.extended.oid, DSDB_EXTENDED_CREATE_OWN_RID_SET) == 0) {
>+		return samldb_extended_create_own_rid_set(module, req);
>+	}
>+
> 	return ldb_next_request(module, req);
> }
> 
>diff --git a/source4/dsdb/samdb/samdb.h b/source4/dsdb/samdb/samdb.h
>index c7260d0..176d065 100644
>--- a/source4/dsdb/samdb/samdb.h
>+++ b/source4/dsdb/samdb/samdb.h
>@@ -281,6 +281,16 @@ struct dsdb_fsmo_extended_op {
> 	struct GUID destination_dsa_guid;
> };
> 
>+/* this takes no data */
>+#define DSDB_EXTENDED_CREATE_OWN_RID_SET "1.3.6.1.4.1.7165.4.4.8"
>+
>+/* this takes a struct dsdb_extended_allocate_rid */
>+#define DSDB_EXTENDED_ALLOCATE_RID "1.3.6.1.4.1.7165.4.4.9"
>+
>+struct dsdb_extended_allocate_rid {
>+	uint32_t rid;
>+};
>+
> /*
>  * passed from the descriptor module in order to
>  * store the recalucated nTSecurityDescriptor without
>diff --git a/source4/setup/schema_samba4.ldif b/source4/setup/schema_samba4.ldif
>index 2e4c16d..04505de 100644
>--- a/source4/setup/schema_samba4.ldif
>+++ b/source4/setup/schema_samba4.ldif
>@@ -226,6 +226,8 @@
> #Allocated: DSDB_EXTENDED_ALLOCATE_RID_POOL 1.3.6.1.4.1.7165.4.4.5
> #Allocated: DSDB_EXTENDED_SCHEMA_UPGRADE_IN_PROGRESS_OID 1.3.6.1.4.1.7165.4.4.6
> #Allocated: DSDB_EXTENDED_SEC_DESC_PROPAGATION_OID 1.3.6.1.4.1.7165.4.4.7
>+#Allocated: DSDB_EXTENDED_CREATE_OWN_RID_SET 1.3.6.1.4.1.7165.4.4.8
>+#Allocated: DSDB_EXTENDED_ALLOCATE_RID 1.3.6.1.4.1.7165.4.4.9
> 
> 
> ############
>-- 
>1.9.1
>
>
>From 71c2d7e0b46bf6b7fd66e4ad35699b7530600da2 Mon Sep 17 00:00:00 2001
>From: Clive Ferreira <cliveferreira@catalyst.net.nz>
>Date: Thu, 27 Oct 2016 17:28:01 +1300
>Subject: [PATCH 15/24] dbcheck: confirm RID Set presence and consistency
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=9954
>(cherry picked from commit 7fd5be535ade5ed119d869c8f215aa605aba2125)
>---
> python/samba/dbchecker.py | 132 ++++++++++++++++++++++++++++++++++++++++++++++
> selftest/knownfail        |   1 -
> 2 files changed, 132 insertions(+), 1 deletion(-)
>
>diff --git a/python/samba/dbchecker.py b/python/samba/dbchecker.py
>index ef54174..72b2eea 100644
>--- a/python/samba/dbchecker.py
>+++ b/python/samba/dbchecker.py
>@@ -32,6 +32,7 @@ from samba.dcerpc import security
> from samba.descriptor import get_wellknown_sds, get_diff_sds
> from samba.auth import system_session, admin_session
> from samba.netcmd import CommandError
>+from samba.netcmd.fsmo import get_fsmo_roleowner
> 
> 
> class dbcheck(object):
>@@ -79,6 +80,7 @@ class dbcheck(object):
>         self.fix_base64_userparameters = False
>         self.fix_utf8_userparameters = False
>         self.fix_doubled_userparameters = False
>+        self.fix_sid_rid_set_conflict = False
>         self.reset_well_known_acls = reset_well_known_acls
>         self.reset_all_well_known_acls = False
>         self.in_transaction = in_transaction
>@@ -92,6 +94,7 @@ class dbcheck(object):
>         self.fix_all_missing_objectclass = False
>         self.fix_missing_deleted_objects = False
>         self.fix_replica_locations = False
>+        self.fix_missing_rid_set_master = False
> 
>         self.dn_set = set()
>         self.link_id_cache = {}
>@@ -157,6 +160,27 @@ class dbcheck(object):
>         if len(forest) == 1:
>             self.dns_partitions.append((ldb.Dn(self.samdb, domaindns_zone), forest[0]))
> 
>+        fsmo_dn = ldb.Dn(self.samdb, "CN=RID Manager$,CN=System," + self.samdb.domain_dn())
>+        rid_master = get_fsmo_roleowner(self.samdb, fsmo_dn, "rid")
>+        if ldb.Dn(self.samdb, self.samdb.get_dsServiceName()) == rid_master:
>+            self.is_rid_master = True
>+        else:
>+            self.is_rid_master = False
>+
>+        # To get your rid set
>+        # 1. Get server name
>+        res = self.samdb.search(base=ldb.Dn(self.samdb, self.samdb.get_serverName()),
>+                                scope=ldb.SCOPE_BASE, attrs=["serverReference"])
>+        # 2. Get server reference
>+        self.server_ref_dn = ldb.Dn(self.samdb, res[0]['serverReference'][0])
>+
>+        # 3. Get RID Set
>+        res = self.samdb.search(base=self.server_ref_dn,
>+                                scope=ldb.SCOPE_BASE, attrs=['rIDSetReferences'])
>+        if "rIDSetReferences" in res[0]:
>+            self.rid_set_dn = ldb.Dn(self.samdb, res[0]['rIDSetReferences'][0])
>+        else:
>+            self.rid_set_dn = None
> 
>     def check_database(self, DN=None, scope=ldb.SCOPE_SUBTREE, controls=[], attrs=['*']):
>         '''perform a database check, returning the number of errors found'''
>@@ -1863,6 +1887,114 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
>                     self.err_replica_locations(obj, msg.dn, location)
>                     error_count += 1
> 
>+        if dn == self.server_ref_dn:
>+            # Check we have a valid RID Set
>+            if "*" in attrs or "rIDSetReferences" in attrs:
>+                if "rIDSetReferences" not in obj:
>+                    # NO RID SET reference
>+                    # We are RID master, allocate it.
>+                    error_count += 1
>+
>+                    if self.is_rid_master:
>+                        # Allocate a RID Set
>+                        if self.confirm_all('Allocate the missing RID set for RID master?',
>+                                            'fix_missing_rid_set_master'):
>+
>+                            # We don't have auto-transaction logic on
>+                            # extended operations, so we have to do it
>+                            # here.
>+
>+                            self.samdb.transaction_start()
>+
>+                            try:
>+                                self.samdb.create_own_rid_set()
>+
>+                            except:
>+                                self.samdb.transaction_cancel()
>+                                raise
>+
>+                            self.samdb.transaction_commit()
>+
>+
>+                    elif not self.samdb.am_rodc():
>+                        self.report("No RID Set found for this server: %s, and we are not the RID Master (so can not self-allocate)" % dn)
>+
>+
>+        # Check some details of our own RID Set
>+        if dn == self.rid_set_dn:
>+            res = self.samdb.search(base=self.rid_set_dn, scope=ldb.SCOPE_BASE,
>+                                    attrs=["rIDAllocationPool",
>+                                           "rIDPreviousAllocationPool",
>+                                           "rIDUsedPool",
>+                                           "rIDNextRID"])
>+            if "rIDAllocationPool" not in res[0]:
>+                self.report("No rIDAllocationPool found in %s" % dn)
>+                error_count += 1
>+            else:
>+                next_pool = int(res[0]["rIDAllocationPool"][0])
>+
>+                high = (0xFFFFFFFF00000000 & next_pool) >> 32
>+                low = 0x00000000FFFFFFFF & next_pool
>+
>+                if high <= low:
>+                    self.report("Invalid RID set %d-%s, %d > %d!" % (low, high, low, high))
>+                    error_count += 1
>+
>+                if "rIDNextRID" in res[0]:
>+                    next_free_rid = int(res[0]["rIDNextRID"][0])
>+                else:
>+                    next_free_rid = 0
>+
>+                if next_free_rid == 0:
>+                    next_free_rid = low
>+                else:
>+                    next_free_rid += 1
>+
>+                # Check the remainder of this pool for conflicts.  If
>+                # ridalloc_allocate_rid() moves to a new pool, this
>+                # will be above high, so we will stop.
>+                while next_free_rid <= high:
>+                    sid = "%s-%d" % (self.samdb.get_domain_sid(), next_free_rid)
>+                    try:
>+                        res = self.samdb.search(base="<SID=%s>" % sid, scope=ldb.SCOPE_BASE,
>+                                                attrs=[])
>+                    except ldb.LdbError, (enum, estr):
>+                        if enum != ldb.ERR_NO_SUCH_OBJECT:
>+                            raise
>+                        res = None
>+                    if res is not None:
>+                        self.report("SID %s for %s conflicts with our current RID set in %s" % (sid, res[0].dn, dn))
>+                        error_count += 1
>+
>+                        if self.confirm_all('Fix conflict between SID %s and RID pool in %s by allocating a new RID?'
>+                                            % (sid, dn),
>+                                            'fix_sid_rid_set_conflict'):
>+                            self.samdb.transaction_start()
>+
>+                            # This will burn RIDs, which will move
>+                            # past the conflict.  We then check again
>+                            # to see if the new RID conflicts, until
>+                            # the end of the current pool.  We don't
>+                            # look at the next pool to avoid burning
>+                            # all RIDs in one go in some strange
>+                            # failure case.
>+                            try:
>+                                while True:
>+                                    allocated_rid = self.samdb.allocate_rid()
>+                                    if allocated_rid >= next_free_rid:
>+                                        next_free_rid = allocated_rid + 1
>+                                        break
>+                            except:
>+                                self.samdb.transaction_cancel()
>+                                raise
>+
>+                            self.samdb.transaction_commit()
>+                        else:
>+                            break
>+                    else:
>+                        next_free_rid += 1
>+
>+
>         return error_count
> 
>     ################################################################
>diff --git a/selftest/knownfail b/selftest/knownfail
>index 26bbb6d..cb309a6 100644
>--- a/selftest/knownfail
>+++ b/selftest/knownfail
>@@ -292,7 +292,6 @@
> #ntvfs server blocks copychunk with execute access on read handle
> ^samba4.smb2.ioctl.copy_chunk_bad_access
> ^samba4.drs.getnc_exop.python.*getnc_exop.DrsReplicaPrefixMapTestCase.test_regular_prefix_map_ex_attid.*
>-^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_offline_manual_seized_ridalloc_with_dbcheck
> ^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_offline_ridalloc
> ^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_offline_samba_tool_seized_ridalloc
> ^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_join_time_ridalloc
>-- 
>1.9.1
>
>
>From 545b9a68f6cba2bfdfc082aa7a0aeb966efe7347 Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Thu, 3 Nov 2016 13:30:56 +1300
>Subject: [PATCH 16/24] dbcheck: Correctly initialise keep_transaction in
> missing_parent test
>
>Otherwise there is no point to this variable, we are trying to work out
>if the subsequent modify succeded
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=9954
>(cherry picked from commit 09537a67b0e761c834fb7c14d7e8d55e07fc5156)
>---
> python/samba/dbchecker.py | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/python/samba/dbchecker.py b/python/samba/dbchecker.py
>index 72b2eea..3fcfbc0 100644
>--- a/python/samba/dbchecker.py
>+++ b/python/samba/dbchecker.py
>@@ -685,7 +685,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
>             self.report('Not moving object %s into LostAndFound' % (obj.dn))
>             return
> 
>-        keep_transaction = True
>+        keep_transaction = False
>         self.samdb.transaction_start()
>         try:
>             nc_root = self.samdb.get_nc_root(obj.dn);
>-- 
>1.9.1
>
>
>From 5efcd549eec2951e8bad30c84dd0ed768d54f247 Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Mon, 31 Oct 2016 10:41:39 +1300
>Subject: [PATCH 17/24] dsdb: Create RID Set as SYSTEM
>
>We do not want random users with add-user rights to own the new RID Set for this
>server, and the ridSet class is thankfully system-only.
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=9954
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>(cherry picked from commit fe90dadd2cd13c20484c06318724c592e5cf298e)
>---
> source4/dsdb/samdb/ldb_modules/ridalloc.c | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
>diff --git a/source4/dsdb/samdb/ldb_modules/ridalloc.c b/source4/dsdb/samdb/ldb_modules/ridalloc.c
>index b5c7f52..d3463e6 100644
>--- a/source4/dsdb/samdb/ldb_modules/ridalloc.c
>+++ b/source4/dsdb/samdb/ldb_modules/ridalloc.c
>@@ -347,8 +347,12 @@ static int ridalloc_create_rid_set_ntds(struct ldb_module *module, TALLOC_CTX *m
> 
> 	/* we need this to go all the way to the top of the module
> 	 * stack, as we need all the extra attributes added (including
>-	 * complex ones like ntsecuritydescriptor) */
>-	ret = dsdb_module_add(module, msg, DSDB_FLAG_TOP_MODULE | DSDB_MODIFY_RELAX, parent);
>+	 * complex ones like ntsecuritydescriptor).  We must do this
>+	 * as system, otherwise a user might end up owning the RID
>+	 * set, and that would be bad... */
>+	ret = dsdb_module_add(module, msg,
>+			      DSDB_FLAG_TOP_MODULE | DSDB_FLAG_AS_SYSTEM
>+			      | DSDB_MODIFY_RELAX, parent);
> 	if (ret != LDB_SUCCESS) {
> 		ldb_asprintf_errstring(ldb, "Failed to add RID Set %s - %s",
> 				       ldb_dn_get_linearized(msg->dn),
>-- 
>1.9.1
>
>
>From 6dc91a319d7da0ff23393cf1d0f634b993a8843f Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Mon, 31 Oct 2016 16:19:37 +1300
>Subject: [PATCH 18/24] dsdb: Rework DSDB code to use WERROR
>
>The WERROR codes are more descriptive for DSDB issues, and almost all the code was
>converting from WERROR to NTSTATUS.  This will allow us to better catch specific
>errors like WERR_DS_DRA_MISSING_PARENT
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398
>(cherry picked from commit 46fefb251f61b274cb687f6d1cb0a0a97fb36b44)
>---
> source4/libnet/libnet_become_dc.c |   8 +--
> source4/libnet/libnet_become_dc.h |  16 ++---
> source4/libnet/libnet_vampire.c   | 125 ++++++++++++++++++++++----------------
> source4/libnet/py_net.c           |  10 +--
> 4 files changed, 89 insertions(+), 70 deletions(-)
>
>diff --git a/source4/libnet/libnet_become_dc.c b/source4/libnet/libnet_become_dc.c
>index 9cfb993..fdd2a63 100644
>--- a/source4/libnet/libnet_become_dc.c
>+++ b/source4/libnet/libnet_become_dc.c
>@@ -2684,7 +2684,7 @@ static WERROR becomeDC_drsuapi_pull_partition_recv(struct libnet_BecomeDC_state
> 	struct GUID *source_dsa_invocation_id = NULL;
> 	struct drsuapi_DsReplicaHighWaterMark *new_highwatermark = NULL;
> 	bool more_data = false;
>-	NTSTATUS nt_status;
>+	WERROR werr;
> 
> 	if (!W_ERROR_IS_OK(r->out.result)) {
> 		return r->out.result;
>@@ -2783,9 +2783,9 @@ static WERROR becomeDC_drsuapi_pull_partition_recv(struct libnet_BecomeDC_state
> 	 */
> 	s->_sc.gensec_skey	= &drsuapi_p->gensec_skey;
> 
>-	nt_status = partition->store_chunk(s->callbacks.private_data, &s->_sc);
>-	if (!NT_STATUS_IS_OK(nt_status)) {
>-		return ntstatus_to_werror(nt_status);
>+	werr = partition->store_chunk(s->callbacks.private_data, &s->_sc);
>+	if (!W_ERROR_IS_OK(werr)) {
>+		return werr;
> 	}
> 
> 	return WERR_OK;
>diff --git a/source4/libnet/libnet_become_dc.h b/source4/libnet/libnet_become_dc.h
>index b3b08bd..f050c22 100644
>--- a/source4/libnet/libnet_become_dc.h
>+++ b/source4/libnet/libnet_become_dc.h
>@@ -97,8 +97,8 @@ struct libnet_BecomeDC_Partition {
> 	bool more_data;
> 	uint32_t replica_flags;
> 
>-	NTSTATUS (*store_chunk)(void *private_data,
>-				const struct libnet_BecomeDC_StoreChunk *info);
>+	WERROR (*store_chunk)(void *private_data,
>+			      const struct libnet_BecomeDC_StoreChunk *info);
> };
> 
> struct libnet_BecomeDC_StoreChunk {
>@@ -123,12 +123,12 @@ struct libnet_BecomeDC_Callbacks {
> 				  const struct libnet_BecomeDC_CheckOptions *info);
> 	NTSTATUS (*prepare_db)(void *private_data,
> 			       const struct libnet_BecomeDC_PrepareDB *info);
>-	NTSTATUS (*schema_chunk)(void *private_data,
>-				 const struct libnet_BecomeDC_StoreChunk *info);
>-	NTSTATUS (*config_chunk)(void *private_data,
>-				 const struct libnet_BecomeDC_StoreChunk *info);
>-	NTSTATUS (*domain_chunk)(void *private_data,
>-				 const struct libnet_BecomeDC_StoreChunk *info);
>+	WERROR (*schema_chunk)(void *private_data,
>+			       const struct libnet_BecomeDC_StoreChunk *info);
>+	WERROR (*config_chunk)(void *private_data,
>+			       const struct libnet_BecomeDC_StoreChunk *info);
>+	WERROR (*domain_chunk)(void *private_data,
>+			       const struct libnet_BecomeDC_StoreChunk *info);
> };
> 
> struct libnet_BecomeDC {
>diff --git a/source4/libnet/libnet_vampire.c b/source4/libnet/libnet_vampire.c
>index 8d68f8f..91d951f 100644
>--- a/source4/libnet/libnet_vampire.c
>+++ b/source4/libnet/libnet_vampire.c
>@@ -216,8 +216,8 @@ NTSTATUS libnet_vampire_cb_check_options(void *private_data,
> 	return NT_STATUS_OK;
> }
> 
>-static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s,
>-					       const struct libnet_BecomeDC_StoreChunk *c)
>+static WERROR libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s,
>+					     const struct libnet_BecomeDC_StoreChunk *c)
> {
> 	WERROR status;
> 	struct dsdb_schema_prefixmap *pfm_remote;
>@@ -244,9 +244,13 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 	DEBUG(0,("Analyze and apply schema objects\n"));
> 
> 	s_dsa			= talloc_zero(s, struct repsFromTo1);
>-	NT_STATUS_HAVE_NO_MEMORY(s_dsa);
>+	if (s_dsa == NULL) {
>+		return WERR_NOT_ENOUGH_MEMORY;
>+	}
> 	s_dsa->other_info	= talloc(s_dsa, struct repsFromTo1OtherInfo);
>-	NT_STATUS_HAVE_NO_MEMORY(s_dsa->other_info);
>+	if (s_dsa->other_info == NULL) {
>+		return WERR_NOT_ENOUGH_MEMORY;
>+	}
> 
> 	switch (c->ctr_level) {
> 	case 1:
>@@ -272,19 +276,19 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 		uptodateness_vector		= c->ctr6->uptodateness_vector;
> 		break;
> 	default:
>-		return NT_STATUS_INVALID_PARAMETER;
>+		return WERR_INVALID_PARAMETER;
> 	}
> 	/* We must set these up to ensure the replMetaData is written
> 	 * correctly, before our NTDS Settings entry is replicated */
> 	ok = samdb_set_ntds_invocation_id(s->ldb, &c->dest_dsa->invocation_id);
> 	if (!ok) {
> 		DEBUG(0,("Failed to set cached ntds invocationId\n"));
>-		return NT_STATUS_FOOBAR;
>+		return WERR_INTERNAL_ERROR;
> 	}
> 	ok = samdb_set_ntds_objectGUID(s->ldb, &c->dest_dsa->ntds_guid);
> 	if (!ok) {
> 		DEBUG(0,("Failed to set cached ntds objectGUID\n"));
>-		return NT_STATUS_FOOBAR;
>+		return WERR_INTERNAL_ERROR;
> 	}
> 
> 	status = dsdb_schema_pfm_from_drsuapi_pfm(mapping_ctr, true,
>@@ -292,7 +296,7 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 	if (!W_ERROR_IS_OK(status)) {
> 		DEBUG(0,(__location__ ": Failed to decode remote prefixMap: %s",
> 			 win_errstr(status)));
>-		return werror_to_ntstatus(status);
>+		return status;
> 	}
> 
> 	s_dsa->replica_flags		= DRSUAPI_DRS_WRIT_REP
>@@ -301,14 +305,18 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 	memset(s_dsa->schedule, 0x11, sizeof(s_dsa->schedule));
> 
> 	tmp_dns_name	= GUID_string(s_dsa->other_info, &s_dsa->source_dsa_obj_guid);
>-	NT_STATUS_HAVE_NO_MEMORY(tmp_dns_name);
>+	if (tmp_dns_name == NULL) {
>+		return WERR_NOT_ENOUGH_MEMORY;
>+	}
> 	tmp_dns_name	= talloc_asprintf_append_buffer(tmp_dns_name, "._msdcs.%s", c->forest->dns_name);
>-	NT_STATUS_HAVE_NO_MEMORY(tmp_dns_name);
>+	if (tmp_dns_name == NULL) {
>+		return WERR_NOT_ENOUGH_MEMORY;
>+	}
> 	s_dsa->other_info->dns_name = tmp_dns_name;
> 
> 	if (s->self_made_schema == NULL) {
> 		DEBUG(0,("libnet_vampire_cb_apply_schema: called with out self_made_schema\n"));
>-		return NT_STATUS_INTERNAL_ERROR;
>+		return WERR_INTERNAL_ERROR;
> 	}
> 
> 	schema_ldb = provision_get_schema(s, s->lp_ctx,
>@@ -323,7 +331,7 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 		ret = dsdb_reference_schema(s->ldb, provision_schema, false);
> 		if (ret != LDB_SUCCESS) {
> 			DEBUG(0,("Failed to attach schema from local provision using remote prefixMap."));
>-			return NT_STATUS_UNSUCCESSFUL;
>+			return WERR_INTERNAL_ERROR;
> 		}
> 		talloc_free(schema_ldb);
> 	}
>@@ -345,7 +353,7 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 	if (!W_ERROR_IS_OK(status)) {
> 		DEBUG(0, ("%s: dsdb_repl_resolve_working_schema() failed: %s",
> 			  __location__, win_errstr(status)));
>-		return werror_to_ntstatus(status);
>+		return status;
> 	}
> 
> 	/* free temp objects for 1st conversion phase */
>@@ -360,7 +368,7 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 	ret = dsdb_set_schema(s->ldb, s->self_made_schema);
> 	if (ret != LDB_SUCCESS) {
> 		DEBUG(0,("Failed to attach working schema from DRS.\n"));
>-		return NT_STATUS_FOOBAR;
>+		return WERR_INTERNAL_ERROR;
> 	}
> 
> 	/* we don't want to access the self made schema anymore */
>@@ -370,7 +378,7 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 	partition_dn = ldb_dn_new(s, s->ldb, c->partition->nc.dn);
> 	if (partition_dn == NULL) {
> 		DEBUG(0,("Failed to parse partition DN from DRS.\n"));
>-		return NT_STATUS_FOOBAR;
>+		return WERR_INVALID_PARAMETER;
> 	}
> 
> 	/* Now convert the schema elements again, using the schema we finalised, ready to actually import */
>@@ -389,7 +397,7 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 						 s, &schema_objs);
> 	if (!W_ERROR_IS_OK(status)) {
> 		DEBUG(0,("Failed to convert objects when trying to import over DRS (2nd pass, to store remote schema): %s\n", win_errstr(status)));
>-		return werror_to_ntstatus(status);
>+		return status;
> 	}
> 
> 	if (lpcfg_parm_bool(s->lp_ctx, NULL, "become dc", "dump objects", false)) {
>@@ -406,11 +414,13 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 	status = dsdb_replicated_objects_commit(s->ldb, NULL, schema_objs, &seq_num);
> 	if (!W_ERROR_IS_OK(status)) {
> 		DEBUG(0,("Failed to commit objects: %s\n", win_errstr(status)));
>-		return werror_to_ntstatus(status);
>+		return status;
> 	}
> 
> 	msg = ldb_msg_new(schema_objs);
>-	NT_STATUS_HAVE_NO_MEMORY(msg);
>+	if (msg == NULL) {
>+		return WERR_NOT_ENOUGH_MEMORY;
>+	}
> 	msg->dn = schema_objs->partition_dn;
> 
> 	/* We must ensure a prefixMap has been written.  Unlike other
>@@ -420,7 +430,7 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 	 * prefixMap for this entire operation.  */
> 	ret = ldb_msg_add_value(msg, "prefixMap", &s->prefixmap_blob, &prefixMap_el);
> 	if (ret != LDB_SUCCESS) {
>-		return NT_STATUS_FOOBAR;
>+		return WERR_NOT_ENOUGH_MEMORY;
> 	}
> 	/* We want to know if a prefixMap was written already, as it
> 	 * would mean that the above comment was not true, and we have
>@@ -430,7 +440,7 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 	ret = dsdb_modify(s->ldb, msg, DSDB_FLAG_AS_SYSTEM);
> 	if (ret != LDB_SUCCESS) {
> 		DEBUG(0,("Failed to add prefixMap: %s\n", ldb_errstring(s->ldb)));
>-		return NT_STATUS_FOOBAR;
>+		return WERR_INTERNAL_ERROR;
> 	}
> 
> 	talloc_free(s_dsa);
>@@ -439,17 +449,17 @@ static NTSTATUS libnet_vampire_cb_apply_schema(struct libnet_vampire_cb_state *s
> 	s->schema = dsdb_get_schema(s->ldb, s);
> 	if (!s->schema) {
> 		DEBUG(0,("Failed to get loaded dsdb_schema\n"));
>-		return NT_STATUS_FOOBAR;
>+		return WERR_INTERNAL_ERROR;
> 	}
> 
>-	return NT_STATUS_OK;
>+	return WERR_OK;
> }
> 
>-NTSTATUS libnet_vampire_cb_schema_chunk(void *private_data,
>-					const struct libnet_BecomeDC_StoreChunk *c)
>+WERROR libnet_vampire_cb_schema_chunk(void *private_data,
>+				      const struct libnet_BecomeDC_StoreChunk *c)
> {
> 	struct libnet_vampire_cb_state *s = talloc_get_type(private_data, struct libnet_vampire_cb_state);
>-	WERROR status;
>+	WERROR werr;
> 	const struct drsuapi_DsReplicaOIDMapping_Ctr *mapping_ctr;
> 	uint32_t nc_object_count;
> 	uint32_t nc_total_received = 0;
>@@ -477,7 +487,7 @@ NTSTATUS libnet_vampire_cb_schema_chunk(void *private_data,
> 		linked_attributes_count		= c->ctr6->linked_attributes_count;
> 		break;
> 	default:
>-		return NT_STATUS_INVALID_PARAMETER;
>+		return WERR_INVALID_PARAMETER;
> 	}
> 
> 	if (!s->schema_part.first_object) {
>@@ -495,7 +505,6 @@ NTSTATUS libnet_vampire_cb_schema_chunk(void *private_data,
> 	}
> 
> 	if (!s->self_made_schema) {
>-		WERROR werr;
> 		struct drsuapi_DsReplicaOIDMapping_Ctr mapping_ctr_without_schema_info;
> 		/* Put the DRS prefixmap aside for the schema we are
> 		 * about to load in the provision, and into the one we
>@@ -510,7 +519,7 @@ NTSTATUS libnet_vampire_cb_schema_chunk(void *private_data,
> 		}
> 		werr = dsdb_get_drsuapi_prefixmap_as_blob(&mapping_ctr_without_schema_info, s, &s->prefixmap_blob);
> 		if (!W_ERROR_IS_OK(werr)) {
>-			return werror_to_ntstatus(werr);
>+			return werr;
> 		}
> 
> 		/* Set up two manually-constructed schema - the local
>@@ -518,16 +527,18 @@ NTSTATUS libnet_vampire_cb_schema_chunk(void *private_data,
> 		 * one, which will then in turn be used to build the
> 		 * other. */
> 		s->self_made_schema = dsdb_new_schema(s);
>-		NT_STATUS_HAVE_NO_MEMORY(s->self_made_schema);
>+		if (s->self_made_schema == NULL) {
>+			return WERR_NOT_ENOUGH_MEMORY;
>+		}
> 
>-		status = dsdb_load_prefixmap_from_drsuapi(s->self_made_schema, mapping_ctr);
>-		if (!W_ERROR_IS_OK(status)) {
>-			return werror_to_ntstatus(status);
>+		werr = dsdb_load_prefixmap_from_drsuapi(s->self_made_schema, mapping_ctr);
>+		if (!W_ERROR_IS_OK(werr)) {
>+			return werr;
> 		}
> 	} else {
>-		status = dsdb_schema_pfm_contains_drsuapi_pfm(s->self_made_schema->prefixmap, mapping_ctr);
>-		if (!W_ERROR_IS_OK(status)) {
>-			return werror_to_ntstatus(status);
>+		werr = dsdb_schema_pfm_contains_drsuapi_pfm(s->self_made_schema->prefixmap, mapping_ctr);
>+		if (!W_ERROR_IS_OK(werr)) {
>+			return werr;
> 		}
> 	}
> 
>@@ -546,11 +557,11 @@ NTSTATUS libnet_vampire_cb_schema_chunk(void *private_data,
> 		return libnet_vampire_cb_apply_schema(s, c);
> 	}
> 
>-	return NT_STATUS_OK;
>+	return WERR_OK;
> }
> 
>-NTSTATUS libnet_vampire_cb_store_chunk(void *private_data,
>-			     const struct libnet_BecomeDC_StoreChunk *c)
>+WERROR libnet_vampire_cb_store_chunk(void *private_data,
>+				     const struct libnet_BecomeDC_StoreChunk *c)
> {
> 	struct libnet_vampire_cb_state *s = talloc_get_type(private_data, struct libnet_vampire_cb_state);
> 	WERROR status;
>@@ -575,9 +586,13 @@ NTSTATUS libnet_vampire_cb_store_chunk(void *private_data,
> 	struct ldb_dn *nc_root = NULL;
> 
> 	s_dsa			= talloc_zero(s, struct repsFromTo1);
>-	NT_STATUS_HAVE_NO_MEMORY(s_dsa);
>+	if (s_dsa == NULL) {
>+		return WERR_NOT_ENOUGH_MEMORY;
>+	}
> 	s_dsa->other_info	= talloc(s_dsa, struct repsFromTo1OtherInfo);
>-	NT_STATUS_HAVE_NO_MEMORY(s_dsa->other_info);
>+	if (s_dsa->other_info == NULL) {
>+		return WERR_NOT_ENOUGH_MEMORY;
>+	}
> 
> 	switch (c->ctr_level) {
> 	case 1:
>@@ -607,7 +622,7 @@ NTSTATUS libnet_vampire_cb_store_chunk(void *private_data,
> 		uptodateness_vector		= c->ctr6->uptodateness_vector;
> 		break;
> 	default:
>-		return NT_STATUS_INVALID_PARAMETER;
>+		return WERR_INVALID_PARAMETER;
> 	}
> 
> 	switch (c->req_level) {
>@@ -634,7 +649,7 @@ NTSTATUS libnet_vampire_cb_store_chunk(void *private_data,
> 		req_replica_flags = c->req10->replica_flags;
> 		break;
> 	default:
>-		return NT_STATUS_INVALID_PARAMETER;
>+		return WERR_INVALID_PARAMETER;
> 	}
> 
> 	if (req_replica_flags & DRSUAPI_DRS_CRITICAL_ONLY) {
>@@ -654,9 +669,13 @@ NTSTATUS libnet_vampire_cb_store_chunk(void *private_data,
> 	memset(s_dsa->schedule, 0x11, sizeof(s_dsa->schedule));
> 
> 	tmp_dns_name	= GUID_string(s_dsa->other_info, &s_dsa->source_dsa_obj_guid);
>-	NT_STATUS_HAVE_NO_MEMORY(tmp_dns_name);
>+	if (tmp_dns_name == NULL) {
>+		return WERR_NOT_ENOUGH_MEMORY;
>+	}
> 	tmp_dns_name	= talloc_asprintf_append_buffer(tmp_dns_name, "._msdcs.%s", c->forest->dns_name);
>-	NT_STATUS_HAVE_NO_MEMORY(tmp_dns_name);
>+	if (tmp_dns_name == NULL) {
>+		return WERR_NOT_ENOUGH_MEMORY;
>+	}
> 	s_dsa->other_info->dns_name = tmp_dns_name;
> 
> 	/* we want to show a count per partition */
>@@ -670,7 +689,7 @@ NTSTATUS libnet_vampire_cb_store_chunk(void *private_data,
> 	partition_dn = ldb_dn_new(s, s->ldb, c->partition->nc.dn);
> 	if (partition_dn == NULL) {
> 		DEBUG(0,("Failed to parse partition DN from DRS.\n"));
>-		return NT_STATUS_FOOBAR;
>+		return WERR_INVALID_PARAMETER;
> 	}
> 
> 	if (is_exop) {
>@@ -688,7 +707,7 @@ NTSTATUS libnet_vampire_cb_store_chunk(void *private_data,
> 		if (ret != LDB_SUCCESS) {
> 			DEBUG(0,(__location__ ": Failed to find nc_root for %s\n",
> 				 ldb_dn_get_linearized(partition_dn)));
>-			return NT_STATUS_INTERNAL_ERROR;
>+			return WERR_INTERNAL_ERROR;
> 		}
> 	} else {
> 		if (nc_object_count) {
>@@ -706,7 +725,7 @@ NTSTATUS libnet_vampire_cb_store_chunk(void *private_data,
> 	schema = dsdb_get_schema(s->ldb, NULL);
> 	if (!schema) {
> 		DEBUG(0,(__location__ ": Schema is not loaded yet!\n"));
>-		return NT_STATUS_INTERNAL_ERROR;
>+		return WERR_INTERNAL_ERROR;
> 	}
> 
> 	if (req_replica_flags & DRSUAPI_DRS_FULL_SYNC_IN_PROGRESS) {
>@@ -732,7 +751,7 @@ NTSTATUS libnet_vampire_cb_store_chunk(void *private_data,
> 						 s, &objs);
> 	if (!W_ERROR_IS_OK(status)) {
> 		DEBUG(0,("Failed to convert objects: %s\n", win_errstr(status)));
>-		return werror_to_ntstatus(status);
>+		return status;
> 	}
> 
> 	if (lpcfg_parm_bool(s->lp_ctx, NULL, "become dc", "dump objects", false)) {
>@@ -748,7 +767,7 @@ NTSTATUS libnet_vampire_cb_store_chunk(void *private_data,
> 	status = dsdb_replicated_objects_commit(s->ldb, NULL, objs, &seq_num);
> 	if (!W_ERROR_IS_OK(status)) {
> 		DEBUG(0,("Failed to commit objects: %s\n", win_errstr(status)));
>-		return werror_to_ntstatus(status);
>+		return status;
> 	}
> 
> 	talloc_free(s_dsa);
>@@ -759,19 +778,19 @@ NTSTATUS libnet_vampire_cb_store_chunk(void *private_data,
> 
> 		if (!linked_attributes[i].identifier) {
> 			DEBUG(0, ("No linked attribute identifier\n"));
>-			return NT_STATUS_FOOBAR;
>+			return WERR_INTERNAL_ERROR;
> 		}
> 
> 		if (!linked_attributes[i].value.blob) {
> 			DEBUG(0, ("No linked attribute value\n"));
>-			return NT_STATUS_FOOBAR;
>+			return WERR_INTERNAL_ERROR;
> 		}
> 
> 		sa = dsdb_attribute_by_attributeID_id(s->schema,
> 						      linked_attributes[i].attid);
> 		if (!sa) {
> 			DEBUG(0, ("Unable to find attribute via attribute id %d\n", linked_attributes[i].attid));
>-			return NT_STATUS_FOOBAR;
>+			return WERR_INTERNAL_ERROR;
> 		}
> 
> 		if (lpcfg_parm_bool(s->lp_ctx, NULL, "become dc", "dump objects", false)) {
>@@ -783,6 +802,6 @@ NTSTATUS libnet_vampire_cb_store_chunk(void *private_data,
> 		}
> 	}
> 
>-	return NT_STATUS_OK;
>+	return WERR_OK;
> }
> 
>diff --git a/source4/libnet/py_net.c b/source4/libnet/py_net.c
>index 48009b2..9259fe6 100644
>--- a/source4/libnet/py_net.c
>+++ b/source4/libnet/py_net.c
>@@ -390,8 +390,8 @@ static PyObject *py_net_replicate_chunk(py_net_Object *self, PyObject *args, PyO
> 	struct replicate_state *s;
> 	unsigned level;
> 	unsigned req_level = 0;
>-	NTSTATUS (*chunk_handler)(void *private_data, const struct libnet_BecomeDC_StoreChunk *c);
>-	NTSTATUS status;
>+	WERROR (*chunk_handler)(void *private_data, const struct libnet_BecomeDC_StoreChunk *c);
>+	WERROR werr;
> 
> 	if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OIO|OIO",
> 					 discard_const_p(char *, kwnames),
>@@ -482,9 +482,9 @@ static PyObject *py_net_replicate_chunk(py_net_Object *self, PyObject *args, PyO
> 
> 	s->chunk.ctr_level = level;
> 
>-	status = chunk_handler(s->vampire_state, &s->chunk);
>-	if (!NT_STATUS_IS_OK(status)) {
>-		PyErr_Format(PyExc_TypeError, "Failed to process chunk: %s", nt_errstr(status));
>+	werr = chunk_handler(s->vampire_state, &s->chunk);
>+	if (!W_ERROR_IS_OK(werr)) {
>+		PyErr_Format(PyExc_TypeError, "Failed to process chunk: %s", win_errstr(werr));
> 		return NULL;
> 	}
> 
>-- 
>1.9.1
>
>
>From 5f712dad77c21a383cf3e9f4dfb2fc9d632390be Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Mon, 31 Oct 2016 16:25:51 +1300
>Subject: [PATCH 19/24] dsdb: Catch errors in extended operations (like
> allocating a RID Set)
>
>There are cases where allocating a RID Set can reasonably fail.  Catch those nicely.
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398
>(cherry picked from commit f72da5ba51ae8bf9f3f54bed36b4572cd1b57adb)
>---
> source4/libnet/py_net.c | 21 +++++++++++++++++++--
> 1 file changed, 19 insertions(+), 2 deletions(-)
>
>diff --git a/source4/libnet/py_net.c b/source4/libnet/py_net.c
>index 9259fe6..44bede6 100644
>--- a/source4/libnet/py_net.c
>+++ b/source4/libnet/py_net.c
>@@ -392,6 +392,8 @@ static PyObject *py_net_replicate_chunk(py_net_Object *self, PyObject *args, PyO
> 	unsigned req_level = 0;
> 	WERROR (*chunk_handler)(void *private_data, const struct libnet_BecomeDC_StoreChunk *c);
> 	WERROR werr;
>+	enum drsuapi_DsExtendedError extended_ret = DRSUAPI_EXOP_ERR_NONE;
>+	enum drsuapi_DsExtendedOperation exop = DRSUAPI_EXOP_NONE;
> 
> 	if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OIO|OIO",
> 					 discard_const_p(char *, kwnames),
>@@ -412,7 +414,10 @@ static PyObject *py_net_replicate_chunk(py_net_Object *self, PyObject *args, PyO
> 			return NULL;
> 		}
> 		s->chunk.ctr1                         = pytalloc_get_ptr(py_ctr);
>-		s->partition.nc                       = *s->chunk.ctr1->naming_context;
>+		if (s->chunk.ctr1->naming_context != NULL) {
>+			s->partition.nc = *s->chunk.ctr1->naming_context;
>+		}
>+		extended_ret = s->chunk.ctr1->extended_ret;
> 		s->partition.more_data                = s->chunk.ctr1->more_data;
> 		s->partition.source_dsa_guid          = s->chunk.ctr1->source_dsa_guid;
> 		s->partition.source_dsa_invocation_id = s->chunk.ctr1->source_dsa_invocation_id;
>@@ -423,7 +428,10 @@ static PyObject *py_net_replicate_chunk(py_net_Object *self, PyObject *args, PyO
> 			return NULL;
> 		}
> 		s->chunk.ctr6                         = pytalloc_get_ptr(py_ctr);
>-		s->partition.nc                       = *s->chunk.ctr6->naming_context;
>+		if (s->chunk.ctr6->naming_context != NULL) {
>+			s->partition.nc = *s->chunk.ctr6->naming_context;
>+		}
>+		extended_ret = s->chunk.ctr6->extended_ret;
> 		s->partition.more_data                = s->chunk.ctr6->more_data;
> 		s->partition.source_dsa_guid          = s->chunk.ctr6->source_dsa_guid;
> 		s->partition.source_dsa_invocation_id = s->chunk.ctr6->source_dsa_invocation_id;
>@@ -447,6 +455,7 @@ static PyObject *py_net_replicate_chunk(py_net_Object *self, PyObject *args, PyO
> 			}
> 
> 			s->chunk.req5 = pytalloc_get_ptr(py_req);
>+			exop = s->chunk.req5->extended_op;
> 			break;
> 		case 8:
> 			if (!py_check_dcerpc_type(py_req, "samba.dcerpc.drsuapi", "DsGetNCChangesRequest8")) {
>@@ -454,6 +463,7 @@ static PyObject *py_net_replicate_chunk(py_net_Object *self, PyObject *args, PyO
> 			}
> 
> 			s->chunk.req8 = pytalloc_get_ptr(py_req);
>+			exop = s->chunk.req8->extended_op;
> 			break;
> 		case 10:
> 			if (!py_check_dcerpc_type(py_req, "samba.dcerpc.drsuapi", "DsGetNCChangesRequest10")) {
>@@ -461,12 +471,19 @@ static PyObject *py_net_replicate_chunk(py_net_Object *self, PyObject *args, PyO
> 			}
> 
> 			s->chunk.req10 = pytalloc_get_ptr(py_req);
>+			exop = s->chunk.req10->extended_op;
> 			break;
> 		default:
> 			PyErr_Format(PyExc_TypeError, "Bad req_level %u in replicate_chunk", req_level);
> 			return NULL;
> 		}
> 	}
>+
>+	if (exop != DRSUAPI_EXOP_NONE && extended_ret != DRSUAPI_EXOP_ERR_SUCCESS) {
>+		PyErr_Format(PyExc_RuntimeError, "Remote EXOP %d failed with %d", exop, extended_ret);
>+		return NULL;
>+	}
>+
> 	s->chunk.req_level = req_level;
> 
> 	chunk_handler = libnet_vampire_cb_store_chunk;
>-- 
>1.9.1
>
>
>From 076f94e6bfb10c6612e68198ee0ef22c470882c0 Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Tue, 1 Nov 2016 15:23:58 +1300
>Subject: [PATCH 20/24] python: create NTSTATUSError, HRESULTError and
> WERRORError
>
>The advantage of these over the previous use of just RuntimeError is that we can
>catch just the errors we want, without having to catch all possible RuntimeError
>cases and assume they decode to a tuple
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398
>(cherry picked from commit c8505c53da3e21f31454f121efd5961d95349a38)
>---
> python/pyglue.c                | 20 ++++++++++++++++++++
> python/samba/__init__.py       |  4 ++++
> source4/libcli/util/pyerrors.h | 15 ++++++++++++---
> 3 files changed, 36 insertions(+), 3 deletions(-)
>
>diff --git a/python/pyglue.c b/python/pyglue.c
>index 81244a2..938a9f0 100644
>--- a/python/pyglue.c
>+++ b/python/pyglue.c
>@@ -24,6 +24,9 @@
> #include "lib/socket/netif.h"
> 
> void init_glue(void);
>+static PyObject *PyExc_NTSTATUSError;
>+static PyObject *PyExc_WERRORError;
>+static PyObject *PyExc_HRESULTError;
> 
> static PyObject *py_generate_random_str(PyObject *self, PyObject *args)
> {
>@@ -294,5 +297,22 @@ void init_glue(void)
> 
> 	PyModule_AddObject(m, "version",
> 					   PyString_FromString(SAMBA_VERSION_STRING));
>+	PyExc_NTSTATUSError = PyErr_NewException(discard_const_p(char, "samba.NTSTATUSError"), PyExc_RuntimeError, NULL);
>+	if (PyExc_NTSTATUSError != NULL) {
>+		Py_INCREF(PyExc_NTSTATUSError);
>+		PyModule_AddObject(m, "NTSTATUSError", PyExc_NTSTATUSError);
>+	}
>+
>+	PyExc_WERRORError = PyErr_NewException(discard_const_p(char, "samba.WERRORError"), PyExc_RuntimeError, NULL);
>+	if (PyExc_WERRORError != NULL) {
>+		Py_INCREF(PyExc_WERRORError);
>+		PyModule_AddObject(m, "WERRORError", PyExc_WERRORError);
>+	}
>+
>+	PyExc_HRESULTError = PyErr_NewException(discard_const_p(char, "samba.HRESULTError"), PyExc_RuntimeError, NULL);
>+	if (PyExc_HRESULTError != NULL) {
>+		Py_INCREF(PyExc_HRESULTError);
>+		PyModule_AddObject(m, "HRESULTError", PyExc_HRESULTError);
>+	}
> }
> 
>diff --git a/python/samba/__init__.py b/python/samba/__init__.py
>index 7cfbc4c..8c75a48 100644
>--- a/python/samba/__init__.py
>+++ b/python/samba/__init__.py
>@@ -399,3 +399,7 @@ generate_random_password = _glue.generate_random_password
> strcasecmp_m = _glue.strcasecmp_m
> strstr_m = _glue.strstr_m
> is_ntvfs_fileserver_built = _glue.is_ntvfs_fileserver_built
>+
>+NTSTATUSError = _glue.NTSTATUSError
>+HRESULTError = _glue.HRESULTError
>+WERRORError = _glue.WERRORError
>diff --git a/source4/libcli/util/pyerrors.h b/source4/libcli/util/pyerrors.h
>index ef99713..9228c34 100644
>--- a/source4/libcli/util/pyerrors.h
>+++ b/source4/libcli/util/pyerrors.h
>@@ -28,11 +28,20 @@
> 
> #define PyErr_FromString(str) Py_BuildValue("(s)", discard_const_p(char, str))
> 
>-#define PyErr_SetWERROR(err) \
>-	PyErr_SetObject(PyExc_RuntimeError, PyErr_FromWERROR(err))
>+#define PyErr_SetWERROR(werr) \
>+        PyErr_SetObject(PyObject_GetAttrString(PyImport_ImportModule("samba"),\
>+					       "WERRORError"),	\
>+			PyErr_FromWERROR(werr))
>+
>+#define PyErr_SetHRESULT(hresult) \
>+        PyErr_SetObject(PyObject_GetAttrString(PyImport_ImportModule("samba"),\
>+					       "HRESULTError"),	\
>+			PyErr_FromHRESULT(hresult))
> 
> #define PyErr_SetNTSTATUS(status) \
>-        PyErr_SetObject(PyExc_RuntimeError, PyErr_FromNTSTATUS(status))
>+        PyErr_SetObject(PyObject_GetAttrString(PyImport_ImportModule("samba"),\
>+					       "NTSTATUSError"),	\
>+			PyErr_FromNTSTATUS(status))
> 
> #define PyErr_NTSTATUS_IS_ERR_RAISE(status) \
> 	if (NT_STATUS_IS_ERR(status)) { \
>-- 
>1.9.1
>
>
>From 10c2919324c0272369c196ec440ab8990b266ab5 Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Tue, 1 Nov 2016 16:03:09 +1300
>Subject: [PATCH 21/24] pyerrors: Add
> PyErr_Set{WERROR,HRESULT,NTSTATUS}_and_string()
>
>This varient allows control of the text explaination string
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398
>(cherry picked from commit e737171f6ef172de559b41d54989eca0d7663b4e)
>---
> source4/libcli/util/pyerrors.h | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
>diff --git a/source4/libcli/util/pyerrors.h b/source4/libcli/util/pyerrors.h
>index 9228c34..c3b3076 100644
>--- a/source4/libcli/util/pyerrors.h
>+++ b/source4/libcli/util/pyerrors.h
>@@ -43,6 +43,21 @@
> 					       "NTSTATUSError"),	\
> 			PyErr_FromNTSTATUS(status))
> 
>+#define PyErr_SetWERROR_and_string(werr, string) \
>+        PyErr_SetObject(PyObject_GetAttrString(PyImport_ImportModule("samba"),\
>+					       "WERRORError"),	\
>+			Py_BuildValue("(i,s)", W_ERROR_V(werr), string))
>+
>+#define PyErr_SetHRESULT_and_string(hresult, string) \
>+        PyErr_SetObject(PyObject_GetAttrString(PyImport_ImportModule("samba"),\
>+					       "HRESULTError"),	\
>+			Py_BuildValue("(i,s)", HRES_ERROR_V(hresult), string))
>+
>+#define PyErr_SetNTSTATUS_and_string(status, string)				\
>+        PyErr_SetObject(PyObject_GetAttrString(PyImport_ImportModule("samba"),\
>+					       "NTSTATUSError"),	\
>+			Py_BuildValue("(i,s)", NT_STATUS_V(status), string))
>+
> #define PyErr_NTSTATUS_IS_ERR_RAISE(status) \
> 	if (NT_STATUS_IS_ERR(status)) { \
> 		PyErr_SetNTSTATUS(status); \
>-- 
>1.9.1
>
>
>From cbe4b946fa07ec88e81e5f83ad73fd0ce8899548 Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Tue, 1 Nov 2016 16:09:20 +1300
>Subject: [PATCH 22/24] python: Add DsExtendedError Exception
>
>This will be used for checking errors during a GetNCChanges EXOP like
>RID Set allocation.
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398
>(cherry picked from commit e51256c7d58040eeee02fc189b55afbc58379f81)
>---
> python/pyglue.c          | 8 ++++++++
> python/samba/__init__.py | 1 +
> 2 files changed, 9 insertions(+)
>
>diff --git a/python/pyglue.c b/python/pyglue.c
>index 938a9f0..dbe7eb4 100644
>--- a/python/pyglue.c
>+++ b/python/pyglue.c
>@@ -27,6 +27,7 @@ void init_glue(void);
> static PyObject *PyExc_NTSTATUSError;
> static PyObject *PyExc_WERRORError;
> static PyObject *PyExc_HRESULTError;
>+static PyObject *PyExc_DsExtendedError;
> 
> static PyObject *py_generate_random_str(PyObject *self, PyObject *args)
> {
>@@ -314,5 +315,12 @@ void init_glue(void)
> 		Py_INCREF(PyExc_HRESULTError);
> 		PyModule_AddObject(m, "HRESULTError", PyExc_HRESULTError);
> 	}
>+
>+	PyExc_DsExtendedError = PyErr_NewException(discard_const_p(char, "samba.DsExtendedError"), PyExc_RuntimeError, NULL);
>+	if (PyExc_DsExtendedError != NULL) {
>+		Py_INCREF(PyExc_DsExtendedError);
>+		PyModule_AddObject(m, "DsExtendedError", PyExc_DsExtendedError);
>+	}
>+
> }
> 
>diff --git a/python/samba/__init__.py b/python/samba/__init__.py
>index 8c75a48..5f91531 100644
>--- a/python/samba/__init__.py
>+++ b/python/samba/__init__.py
>@@ -403,3 +403,4 @@ is_ntvfs_fileserver_built = _glue.is_ntvfs_fileserver_built
> NTSTATUSError = _glue.NTSTATUSError
> HRESULTError = _glue.HRESULTError
> WERRORError = _glue.WERRORError
>+DsExtendedError = _glue.DsExtendedError
>-- 
>1.9.1
>
>
>From d721439e72218d0097ad68bd4b17573ab81d2acc Mon Sep 17 00:00:00 2001
>From: Andrew Bartlett <abartlet@samba.org>
>Date: Tue, 1 Nov 2016 12:38:48 +1300
>Subject: [PATCH 23/24] python-libnet: Use new NTSTATUSError, WERRORError and
> DsExtendedError exceptions
>
>This will allow callers to catch specific errors rather than RuntimeException
>
>As this slightly changes the exception, the timecmd test must be updated.
>
>Signed-off-by: Andrew Bartlett <abartlet@samba.org>
>Reviewed-by: Garming Sam <garming@catalyst.net.nz>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=12398
>(cherry picked from commit 2a49c7451949d42e5a4d6fce3ce607f05f9e7b71)
>---
> python/samba/tests/samba_tool/timecmd.py |   2 +-
> source4/libnet/py_net.c                  | 116 +++++++++++++++++++++++++++----
> 2 files changed, 103 insertions(+), 15 deletions(-)
>
>diff --git a/python/samba/tests/samba_tool/timecmd.py b/python/samba/tests/samba_tool/timecmd.py
>index 310f861..68dcb06 100644
>--- a/python/samba/tests/samba_tool/timecmd.py
>+++ b/python/samba/tests/samba_tool/timecmd.py
>@@ -39,5 +39,5 @@ class TimeCmdTestCase(SambaToolCmdTest):
>         """Run time against a non-existent server, and make sure it fails"""
>         (result, out, err) = self.runcmd("time", "notaserver")
>         self.assertEquals(result, -1, "check for result code")
>-        self.assertTrue(err.strip().endswith("NT_STATUS_OBJECT_NAME_NOT_FOUND"), "ensure right error string")
>+        self.assertNotEqual(err.strip().find("NT_STATUS_OBJECT_NAME_NOT_FOUND"), -1, "ensure right error string")
>         self.assertEquals(out, "", "ensure no output returned")
>diff --git a/source4/libnet/py_net.c b/source4/libnet/py_net.c
>index 44bede6..3e70c79 100644
>--- a/source4/libnet/py_net.c
>+++ b/source4/libnet/py_net.c
>@@ -38,6 +38,72 @@
> 
> void initnet(void);
> 
>+static void PyErr_SetDsExtendedError(enum drsuapi_DsExtendedError ext_err, const char *error_description)
>+{
>+	PyObject *error = PyObject_GetAttrString(PyImport_ImportModule("samba"),
>+						 "DsExtendedError");
>+	if (error_description == NULL) {
>+		switch (ext_err) {
>+			/* Copied out of ndr_drsuapi.c:ndr_print_drsuapi_DsExtendedError() */
>+			case DRSUAPI_EXOP_ERR_NONE:
>+				error_description = "DRSUAPI_EXOP_ERR_NONE";
>+				break;
>+			case DRSUAPI_EXOP_ERR_SUCCESS:
>+				error_description = "DRSUAPI_EXOP_ERR_SUCCESS";
>+				break;
>+			case DRSUAPI_EXOP_ERR_UNKNOWN_OP:
>+				error_description = "DRSUAPI_EXOP_ERR_UNKNOWN_OP";
>+				break;
>+			case DRSUAPI_EXOP_ERR_FSMO_NOT_OWNER:
>+				error_description = "DRSUAPI_EXOP_ERR_FSMO_NOT_OWNER";
>+				break;
>+			case DRSUAPI_EXOP_ERR_UPDATE_ERR:
>+				error_description = "DRSUAPI_EXOP_ERR_UPDATE_ERR";
>+				break;
>+			case DRSUAPI_EXOP_ERR_EXCEPTION:
>+				error_description = "DRSUAPI_EXOP_ERR_EXCEPTION";
>+				break;
>+			case DRSUAPI_EXOP_ERR_UNKNOWN_CALLER:
>+				error_description = "DRSUAPI_EXOP_ERR_UNKNOWN_CALLER";
>+				break;
>+			case DRSUAPI_EXOP_ERR_RID_ALLOC:
>+				error_description = "DRSUAPI_EXOP_ERR_RID_ALLOC";
>+				break;
>+			case DRSUAPI_EXOP_ERR_FSMO_OWNER_DELETED:
>+				error_description = "DRSUAPI_EXOP_ERR_FSMO_OWNER_DELETED";
>+				break;
>+			case DRSUAPI_EXOP_ERR_FMSO_PENDING_OP:
>+				error_description = "DRSUAPI_EXOP_ERR_FMSO_PENDING_OP";
>+				break;
>+			case DRSUAPI_EXOP_ERR_MISMATCH:
>+				error_description = "DRSUAPI_EXOP_ERR_MISMATCH";
>+				break;
>+			case DRSUAPI_EXOP_ERR_COULDNT_CONTACT:
>+				error_description = "DRSUAPI_EXOP_ERR_COULDNT_CONTACT";
>+				break;
>+			case DRSUAPI_EXOP_ERR_FSMO_REFUSING_ROLES:
>+				error_description = "DRSUAPI_EXOP_ERR_FSMO_REFUSING_ROLES";
>+				break;
>+			case DRSUAPI_EXOP_ERR_DIR_ERROR:
>+				error_description = "DRSUAPI_EXOP_ERR_DIR_ERROR";
>+				break;
>+			case DRSUAPI_EXOP_ERR_FSMO_MISSING_SETTINGS:
>+				error_description = "DRSUAPI_EXOP_ERR_FSMO_MISSING_SETTINGS";
>+				break;
>+			case DRSUAPI_EXOP_ERR_ACCESS_DENIED:
>+				error_description = "DRSUAPI_EXOP_ERR_ACCESS_DENIED";
>+				break;
>+			case DRSUAPI_EXOP_ERR_PARAM_ERROR:
>+				error_description = "DRSUAPI_EXOP_ERR_PARAM_ERROR";
>+				break;
>+		}
>+	}
>+	PyErr_SetObject(error,
>+			Py_BuildValue(discard_const_p(char, "(i,s)"),
>+				      ext_err,
>+				      error_description));
>+}
>+
> static PyObject *py_net_join_member(py_net_Object *self, PyObject *args, PyObject *kwargs)
> {
> 	struct libnet_Join_member r;
>@@ -65,7 +131,10 @@ static PyObject *py_net_join_member(py_net_Object *self, PyObject *args, PyObjec
> 
> 	status = libnet_Join_member(self->libnet_ctx, mem_ctx, &r);
> 	if (NT_STATUS_IS_ERR(status)) {
>-		PyErr_SetString(PyExc_RuntimeError, r.out.error_string?r.out.error_string:nt_errstr(status));
>+		PyErr_SetNTSTATUS_and_string(status,
>+					     r.out.error_string
>+					     ? r.out.error_string
>+					     : nt_errstr(status));
> 		talloc_free(mem_ctx);
> 		return NULL;
> 	}
>@@ -115,8 +184,10 @@ static PyObject *py_net_change_password(py_net_Object *self, PyObject *args, PyO
> 
> 	status = libnet_ChangePassword(self->libnet_ctx, mem_ctx, &r);
> 	if (NT_STATUS_IS_ERR(status)) {
>-		PyErr_SetString(PyExc_RuntimeError,
>-				r.generic.out.error_string?r.generic.out.error_string:nt_errstr(status));
>+		PyErr_SetNTSTATUS_and_string(status,
>+					     r.generic.out.error_string
>+					     ? r.generic.out.error_string
>+					     : nt_errstr(status));
> 		talloc_free(mem_ctx);
> 		return NULL;
> 	}
>@@ -164,8 +235,10 @@ static PyObject *py_net_set_password(py_net_Object *self, PyObject *args, PyObje
> 
> 	status = libnet_SetPassword(self->libnet_ctx, mem_ctx, &r);
> 	if (NT_STATUS_IS_ERR(status)) {
>-		PyErr_SetString(PyExc_RuntimeError,
>-				r.generic.out.error_string?r.generic.out.error_string:nt_errstr(status));
>+		PyErr_SetNTSTATUS_and_string(status,
>+					     r.generic.out.error_string
>+					     ? r.generic.out.error_string
>+					     : nt_errstr(status));
> 		talloc_free(mem_ctx);
> 		return NULL;
> 	}
>@@ -205,8 +278,10 @@ static PyObject *py_net_time(py_net_Object *self, PyObject *args, PyObject *kwar
> 
> 	status = libnet_RemoteTOD(self->libnet_ctx, mem_ctx, &r);
> 	if (!NT_STATUS_IS_OK(status)) {
>-		PyErr_SetString(PyExc_RuntimeError,
>-				r.generic.out.error_string?r.generic.out.error_string:nt_errstr(status));
>+		PyErr_SetNTSTATUS_and_string(status,
>+					     r.generic.out.error_string
>+					     ? r.generic.out.error_string
>+					     : nt_errstr(status));
> 		talloc_free(mem_ctx);
> 		return NULL;
> 	}
>@@ -246,7 +321,10 @@ static PyObject *py_net_user_create(py_net_Object *self, PyObject *args, PyObjec
> 
> 	status = libnet_CreateUser(self->libnet_ctx, mem_ctx, &r);
> 	if (!NT_STATUS_IS_OK(status)) {
>-		PyErr_SetString(PyExc_RuntimeError, r.out.error_string?r.out.error_string:nt_errstr(status));
>+		PyErr_SetNTSTATUS_and_string(status,
>+					     r.out.error_string
>+					     ? r.out.error_string
>+					     : nt_errstr(status));
> 		talloc_free(mem_ctx);
> 		return NULL;
> 	}
>@@ -280,7 +358,10 @@ static PyObject *py_net_user_delete(py_net_Object *self, PyObject *args, PyObjec
> 
> 	status = libnet_DeleteUser(self->libnet_ctx, mem_ctx, &r);
> 	if (!NT_STATUS_IS_OK(status)) {
>-		PyErr_SetString(PyExc_RuntimeError, r.out.error_string?r.out.error_string:nt_errstr(status));
>+		PyErr_SetNTSTATUS_and_string(status,
>+					   r.out.error_string
>+					  ? r.out.error_string
>+					  : nt_errstr(status));
> 		talloc_free(mem_ctx);
> 		return NULL;
> 	}
>@@ -358,8 +439,10 @@ static PyObject *py_net_replicate_init(py_net_Object *self, PyObject *args, PyOb
> 				    s,
> 				    &s->gensec_skey);
> 	if (!NT_STATUS_IS_OK(status)) {
>-		PyErr_Format(PyExc_RuntimeError, "Unable to get session key from drspipe: %s",
>-			     nt_errstr(status));
>+		char *error_string = talloc_asprintf(s,
>+						     "Unable to get session key from drspipe: %s",
>+						     nt_errstr(status));
>+		PyErr_SetNTSTATUS_and_string(status, error_string);
> 		talloc_free(s);
> 		return NULL;
> 	}
>@@ -480,7 +563,7 @@ static PyObject *py_net_replicate_chunk(py_net_Object *self, PyObject *args, PyO
> 	}
> 
> 	if (exop != DRSUAPI_EXOP_NONE && extended_ret != DRSUAPI_EXOP_ERR_SUCCESS) {
>-		PyErr_Format(PyExc_RuntimeError, "Remote EXOP %d failed with %d", exop, extended_ret);
>+		PyErr_SetDsExtendedError(extended_ret, NULL);
> 		return NULL;
> 	}
> 
>@@ -501,7 +584,12 @@ static PyObject *py_net_replicate_chunk(py_net_Object *self, PyObject *args, PyO
> 
> 	werr = chunk_handler(s->vampire_state, &s->chunk);
> 	if (!W_ERROR_IS_OK(werr)) {
>-		PyErr_Format(PyExc_TypeError, "Failed to process chunk: %s", win_errstr(werr));
>+		char *error_string
>+			= talloc_asprintf(NULL,
>+					  "Failed to process 'chunk' of DRS replicated objects: %s",
>+					  win_errstr(werr));
>+		PyErr_SetWERROR_and_string(werr, error_string);
>+		TALLOC_FREE(error_string);
> 		return NULL;
> 	}
> 
>@@ -542,7 +630,7 @@ static PyObject *py_net_finddc(py_net_Object *self, PyObject *args, PyObject *kw
> 	status = finddcs_cldap(io, io,
> 			       lpcfg_resolve_context(self->libnet_ctx->lp_ctx), self->ev);
> 	if (NT_STATUS_IS_ERR(status)) {
>-		PyErr_SetString(PyExc_RuntimeError, nt_errstr(status));
>+		PyErr_SetNTSTATUS(status);
> 		talloc_free(mem_ctx);
> 		return NULL;
> 	}
>-- 
>1.9.1
>
>
>From 05cfb82abd11f25d5aa71aae877c580411518e2d Mon Sep 17 00:00:00 2001
>From: Garming Sam <garming@catalyst.net.nz>
>Date: Tue, 1 Nov 2016 16:29:53 +1300
>Subject: [PATCH 24/24] samba_tool/fsmo: Allocate RID Set when seizing RID
> manager
>
>Seizing the role without allocating a RID set for itself is likely prone
>to cause issues.
>
>Pair-programmed-with: Clive Ferreira <cliveferreira@catalyst.net.nz>
>
>Signed-off-by: Clive Ferreira <cliveferreira@catalyst.net.nz>
>Signed-off-by: Garming Sam <garming@catalyst.net.nz>
>Reviewed-by: Andrew Bartlett <abartlet@samba.org>
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=9954
>
>Autobuild-User(master): Garming Sam <garming@samba.org>
>Autobuild-Date(master): Fri Nov  4 08:37:05 CET 2016 on sn-devel-144
>
>(cherry picked from commit 815658d2db46e4accdd35f5925585ec1f1c3d74f)
>---
> python/samba/netcmd/fsmo.py | 29 +++++++++++++++++++++++++++--
> selftest/knownfail          |  1 -
> 2 files changed, 27 insertions(+), 3 deletions(-)
>
>diff --git a/python/samba/netcmd/fsmo.py b/python/samba/netcmd/fsmo.py
>index 1351654..62b3e43 100644
>--- a/python/samba/netcmd/fsmo.py
>+++ b/python/samba/netcmd/fsmo.py
>@@ -295,12 +295,37 @@ You must provide an Admin user and password."""),
>             m["fSMORoleOwner"]= ldb.MessageElement(
>                 serviceName, ldb.FLAG_MOD_REPLACE,
>                 "fSMORoleOwner")
>+
>+            samdb.transaction_start()
>             try:
>                 samdb.modify(m)
>+                if role == "rid":
>+                    # We may need to allocate the initial RID Set
>+                    samdb.create_own_rid_set()
>+
>             except LdbError, (num, msg):
>-                raise CommandError("Failed to seize '%s' role: %s" %
>-                                   (role, msg))
>+                if role == "rid" and num == ldb.ERR_ENTRY_ALREADY_EXISTS:
>+
>+                    # Try again without the RID Set allocation
>+                    # (normal).  We have to manage the transaction as
>+                    # we do not have nested transactions and creating
>+                    # a RID set touches multiple objects. :-(
>+                    samdb.transaction_cancel()
>+                    samdb.transaction_start()
>+                    try:
>+                        samdb.modify(m)
>+                    except LdbError, (num, msg):
>+                        samdb.transaction_cancel()
>+                        raise CommandError("Failed to seize '%s' role: %s" %
>+                                           (role, msg))
>+
>+                else:
>+                    samdb.transaction_cancel()
>+                    raise CommandError("Failed to seize '%s' role: %s" %
>+                                       (role, msg))
>+            samdb.transaction_commit()
>             self.outf.write("FSMO seize of '%s' role successful\n" % role)
>+
>             return True
> 
>     def seize_dns_role(self, role, samdb, credopts, sambaopts,
>diff --git a/selftest/knownfail b/selftest/knownfail
>index cb309a6..7c42777 100644
>--- a/selftest/knownfail
>+++ b/selftest/knownfail
>@@ -293,7 +293,6 @@
> ^samba4.smb2.ioctl.copy_chunk_bad_access
> ^samba4.drs.getnc_exop.python.*getnc_exop.DrsReplicaPrefixMapTestCase.test_regular_prefix_map_ex_attid.*
> ^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_offline_ridalloc
>-^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_offline_samba_tool_seized_ridalloc
> ^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_join_time_ridalloc
> ^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_rid_set_dbcheck_after_seize
> ^samba4.drs.ridalloc_exop.python.*ridalloc_exop.DrsReplicaSyncTestCase.test_rid_set_dbcheck
>-- 
>1.9.1
>

Flags:

abartlet: review+

Actions: View

Attachments on bug 12453: 12700