On Samba 4.12.0, I receive the error "ndr_pull_uint32: ndr_pull_error(Buffer Size Error): Pull bytes 4 (../../librpc/ndr/ndr_basic.c:152) at ../../librpc/ndr/ndr_basic.c:152" when opening the DNS Manager in MMC.
Created attachment 15838 [details] Error at log-level 10 This jumped out at me from the log-level 10 output. I'm happy to send the full logs but would rather do that somewhere more private.
I'm thinking the DNS RPC server isn't running: samba-tool dns serverinfo dc01 Failed to connect host xxx.xxx.xxx.xxx on port 135 - NT_STATUS_CONNECTION_REFUSED Failed to connect host xxx.xxx.xxx.xxx (dc01) on port 135 - NT_STATUS_CONNECTION_REFUSED. ERROR: Connecting to DNS RPC server dc01 failed with (3221226038, 'The transport-connection attempt was refused by the remote system.')
I've marked the bug private to allow Alex to upload a log. I'll mark that attachment private when the log comes in and re-open the bug.
Created attachment 15863 [details] Log Level 10
Re-opening bug now attachment is private.
I was unable to reproduce the issue on a freshly provisioned DC, so I thought there may be some corruption in my DB that was causing the issue. To investigate, I exported the DB using samba-tool domain backup online (and offline a 2nd time) and tried to restore into an empty VM, but that also failed: Removing computer account: CN=XXX,OU=Domain Controllers,DC=XXX,DC=XXX,DC=XXX (and any child objects) ndr_pull_uint32: ndr_pull_error(Buffer Size Error): Pull bytes 4 (../../librpc/ndr/ndr_basic.c:181) at ../../librpc/ndr/ndr_basic.c:181 ERROR(runtime): uncaught exception - (9002, 'WERR_DNS_ERROR_RCODE_SERVER_FAILURE') File "/usr/local/samba/lib/python3.7/site-packages/samba/netcmd/__init__.py", line 186, in _run return self.run(*args, **kwargs) File "/usr/local/samba/lib/python3.7/site-packages/samba/netcmd/domain_backup.py", line 607, in run remove_dc(samdb, logger, cn) File "/usr/local/samba/lib/python3.7/site-packages/samba/remove_dc.py", line 449, in remove_dc remove_dns_account=True) File "/usr/local/samba/lib/python3.7/site-packages/samba/remove_dc.py", line 371, in offline_remove_ntds_dc remove_dns_account=remove_dns_account) File "/usr/local/samba/lib/python3.7/site-packages/samba/remove_dc.py", line 283, in offline_remove_server remove_dns_references(samdb, logger, dnsHostName) File "/usr/local/samba/lib/python3.7/site-packages/samba/remove_dc.py", line 114, in remove_dns_references samdb.dns_replace(dnsHostName, []) File "/usr/local/samba/lib/python3.7/site-packages/samba/samdb.py", line 1256, in dns_replace return dsdb_dns.replace(self, dns_name, new_records) A transaction is still active in ldb context [0x22ac3c0] on /sambarestore/private/sam.ldb I'm happy to send a copy, minus secrets, of my AD DB for investigation if that would help?
I'm sorry for the delay in looking into this. I strongly suspect that our recent work to address fuzzing results from Google's ozz-fuzz has caused Samba to no longer parse previously valid DNS records. To locate good examples, use ldbsearch with the --show-binary option on the DNS partitions, then show us the attribute values without --show-binary (eg as base64). These are unlikely to be sensitive beyond IP addresses and names.
Created attachment 15883 [details] ldbsearch data ldbsearch of the domain and forest partitions, _bin signifies with --show-binary option, compressed into .tar.xz file.
(In reply to Andrew Bartlett from comment #8) No worries Andrew, I hope you are safe and well! I suspected as much and for good measure removed all dNSTombstoned records just in case it was an older one hanging about but no good. I've submitted what I think is what you asked for :)
Is there a way I can filter for the "dodgy" records?
I've set the ldbsearch data to be private as I don't think the world needs to see the main data. The key data is: dNSProperty:: AAAAAMgAAAAAAAAAAQAAAJIAAAAAAAAA dNSProperty:: AAAAAFDogwEAAAAAAQAAAJAAAAB45YMB $ make -j && bin/ndrdump dnsp dnsp_DnsProperty struct --base64-input --input=AAAAAMgAAAAAAAAAAQAAAJIAAAAAAAAA PYTHONHASHSEED=1 WAF_MAKE=1 ./buildtools/bin/waf build Waf: Entering directory `/home/abartlet/samba/bin/default' Selected embedded Heimdal build Waf: Leaving directory `/home/abartlet/samba/bin/default' Build commands will be stored in bin/default/compile_commands.json 'build' finished successfully (11.502s) pull returned Success dnsp_DnsProperty: struct dnsp_DnsProperty wDataLength : 0x00000000 (0) namelength : 0x000000c8 (200) flag : 0x00000000 (0) version : 0x00000001 (1) id : DSPROPERTY_ZONE_NS_SERVERS_DA (146) data : union dnsPropertyData(case 0) name : 0x00000000 (0) dump OK $ make -j && bin/ndrdump dnsp dnsp_DnsProperty struct --base64-input --input=AAAAAFDogwEAAAAAAQAAAJAAAAB45YMB PYTHONHASHSEED=1 WAF_MAKE=1 ./buildtools/bin/waf build Waf: Entering directory `/home/abartlet/samba/bin/default' Selected embedded Heimdal build Waf: Leaving directory `/home/abartlet/samba/bin/default' Build commands will be stored in bin/default/compile_commands.json 'build' finished successfully (9.527s) pull returned Success dnsp_DnsProperty: struct dnsp_DnsProperty wDataLength : 0x00000000 (0) namelength : 0x0183e850 (25421904) flag : 0x00000000 (0) version : 0x00000001 (1) id : DSPROPERTY_ZONE_SCAVENGING_SERVERS_DA (144) data : union dnsPropertyData(case 0) name : 0x0183e578 (25421176) dump OK Reverting ae9a25d1ebe8101ea36f3573cc67261c2d79f4da fixes it, so we need to find a way to still avoid bug 14206.
Alex, Can you describe the history of your domain? Was it running Windows at some point? I'm just trying to understand how these values were created, so I can address this properly. Thanks!
Just to clarify my plan here, I'm assuming this was a Microsoft domain or there has been a Microsoft DC there at some point. If so, I'll ask Microsoft what such a dnsProperty is meant to mean. The value() property in our IDL is an extension and is the cause of the original fuzzing issue. We may simply revert the problem patch and instead move that length calculation to the callers.
I thought I'd rebuilt the domain after the Windows DC we were running had issues but it seems not. Since samba and bind don't use Trust Anchors I'm going to say yes this must be left over from a Windows DC. Thank you for your help with this issue btw!
I've just removed the Trust Anchors container and purged the tombstone from all DCs for good measure, and restarted all services but the DNS manager still isn't working for our DCs that have upgraded to 4.12.0. Are there any other records? Is there a such I can run that just shows records which cause the code to trip up? Cheers!
So git revert fee5c6a4247aeac71318186bbff7708d25de5912 definitely fixes the problem for now :)
I've asked Microsoft about this. If we don't get time/resources to fix this properly then a revert would be an acceptable option.
I'm seeing the same error with DNS updates (and also when using DNS Manager). 21-Apr-2020 16:54:00.536 samba_dlz: starting transaction on zone dom.domain.com 21-Apr-2020 16:54:00.538 client @0xffff88a203e0 10.10.50.10#54757/key H-MSN-SMBDC-01\$\@DOM.DOMAIN.COM: updating zone 'dom.domain.com/NONE': prerequisites are OK 21-Apr-2020 16:54:00.538 samba_dlz: Starting GENSEC mechanism spnego 21-Apr-2020 16:54:00.539 samba_dlz: Starting GENSEC submechanism gssapi_krb5 21-Apr-2020 16:54:00.614 samba_dlz: gensec_gssapi: NO credentials were delegated 21-Apr-2020 16:54:00.614 samba_dlz: GSSAPI Connection will be cryptographically sealed 21-Apr-2020 16:54:00.641 samba_dlz: Successful AuthZ: [(null),krb5] user [DOMAIN]\[H-MSN-SMBDC-01$] [S-1-5-21-692937900-3436463876-1478695411-1195] at [Tue, 21 Apr 2020 16:54:00.641117 CDT] Remote host [NULL] local host [NULL] 21-Apr-2020 16:54:00.641 samba_dlz: {"timestamp": "2020-04-21T16:54:00.641407-0500", "type": "Authorization", "Authorization": {"version": {"major": 1, "minor": 1}, "localAddress": null, "remoteAddress": null, "serviceDescription": null, "authType": "krb5", "domain": "DOMAIN", "account": "H-MSN-SMBDC-01$", "sid": "S-1-5-21-692937900-3436463876-1478695411-1195", "sessionId": "4ba455bd-18fb-4ce1-b1b5-0f6a7a619b97", "logonServer": "H-MSN-SMBDC-01", "transportProtection": "SEAL", "accountFlags": "0x00002100"}} 21-Apr-2020 16:54:00.643 samba_dlz: allowing update of signer=H-MSN-SMBDC-01\$\@DOM.DOMAIN.COM name=dom.domain.com tcpaddr=10.10.50.10 type=NS key=2508936627.sig-h-msn-smbdc-01.dom.domain.com/160/0 21-Apr-2020 16:54:00.683 client @0xffff88a203e0 10.10.50.10#54757/key H-MSN-SMBDC-01\$\@DOM.DOMAIN.COM: updating zone 'dom.domain.com/NONE': update section prescan OK 21-Apr-2020 16:54:00.685 client @0xffff88a203e0 10.10.50.10#54757/key H-MSN-SMBDC-01\$\@DOM.DOMAIN.COM: updating zone 'dom.domain.com/NONE': adding an RR at 'dom.domain.com' NS h-msn-smbdc-01.dom.domain.com. 21-Apr-2020 16:54:00.687 samba_dlz: ndr_pull_uint32: ndr_pull_error(Buffer Size Error): Pull bytes 4 (../../librpc/ndr/ndr_basic.c:152) at ../../librpc/ndr/ndr_basic.c:152 21-Apr-2020 16:54:00.687 samba_dlz: failed to modify DC=@,DC=dom.domain.com,CN=MicrosoftDNS,DC=DomainDnsZones,DC=dom,DC=domain,DC=com - WERR_DNS_ERROR_RCODE_SERVER_FAILURE 21-Apr-2020 16:54:00.696 client @0xffff88a203e0 10.10.50.10#54757/key H-MSN-SMBDC-01\$\@DOM.DOMAIN.COM: updating zone 'dom.domain.com/NONE': rolling back 21-Apr-2020 16:54:00.696 samba_dlz: DSDB Transaction [rollback] at [Tue, 21 Apr 2020 16:54:00.696638 CDT] duration [159812] 21-Apr-2020 16:54:00.696 samba_dlz: {"timestamp": "2020-04-21T16:54:00.696789-0500", "type": "dsdbTransaction", "dsdbTransaction": {"version": {"major": 1, "minor": 0}, "action": "rollback", "transactionId": "3095c3eb-c0e1-4b63-93bb-bc986fd9c1ea", "duration": 159812}} 21-Apr-2020 16:54:00.697 samba_dlz: cancelling transaction on zone dom.domain.com Is there a way to determine what records are causing the issue?
(In reply to Derek Lambert from comment #19) G'Day Derek, You can see that DC=@,DC=dom.domain.com,CN=MicrosoftDNS,DC=DomainDnsZones,DC=dom,DC=domain,DC=com is the record we have a problem with. Have you ever had a Windows DC in that domain? Thanks, Andrew Bartlett
Yes, I currently do. I'm trying to migrate them to Samba :)
In ADSI Edit the dNSProperty for DC=@,DC=dom.domain.com,CN=MicrosoftDNS,DC=DomainDnsZones,DC=dom,DC=domain,DC=com is <not set>.
(In reply to Derek Lambert from comment #21) So, we have a query out with Microsoft as to what the correct parsing is for this value. We tightened up our parsing code for Samba 4.12 in response to some fuzzing work and in this case it was tightened up too much. I'm thinking that we will end up with two different NDR structures, one with length 0 (and enforced with range(0,0) and one with the data.
(In reply to Andrew Bartlett from comment #20) It looks like one of his DCs is trying to update its A record for the root of the domain. I don't think DC=@,DC=dom.domain.com,CN=MicrosoftDNS,DC=DomainDnsZones,DC=dom,DC=domain,DC=com is the actual problem here, it just happens to be the update that has been logged and is failing, like any DNS update would do when there's corrupt data. The actual corrupt data will lie somewhere else (like with me, in CN=TrustAnchors) no?
I got an answer from MS and they say to just ignore the dud attribute value. So a fix here shouldn't be too hard, I just need some time.
Alex, Would you like to be credited? Is so, can you let me know what form? (Name, name and e-mail, name and affiliation/company)? Thanks!
Andrew, That's very kind of you. Just Alex MacCuish is fine :) Keep safe and well!
Created attachment 15984 [details] backported patch for 4.12 (only)
Comment on attachment 15984 [details] backported patch for 4.12 (only) Karolin, Please merge for 4.12.next
Alex, I'm very sorry, we didn't get this tagged in time for the 4.12.3 release just made. It will I hope be in the next 4.12 release currently scheduled for July 2 2020. Sorry!
(In reply to Andrew Bartlett from comment #29) Pushed to autobuild-v4-12-test.
(In reply to Karolin Seeger from comment #31) Pushed to v4-12-test. Closing out bug report. Thanks!