Bug 11128 - ndr_push_error unexpected blob length is too large
ndr_push_error unexpected blob length is too large
Status: RESOLVED FIXED
Product: Samba 4.1 and newer
Classification: Unclassified
Component: DNS server
4.1.17
All All
: P5 normal
: ---
Assigned To: Kai Blin
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2015-03-03 12:12 UTC by Tim
Modified: 2016-03-08 13:34 UTC (History)
8 users (show)

See Also:


Attachments
EDNS Problem (1.05 KB, application/octet-stream)
2015-03-15 10:04 UTC, bugs
no flags Details
Possible patches for master (14.87 KB, patch)
2015-08-19 04:01 UTC, Stefan Metzmacher
metze: review? (kai)
slow: review? (slow)
metze: review? (garming)
metze: review? (abartlet)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tim 2015-03-03 12:12:42 UTC
Setting log level to 3 on an AD DC (SerNet packages) throws up the following errors in /var/log/samba/log.samba every 2-3 minutes:
[2015/03/03 12:50:14.432182,  1] ../librpc/ndr/ndr.c:489(ndr_push_error)
  ndr_push_error(6): Invalid...Unexpected blob length is too large


Every ndr_push_error seems to have a dependency to forwarding DNS queries:

Snippet:
# cat /var/log/samba/log.samba | grep -B2 ndr_push_error

[2015/03/03 12:39:19.917410,  2] ../source4/dns_server/dns_query.c:629(dns_server_process_query_send)
  Not authoritative for 'aus4.vips.phx1.mozilla.com', forwarding
[2015/03/03 12:39:19.939079,  1] ../librpc/ndr/ndr.c:489(ndr_push_error)
  ndr_push_error(6): Invalid...Unexpected blob length is too large
--
[2015/03/03 12:43:20.092877,  2] ../source4/dns_server/dns_query.c:629(dns_server_process_query_send)
  Not authoritative for 'snippets.mozilla.com', forwarding
[2015/03/03 12:43:20.114941,  1] ../librpc/ndr/ndr.c:489(ndr_push_error)
  ndr_push_error(6): Invalid...Unexpected blob length is too large
--
[2015/03/03 12:44:14.379673,  2] ../source4/dns_server/dns_query.c:629(dns_server_process_query_send)
  Not authoritative for 'aus4.vips.phx1.mozilla.com', forwarding
[2015/03/03 12:44:14.385740,  1] ../librpc/ndr/ndr.c:489(ndr_push_error)
  ndr_push_error(6): Invalid...Unexpected blob length is too large
[2015/03/03 12:44:14.385802,  1] ../source4/dns_server/dns_server.c:257(dns_process_recv)
  Failed to push packet: Length Error!
[2015/03/03 12:44:14.400388,  1] ../librpc/ndr/ndr.c:489(ndr_push_error)
  ndr_push_error(6): Invalid...Unexpected blob length is too large
--
[2015/03/03 12:48:14.442329,  2] ../source4/dns_server/dns_query.c:629(dns_server_process_query_send)
  Not authoritative for 'services.addons.mozilla.org', forwarding
[2015/03/03 12:48:14.463250,  1] ../librpc/ndr/ndr.c:489(ndr_push_error)
  ndr_push_error(6): Invalid...Unexpected blob length is too large
--
[2015/03/03 12:48:15.473271,  2] ../source4/dns_server/dns_query.c:629(dns_server_process_query_send)
  Not authoritative for 'versioncheck.addons.mozilla.org', forwarding
[2015/03/03 12:48:15.494935,  1] ../librpc/ndr/ndr.c:489(ndr_push_error)
  ndr_push_error(6): Invalid...Unexpected blob length is too large
--
[2015/03/03 12:48:16.277772,  2] ../source4/dns_server/dns_query.c:629(dns_server_process_query_send)
  Not authoritative for 'versioncheck.addons.mozilla.org', forwarding
[2015/03/03 12:48:16.298855,  1] ../librpc/ndr/ndr.c:489(ndr_push_error)
  ndr_push_error(6): Invalid...Unexpected blob length is too large
--
[2015/03/03 12:50:14.410407,  2] ../source4/dns_server/dns_query.c:629(dns_server_process_query_send)
  Not authoritative for 'blocklist.addons.mozilla.org', forwarding
[2015/03/03 12:50:14.432182,  1] ../librpc/ndr/ndr.c:489(ndr_push_error)
  ndr_push_error(6): Invalid...Unexpected blob length is too large

There are also other domains queried than mozilla.org
These errors are not expected to be there. I can only say that sometimes DNS queries on clients "feel" slow if this matters.

Samba version: 4.1.17 SerNet
OS: CentOS 7
Hardware: HP Proliant DL380G6
Comment 1 David Disseldorp 2015-03-03 13:11:26 UTC
@Tim, please provide a network trace covering the dns request and attach it to the bug - https://wiki.samba.org/index.php/Capture_Packets

Attachments can be made private to the Samba Team if necessary.
Comment 2 bugs 2015-03-13 12:47:20 UTC
Someone already made a network trace .... see https://lists.samba.org/archive/samba-technical/2013-February/090263.html
Comment 3 David Disseldorp 2015-03-13 12:49:12 UTC
(In reply to bugs from comment #2)

To quote Andrew's response:

"Just as a heads up: When we ask for a network trace, we actually mean
the PCAP file, not the parsed output.  This allows us to inspect it in
detail using wireshark."
Comment 4 Tim 2015-03-13 16:43:42 UTC
Strange... I can't reproduce it for now. I got an update from sernet and version went from 4.1.17-SerNet-RedHat-10.el7 to 4.1.17-SerNet-RedHat-11.el7.

I will observe it.
Comment 5 bugs 2015-03-14 07:37:44 UTC
I compiled stable 4_2 two days ago. I get problems with long txt records (DKIM). @David: OK, next time on my pc , i'll get your a pcap file.
Comment 6 bugs 2015-03-15 10:04:30 UTC
Created attachment 10872 [details]
EDNS Problem
Comment 7 bugs 2015-03-15 10:05:21 UTC
i suppose there is a problem with EDNS. samba sends requests with EDNS flag on and seems to complain when answer payload is bigger than 512 bytes.

pcap-file attached
Comment 8 David Disseldorp 2015-03-16 11:37:38 UTC
Thanks for the trace. As you've indicated, it appears that EDNS0 payload marshalling / unmarshalling is broken.

The corresponding librpc/idl/dns.idl is:

184         typedef [public] struct {
185                 uint16     option_code;
186                 uint16     option_length;
187                 uint8      option_data[option_length];
188         } dns_opt_record;
...
229         typedef [nodiscriminant,public,flag(NDR_NOALIGN)] union {
...
241                 [case(DNS_QTYPE_OPT)]   dns_opt_record   opt_record;
...
245         } dns_rdata;
246 
247         typedef [flag(LIBNDR_PRINT_ARRAY_HEX|NDR_NOALIGN),nopush,nopull] struct {
248                 dns_string name;
249                 dns_qtype  rr_type;
250                 dns_qclass rr_class;
251                 uint32     ttl;
252                 uint16     length; /* Should be set to either UINT16_MAX or 0 */
253                 [switch_is(rr_type)] dns_rdata rdata;
254                 DATA_BLOB  unexpected;
255         } dns_res_rec;

This resembles the rfc2671 specification...

4.3. The fixed part of an OPT RR is structured as follows:


     Field Name   Field Type     Description
     ------------------------------------------------------
     NAME         domain name    empty (root domain)
     TYPE         u_int16_t      OPT
     CLASS        u_int16_t      sender's UDP payload size
     TTL          u_int32_t      extended RCODE and flags
     RDLEN        u_int16_t      describes RDATA
     RDATA        octet stream   {attribute,value} pairs

4.4. The variable part of an OPT RR is encoded in its RDATA and is

structured as zero or more of the following:


                +0 (MSB)                            +1 (LSB)
     +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
  0: |                          OPTION-CODE                          |
     +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
  2: |                         OPTION-LENGTH                         |
     +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
  4: |                                                               |
     /                          OPTION-DATA                          /
     /                                                               /
     +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+

   OPTION-CODE    (Assigned by IANA.)

   OPTION-LENGTH  Size (in octets) of OPTION-DATA.

   OPTION-DATA    Varies per OPTION-CODE.

However, note that rfc2671 states RDATA can be composed of *zero* or more attribute/value pairs. The Samba IDL appears to assume the presence of a single attribute/value pair, which is not included in the attached trace.

IIUC, a suitable fix should see the marshalling / unmarshalling code take into account the RDLEN value in the fixed part of the OPT RR structure.

@Kai, any chance you could take it from here?
Comment 9 Kai Blin 2015-03-17 07:02:24 UTC
Interesting. I'd have expected that's just what that IDL would be doing. I'll have to poke one of the IDL cracks, as I'm a bit stumped what to do here.
Comment 10 Stefan Metzmacher 2015-08-19 04:01:43 UTC
Created attachment 11351 [details]
Possible patches for master
Comment 11 Stefan Metzmacher 2016-01-04 08:21:45 UTC
(In reply to Stefan Metzmacher from comment #10)

Garming, this seems to be the fix for the problem you're trying to solve
in https://git.samba.org/?p=garming/samba.git;a=shortlog;h=refs/heads/dns-txt-4
Comment 12 Andrew Bartlett 2016-01-04 08:29:25 UTC
I expect to look at this with Garming this week, he has become quite familiar with the DNS parser of late.
Comment 13 Stefan Metzmacher 2016-03-08 13:34:03 UTC
This should be fixed with Samba 4.3.6, 4.2.9, 4.1.23 and 4.4.0rc4