Bug 9832 - libtalloc 2.0.8 double free error
libtalloc 2.0.8 double free error
Status: RESOLVED FIXED
Product: TALLOC
Classification: Unclassified
Component: libtalloc
unspecified
x64 FreeBSD
: P2 major
: ---
Assigned To: Karolin Seeger
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-04-29 16:31 UTC by Christopher Harrison
Modified: 2013-11-26 09:57 UTC (History)
3 users (show)

See Also:


Attachments
Patch (823 bytes, patch)
2013-04-29 16:41 UTC, Volker Lendecke
no flags Details
Patch with cp-info (984 bytes, patch)
2013-04-30 09:09 UTC, Volker Lendecke
metze: review+
Details
harrison.vcf (227 bytes, text/x-vcard)
2013-04-30 15:26 UTC, Christopher Harrison
no flags Details
Patch (1.13 KB, patch)
2013-05-07 08:33 UTC, Volker Lendecke
no flags Details
New patch for the second issue (819 bytes, patch)
2013-05-07 08:39 UTC, Volker Lendecke
no flags Details
2nd patch with cherry-pick information (1.08 KB, patch)
2013-05-28 12:42 UTC, Stefan Metzmacher
jra: review+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Christopher Harrison 2013-04-29 16:31:25 UTC
Talloc appears to have a double free bug which causes samba4 to crash.   Here is the stack trace from a running S4 which experienced the problem.

(gdb) bt
#0  0x00000008047f124c in thr_kill () from /lib/libc.so.7
#1  0x000000080489385b in abort () from /lib/libc.so.7
#2  0x000000080084803e in smb_panic_default (why=Could not find the frame base for "smb_panic_default".
) at ../lib/util/fault.c:149
#3  0x0000000800848078 in smb_panic (
    why=0x803f574d8 "Bad talloc magic value - access after free")
    at ../lib/util/fault.c:162
#4  0x0000000803f4fe3c in talloc_vasprintf ()
   from /usr/local/lib/libtalloc.so.2
#5  0x0000000803f4ff38 in talloc_asprintf () from /usr/local/lib/libtalloc.so.2
#6  0x0000000816eb4506 in wbsrv_call_loop (subreq=0x0)
    at ../source4/winbind/wb_server.c:78
#7  0x0000000801505a99 in tstream_read_pdu_blob_done (subreq=0x0)
    at ../libcli/util/tstream.c:117
#8  0x0000000807084371 in tstream_readv_done (subreq=0x0)
    at ../lib/tsocket/tsocket.c:604
#9  0x00000008070883f9 in tstream_bsd_readv_handler (private_data=0x821ec8260)
    at ../lib/tsocket/tsocket_bsd.c:1769
#10 0x0000000807087b24 in tstream_bsd_fde_handler (ev=0x80d82d760,
    fde=0x80dbe1260, flags=1, private_data=0x80dbe0de0)
    at ../lib/tsocket/tsocket_bsd.c:1487
#11 0x000000080458e0e2 in std_event_loop_once ()
   from /usr/local/lib/libtevent.so.0
#12 0x000000080458addc in _tevent_loop_once ()
---Type <return> to continue, or q <return> to quit---
   from /usr/local/lib/libtevent.so.0
#13 0x000000080458ae4b in tevent_common_loop_wait ()
   from /usr/local/lib/libtevent.so.0
#14 0x0000000810b52408 in standard_new_task (ev=0x80d82d760, 
    lp_ctx=0x80d841860, service_name=0x816ecb604 "winbind", 
    new_task=0x8015055e0 <task_server_callback>, private_data=0x80dbf33f0)
    at ../source4/smbd/process_standard.c:186
#15 0x0000000801505790 in task_server_startup (event_ctx=0x80d82d760, 
    lp_ctx=0x80d841860, service_name=0x816ecb604 "winbind", 
    model_ops=0x810d52a60, task_init=0x816eb47d0 <winbind_task_init>)
    at ../source4/smbd/service_task.c:112
#16 0x0000000801503db1 in server_service_init (name=0x80d873c70 "winbind", 
    event_context=0x80d82d760, lp_ctx=0x80d841860, model_ops=0x810d52a60)
    at ../source4/smbd/service.c:63
#17 0x0000000801503f06 in server_service_startup (event_ctx=0x80d82d760, 
    lp_ctx=0x80d841860, model=0x410421 "standard", server_services=0x80d82d460)
    at ../source4/smbd/service.c:95
#18 0x000000000040ac7c in binary_smbd_main (binary_name=0x410232 "samba", 
    argc=2, argv=0x7fffffffd6f0) at ../source4/smbd/server.c:477
#19 0x000000000040ad62 in main (argc=2, argv=0x7fffffffd6f0)
    at ../source4/smbd/server.c:497
(gdb)
Comment 1 Volker Lendecke 2013-04-29 16:41:06 UTC
Created attachment 8828 [details]
Patch

Can you try this?
Comment 2 Christopher Harrison 2013-04-29 18:44:13 UTC
(In reply to comment #1)
> Created attachment 8828 [details]
> Patch
> 
> Can you try this?

Testing it now
Comment 3 Stefan Metzmacher 2013-04-30 06:33:23 UTC
Comment on attachment 8828 [details]
Patch

Reviewed-by: Stefan Metzmacher <metze@samba.org>

Volker, please push this to master :-)
Comment 4 Volker Lendecke 2013-04-30 09:09:27 UTC
Created attachment 8832 [details]
Patch with cp-info
Comment 5 Christopher Harrison 2013-04-30 15:26:13 UTC
Problem still exists.

 From the samba log:
single_terminate: reason[wbsrv_call_loop: tstream_read_pdu_blob_recv() - 
NT_STATUS_CONNECTION_DISCONNECTED]
talloc: access after free error - first free may be at 
../source4/winbind/wb_samba3_protocol.c:361

(gdb) bt
#0  0x00000008047f124c in thr_kill () from /lib/libc.so.7
#1  0x000000080489385b in abort () from /lib/libc.so.7
#2  0x000000080084803e in smb_panic_default (why=Could not find the 
frame base for "smb_panic_default".
) at ../lib/util/fault.c:149
#3  0x0000000800848078 in smb_panic (
     why=0x803f574d8 "Bad talloc magic value - access after free")
     at ../lib/util/fault.c:162
#4  0x0000000803f4fe3c in talloc_vasprintf ()
    from /usr/local/lib/libtalloc.so.2
#5  0x0000000803f4ff38 in talloc_asprintf () from 
/usr/local/lib/libtalloc.so.2
#6  0x0000000816eb4506 in wbsrv_call_loop (subreq=0x0)
     at ../source4/winbind/wb_server.c:78
#7  0x0000000801505a99 in tstream_read_pdu_blob_done (subreq=0x0)
     at ../libcli/util/tstream.c:117
#8  0x0000000807084371 in tstream_readv_done (subreq=0x0)
     at ../lib/tsocket/tsocket.c:604
#9  0x00000008070883f9 in tstream_bsd_readv_handler 
(private_data=0x821eca060)
     at ../lib/tsocket/tsocket_bsd.c:1769
#10 0x0000000807087b24 in tstream_bsd_fde_handler (ev=0x80d82d760,
     fde=0x820c7e5e0, flags=1, private_data=0x820c7df20)
     at ../lib/tsocket/tsocket_bsd.c:1487
#11 0x000000080458e0e2 in std_event_loop_once ()

On 04/30/13 04:13, samba-bugs@samba.org wrote:
> https://bugzilla.samba.org/show_bug.cgi?id=9832
>
> Stefan (metze) Metzmacher<metze@samba.org>  changed:
>
>             What    |Removed                     |Added
> ----------------------------------------------------------------------------
>           AssignedTo|idra@samba.org              |ks@sernet.de
>
Comment 6 Christopher Harrison 2013-04-30 15:26:13 UTC
Created attachment 8840 [details]
harrison.vcf
Comment 7 Christopher Harrison 2013-04-30 15:55:13 UTC
talloc: access after free error - first free may be at ../source4/winbind/wb_samba3_protocol.c:361
Bad talloc magic value - access after free
PANIC: Bad talloc magic value - access after free
Comment 8 Karolin Seeger 2013-05-06 19:19:27 UTC
Pushed patch to autobuild-v4-0-test.

Re-assigning to Volker.
Volker, is this still an issue?
Comment 9 Volker Lendecke 2013-05-07 08:11:34 UTC
(In reply to comment #8)
> Pushed patch to autobuild-v4-0-test.
> 
> Re-assigning to Volker.
> Volker, is this still an issue?

Without more information we can't know. Apparently the patch did not fix the issue, but we need a backtrace.
Comment 10 Volker Lendecke 2013-05-07 08:12:05 UTC
Ah, sorry, we do have a backtrace
Comment 11 Karolin Seeger 2013-05-07 08:26:25 UTC
(In reply to comment #9)
> (In reply to comment #8)
> > Pushed patch to autobuild-v4-0-test.
> > 
> > Re-assigning to Volker.
> > Volker, is this still an issue?
> 
> Without more information we can't know. Apparently the patch did not fix the
> issue, but we need a backtrace.

But the patch is needed anyway?
Comment 12 Volker Lendecke 2013-05-07 08:33:29 UTC
Created attachment 8858 [details]
Patch

More fixes for invalid talloc hierarchy.
Comment 13 Volker Lendecke 2013-05-07 08:35:52 UTC
The patch I uploaded is correct and does fix the issue reported earlier. The second backtrace is a different bug of the same nature. I will submit a better prepared patch when this second issue has gone through master.
Comment 14 Volker Lendecke 2013-05-07 08:39:26 UTC
Created attachment 8859 [details]
New patch for the second issue

Sorry for the confusion. I had missed the fact that the first patch had not yet been submitted to 4-0-test. I have fixed the first issue again with attachment 8858 [details], which won't apply to the just pushed patch 8832.
Comment 15 Karolin Seeger 2013-05-07 08:50:49 UTC
(In reply to comment #14)
> Created attachment 8859 [details]
> New patch for the second issue
> 
> Sorry for the confusion. I had missed the fact that the first patch had not yet
> been submitted to 4-0-test. I have fixed the first issue again with attachment
> 8858 [details], which won't apply to the just pushed patch 8832.

The first patch had been pushed to autobuild-v4-0-test yesterday, but unfortunately the autobuild failed and was re-started this morning.
Comment 16 Christopher Harrison 2013-05-14 15:52:05 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > Created attachment 8859 [details] [details]
> > New patch for the second issue
> > 
> > Sorry for the confusion. I had missed the fact that the first patch had not yet
> > been submitted to 4-0-test. I have fixed the first issue again with attachment
> > 8858 [details], which won't apply to the just pushed patch 8832.
> 
> The first patch had been pushed to autobuild-v4-0-test yesterday, but
> unfortunately the autobuild failed and was re-started this morning.

Any progress?   It looks like this is still a problem

talloc: access after free error - first free may be at ../source4/winbind/wb_samba3_protocol.c:361
Bad talloc magic value - access after free
PANIC: Bad talloc magic value - access after free
Comment 17 Christopher Harrison 2013-05-14 17:03:30 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > Created attachment 8859 [details] [details]
> > New patch for the second issue
> > 
> > Sorry for the confusion. I had missed the fact that the first patch had not yet
> > been submitted to 4-0-test. I have fixed the first issue again with attachment
> > 8858 [details], which won't apply to the just pushed patch 8832.
> 
> The first patch had been pushed to autobuild-v4-0-test yesterday, but
> unfortunately the autobuild failed and was re-started this morning.

Any progress?   It looks like this is still a problem

talloc: access after free error - first free may be at ../source4/winbind/wb_samba3_protocol.c:361
Bad talloc magic value - access after free
PANIC: Bad talloc magic value - access after free
Comment 18 sascha 2013-05-15 08:39:11 UTC
hi all,

i can confirm this bug with winbind enabled via nsswitch. a samaba4  stand-alone install doesn't stuck with this bug.

i applied both patches and thing are fine now. applying both patches fixed the issue.


thanks!
Comment 19 Volker Lendecke 2013-05-15 09:16:36 UTC
(In reply to comment #17)
> (In reply to comment #15)
> > (In reply to comment #14)
> > > Created attachment 8859 [details] [details] [details]
> > > New patch for the second issue
> > > 
> > > Sorry for the confusion. I had missed the fact that the first patch had not yet
> > > been submitted to 4-0-test. I have fixed the first issue again with attachment
> > > 8858 [details], which won't apply to the just pushed patch 8832.
> > 
> > The first patch had been pushed to autobuild-v4-0-test yesterday, but
> > unfortunately the autobuild failed and was re-started this morning.
> 
> Any progress?   It looks like this is still a problem
> 
> talloc: access after free error - first free may be at
> ../source4/winbind/wb_samba3_protocol.c:361
> Bad talloc magic value - access after free
> PANIC: Bad talloc magic value - access after free

Did you apply both patches?

https://bugzilla.samba.org/attachment.cgi?id=8832 and 
https://bugzilla.samba.org/attachment.cgi?id=8859 ?
Comment 20 sascha 2013-05-15 09:24:09 UTC
Yes, I did apply both patches to FreeBSD ports version 4.0.4_1 and things are just fine.
Comment 21 Volker Lendecke 2013-05-15 09:26:11 UTC
(In reply to comment #20)
> Yes, I did apply both patches to FreeBSD ports version 4.0.4_1 and things are
> just fine.

This question was more for  Christopher Harrison  :-)
Comment 22 Christopher Harrison 2013-05-15 14:26:55 UTC
I applied one of them but not both. Let me apply both and get back to 
you later if this fixed the issue.
     -C

On 5/15/13 4:26 AM, samba-bugs@samba.org wrote:
> https://bugzilla.samba.org/show_bug.cgi?id=9832
>
> --- Comment #21 from Volker Lendecke <vl@samba.org> 2013-05-15 09:26:11 UTC ---
> (In reply to comment #20)
>> Yes, I did apply both patches to FreeBSD ports version 4.0.4_1 and things are
>> just fine.
> This question was more for  Christopher Harrison  :-)
>
Comment 23 Stefan Metzmacher 2013-05-28 12:42:31 UTC
Created attachment 8927 [details]
2nd patch with cherry-pick information
Comment 24 Jeremy Allison 2013-05-30 18:18:08 UTC
Comment on attachment 8927 [details]
2nd patch with cherry-pick information

LGTM.
Comment 25 Jeremy Allison 2013-05-30 18:18:39 UTC
Re-assigning to Karolin for inclusion in 4.0.next.
Jeremy.
Comment 26 Karolin Seeger 2013-06-03 10:17:54 UTC
Pushed to autobuild-v4-0-test.
Comment 27 Karolin Seeger 2013-06-04 07:49:19 UTC
Pushed to v4-0-test.
Closing out bug report.

Thanks!
Comment 28 Kinglok, Fong 2013-11-25 04:35:49 UTC
Hi,

This floods my server log running DC on 4.1.2.  I don’t know if it is the same problem like bug 9832?

Kinglok, Fong



======================================================= 

[2013/11/25 10:53:03.708423,  0] ../source3/lib/dumpcore.c:317(dump_core)
  dumping core in /usr/local/samba/var/cores/smbd
[2013/11/25 10:53:04.106077,  0] ../source3/lib/popt_common.c:67(popt_s3_talloc_log_fn)
  talloc: access after free error - first free may be at ../source3/smbd/open.c:1569
[2013/11/25 10:53:04.106163,  0] ../source3/lib/popt_common.c:67(popt_s3_talloc_log_fn)
  Bad talloc magic value - access after free
[2013/11/25 10:53:04.106194,  0] ../source3/lib/util.c:785(smb_panic_s3)
  PANIC (pid 20650): Bad talloc magic value - access after free
[2013/11/25 10:53:04.106573,  0] ../source3/lib/util.c:896(log_stack_trace)
  BACKTRACE: 22 stack frames:
   #0 /usr/local/samba/lib/libsmbconf.so.0(log_stack_trace+0x1f) [0x7f8ffd32f2d2]
   #1 /usr/local/samba/lib/libsmbconf.so.0(smb_panic_s3+0x6c) [0x7f8ffd32f14b]
   #2 /usr/local/samba/lib/libsamba-util.so.0(smb_panic+0x28) [0x7f8ffed9f17f]
   #3 /usr/local/samba/lib/private/libtalloc.so.2(+0x241d) [0x7f8ffe3c041d]
   #4 /usr/local/samba/lib/private/libtalloc.so.2(+0x2499) [0x7f8ffe3c0499]
   #5 /usr/local/samba/lib/private/libtalloc.so.2(+0x2516) [0x7f8ffe3c0516]
   #6 /usr/local/samba/lib/private/libtalloc.so.2(talloc_get_name+0x18) [0x7f8ffe3c1fb6]
   #7 /usr/local/samba/lib/private/libtalloc.so.2(_talloc_get_type_abort+0x4c) [0x7f8ffe3c2136]
   #8 /usr/local/samba/lib/libsmbconf.so.0(+0x33ccf) [0x7f8ffd33accf]
   #9 /usr/local/samba/lib/private/libtevent.so.0(tevent_common_loop_immediate+0x1f5) [0x7f8ffe5cc358]
   #10 /usr/local/samba/lib/libsmbconf.so.0(run_events_poll+0x56) [0x7f8ffd34b3f9]
   #11 /usr/local/samba/lib/libsmbconf.so.0(+0x44abd) [0x7f8ffd34babd]
   #12 /usr/local/samba/lib/private/libtevent.so.0(_tevent_loop_once+0xf4) [0x7f8ffe5cb492]
   #13 /usr/local/samba/lib/private/libsmbd_base.so(smbd_process+0x12ef) [0x7f8ffe94237b]
   #14 /usr/local/samba/sbin/smbd(+0xa12d) [0x7f8fff40512d]
   #15 /usr/local/samba/lib/libsmbconf.so.0(run_events_poll+0x55f) [0x7f8ffd34b902]
   #16 /usr/local/samba/lib/libsmbconf.so.0(+0x44bce) [0x7f8ffd34bbce]
   #17 /usr/local/samba/lib/private/libtevent.so.0(_tevent_loop_once+0xf4) [0x7f8ffe5cb492]
   #18 /usr/local/samba/sbin/smbd(+0xad97) [0x7f8fff405d97]
   #19 /usr/local/samba/sbin/smbd(main+0x1753) [0x7f8fff40763b]
   #20 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f8ffbbc1ead]
   #21 /usr/local/samba/sbin/smbd(+0x5ed9) [0x7f8fff400ed9]
[2013/11/25 10:53:04.106797,  0] ../source3/lib/dumpcore.c:317(dump_core)
  dumping core in /usr/local/samba/var/cores/smbd
Comment 29 Volker Lendecke 2013-11-25 11:23:28 UTC
(In reply to comment #28)
> Hi,
> 
> This floods my server log running DC on 4.1.2.  I don’t know if it is the same
> problem like bug 9832?
> 
> Kinglok, Fong
> 
> 
> 
> ======================================================= 
> 
> [2013/11/25 10:53:03.708423,  0] ../source3/lib/dumpcore.c:317(dump_core)
>   dumping core in /usr/local/samba/var/cores/smbd
> [2013/11/25 10:53:04.106077,  0]
> ../source3/lib/popt_common.c:67(popt_s3_talloc_log_fn)
>   talloc: access after free error - first free may be at
> ../source3/smbd/open.c:1569
> [2013/11/25 10:53:04.106163,  0]
> ../source3/lib/popt_common.c:67(popt_s3_talloc_log_fn)
>   Bad talloc magic value - access after free
> [2013/11/25 10:53:04.106194,  0] ../source3/lib/util.c:785(smb_panic_s3)
>   PANIC (pid 20650): Bad talloc magic value - access after free
> [2013/11/25 10:53:04.106573,  0] ../source3/lib/util.c:896(log_stack_trace)
>   BACKTRACE: 22 stack frames:
>    #0 /usr/local/samba/lib/libsmbconf.so.0(log_stack_trace+0x1f)
> [0x7f8ffd32f2d2]
>    #1 /usr/local/samba/lib/libsmbconf.so.0(smb_panic_s3+0x6c) [0x7f8ffd32f14b]
>    #2 /usr/local/samba/lib/libsamba-util.so.0(smb_panic+0x28) [0x7f8ffed9f17f]
>    #3 /usr/local/samba/lib/private/libtalloc.so.2(+0x241d) [0x7f8ffe3c041d]
>    #4 /usr/local/samba/lib/private/libtalloc.so.2(+0x2499) [0x7f8ffe3c0499]
>    #5 /usr/local/samba/lib/private/libtalloc.so.2(+0x2516) [0x7f8ffe3c0516]
>    #6 /usr/local/samba/lib/private/libtalloc.so.2(talloc_get_name+0x18)
> [0x7f8ffe3c1fb6]
>    #7 /usr/local/samba/lib/private/libtalloc.so.2(_talloc_get_type_abort+0x4c)
> [0x7f8ffe3c2136]
>    #8 /usr/local/samba/lib/libsmbconf.so.0(+0x33ccf) [0x7f8ffd33accf]
>    #9
> /usr/local/samba/lib/private/libtevent.so.0(tevent_common_loop_immediate+0x1f5)
> [0x7f8ffe5cc358]
>    #10 /usr/local/samba/lib/libsmbconf.so.0(run_events_poll+0x56)
> [0x7f8ffd34b3f9]
>    #11 /usr/local/samba/lib/libsmbconf.so.0(+0x44abd) [0x7f8ffd34babd]
>    #12 /usr/local/samba/lib/private/libtevent.so.0(_tevent_loop_once+0xf4)
> [0x7f8ffe5cb492]
>    #13 /usr/local/samba/lib/private/libsmbd_base.so(smbd_process+0x12ef)
> [0x7f8ffe94237b]
>    #14 /usr/local/samba/sbin/smbd(+0xa12d) [0x7f8fff40512d]
>    #15 /usr/local/samba/lib/libsmbconf.so.0(run_events_poll+0x55f)
> [0x7f8ffd34b902]
>    #16 /usr/local/samba/lib/libsmbconf.so.0(+0x44bce) [0x7f8ffd34bbce]
>    #17 /usr/local/samba/lib/private/libtevent.so.0(_tevent_loop_once+0xf4)
> [0x7f8ffe5cb492]
>    #18 /usr/local/samba/sbin/smbd(+0xad97) [0x7f8fff405d97]
>    #19 /usr/local/samba/sbin/smbd(main+0x1753) [0x7f8fff40763b]
>    #20 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f8ffbbc1ead]
>    #21 /usr/local/samba/sbin/smbd(+0x5ed9) [0x7f8fff400ed9]
> [2013/11/25 10:53:04.106797,  0] ../source3/lib/dumpcore.c:317(dump_core)
>   dumping core in /usr/local/samba/var/cores/smbd

But 10284 has a patch that fixes a double-free: https://bugzilla.samba.org/attachment.cgi?id=9466. Maybe your is the same?
Comment 30 Kinglok, Fong 2013-11-26 09:57:14 UTC
After applying the patch (https://bugzilla.samba.org/attachment.cgi?id=9466),  I confirm the problem is solved.

Thanks!