Bug 11245 - FreeIPA trusts to AD broken due to Samba 4.2 failure to run LSARPC pipe externally
Summary: FreeIPA trusts to AD broken due to Samba 4.2 failure to run LSARPC pipe exter...
Status: RESOLVED FIXED
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: DCE-RPCs and pipes (show other bugs)
Version: 4.2.1
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Karolin Seeger
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks: 11061
  Show dependency treegraph
 
Reported: 2015-04-30 10:05 UTC by Alexander Bokovoy
Modified: 2015-06-06 20:04 UTC (History)
2 users (show)

See Also:


Attachments
patch for lsad startup (as in master) (1.39 KB, patch)
2015-05-28 07:46 UTC, Guenther Deschner
ab: review+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Bokovoy 2015-04-30 10:05:02 UTC
(a copy of Red Hat bugzilla's bug 1217346 which is the original bug)

With Samba 4.2.1 I'm unable to even initiate trust to AD with FreeIPA because Samba cannot start lsasd daemon with external LSARPC pipe (and other pipes too).

On FreeIPA's python code trying to establish connection to lsarpc pipe, I'm getting:

[2015/04/30 10:26:46.527638,  2, pid=1257, effective(1981400000, 1981400000), real(1981400000, 0), class=rpc_srv] ../source3/rpc_server/rpc_ncacn_np.c:773(make_external_rpc_pipe)
tstream_npa_connect_recv  to /run/samba/ncalrpc/np for pipe lsarpc and user F22\admin failed: No such file or directory
..
NT error packet at ../source3/smbd/nttrans.c(299) cmd=162 (SMBntcreateX) NT_STATUS_OBJECT_NAME_NOT_FOUND


When Samba starts, it forks out an lsasd daemon off smbd. That daemon creates TCP sockets and forks out handlers for specific pipes. However, it looks like the socket setup succeeds and then a failure is returned by dcerpc_binding_vector_add_port() which causes rpc_create_tcpip_sockets() to bail out and return to lsasd initialization code, causing it to exit.

Using a modified systemd unit for smb.service I was able to catch strace what is happening:
write(14, "  bind succeeded on port 1024\n", 30) = 30
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=127, ...}) = 0
getgid()                                = 0
getuid()                                = 0
getegid()                               = 0
geteuid()                               = 0
geteuid()                               = 0
write(14, "[2015/04/29 18:46:36.028567, 10, pid=1331, effective(0, 0), real(0, 0)] ../source3/rpc_server/rpc_server.c:628(create_tcpip_socket)\n", 132) = 132
geteuid()                               = 0
write(14, "  Opened tcpip socket fd 16 for port 1024\n", 42) = 42
socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 38
setsockopt(38, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
setsockopt(38, SOL_SOCKET, SO_REUSEPORT, [1], 4) = 0
bind(38, {sa_family=AF_INET, sin_port=htons(1024), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=127, ...}) = 0
getgid()                                = 0
getuid()                                = 0
getegid()                               = 0
geteuid()                               = 0
geteuid()                               = 0
write(14, "[2015/04/29 18:46:36.029143, 10, pid=1331, effective(0, 0), real(0, 0)] ../source3/lib/util_sock.c:531(open_socket_in)\n", 119) = 119
geteuid()                               = 0
write(14, "  bind succeeded on port 1024\n", 30) = 30
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=127, ...}) = 0
getgid()                                = 0
getuid()                                = 0
getegid()                               = 0
geteuid()                               = 0
geteuid()                               = 0
write(14, "[2015/04/29 18:46:36.029404, 10, pid=1331, effective(0, 0), real(0, 0)] ../source3/rpc_server/rpc_server.c:628(create_tcpip_socket)\n", 132) = 132
geteuid()                               = 0
write(14, "  Opened tcpip socket fd 38 for port 1024\n", 42) = 42
close(38)                               = 0
close(35)                               = 0
close(3)                                = 0
close(31)                               = 0
exit_group(1)                           = ?
+++ exited with 1 +++

As you can see, socket is created and then closed/exited.

The code is roughly here:
in start_lsasd():
        ok = lsasd_create_sockets(ev_ctx, msg_ctx, listen_fd, &listen_fd_size);
        if (!ok) {
                exit(1);
        }

in lsasd_create_sockets():
        /* Create only one tcpip listener for all services */
        status = rpc_create_tcpip_sockets(&ndr_table_lsarpc,
                                          v_orig,
                                          0,
                                          listen_fd,
                                          listen_fd_size);
        if (!NT_STATUS_IS_OK(status)) {
                goto done;
        }


in rpc_create_tcpip_sockets():
                for (sock_ptr = sock_addr;
                     next_token_talloc(talloc_tos(), &sock_ptr, &sock_tok, " \t,");
                    ) {
                        struct sockaddr_storage ss;
                        int fd;

                        /* open an incoming socket */
                        if (!interpret_string_addr(&ss,
                                                   sock_tok,
                                                   AI_NUMERICHOST|AI_PASSIVE)) {
                                continue;
                        }

                        fd = create_tcpip_socket(&ss, &p);
                        if (fd < 0 || p == 0) {
                                status = NT_STATUS_UNSUCCESSFUL;
                                if (fd != -1) {
                                        close(fd);
                                }
                                goto done;
                        }
                        listen_fd[*listen_fd_size] = fd;
                        (*listen_fd_size)++;

                        if (bvec != NULL) {
                                status = dcerpc_binding_vector_add_port(iface,
                                                                        bvec,
                                                                        sock_ptr,
                                                                        p);
                                if (!NT_STATUS_IS_OK(status)) {
                                        close(fd);
                                        goto done;
                                }
                        }
                }

the way I interpret the strace output is that dcerpc_binding_vector_add_port() fails. It looks like the code in commit 52ccd28ca75bef4f7ac2489389a5aebf5db2b34a may be responsible here.

Commit 52ccd28ca75bef4f7ac2489389a5aebf5db2b34a
s3:dcerpc_ep: make use of dcerpc_binding helper functions
We should not dereference 'struct dcerpc_binding'.

$ git tag --contains 52ccd28ca75bef4f7ac2489389a5aebf5db2b34a |grep samba
samba-4.2.0
samba-4.2.0rc1
samba-4.2.0rc2
samba-4.2.0rc3
samba-4.2.0rc4
samba-4.2.0rc5
samba-4.2.1

Guenther and I found actual issue -- rpc_create_tcpip_sockets() passed empty string (non-NULL, '') to dcerpc_binding_vector_add_port() and then dcerpc_binding_vector_add_port() tried to set binding's option 'host' to this string.

With the commit 017338a1 which is also 4.2 only, setting empty string to special properties of the binding is not possible:

+               if (value[0] == '\0') {
+                       return NT_STATUS_INVALID_PARAMETER_MIX;
+               }
+

Following patch fixed the issue for us:
commit 1ff9ffea061e4bdecea65749d8d0e3c082e25d77
Author:     Günther Deschner <gd@samba.org>
AuthorDate: Thu Apr 30 11:20:58 2015 +0200
Commit:     Günther Deschner <gd@samba.org>
CommitDate: Thu Apr 30 11:22:26 2015 +0200

    s3-rpc_server: fix rpc_create_tcpip_sockets() processing of interfaces.
    
    We were supplying an empty value for the "host" binding string option, causing
    dcerpc_binding_vector_add_port() call to fail.
    
    Guenther
    
    Signed-off-by: Günther Deschner <gd@samba.org>
    Pair-Programmed-With: Alexander Bokovoy <ab@samba.org>
---
 source3/rpc_server/rpc_sock_helper.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/source3/rpc_server/rpc_sock_helper.c b/source3/rpc_server/rpc_sock_helper.c
index 8f371b8..7aced63 100644
--- a/source3/rpc_server/rpc_sock_helper.c
+++ b/source3/rpc_server/rpc_sock_helper.c
@@ -138,7 +138,7 @@ NTSTATUS rpc_create_tcpip_sockets(const struct ndr_interface_table *iface,
 			if (bvec != NULL) {
 				status = dcerpc_binding_vector_add_port(iface,
 									bvec,
-									sock_ptr,
+									sock_tok,
 									p);
 				if (!NT_STATUS_IS_OK(status)) {
 					close(fd);

	

    Collapse All Comments
    Expand All Comments
    Add Comment
    Unwrap comments
    Show CC Changes
Comment 1 Jeremy Allison 2015-04-30 16:20:49 UTC
Any chance of a test case also to stop us regressing on this ?
Comment 2 Alexander Bokovoy 2015-04-30 16:22:05 UTC
Yes, test lsasd without 'bind interfaces only=yes' :)

I wonder though if this is going to work under make test.
Comment 3 Andrew Bartlett 2015-05-01 09:39:00 UTC
Shouldn't this be marked as a duplicate of bug 11061.  It certainly should be a blocker of it.
Comment 4 Alexander Bokovoy 2015-05-01 15:10:05 UTC
(In reply to Andrew Bartlett from comment #3)
I don't think bug 11061 is relevant here because you never run s3 lsasd external daemon in that configuration. Initially it looked very similar but you need to verify the logs (they are double gzipped) to see that bug 11061 is related to s4 pipe processing, not s3.
Comment 5 Guenther Deschner 2015-05-28 07:46:58 UTC
Created attachment 11098 [details]
patch for lsad startup (as in master)
Comment 6 Alexander Bokovoy 2015-05-28 09:24:46 UTC
Comment on attachment 11098 [details]
patch for lsad startup (as in master)

Patch is OK.
Comment 7 Guenther Deschner 2015-05-28 09:39:09 UTC
Karolin, please add to v4-2-test. Thanks!
Comment 8 Karolin Seeger 2015-06-01 19:05:07 UTC
(In reply to Guenther Deschner from comment #7)
Pushed to autobuild-v4-2-test.
Comment 9 Karolin Seeger 2015-06-06 20:04:32 UTC
(In reply to Karolin Seeger from comment #8)
Pushed to v4-2-test.
Closing out bug report.

Thanks!