Originally reported at: http://www.freebsd.org/cgi/query-pr.cgi?pr=187482 and http://www.freebsd.org/cgi/query-pr.cgi?pr=183285 The symptoms while building 4.1.6 are: cc: warning: argument unused during compilation: '-undefined dynamic_lookup' [3542/3756] Linking default/source3/winbindd/libidmap-rfc2307.so cc: warning: argument unused during compilation: '-undefined dynamic_lookup' [3543/3756] Linking default/source4/ldap_server/libservice-ldap.so /usr/local/lib/libsamdb.so: undefined reference to `sam_rid_crypt@SAMBA_4.1.4' /usr/local/lib/libsamba-credentials.so: undefined reference to `ms_suptypes_to_ietf_enctypes@SAMBA_4.1.4' /usr/local/lib/libgensec.so: undefined reference to `gssapi_obtain_pac_blob@SAMBA_4.1.4' /usr/local/lib/libndr-standard.so: undefined reference to `ndr_push_kerb_EncTypes@SAMBA_4.1.4' /usr/local/lib/libsamdb.so: undefined reference to `werror_to_ntstatus@SAMBA_4.1.4' /usr/local/lib/libsamdb.so: undefined reference to `dsdb_write_prefixes_from_schema_to_ldb@SAMBA_4.1.4' /usr/local/lib/libsamdb.so: undefined reference to `dsdb_schema_set_el_from_ldb_msg_dups@SAMBA_4.1.4' /usr/local/lib/libgensec.so: undefined reference to `packet_set_private@SAMBA_4.1.4' /usr/local/lib/libgensec.so: undefined reference to `asn1_start_tag@SAMBA_4.1.4' /usr/local/lib/libgensec.so: undefined reference to `socket_get_peer_addr@SAMBA_4.1.4' /usr/local/lib/libgensec.so: undefined reference to `socket_get_fd@SAMBA_4.1.4' /usr/local/lib/libsamba-credentials.so: undefined reference to `smb_krb5_keyblock_init_contents@SAMBA_4.1.4' /usr/local/lib/libsamdb.so: undefined reference to `dom_sid_in_domain@SAMBA_4.1.4' /usr/local/lib/libndr-krb5pac.so: undefined reference to `ndr_pull_dom_sid2@SAMBA_4.1.4' ...... /usr/local/lib/libsamdb.so: undefined reference to `dsdb_get_fsmo_role_info@SAMBA_4.1.4' /usr/local/lib/libgensec.so: undefined reference to `spnego_free_data@SAMBA_4.1.4' /usr/local/lib/libsamdb.so: undefined reference to `security_token_set_privilege@SAMBA_4.1.4' /usr/local/lib/libsamdb.so: undefined reference to `security_token_initialise@SAMBA_4.1.4' cc: error: linker command failed with exit code 1 (use -v to see invocation) Waf: Leaving directory `/usr/ports/net/samba41/work/samba-4.1.6/bin' Build failed: -> task failed (err #1): {task: cc_link popt_common_5.o,popt_credentials_6.o,ntlm_auth_1.o -> ntlm_auth4} [: xTry: unexpected operator *** Error code 1 Basically, the root of the problem is the approach that used in FreeBSD to build 3d parity software - everything is built on a life system, without isolation of the build environment(been discussed million of times). Distribution packages, actually, built in such an environment(poudriere), but it's not used by end-users for local installations. So, during the build, if previous version of Samba was already installed all the share libraries present in /usr/local/lib, /usr/local/lib/samba and /usr/local/lib/shared-modules directories, that are known to ldconfig. Actually, /usr/local prefix is arbitrary, could be smth. else in the particular installation. The typical generated linking command for samba-4x looks something like: cc default/source4/auth/samba_server_gensec_8.o default/source4/lib/cmdline/popt_common_5.o default/source4/ntptr/simple_ldb/ntptr_simple_ldb_1.o default/source4/ntptr/ntptr_base_2.o default/source4/ntptr/ntptr_interface_2.o default/source4/smbd/server_9.o default/source4/rpc_server/common/forward_3.o default/source4/rpc_server/common/reply_3.o default/source4/rpc_server/dcesrv_auth_3.o default/source4/rpc_server/common/loadparm_3.o -o /usr/ports/net/samba41/work/samba-4.1.6/bin/default/source4/smbd/samba -pie -fstack-protector -Wl,-no-undefined -Wl,--export-dynamic -pthread -Wl,--as-needed -Wl,-rpath,/usr/local/lib -Wl,-rpath,/usr/local/lib -Wl,-rpath,/usr/local/lib -Wl,-rpath,/usr/local/lib -Wl,-rpath,/usr/local/lib -Wl,-rpath,/usr/local/lib -Wl,-rpath,/usr/local/lib -L/usr/local/lib -pthread -Ldefault/lib/socket -Ldefault/source4/libcli/ldap -Ldefault/nsswitch -Ldefault/source4/lib/socket -Ldefault/lib/addns -Ldefault/auth -Ldefault/libcli/ldap -Ldefault/libds/common -Ldefault/source4/ntvfs/posix -Ldefault/libcli/smb -Ldefault/libcli/cldap -Ldefault/source4/libcli -Ldefault/libcli/nbt -Ldefault/source4/libcli/wbclient -Ldefault/source4/auth -Ldefault/nsswitch/libwbclient -Ldefault/lib -Ldefault/libcli/named_pipe_auth -Ldefault/source4/lib/messaging -Ldefault/libcli/registry -Ldefault/source4/librpc -Ldefault/lib/krb5_wrap -Ldefault/source4/auth/kerberos -Ldefault/libcli/util -Ldefault/source4/heimdal_build -Ldefault/source4/dsdb -Ldefault/lib/dbwrap -Ldefault/auth/gensec -Ldefault/source4/lib/events -Ldefault/libcli/security -Ldefault/source4/ntvfs -Ldefault/source4/auth/ntlm -Ldefault/source4/param -Ldefault/librpc -Ldefault/source4/smbd -Ldefault/lib/tdb_wrap -Ldefault/libcli/auth -Ldefault/lib/ldb-samba -Ldefault/lib/param -Ldefault/source4/lib/registry -Ldefault/auth/credentials -Ldefault/source4/cluster -Ldefault/lib/replace -Ldefault/lib/util -Wl,-Bdynamic -lsamba-util -lreplace -lcluster -lsamba-credentials -lregistry -lsamba-hostconfig -lldbsamba -lcliauth -ltdb-wrap -lservice -lndr -lshares -lauth4 -lntvfs -lutil_tdb -lsamba-security -lprocess_model -levents -lgensec -lutil_setid -ldbwrap -lsamdb-common -lcom_err-samba4 -lgssapi-samba4 -lerrors -lauthkrb5 -lkrb5samba -ldcerpc -lutil_reg -ldcerpc-samba -lserver-role -lndr-samba -lasn1util -lndr-nbt -lndr-samba4 -ltevent-util -lMESSAGING -lnpa_tstream -lsamba-sockets -lsamdb -lsamba-modules -lwbclient -ldcerpc-samba4 -lauth_unix_token -lLIBWBCLIENT_OLD -lcli-nbt -lroken-samba4 -lsmbclient-raw -lcli_cldap -lcli_smb_common -lposix_eadb -lndr-standard -lutil_ntdb -lflag_mapping -lcli-ldap-common -lkrb5-samba4 -lasn1-samba4 -lhcrypto-samba4 -lwind-samba4 -lauth_sam_reply -lndr-krb5pac -laddns -ldcerpc-binding -lnetif -lwinbind-client -lcli-ldap -lsmb_transport -lheimbase-samba4 -lhx509-samba4 -linterfaces -linotify -lpam -lsasl2 -lutil -lz -lgcrypt -lgnutls -lntdb -lmd -lrt -lldb -ltalloc -lexecinfo -ltdb -lpopt -lpyldb-util -lcrypt -lpthread -ltevent -lm -lintl -lpython2.7 In particular, '-L/usr/local/lib' repeated multiple times and precedes local to the build -L flags, which point to the newer versions of the libs. Hence older versions of libraries are picked up and build failed with the error above. While Samba4 was in alpha stage and used Perl for build process I've submitted a patch, that sorted -L directories, based on leading '/', putting absolute paths to the end of the -L list. Unfortunately, mu Python and WAF Fu are not good enough to repeat this success. Manual editing of the line above using such an approach: cc default/source4/auth/samba_server_gensec_8.o default/source4/lib/cmdline/popt_common_5.o default/source4/ntptr/simple_ldb/ntptr_simple_ldb_1.o default/source4/ntptr/ntptr_base_2.o default/source4/ntptr/ntptr_interface_2.o default/source4/smbd/server_9.o default/source4/rpc_server/common/forward_3.o default/source4/rpc_server/common/reply_3.o default/source4/rpc_server/dcesrv_auth_3.o default/source4/rpc_server/common/loadparm_3.o -o /usr/ports/net/samba41/work/samba-4.1.6/bin/default/source4/smbd/samba -pie -fstack-protector -Wl,-no-undefined -Wl,--export-dynamic -pthread -Wl,--as-needed -Wl,-rpath,/usr/local/lib -pthread -Ldefault/lib/socket -Ldefault/source4/libcli/ldap -Ldefault/nsswitch -Ldefault/source4/lib/socket -Ldefault/lib/addns -Ldefault/auth -Ldefault/libcli/ldap -Ldefault/libds/common -Ldefault/source4/ntvfs/posix -Ldefault/libcli/smb -Ldefault/libcli/cldap -Ldefault/source4/libcli -Ldefault/libcli/nbt -Ldefault/source4/libcli/wbclient -Ldefault/source4/auth -Ldefault/nsswitch/libwbclient -Ldefault/lib -Ldefault/libcli/named_pipe_auth -Ldefault/source4/lib/messaging -Ldefault/libcli/registry -Ldefault/source4/librpc -Ldefault/lib/krb5_wrap -Ldefault/source4/auth/kerberos -Ldefault/libcli/util -Ldefault/source4/heimdal_build -Ldefault/source4/dsdb -Ldefault/lib/dbwrap -Ldefault/auth/gensec -Ldefault/source4/lib/events -Ldefault/libcli/security -Ldefault/source4/ntvfs -Ldefault/source4/auth/ntlm -Ldefault/source4/param -Ldefault/librpc -Ldefault/source4/smbd -Ldefault/lib/tdb_wrap -Ldefault/libcli/auth -Ldefault/lib/ldb-samba -Ldefault/lib/param -Ldefault/source4/lib/registry -Ldefault/auth/credentials -Ldefault/source4/cluster -Ldefault/lib/replace -Ldefault/lib/util *-L/usr/local/lib* *-Wl,-rpath,/usr/local/lib* -Wl,-Bdynamic -lsamba-util -lreplace -lcluster -lsamba-credentials -lregistry -lsamba-hostconfig -lldbsamba -lcliauth -ltdb-wrap -lservice -lndr -lshares -lauth4 -lntvfs -lutil_tdb -lsamba-security -lprocess_model -levents -lgensec -lutil_setid -ldbwrap -lsamdb-common -lcom_err-samba4 -lgssapi-samba4 -lerrors -lauthkrb5 -lkrb5samba -ldcerpc -lutil_reg -ldcerpc-samba -lserver-role -lndr-samba -lasn1util -lndr-nbt -lndr-samba4 -ltevent-util -lMESSAGING -lnpa_tstream -lsamba-sockets -lsamdb -lsamba-modules -lwbclient -ldcerpc-samba4 -lauth_unix_token -lLIBWBCLIENT_OLD -lcli-nbt -lroken-samba4 -lsmbclient-raw -lcli_cldap -lcli_smb_common -lposix_eadb -lndr-standard -lutil_ntdb -lflag_mapping -lcli-ldap-common -lkrb5-samba4 -lasn1-samba4 -lhcrypto-samba4 -lwind-samba4 -lauth_sam_reply -lndr-krb5pac -laddns -ldcerpc-binding -lnetif -lwinbind-client -lcli-ldap -lsmb_transport -lheimbase-samba4 -lhx509-samba4 -linterfaces -linotify -lpam -lsasl2 -lutil -lz -lgcrypt -lgnutls -lntdb -lmd -lrt -lldb -ltalloc -lexecinfo -ltdb -lpopt -lpyldb-util -lcrypt -lpthread -ltevent -lm -lintl -lpython2.7 Allowed to link samba binary in the same environment without problems, giving working Samba4x. Jelmer, --rpath seems doesn't play any role in this problem, I've tried both unsorted and sorted builds - -L/usr/local/lib seems to be the key. Please, assist in addressing this issue.
Created attachment 9814 [details] patch for v4-1-test, picked from master This was fixed in master quite some time ago. It seems we have forgotten to backport... Sorry for that. Please test this patch!
Hi, Michael! (In reply to comment #1) > Created attachment 9814 [details] > patch for v4-1-test, picked from master > > This was fixed in master quite some time ago. > It seems we have forgotten to backport... > Sorry for that. > > Please test this patch! Sorry, that patch doesn't help. It deals with rpath, while the problem is in the -L path. From other side, I can see that amount of excessive -Wl,--rpath,/use/local/lib has disappeared, suppressed into one line, which is an improvement as well. So please, push this patch into 40 and 41 branches. What I could figure out is that offending -L appear for the targets which do link against Python. # python-config --libs -L/usr/local/lib -pthread -lintl -lutil -lm -lpython2.7 And that exactly the combination that appears in the combined linking flags: 12:19:15 runner cc default/source4/lib/cmdline/popt_common_5.o default/source4/lib/cmdline/popt_credentials_6.o default/source4/utils/ntlm_auth_1.o -o /usr/ports/net/samba41/work/samba-4.1.6/bin/default/source4/utils/ntlm_auth4 -pie -fstack-protector -Wl,-no-undefined -Wl,--export-dynamic -pthread -Wl,--as-needed ***-L/usr/local/lib -pthread*** -Wl,-rpath,/usr/local/lib -Ldefault/lib/socket -Ldefault/source4/libcli/ldap -Ldefault/nsswitch -Ldefault/libcli/smb -Ldefault/source4/lib/socket -Ldefault/libcli/cldap -Ldefault/source4/libcli -Ldefault/libcli/nbt -Ldefault/lib/addns -Ldefault/source4/libcli/wbclient -Ldefault/source4/auth -Ldefault/nsswitch/libwbclient -Ldefault/auth -Ldefault/libcli/named_pipe_auth -Ldefault/source4/cluster -Ldefault/source4/librpc -Ldefault/libcli/util -Ldefault/libcli/ldap -Ldefault/libds/common -Ldefault/lib -Ldefault/lib/krb5_wrap -Ldefault/source4/auth/ntlm -Ldefault/source4/auth/kerberos -Ldefault/lib/tdb_wrap -Ldefault/lib/dbwrap -Ldefault/auth/gensec -Ldefault/source4/lib/events -Ldefault/libcli/auth -Ldefault/lib/util -Ldefault/source4/smbd -Ldefault/libcli/security -Ldefault/lib/ldb-samba -Ldefault/source4/lib/messaging -Ldefault/lib/param -Ldefault/librpc -Ldefault/auth/credentials -Ldefault/lib/replace -Ldefault/source4/heimdal_build -Ldefault/source4/dsdb -Ldefault/source4/lib/cmdline -Wl,-Bdynamic -lcmdline-credentials -lsamdb-common -lroken-samba4 -lgssapi-samba4 -lreplace -lsamba-credentials -lndr-samba -lsamba-hostconfig -lMESSAGING -lldbsamba -lsamba-security -lservice -lutil_tdb -lndr-nbt -lcliauth -levents -lgensec -lsamba-util -lcom_err-samba4 -ldbwrap -ltdb-wrap -lauthkrb5 -lauth4 -lkrb5samba -lsamba-sockets -lndr -lflag_mapping -lcli-ldap-common -lkrb5-samba4 -lasn1-samba4 -lhcrypto-samba4 -lwind-samba4 -lerrors -lndr-krb5pac -lndr-standard -lasn1util -lserver-role -lndr-samba4 -ldcerpc -lcluster -ltevent-util -lnpa_tstream -lprocess_model -lsamba-modules -lsamdb -lutil_setid -lutil_ntdb -lauth_sam_reply -lwbclient -ldcerpc-samba4 -lauth_unix_token -lLIBWBCLIENT_OLD -lheimbase-samba4 -lhx509-samba4 -laddns -lcli-nbt -lsmbclient-raw -ldcerpc-binding -lcli_cldap -lnetif -lcli_smb_common -ldcerpc-samba -lwinbind-client -lcli-ldap -linterfaces -lsmb_transport -lpam -lsasl2 -lmd -lgnutls -lgcrypt -lz -lntdb -lrt -lldb -ltalloc -lexecinfo -lcrypt -ltdb -lpopt -lpyldb-util ***-lutil -lpthread -ltevent -lm -lintl -lpython2.7*** Compare it to any other linking command: 12:19:15 runner cc default/source3/client/client_149.o default/source3/client/clitar_149.o default/source3/client/dnsbrowse_149.o default/libcli/smbreadline/smbreadline_1.o -o /usr/ports/net/samba41/work/samba-4.1.6/bin/default/source3/client/smbclient -pie -fstack-protector -Wl,-no-undefined -Wl,--export-dynamic -pthread -Wl,--as-needed ****** -Wl,-rpath,/usr/local/lib -Ldefault/source4/lib/socket -Ldefault/libcli/nbt -Ldefault/auth -Ldefault/libds/common -Ldefault/source4/libcli/ldap -Ldefault/nsswitch -Ldefault/source4/auth/kerberos -Ldefault/lib/ldb-samba -Ldefault/lib/param -Ldefault/source4/dsdb -Ldefault/source4/lib/events -Ldefault/libcli/registry -Ldefault/lib/tdb_wrap -Ldefault/auth/gensec -Ldefault/lib/krb5_wrap -Ldefault/libcli/cldap -Ldefault/libcli/ldap -Ldefault/nsswitch/libwbclient -Ldefault/lib/addns -Ldefault/source4/heimdal_build -Ldefault/libcli/auth -Ldefault/auth/credentials -Ldefault/lib -Ldefault/librpc -Ldefault/libcli/smb -Ldefault/libcli/util -Ldefault/libcli/security -Ldefault/lib/dbwrap -Ldefault/lib/ccan -Ldefault/lib/replace -Ldefault/lib/socket -Ldefault/lib/util -Ldefault/source3 -L/usr/local/lib -Wl,-Bdynamic -lpopt_samba3 -lsamba-util -linterfaces -ltevent-util -lreplace -lmsrpc3 -lccan -llibsmb -ldbwrap -lsamba-security -lerrors -lsmbconf -lcli_smb_common -lsamba3-util -lutil_tdb -ldcerpc-samba -lndr-standard -lsamba-sockets -lndr -lutil_cmdline -lutil_setid -lndr-samba -lsamba-credentials -lgse -ldcerpc-binding -lcliauth -lkrb5-samba4 -laddns -lwbclient -lgssapi-samba4 -lcli-ldap-common -lcli_cldap -lkrb5samba -lgensec -ltdb-wrap -lutil_ntdb -lsmbregistry -lCHARSET3 -lutil_reg -lsmb_transport -lndr-nbt -lroken-samba4 -levents -lsecrets3 -lndr-krb5pac -lasn1util -lsamdb-common -lcom_err-samba4 -lsamba-hostconfig -lldbsamba -lauthkrb5 -lheimbase-samba4 -lasn1-samba4 -lhx509-samba4 -lhcrypto-samba4 -lwind-samba4 -lwinbind-client -lcli-ldap -lsamba-modules -lsamdb -lsmbd_shim -lflag_mapping -lserver-role -lauth_sam_reply -lcli-nbt -lnetif -lmd -lutil -lpyldb-util -lldb -ltalloc -lz -lgcrypt -lgnutls -lsasl2 -lntdb -lrt -lexecinfo -lncurses -lcrypt -ltdb -lpopt -lpthread -ltevent -lreadline Flags -pie -fstack-protector -Wl,-no-undefined -Wl,--export-dynamic -pthread -Wl,--as-needed seems to be common, then Python staff goes in, then internal libs are linked in. So, we need to find where python_LDFLAGS are inserted into the liking flags and push it backwards.
Not sure, really, but maybe it all defined in Tools/python.py: @feature('pyext') @before('apply_incpaths', 'apply_lib_vars', 'apply_type_vars', 'apply_bundle') @after('vars_target_cshlib') def init_pyext(self): self.default_install_path = '${PYTHONARCHDIR}' self.uselib = self.to_list(getattr(self, 'uselib', '')) if not 'PYEXT' in self.uselib: self.uselib.append('PYEXT') self.env['MACBUNDLE'] = True @before('apply_link', 'apply_lib_vars', 'apply_type_vars') @after('apply_bundle') @feature('pyext') def pyext_shlib_ext(self): # override shlib_PATTERN set by the osx module self.env['shlib_PATTERN'] = self.env['pyext_PATTERN'] @before('apply_incpaths', 'apply_lib_vars', 'apply_type_vars') @feature('pyembed') def init_pyembed(self): self.uselib = self.to_list(getattr(self, 'uselib', '')) if not 'PYEMBED' in self.uselib: self.uselib.append('PYEMBED') Here?
(In reply to comment #2) > Hi, Michael! > > (In reply to comment #1) > > Created attachment 9814 [details] [details] > > patch for v4-1-test, picked from master > > > > This was fixed in master quite some time ago. > > It seems we have forgotten to backport... > > Sorry for that. > > > > Please test this patch! > > Sorry, that patch doesn't help. It deals with rpath, while the problem is in > the -L path. That is a pity. ;-) I need to look into your findings... > From other side, I can see that amount of excessive > -Wl,--rpath,/use/local/lib has disappeared, suppressed into one line, which is > an improvement as well. So please, push this patch into 40 and 41 branches. I created Bug #10548 for that purpose. Cheers - Michael
Cool. Do you have any idea how to fix that Python injected paths as well?
Created attachment 10375 [details] Patch for master Better late than never... This is a patch for master that should (among other,similar build problems) fix this issue.
Comment on attachment 10375 [details] Patch for master Hm, I thought buildtools/wafadmin/ should be left untouched as it contains upstream code and all local modifications had to go to wafsamba? Looking at git log I seem to be wrong, but then...
Once we've upstreamed a bug report + patch I think it's ok to change waf downstream.
@Ralph: Sure we should leave wafadmin untouched if possible. But if it is a true bug in upstream, wafadmin needs to be fixed. (Look at patch 64f5e24100a764ec198cab9a8d2c43fa86e7027c: I fixed that for samba and ita pushed it upstream.) My fix here is not (yet) the one necessarily proposed for pushing, but meant as a demonstration how the thing can be solved. And this issue may also be an upstream bug. Not 100% certain yet. Anyhow. wafadmin was for me the easier place to do the fix for a start. Cheers - Michael
*** This bug has been marked as a duplicate of bug 10548 ***
The problem I've originally reported isn't the same as #10548 and still (as of 4.4 and 4.5) persists. See https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=213345
Created attachment 12791 [details] Patch to fix order of CFLAGS inclusion
Patch is provided by Igor Pavlov <igor.arabesc.pavlov _at_ gmail.com>