Bug 13343 - build errors with cc from developerstudio 12.5 on Solaris
Summary: build errors with cc from developerstudio 12.5 on Solaris
Status: RESOLVED FIXED
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: Build (show other bugs)
Version: 4.8.0
Hardware: Sparc Solaris
: P5 major (vote)
Target Milestone: ---
Assignee: Stefan Metzmacher
QA Contact: Samba QA Contact
URL:
Keywords:
: 13383 13403 (view as bug list)
Depends on:
Blocks: 13344 13345
  Show dependency treegraph
 
Reported: 2018-03-20 20:17 UTC by Andreas Leitgeb
Modified: 2020-12-11 12:13 UTC (History)
9 users (show)

See Also:


Attachments
Patches for v4-8-test (for bugs #13343, #13344, #13345) (24.31 KB, patch)
2018-04-19 10:14 UTC, Stefan Metzmacher
bjacke: review+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andreas Leitgeb 2018-03-20 20:17:56 UTC
Oracle Developer Studio 12.5 -- cc doesn't like functions declared as returning "void" to contain statements like   return some_other_function(...), even if the other function is void as well.

This pattern shows up in two modules:
  lib/crypto/aes.c  (lines 255, 257, 264, 266)
  lib/ldb/tests/ldb_mod_op_test.c (lines 1917, 1922, 1927, 1932)
The quickfixes are to either just remove the "return" keywords,
or to move the "return" after the function call.


A different error occurs about some "linux'ism": SOCK_NONBLOCK.
  source3/modules/vfs_virusfilter_utils.c
uses this symbol, but it seems like  O_NONBLOCK could be better.


Finally there are linking errors:
default/lib/pthreadpool/pthreadpooltest_cmocka
  inet_addr    default/lib/async_req/async_connect_send_test_2.o

default/source4/heimdal_build/samba4kpasswd 
  rep_gethostbyname       default/source4/heimdal_build/libroken-samba4.so
  libintl_bindtextdomain  default/source4/heimdal_build/libkrb5-samba4.so
  libintl_dgettext        default/source4/heimdal_build/libkrb5-samba4.so

default/source3/libmessages-dgm-samba4.so
  libintl_gettext         default/source4/heimdal/kuser/kinit_134.o
  libintl_textdomain      default/source4/heimdal/kuser/kinit_134.o
  rep_gethostbyname       default/source4/heimdal_build/libroken-samba4.so
  libintl_bindtextdomain  default/source4/heimdal/kuser/kinit_134.o
  libintl_dgettext        default/source4/heimdal_build/libkrb5-samba4.so

These are those I saw so far. Almost each time I restart the build it showed me link errors for different targets.  I can't really claim I understood the new (in 4.x) build system...
Comment 1 Andreas Leitgeb 2018-03-20 20:23:50 UTC
Some extra info:  system seems to have libintl.h  and a libintl.so.1 but the lib contains the bare symbols like "bindtextdomain" (according to "nm" utility) without the libintl_ prefix. Maybe some #define-sorcery went wrong there?
Comment 2 Andreas Leitgeb 2018-03-20 20:35:55 UTC
Another detail: the failed dependency for inet_addr has an extra
remark by the linker:

  inet_addr   default/lib/async_req/async_connect_send_test_2.o
       (symbol belongs to implicit dependency /usr/lib/libnsl.so.1)

Sorry for not including that in original report. I don't even understand
this implicit dependency. According to "nm async_connect_send_test_2.o"
it does have a (pretty explicit) reference: "...|UNDEF  |inet_addr".
Comment 3 Andreas Leitgeb 2018-03-20 20:54:02 UTC
Another observation...

The problem about libintl has already been mentioned in bugreport 
  https://bugzilla.samba.org/show_bug.cgi?id=8259

If I redo the linker and add -lintl then all the libintl_... references
are satisfied - apparently the "libintl_" prefix is some linker magic.

but the reference to rep_gethostbyname remains unresolved.  The #define 
sorcery seems to map gethostbyname to rep_gethostbyname, but then seems 
not even to compile its own replacement for gethostbyname, nor pass it
to the linker.
Comment 4 Andreas Leitgeb 2018-03-20 21:01:22 UTC
(In reply to Andreas Leitgeb from comment #3)
ah, well, no linker magic.

It seems to pick up the libintl.h include from /usr/local/ and apparently doesn't pick up any libintl.so at all, unless I manually add -lintl to the line, after which it picks up the /usr/local/lib/libintl.so and that part goes ok.
Comment 5 Andreas Leitgeb 2018-04-10 19:49:11 UTC
More about the libintl-problem:

waf has a test to check for whether a "-l<lib>" is necessary to compile and link a testprog using certain functions.

The testprog contains e.g. this snippet in main(void)'s body:

   #ifndef dgettext
   void *_x; _x=(void *)&dgettext; return (int)_x;
   #endif

but in the /usr/local/lib variant of libintl.h, dgettext is a macro, so the test is essentially empty and an empty main() can trivially be built without any -l...

I don't really know how to easily improve the test, to detect if a real call to dgettext would need -lintl, but I'd suggest to just disregard the possibility of dgettext ever working without -lintl:

thus in lib/replace/wscript:
  remove the if-block   conf.CHECK_FUNCS_IN('dgettext gettext', '', checklibc=True, headers='libintl.h')
where it passes "checklibc=True", and turn the "elif" (with "checklibc=False") into an "if".

For my own use, I changed the script to set conf.env.intl_libs='intl' even if the check (falsely) claims it's not needed, and it gets further.

With all my local fixes, I now get as far as where it attempts to link a tool "winbindd", and cannot find a symbol ads_keytab_open, but my configure had option "--without-ads", so I'm more surprised that it *wants* that symbol than that it doesn't exist.
Comment 6 Jeremy Allison 2018-04-10 19:52:42 UTC
(In reply to Andreas Leitgeb from comment #5)

> thus in lib/replace/wscript:
>  remove the if-block   conf.CHECK_FUNCS_IN('dgettext gettext', '',
> checklibc=True, headers='libintl.h')
> where it passes "checklibc=True", and turn the "elif" (with "checklibc=False") > into an "if"

Can you post this as a git format patch please ?

> With all my local fixes, I now get as far as where it attempts to link a tool 
> "winbindd", and cannot find a symbol ads_keytab_open, but my configure had 
> option "--without-ads", so I'm more surprised that it *wants* that symbol than > that it doesn't exist.

Known bug that got fixed recently. Track git master for more up to date results.
Comment 7 Andreas Leitgeb 2018-04-10 21:07:24 UTC
After forking samba on github, cloning it and preparing the changes in the git-tree I noticed the comments saying that on glibc systems dgettext and friends are indeed already included in libc.
Thus, my previously suggested "fix" would likely break it on lots of systems.

Obviously it isn't quite as easy to fix generally. Maybe we need to change buildtools/wafsamba/samba_autoconf.py function CHECK_VARIABLE to allow to skip the macro-check, then for dgettext,etc we'd specifically disable the macro-check, and the test would properly test for necessity of -lintl, even though the actually resulting symbols happen to have a different name.

My next experiment was to remove the #ifndef-#endif-wrapper from said function.
most tests went fine anyway (for all non-macro functions, of course), but even some macros ended up fine: the first test failed, but the immediately following test using the symbol as an int lvalue succeeded and correct conclusions were drawn ...  except for IPV6, which wasn't any longer correctly detected after my change.

Will continue tomorrow...
Comment 8 Stefan Metzmacher 2018-04-12 18:48:58 UTC
*** Bug 13383 has been marked as a duplicate of this bug. ***
Comment 9 Stefan Metzmacher 2018-04-19 10:14:20 UTC
Created attachment 14156 [details]
Patches for v4-8-test (for bugs #13343, #13344, #13345)
Comment 10 Karolin Seeger 2018-04-20 09:55:21 UTC
Pushed to autobuild-v4-8-test.
Comment 11 Andreas Leitgeb 2018-04-24 16:23:51 UTC
I applied both the uploaded patch and my local solaris-specific fix for linking with -lintl to a freshly extracted samba-4.8.0 tree, and it got much further this time.

Unfortunately it still fails with:

[2877/2990] Linking default/source3/rpcclient/rpcclient
ld: warning: global symbol '_END_' has non-global binding:
        (file /usr/local/lib/libgcrypt.so.11 value=LOCL);
ld: warning: global symbol '_START_' has non-global binding:
        (file /usr/local/lib/libgcrypt.so.11 value=LOCL);
ld: warning: global symbol '_END_' has non-global binding:
        (file /usr/local/lib/libgpg-error.so.0 value=LOCL);
ld: warning: global symbol '_START_' has non-global binding:
        (file /usr/local/lib/libgpg-error.so.0 value=LOCL);
Undefined                       first referenced
 symbol                             in file
ads_keytab_open                     default/source3/libads-samba4.so
ld: fatal: symbol referencing errors

Those ld-warnings also happend for almost all other "Linking"-actions.

configure call was: ./buildtools/bin/waf configure --prefix=/opt/samba --without-ldap --disable-avahi --disable-cups --disable-iprint --without-ads --disable-python --without-ad-dc
Comment 12 Karolin Seeger 2018-04-25 06:44:22 UTC
Pushed to v4-8-test.
Re-assiging to metze for further investigation.
Comment 13 Stefan Metzmacher 2018-04-25 07:40:17 UTC
*** Bug 13403 has been marked as a duplicate of this bug. ***
Comment 14 Albert Chin (temp failure) 2018-12-26 22:09:49 UTC
I built 4.8.8 on Solaris 10/Intel with Solaris Studio 12.2 and get:
21:40:55 runner c99 default/lib/async_req/async_sock_1.o default/lib/async_req/async_connect_send_test_2.o -o /opt/build/china/samba-4.8.8/bin/default/lib/async_req/async_connect_send_test -L/opt/TWWfsw/libgnutls212/lib -R/opt/TWWfsw/libgnutls212/lib -lpthread -Ldefault/libcli/util -Ldefault/lib/replace -Ldefault/lib/talloc -Ldefault/lib/util -Ldefault/lib/tevent -L/usr/local/lib -Bdynamic -ltevent -liov-buf-samba4 -ltalloc -lsocket-blocking-samba4 -ltevent-util -lreplace-samba4 -lsamba-errors -lsamba-debug-samba4 -ltime-basic-samba4 -lsocket -lcrypt -lrt
Undefined                       first referenced
 symbol                             in file
inet_addr                           default/lib/async_req/async_connect_send_test_2.o  (symbol belongs to implicit dependency /lib/libnsl.so.1)
ld: fatal: symbol referencing errors. No output written to /opt/build/china/samba-4.8.8/bin/default/lib/async_req/async_connect_send_test
Waf: Leaving directory `/opt/build/china/samba-4.8.8/bin'
Build failed:  -> task failed (err #2): 
        {task: cc_link async_sock_1.o,async_connect_send_test_2.o -> async_connect_send_test}

When configuring Samba on this platform, I saw:
Checking for libnsl                                           : not found 

bin/config.log shows:
Checking for libnsl
/opt/TWWfsw/pkgconfig02/bin/pkg-config --cflags --libs libnsl
Package libnsl was not found in the pkg-config search path.
Perhaps you should add the directory containing `libnsl.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libnsl' found
not found

Naturally, there is no libnsl.pc.

According to inet_addr(3):
SYNOPSIS
     cc [ flag... ] file... -lsocket  -lnsl  [ library... ]

So do I need to modify lib/replace/wscript to get for inet_addr() in both -lsocket and -lnsl?

The problem is the lack of -lnsl in linking.

With Solaris Studio 12.3 on Solaris 11/Intel, I get:
21:52:58 runner c99 default/source4/heimdal/lib/vers/print_version_129.o default/source4/heimdal_build/version_129.o default/source4/heimdal/kuser/kgetcred_136.o -o /opt/build/china/samba-4.8.8/bin/default/source4/heimdal_build/samba4kgetcred -L/opt/TWWfsw/libgnutls212/lib -R/opt/TWWfsw/libgnutls212/lib -lpthread -Ldefault/lib/replace -Ldefault/source4/heimdal_build -L/usr/local/lib -Bdynamic -lkrb5-samba4 -lroken-samba4 -lasn1-samba4 -lhcrypto-samba4 -lreplace-samba4 -lheimntlm-samba4 -lheimbase-samba4 -lcom_err-samba4 -lhx509-samba4 -lwind-samba4 -lrt -lcrypt -lresolv -lsocket
Undefined                       first referenced
 symbol                             in file
rep_gethostbyname                   default/source4/heimdal_build/libroken-samba4.so
ld: fatal: symbol referencing errors. No output written to /opt/build/china/samba-4.8.8/bin/default/source4/heimdal_build/samba4kgetcred
Waf: Leaving directory `/opt/build/china/samba-4.8.8/bin'
Build failed:  -> task failed (err #2): 
        {task: cc_link print_version_129.o,version_129.o,kgetcred_136.o -> samba4kgetcred}
Comment 15 Albert Chin (temp failure) 2018-12-26 22:25:17 UTC
Is something like the following suppose to check for inet_addr in -lnsl?
--- lib/replace/wscript.orig    2018-12-26 18:57:06.381209812 +0000
+++ lib/replace/wscript 2018-12-26 22:09:07.168376030 +0000
@@ -189,6 +191,9 @@
                            getaddrinfo getnameinfo freeaddrinfo gai_strerror socketpair''',
                         'socket nsl', checklibc=True,
                         headers='sys/socket.h netinet/in.h arpa/inet.h netdb.h')
+    conf.CHECK_FUNCS_IN('inet_addr',
+                        'nsl', checklibc=True,
+                        headers='sys/socket.h netinet/in.h arpa/inet.h netdb.h')
 
     # Some old Linux systems have broken header files and
     # miss the IPV6_V6ONLY define in netinet/in.h,

I tried this on the Solaris 10/Intel system and I never saw -lnsl being checked. From bin/config.log:
Checking for inet_addr
[1/2] Compiling test.c
['c99', '-g', '-xs', '-D__EXTENSIONS__', '-I/opt/TWWfsw/libgnutls212/include', '-Idefault', '-I..', '-Idefault', '-I..', '-I/usr/local/include', '-D_SAMBA_BUILD_=4', '-DHAVE_CONFIG_H=1', '-D_STDC_C99=1', '-D_XPG6=1', '-D_GNU_SOURCE=1', '-D_XOPEN_SOURCE_EXTENDED=1', '../test.c', '-c', '-o', 'default/test_1.o']
[2/2] Linking default/testprog
Undefined                       first referenced
 symbol                             in file
inet_addr                           default/test_1.o
ld: fatal: symbol referencing errors. No output written to /opt/build/china/samba-4.8.8/bin/.conf_check_0/testbuild/default/testprog
['c99', 'default/test_1.o', '-o', '/opt/build/china/samba-4.8.8/bin/.conf_check_0/testbuild/default/testprog', '-L/opt/TWWfsw/libgnutls212/lib', '-R/opt/TWWfsw/libgnutls212/lib', '-L/usr/local/lib']
command returned 'Build failed:  -> task failed (err #2): \n\t{task: cc_link test_1.o -> testprog}'==>
...
Checking for inet_addr
[1/2] Compiling test.c
['c99', '-g', '-xs', '-D__EXTENSIONS__', '-I/opt/TWWfsw/libgnutls212/include', '-Idefault', '-I..', '-Idefault', '-I..', '-I/usr/local/include', '-D_SAMBA_BUILD_=4', '-DHAVE_CONFIG_H=1', '-D_STDC_C99=1', '-D_XPG6=1', '-D_GNU_SOURCE=1', '-D_XOPEN_SOURCE_EXTENDED=1', '../test.c', '-c', '-o', 'default/test_1.o']
[2/2] Linking default/testprog
Undefined                       first referenced
 symbol                             in file
inet_addr                           default/test_1.o
ld: fatal: symbol referencing errors. No output written to /opt/build/china/samba-4.8.8/bin/.conf_check_0/testbuild/default/testprog
['c99', 'default/test_1.o', '-o', '/opt/build/china/samba-4.8.8/bin/.conf_check_0/testbuild/default/testprog', '-L/opt/TWWfsw/libgnutls212/lib', '-R/opt/TWWfsw/libgnutls212/lib', '-L/usr/local/lib']
command returned 'Build failed:  -> task failed (err #2): \n\t{task: cc_link test_1.o -> testprog}'==>
Comment 16 Björn Jacke 2019-02-09 10:35:54 UTC
ACK, conf.CHECK_FUNCS_IN is supposed to do that check and I can confirm that this is not working properly on Solars. And it's not really obvious to see why. The move to waf some years ago really kicked out a huge number of the platform/compiler combinations that we perfectly supported before unfortunately.
Comment 17 Andreas Leitgeb 2019-02-15 10:09:54 UTC
" And it's not really obvious to see why."

not obvious, indeed.

The problem consists of these parts:

The system has a libintl in /usr/lib with a symbol "dgettext" in it, and a header file that declares such a function.

There is also a GNU package libintl, which installs in /usr/local/lib, but has different names for the functions. It's header file uses #define to map dgettext to the alternative name "libintl_dgettext".

samba is meant to use the GNU version (iirc, the system version failed somewhere else but forgot details), so it gets to see the macros for dgettext.  The macro, however causes the test-code generated in conf.CHECK_FUNCS_IN to produce an effectively empty main() body, and thus linking of an empty main() works fine even without any "-lintl" option. And thus, the waf believes that there is no need for passing "-lintl" to the linker.

I don't think that this can be solved generally in conf.CHECK_FUNCS_IN.

Probably the only possible fix is to add some special workaround code for solaris and gnu libintl, to just force the "-lintl" to be passed regardless of CHECK_FUNCS_IN result.

When I started the thread I still struggled with "cc" (thus the topic), but later jumped to "gcc", and with a manual patch to enforce -lintl option I eventually was able to compile it all.   Most other problems mentioned in this thread disappeared with some version of samba that was released in the meantime.  And my boss might fire me, if I even attempted to replace currently running samba with something even newer, as long as current version works ;-)
Comment 18 Björn Jacke 2019-02-15 22:43:45 UTC
On 15.02.19 11:09, samba-bugs@samba.org wrote:
> Probably the only possible fix is to add some special workaround code for
> solaris and gnu libintl, to just force the "-lintl" to be passed regardless of
> CHECK_FUNCS_IN result.

this would be definitely the most wrong approach. This will fix things
for the one setup and break other setups. The only way is to fix all the
tests and all the compile switches properly. I made some fixes for all
that which are not upstream yet. We should really involve a Unix machine
with a non-gnu compiler into our check-in process so that we are forced
not to ignore the non-Linux world out there.
Comment 19 Björn Jacke 2019-09-18 11:51:52 UTC
the mentioned waf configure test fixes are upstream already and I made successfull builds with the studion compiler. Have in mind that the poratbility for less mainstream platforms is a moving target and very now and then new fixes might be required there.
Comment 20 Samba QA Contact 2020-11-10 08:18:08 UTC
This bug was referenced in samba master:

96e2cf7905e146a40f788899e4759691aac6c942