Due to the "--as-needed" option, Samba 3.5. is not linked against libncurses any longer. On RHEL5 x86_64, libreadline is not linked against libncurses (but should be). That's why 'net' is not working on RHEL5-x86_64: net net: symbol lookup error: /usr/lib64/libreadline.so.5: undefined symbol: PC This is not a Samba bug, but it would be nice to be able to disable "--as-needed".
Created attachment 5456 [details] Patch created by Metze Patch has been pushed to master as 22d316926b9589608d332143c1fa134229b75b3c.
this is not required, see the commit message of e1ebadb85b250f950d0f5415eda83209f94378c1: <quote> Also change order of options so that user supplied LDFLAGS are put *after* the automatic --aѕ-needed flag. This way it's pollible to force not use as-needed by setting LDFLAGS environment variable to "-Wl,--no-as-needed". </quote> I think we don't need the extra configure option. IMHO on those broken RHEL5 installations people should just manually set LDFLAGS accordingly.
gd, simo, andreas: it would be cool if you convince the maintainer of readline to fix the readline linking mess in RHEL5! ;-)
(In reply to comment #3) > gd, simo, andreas: it would be cool if you convince the maintainer of readline > to fix the readline linking mess in RHEL5! ;-) > Björn, can you open a bug on RH bugzilla ? I will then be able to ask for it to be fixed.
Simo: there is already one: https://bugzilla.redhat.com/show_bug.cgi?id=499837
(In reply to comment #5) > Simo: there is already one: https://bugzilla.redhat.com/show_bug.cgi?id=499837 Ok, I've just read the comments, and they have a good reason not to link readline with ncurses, this means you will have to make sure samba on RHEL5 is linked with libncurses or libtermcap and not leave them out. To me this looks really a samba bug and we should provide means to fix it. Simo.
unresolved symbols of a system library and missing dependencies of it are a bug of samba? ;-) You would need a cristal ball to link against all the libs that other libs might have failed to link against. It's just a hack of this specific readline package.
Björn, good or bad as it is, there is a clear explanation in the RH bug as why that can't be changed for now. We have 2 options, take that in account, or leave 3.5.0 broken on RHEL5. Asking RH to change RHEL5 in a way that would break other existing applications is simply not possible. As the bug also says this is not a problem in RHEL 6, so this is just another quirk for an old, established platform we need to take in account when building samba. Simo.
you just have to disable --as-needed and use --allow-shlib-undefined on RHEL5 by setting before configure LDFLAGS="-Wl,--allow-shlib-undefined,--no-as-needed". We might mention this in the release notes, in the wiki or so. Automatic handling would require a configure check for this specific underlinking issue.
*** This bug has been marked as a duplicate of bug 6984 ***
(In reply to comment #2) > this is not required, see the commit message of > e1ebadb85b250f950d0f5415eda83209f94378c1: > > <quote> > Also change order of options so that user supplied LDFLAGS are put *after* the > automatic --aѕ-needed flag. This way it's pollible to force not use as-needed > by setting LDFLAGS environment variable to "-Wl,--no-as-needed". > </quote> Apparently, according to Simos comments, it not as simple as saying that rhel's libreadline is broken. And I think it is very inconvenient to have to specify chatty ld flags via the environment. A single configure option like in Metze's patch is much easier to use and also much easier to find! I'd vote for the --enable/disable-as-needed patch !! Michael > I think we don't need the extra configure option. IMHO on those broken RHEL5 > installations people should just manually set LDFLAGS accordingly.
I think this is not a duplicate of bug 6984. Rather, 6984 can be a considert a special incarnation of this bug. I vote for having the use of --as-needed configurable via a ./configure switch. This bahaviour was introduced with 3.5, and there is no use to make it more difficult than necessary to build on RHEL5 and possibly other platforms. It is a valid requist to want to be able to turn of the use of --as-needed. ==> I am reopening this bug. Michael
It's not that simple. The rhel5 build is only working accidentally if as-needed is disabled. As soon as ncurses linking flags is not in the global LDFALGS any more (there is not much reason for having it there I think...). So it's just a matter of time until simply disabling as-needed will not be enough. The proper fix for the "special" rhel5 readline lib is to check if linking a test prog against readline without linking against ncurses results in a broken testprog. Then test if the testprog is working with readline also linked in. You actually need to add the ncurses linker flag manually for a proper fix *and* make sure to set --no-as-needed. Given that, the --disable-as-needed will be of much help. The switch alone will anyway not help those people who try to build there and see that failure.
(In reply to comment #13) > It's not that simple. The rhel5 build is only working accidentally if as-needed > is disabled. As soon as ncurses linking flags is not in the global LDFALGS any > more (there is not much reason for having it there I think...). So it's just a > matter of time until simply disabling as-needed will not be enough. > > The proper fix for the "special" rhel5 readline lib is to check if linking a > test prog against readline without linking against ncurses results in a broken > testprog. Then test if the testprog is working with readline also linked in. > You actually need to add the ncurses linker flag manually for a proper fix > *and* make sure to set --no-as-needed. > > Given that, the --disable-as-needed will be of much help. The switch alone will > anyway not help those people who try to build there and see that failure. Ah! Seems like there is someone with a clue how to fix this properly! ... :-) Cheers - Michael
Please add Metzes patch: An other workaround would be to add something like extern void tgetent(); static void (*dummyptr)() = tgetent; into somewhere at lib/replace. This would force the linker to add -lncurses (termlib, termcap) because of an explicit dependency. Example code to be compiled on Centos5/RHEL5 --- #ifdef FIX extern void tgetent(); static void (*dummyptr)() = tgetent; #endif int main() { readline(); } ---- Works: gcc aa.c -lreadline -lncurses -- Does not work: gcc -Wl,--as-needed aa.c -lreadline -lncurses /usr/lib/gcc/x86_64-redhat-linux/4.1.1/../../../../lib64/libreadline.so: undefined reference to `PC' /usr/lib/gcc/x86_64-redhat-linux/4.1.1/../../../../lib64/libreadline.so: undefined reference to `tgetflag' /usr/lib/gcc/x86_64-redhat-linux/4.1.1/../../../../lib64/libreadline.so: undefin ..... --- Works: gcc -DFIX -Wl,--as-needed aa.c -lreadline -lncurses ----
Created attachment 5708 [details] Alternative approach working around this isse
Created attachment 5709 [details] Simple test case to demonstrate the GNU LD Bug
has this bug been fixed in the latest 3.5.3?
(In reply to comment #18) > has this bug been fixed in the latest 3.5.3? > No, unfortunately not yet. Opinions seem to differ...
(In reply to comment #16) > Created an attachment (id=5708) [details] > Alternative approach working around this isse Any volunteer for patch review?
It's a fun way to solve this. We should have a configure test for it though or a huge comment that this is a RHEL5 specific thing. Otherwise it will cause grey hair in a couple of years when people try to understand why this is done. Volker
I don't like this patch as introduce an explicit dependency on all systems
Right. That's why I mentioned the configure check. Volker
Yes indeed, my patch was more or less a joke. Hint: The testcase appended can be extended to work as a configure check.
the check from comment 17 checks for some bug in ld but that seems not to be related to an underlinked readline.
(In reply to comment #25) > the check from comment 17 checks for some bug in ld but that seems not to be > related to an underlinked readline. > Of course it is. The linker line reads: .. objects.o ... --as-needed -lreadline -lncurses. -lncurses is wrongly removed by ld because there is no direct symbol resolving in the objects. It does not honor second-order dependent symbols from readline library, even if stated on the command line. this is why my silly patch proposal works ... It is an LD bug.
Isn't that exactly what --as-needed is for? It should skip unused libraries which are provided on the cmdline. And we use it because cups-options and krb5-config sometimes list unused libraries.
Yes, but there are subtile cases. --as-needed should only delete libraries which are unused. But it does delete libraries which are needed, in the special case of secon order dependencies. Please consider the example attached: On rhel5 it does On RHEL5 + gcc -c -fPIC sha.c + gcc -c -fPIC shb.c + gcc --shared -fPIC -o liba.so sha.c + gcc --shared -fPIC -o libb.so shb.c + echo if this fails, gnu ld has a bug if this fails, gnu ld has a bug + gcc -o a.out a.c -Wl,--as-needed -L. -la -lb ./liba.so: undefined reference to `b' collect2: ld gab 1 als Ende-Status zurück On Ubuntu 10.04 with GNU ld (GNU Binutils for Ubuntu) 2.20.1-system.20100303 + gcc -c -fPIC sha.c + gcc -c -fPIC shb.c + gcc --shared -fPIC -o liba.so sha.c + gcc --shared -fPIC -o libb.so shb.c + echo if this fails, gnu ld has a bug if this fails, gnu ld has a bug + gcc -o a.out a.c -Wl,-as-needed -L. -la -lb No Error -------- In order to translate to the samba/rhel5 case: libb.so == libncurses.so b() == tgetent .... liba.so == libreadline.so a() == readline() a.c == smbclient ... SuSE and others link their readline differently: gcc --shared -fPIC -o libb.so shb.c gcc --shared -fPIC -o liba.so sha.c -L. -lb readline does now have an explicit dependency to ncurses. But not (by design) in RHEL.
In the compile of a.out you link explicitly against libb and liba even though you only use a symbol from liba. You already did the mistake to not link liba against libb so you produce an underlinked library which is the main problem here. If you would have linked liba against libb then liba would have liba in the DT_NEEDED elf header. I already said that a cristal ball would be needed to know which libraries we use lack which dependencies which we would then need to manually add ourself (like ncurses which we don't need to link against ourself in most places).
Created attachment 5736 [details] configure.in patch configure.in patch: Tries to link with readline, if it worked before. if this fails revert detection of --as-needed.
(In reply to comment #29) you miss the point: libreadline *is* underlinked on RHEL.
Comment on attachment 5736 [details] configure.in patch This comment is wrong: +# check wether a libreadline triggers a bug in ld with --as-needed +# i.e. RHEL5 This isn't a bug in ld, it's a bug in libreadline on RHEL5.
Comment on attachment 5736 [details] configure.in patch I guess the patch itself looks good, but I haven't tested it myself.
(In reply to comment #32) > (From update of attachment 5736 [details]) > This comment is wrong: > > +# check wether a libreadline triggers a bug in ld with --as-needed > +# i.e. RHEL5 > > This isn't a bug in ld, it's a bug in libreadline on RHEL5. > This *is* an ld Bug. Please compare the output for instance if "ldd gpg" und "ldd lftp" on Centos5/RHEl5. one of them it is linked with both termcap and readline and the other with ncurses and the same readline. This is by design. You have the choice to use readline with any terminal library you like. I double checked with Florian La Roche, who was with RedHat for some time. If you don't trust me, PLEASE try my demonstration on for instance ubuntu 10.04 with an recent ld , you will see that underlinking does not trigger this kind of bug any more.
maybe it becomes more clear that we do not suffer from a GNU ld bug when we remove the useless linking agains ncurses from the Makefile? The configure check will also need the test for ncurses dependency to be complete, see comment #13. We will probably remove the ncurses linker flags and than simply not using --as-needed will make the undelinked readline fail to link again.
Do not understand me: I am not an advocate for the RedHat design decision. I only want you to understand, what they were trying to achieve: The -lncurses is not useless Which terminal library you do want to link against? You'll have to explicitly state which terminal library you want. For Linux, this is a RedHat "Feature": You have the choice to link readline against -lncurses or -ltermap. Both is possible. Other Linux distros do not support this. And BTW: There are UNIX systems which do not know about library dependencies. For instance having a static libreadline.a with PIC code. So you'll have to test which one works.
The pros and cons of this specially linked library the ABI problems it has and whatever else is not in the scope of this samba bug and should really be discusses elsewhere. All which is of interest at this place is what a complete configure check for that would need, see above. As this bug is about "--as-needed cannot be disabled" and as-needed is just the GNU name of what other linkers do with "-z ignore" I would suggest we add a configure option "--ignore-underlinked-libs" which does almost the same as Metze's initial patch but in turn also disables the -z defs settings.
I proposed a working and complete configure.in patch. So please apply.
Created attachment 5737 [details] Patch without offending comment
The patch should include as comments, just as Volker notes, a clear, unemotional description of the issue. Bugzilla references are good, but given how long this will be with us, and we may change bug tracker in the next 10 years, a long clear description is better.
(In reply to comment #40) > The patch should include as comments, just as Volker notes, a clear, > unemotional description of the issue. Bugzilla references are good, but given > how long this will be with us, and we may change bug tracker in the next 10 > years, a long clear description is better. > I already did it before. Ok, now the scientific way: Take /usr/lib/libreadline.so and /usr/lib/libncurses.so from a RHEL5 system (I will append a tgz to this report) and put it in a directory use this test.c in the same dir. main() { readline(); } Do this on almost any non-RHEL5 system (I tested on SuSE > 11.0, Ubuntu 10.04) cc -Wl,--as-needed test.c -L. -lreadline -lncurses No Problem. No do it on RHEL5: cc -Wl,--as-needed test.c -L. -lreadline -lncurses ... unresolved symbols ... The only relevant parameter I changed (well, forget about the compiler,libc) is the linker. So I have to deduce it is an linker bug. So Metzes negative review is invalid.
Sorry to correct myself: The example works on RHEL6 Beta, Ubuntu 10.04, not on Suse >11.0 > Do this on almost any non-RHEL5 system (I tested on SuSE > 11.0, Ubuntu 10.04) > cc -Wl,--as-needed test.c -L. -lreadline -lncurses >
O.k. you got me: Not a ld bug ... while retesting...
Created attachment 5741 [details] patch Patch with longer description
Comment on attachment 5737 [details] Patch without offending comment Sorry, I already changed the negative review flag into a positive one, but didn't managed to store it (as I didn't noticed that firefox asked me for a password...)
Comment on attachment 5741 [details] patch Can you readd the reference to the bug id?
Created attachment 5742 [details] patch with description and bugid surely I can reference the bugid
I would call this a blocker. It is pretty embarassing that on RHEL5 ./configure;make does not work out of the box. Volker
Karolin, the patch works for me on RHEL5 and looks like the right approach. I am going to push it in master, and would like to see it in 3.5
Signed off and pushed to master
Reassigning to Karolin for inclusion in 3.5.x
Comment on attachment 5742 [details] patch with description and bugid simo, this is no working fix. Did you test it? There needs to be tested more things. Partly this is written above, I know that metze has a good plan how to cleanly add all the required tests at the apropriate places.
(In reply to comment #52) > (From update of attachment 5742 [details]) > simo, this is no working fix. Did you test it? Of course I did. Current master does not build on RHEL5, and this patches makes it build just fine. > There needs to be tested more things. make on RHEL5 tested it just fine ;-) > Partly this is written above, I know that metze has a good plan how to > cleanly add all the required tests at the apropriate places. We can improve these patches if you want, but they do allow me to build on RHEL5. Given nobody seem to have better patches yet, and these patches makes 3.5.x buildable on RHEL, they are good enough to go in. Simo.
Reassigning to Metze as he had an idea for a proper fix if I remember correctly.
RH has supported packages. SerNet has supported current packages. The ability to build from source for RHEL is a nice enhancement but not a blocker. Volker
You claim that it breakes something without giving an explanation why it breaks or at least an example where or does breaks code.
last time I tried with just leaving out --as-needed at least the net binary compiled but refused to work (and complaining about unresolved symbols) unless also --allow-shlib-undefined was also used at compile time. It retested with the current 3.5 release and it actually worked this time however. I'll therefore remove my - flag as it's not broken like I thought. The whole test for as-needed should eventually be moved up in configure.in so that all the other linker tests are being done with the as-needed flags that we use in the final compile.
Ok, as there are two positive reviews now, I just pushed the patch to v3-5-test. Will be included in 3.5.4. Closing out bug report. Thanks a lot! :-)
In FreeBSD, -lkrb5 does not explicitly depend on -lasn1. This leads to initialize_asn1_error_table_r not being available. Volker
Fixed with b9835a1f9d8b81. as-needed is gone :-)
Just FYI (in case anyone else hits this). The fix doesn't work for RedHat4.x, as there the link succeeds (so configure thinks it is OK to use --as-needed), but the *execution* fails (so it actuality isn't). This patch (for 3.5.8) runs the test program too. --- source3/configure.orig 2011-03-06 18:58:41.000000000 +0000 +++ source3/configure 2011-04-06 17:47:09.000000000 +0100 @@ -52315,7 +52315,7 @@ test ! -s conftest.err } && test -s conftest$ac_exeext && { test "$cross_compiling" = yes || - $as_test_x conftest$ac_exeext + ( $as_test_x conftest$ac_exeext && ./conftest$ac_exeext ) }; then { $as_echo "$as_me:$LINENO: result: yes" >&5 $as_echo "yes" >&6; }