Created attachment 15553 [details] Full samba build log Cross-compiling for arm target on x86_64 host (Ubuntu 16.04), with toolchain based on gcc-9.2.0, glibc-2.3.0, heimdal-7.7.0, Python-3.7.5. Samba 4.9.13 and this toolchain (using Python2 in place of Python3) builds fine when cross-compiling. However, the latest Samba v4.11-test[1] with patches[2,3] and configuration[4] results in the following ASN1 syntax error failure (full log attached): [246/412] Processing source4/heimdal/lib/asn1/rfc2459.asn1 /home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/asn1_compile: 1: /home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/asn1_compile: Syntax error: word unexpected (expecting ")") /home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/asn1_compile: 1: /home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/asn1_compile: Syntax error: word unexpected (expecting ")") /home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/asn1_compile: 1: /home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/asn1_compile: Syntax error: word unexpected (expecting ")") /home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/asn1_compile: 1: /home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/asn1_compile: Syntax error: word unexpected (expecting ")") /home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/asn1_compile: 1: /home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/asn1_compile: Syntax error: word unexpected (expecting ")") /home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/asn1_compile: 1: /home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/asn1_compile: Syntax error: word unexpected (expecting ")") Waf: Leaving directory `/home/ubuntu/projects/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/samba-7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0/bin/default' Build failed -> task in 'HEIMDAL_KX509_ASN1_ASN1' failed with exit status 2 (run with -v to display more information) -> task in 'HEIMDAL_HDB_ASN1_ASN1' failed with exit status 2 (run with -v to display more information) -> task in 'HEIMDAL_GSSAPI_ASN1_ASN1' failed with exit status 2 (run with -v to display more information) -> task in 'HEIMDAL_SPNEGO_ASN1_ASN1' failed with exit status 2 (run with -v to display more information) -> task in 'HEIMDAL_RFC2459_ASN1_ASN1' failed with exit status 2 (run with -v to display more information) -> task in 'HEIMDAL_DIGEST_ASN1_ASN1' failed with exit status 2 (run with -v to display more information) The most obvious issue is that the Samba build is using the *bundled* asn1_compile even though the build is configured to use the system version: --bundled-libraries='ALL,!asn1_compile,!compile_et,!zlib' \ I've tried removing the leading "ALL," in case that is confusing Samba but it makes no difference, I have the same failure with only "!asn1_compile,!compile_et". When building 4.9.13 (same --bundled-libraries option) the following is logged: Checking for program compile_et : /home/neil/projects/scratch/alternates/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/toolchain/bin/compile_et Checking for program asn1_compile : /home/neil/projects/scratch/alternates/LibreELEC.tv/build.LibreELEC-RPi2.arm-9.80-devel/toolchain/bin/asn1_compile However I don't see this logged at all when building v4.11-test. Anyone got any ideas? Is --bundled-libraries being ignored? 1. https://github.com/samba-team/samba/tree/7f5334a92c4a378f88c0ee8c5fde46dd087a9dc0 2. https://bugzilla.samba.org/show_bug.cgi?id=13846#c23 3. https://github.com/buildroot/buildroot/blob/8b11b96f41a6ffa76556c9bf03a863955871ee57/package/samba4/0006-heimdal_build-wscript_build-do-not-add-host-include-.patch 4. https://github.com/LibreELEC/LibreELEC.tv/blob/f35b9f6b8ed4445edff20630cc39b25e47c428fb/packages/network/samba/package.mk
That looks like a degradation in 4.11 (unlike some other cross-compile-related issues that came with 4.10.x) - I cross-built Samba 4.10.8 using buildroot.org - worked fine. Then used the buildroot recipe for Samba master - got this asn1_compile error. In my case it just wouldn't run asn1_compile because I was building for AArch64: bin/sh: /home/vagrant/buildroot/output/build/samba4-25b1e478ec5e8ac40d41013a824311824482fa67/bin/asn1_compile: cannot execute binary file: Exec format error Will look further into this...
Yes! Finally!! :) I can confirm that samba-4.10.9 + buildroot patches[1] (all of them, in place of the patch from [2]) is now building for me. Previously I tested building samba-4.10.0 with the various patches discussed in these bugs, but had no success - maybe it was because at the time I was using Python-2.7.13. Or the wrong version of the patches. I also didn't have the Heimdal buildroot work-around patch (a ping on my bug would have been nice, to let me know!) Unfortunately I've now run into #14132[1] so can't actually install anything I've built, but - at least it now builds! :) Thanks for the pointers Uri. Please ping me if you discover anything about the ASN1 regression and want me to test it. Who knows, maybe 4.11.x installs stuff too. :) 1. https://github.com/buildroot/buildroot/tree/8b11b96f41a6ffa76556c9bf03a863955871ee57/package/samba4 2. https://bugzilla.samba.org/show_bug.cgi?id=13846#c23 3. https://bugzilla.samba.org/show_bug.cgi?id=14132
Yes, on further investigation, the asn1_compile binary being used by 4.11.x is the 32-bit arm target, which is of course a bit useless on my x86_64 build host, and probably explains all the syntax errors... not that the build should be creating/using this binary, as it should be using the x86_64 system binary! $ file samba-4.11.1/bin/default/source4/heimdal_build/asn1_compile samba-4.11.1/bin/default/source4/heimdal_build/asn1_compile: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-, for GNU/Linux 4.4.0, not stripped
Presumably this is only an issue because the 4.11.x build is no longer respecting !asn1_compile (etc.) in --bundled-libraries - somewhere a check has been dropped or skipped as the build has been told to use the system binary, and to not build/use the bundled version (which will never work when cross-compiling - that's why we use !asn1_compile). If the --bundled-libraries check is fixed so it works as it did in 4.10.9 then this building & executing target binary on host issue should go away automatically.
That seems to be a regression of 8061983d4882f3ba3f12da71443b035d7b672eec
Created attachment 15558 [details] a proposed fix for master
Neil, Can you please test the attached fix for Samba 4.11? I was able to get past the asn1_compile hurdle on master branch and my buildroot setup using this patch (didn't get to a full build due to some unrelated header issue, but that may be a master-only degradation). Thanks, Uri.
Also, if you do get to test this patch on 4.11, can you use the cross-compilation fix from https://bugzilla.samba.org/show_bug.cgi?id=13846#c23 instead of the buildroot cross-compilation fix? That would validate this fix and allow me to push it upstream. Thanks, Uri.
Hi Uri Many thanks, and I have good news! I managed to cross-compile 4.11.1 x86_64->arm/32-bit with the following 4 patches: 1. https://bugzilla.samba.org/show_bug.cgi?id=14164#c6 (ASN1 fix) 2. https://bugzilla.samba.org/show_bug.cgi?id=13846#c23 (cross-answers fix) 3. https://bugzilla.samba.org/show_bug.cgi?id=13856#c11 (buildroot Heimdal cross-compile fix) 4. https://bugzilla.samba.org/show_bug.cgi?id=13856#c13 (fix "link redeclared" issue) Without the 3rd patch I get the "cross-compile badness" error due to /usr/include/heimdal being included. Without the 4th patch I get the "link redeclared" issue. The patch 3 and 4 failures are both bug #13856. Unfortunately Samba 4.11.1 doesn't actually install anything (bug #14132), but we're getting there... :) Thanks Neil
We shouldn't be using a system asn1_compile to build parts of heimdal, the idea was to build those with the host C compiler and then execute those tools as part of the build. otherwise we end up in unpredictable territory as asn1_compile and compile_et (less likely) could well generate files that rely on aspects of the matching library. If those get out of sync then there could be trouble. So the fix is to make use_hostcc work properly. Sorry!
(In reply to Andrew Bartlett from comment #10) I particular it appears that this allows the use of an old, unused upstream compile_et from MIT kerberos to compile these for Heimdal, triggering other workarounds. For example the issue raised in https://bugzilla.samba.org/show_bug.cgi?id=13856#c13 appears to be from the MIT compile_et removed in this commit: commit 939bb6c500aba6359f80bd27ba6269c23168fc16 Author: Greg Hudson <ghudson@mit.edu> Date: Tue Apr 29 11:03:03 2014 -0400 Remove compile_et.c We have used awk scripts in preference to the C version of compile_et since 1994 (r4426). Get rid of the sources for the C version and all of the Makefile.in references to them.
since 4.18 A) the patch no longer applies as the lines from wscript_configure_system_heimdal have been removed from wscript_configure_system_heimdal B) we are still carrying the wscript_configure_embedded_heimdal insertions of the patch for the aarch64 and arm cross compile build on x86_64