Bug 15033 - 4.16.0 build without PYTHONHASHSEED=1 fails due to gssapi.h header ordering
Summary: 4.16.0 build without PYTHONHASHSEED=1 fails due to gssapi.h header ordering
Status: NEW
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: Build (show other bugs)
Version: 4.16.0
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Samba QA Contact
QA Contact: Samba QA Contact
URL: https://gitlab.com/samba-team/samba/-...
Keywords:
Depends on:
Blocks:
 
Reported: 2022-03-24 14:44 UTC by Rudi Heirbaum
Modified: 2022-06-13 00:47 UTC (History)
4 users (show)

See Also:


Attachments
Patches for v4-16-test (22.96 KB, text/plain)
2022-03-30 09:09 UTC, Stefan Metzmacher
metze: review? (abartlet)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Rudi Heirbaum 2022-03-24 14:44:51 UTC
build bug with LibreELEC + samba

Reference to the following line in the Heimdal code: 

https://github.com/heimdal/heimdal/blob/6b45c3512e37cc2ea8c87f1e96b02bec25967544/lib/gssapi/gssapi/gssapi_krb5.h#L39

During a cross compile we are having inconsistent builds of Samba 4.16.0 with the gssapi.h. 

a) We separately build heimdal_asn1_compile and heimdal_compile_et and are using heimdal/heimdal:master without issue.

b) I have put together the following workaround to deal with the build failure. Not really sure the right way to have this fixed. 
I have confirmed that the only gssapi.h files in the samba build are:
- lib/replace/system/gssapi.h
- third_party/heimdal/lib/gssapi/gssapi.h
- third_party/heimdal/lib/gssapi/gssapi/gssapi.h

https://github.com/LibreELEC/LibreELEC.tv/pull/6241/commits/810be4a5b9d596225a0b1a5ce15c853fd63511ca

== Detail below ==

samba: qualify required gssapi.h
Inconsistent builds with samba 4.16 with the wrong gssapi.h being picked up. This is a workaround, and being raised upstream.

```
[1800/2511] Compiling source3/lib/smbd_shim.c
In file included from ../../third_party/heimdal/lib/gssapi/gssapi/gssapi_krb5.h:40,
                 from ../../lib/replace/system/gssapi.h:43,
                 from ../../lib/krb5_wrap/gss_samba.h:27,
                 from ../../auth/kerberos/pac_utils.h:27,
                 from ../../auth/kerberos/gssapi_pac.c:28:
../../third_party/heimdal/lib/gssapi/gssapi/gssapi_oid.h:9:8: error: unknown type name 'GSSAPI_LIB_VARIABLE'
    9 | extern GSSAPI_LIB_VARIABLE gss_OID_desc __gss_krb5_copy_ccache_x_oid_desc;
      |        ^~~~~~~~~~~~~~~~~~~
```
WORKAROUND PATCH
```
--- a/third_party/heimdal/lib/gssapi/gssapi/gssapi_krb5.h	2022-03-08 14:53:05.742089500 +0000
+++ b/third_party/heimdal/lib/gssapi/gssapi/gssapi_krb5.h	2022-03-08 14:53:05.742089500 +0000
@@ -36,7 +36,7 @@
 #ifndef GSSAPI_KRB5_H_
 #define GSSAPI_KRB5_H_
 
-#include <gssapi.h>
+#include <gssapi/gssapi/gssapi.h>
 #include <gssapi/gssapi_oid.h>
 
 GSSAPI_CPP_START
```

Andrew - firstly - apologies for creating the ticket as an FYI in Heimdal too, and potentially causing additional work - was not my intention. Was as an FYI as I had miss understood from the release notes that the Samba Heimdal was now closely tracking the github Heimdal/Heimdal.

References:
- https://github.com/heimdal/heimdal/issues/984
- https://github.com/LibreELEC/LibreELEC.tv/pull/6241/commits/810be4a5b9d596225a0b1a5ce15c853fd63511ca
- https://github.com/LibreELEC/LibreELEC.tv/pull/6241
Comment 1 Stefan Metzmacher 2022-03-30 09:09:18 UTC
Created attachment 17256 [details]
Patches for v4-16-test
Comment 2 Andrew Bartlett 2022-03-30 18:24:33 UTC
Comment on attachment 17256 [details]
Patches for v4-16-test

BUG: tag is missing and it appears (if you didn't follow all the threads) to be unrelated to the bug.

I'll re-title the bug - what appears to be wrong header paths may well still be related to that, but shows up non-determinsicly due to lake of PYTHONHASHSEED=1 due to direct calling of 'waf' in some build scripts.
Comment 3 Stefan Metzmacher 2022-03-31 09:56:56 UTC
Comment on attachment 17256 [details]
Patches for v4-16-test

The last commit has this added to the commit message:

The last 8 patches are backported to avoid people running into:
https://bugzilla.samba.org/show_bug.cgi?id=15033
and
https://bugzilla.samba.org/show_bug.cgi?id=15037
Comment 4 Rudi Heirbaum 2022-03-31 10:44:26 UTC
A) I have applied the attached patch “Patches for v4-16-test” (with the below changes - so that it applies cleanly) against the 4.16.0.

B) I have adjusted the LibreELEC build as follows, and it now successfully builds consistently. Noting we are still calling waf directly for the install_target. We were already calling configure directly. I have not done any run testing as yet. But have done 2 iterations of the cross-compiled builds (17 targets each) which were “reliably” inconsistently failing before.

diff --git a/packages/network/samba/package.mk b/packages/network/samba/package.mk
index e0b2b5c925..779514dee4 100644
--- a/packages/network/samba/package.mk
+++ b/packages/network/samba/package.mk
@@ -122,12 +122,12 @@ pre_make_target() {
 }
 
 make_target() {
-  ./buildtools/bin/waf build ${PKG_WAF_VERBOSE} --targets=${PKG_SAMBA_TARGET} -j${CONCURRENCY_MAKE_LEVEL}
+  make ${PKG_SAMBA_TARGET} -j${CONCURRENCY_MAKE_LEVEL}
 }
 
 makeinstall_target() {
-  ./buildtools/bin/waf install ${PKG_WAF_VERBOSE} --destdir=${SYSROOT_PREFIX} --targets=smbclient -j${CONCURRENCY_MAKE_LEVEL}
-  ./buildtools/bin/waf install ${PKG_WAF_VERBOSE} --destdir=${INSTALL} --targets=${PKG_SAMBA_TARGET} -j${CONCURRENCY_MAKE_LEVEL}
+  PYTHONHASHSEED=1 WAF_MAKE=1 ./buildtools/bin/waf install ${PKG_WAF_VERBOSE} --destdir=${SYSROOT_PREFIX} --targets=smbclient -j${
CONCURRENCY_MAKE_LEVEL}
+  PYTHONHASHSEED=1 WAF_MAKE=1 ./buildtools/bin/waf install ${PKG_WAF_VERBOSE} --destdir=${INSTALL} --targets=${PKG_SAMBA_TARGET} -
j${CONCURRENCY_MAKE_LEVEL}
 }
 
 copy_directory_of_links() {





=== patch below ===

$ diff attachment.cgi\?id\=17256 patches-for-v4-16-test.patch -Nu
--- attachment.cgi?id=17256     2022-03-31 10:17:59.030079498 +0000
+++ patches-for-v4-16-test.patch        2022-03-30 11:01:49.418320817 +0000
@@ -120,43 +120,6 @@
 +(cd $prefix1 && find .) | sort >p1.txt
 +(cd $prefix2 && find .) | sort >p2.txt
  diff -u p[12].txt
-diff --git a/buildtools/scripts/abi_gen.sh b/buildtools/scripts/abi_gen.sh
-index ddb0a7cc36fd..c66a1b848a88 100755
---- a/buildtools/scripts/abi_gen.sh
-+++ b/buildtools/scripts/abi_gen.sh
-@@ -6,21 +6,21 @@ SHAREDLIB="$1"
- GDBSCRIPT="gdb_syms.$$"
- 
- (
--cat <<EOF
-+      cat <<EOF
- set height 0
- set width 0
- EOF
- 
--# On older linker versions _init|_fini symbols are not hidden.
--objdump --dynamic-syms "${SHAREDLIB}" | \
--    awk '$0 !~ /.hidden/ {if ($2 == "g" && $3 ~ /D(F|O)/ && $4 ~ /(.bss|.rodata|.text)/) print $NF}' | \
--    sort | \
--    while read -r s; do
--    echo "echo $s: "
--    echo p "${s}"
--done
--) > $GDBSCRIPT
-+      # On older linker versions _init|_fini symbols are not hidden.
-+      objdump --dynamic-syms "${SHAREDLIB}" |
-+              awk '$0 !~ /.hidden/ {if ($2 == "g" && $3 ~ /D(F|O)/ && $4 ~ /(.bss|.rodata|.text)/) print $NF}' |
-+              sort |
-+              while read -r s; do
-+                      echo "echo $s: "
-+                      echo p "${s}"
-+              done
-+) >$GDBSCRIPT
- 
- # forcing the terminal avoids a problem on Fedora12
--TERM=none gdb -n -batch -x $GDBSCRIPT "$SHAREDLIB" < /dev/null
-+TERM=none gdb -n -batch -x $GDBSCRIPT "$SHAREDLIB" </dev/null
- rm -f $GDBSCRIPT
 diff --git a/buildtools/scripts/autogen-waf.sh b/buildtools/scripts/autogen-waf.sh
 index 7a6e94c5ec4d..a0ed80c33fa6 100755
 --- a/buildtools/scripts/autogen-waf.sh
@@ -441,36 +404,6 @@
  
 -- 
 2.25.1
-
-
-From 8af7d17c14fb1f664f4a3f69ef0f7a9bbc5a96b1 Mon Sep 17 00:00:00 2001
-From: Stefan Metzmacher <metze@samba.org>
-Date: Mon, 28 Mar 2022 12:49:50 +0200
-Subject: [PATCH 4/8] s4:selftest/provisions: make use of 'make testenv' and
- avoid direct waf
-
-Signed-off-by: Stefan Metzmacher <metze@samba.org>
-Reviewed-by: Andrew Bartlett <abartlet@samba.org>
-(cherry picked from commit 0be4f5672338802042b06308c5cf0ea04bcaf48e)
----
- .../selftest/provisions/release-4-1-0rc3/steps-to-reproduce.txt | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/source4/selftest/provisions/release-4-1-0rc3/steps-to-reproduce.txt b/source4/selftest/provisions/release-4-1-0rc3/steps-to-reproduce.txt
-index fc0dfddf8ac2..19466e252f6b 100644
---- a/source4/selftest/provisions/release-4-1-0rc3/steps-to-reproduce.txt
-+++ b/source4/selftest/provisions/release-4-1-0rc3/steps-to-reproduce.txt
-@@ -6,7 +6,7 @@ patch -p1 < rename-domains.patch
- 
- # MASTER_SRC needs to point at where the schema files can be found, not the 4.1.0rc1 checkout
- 
--SELFTEST_TESTENV=promoted_dc:local ./buildtools/bin/waf test --testenv
-+SELFTEST_TESTENV=promoted_dc:local make testenv
- 
- # then in the testenv, run:
- 
--- 
-2.25.1
 
 
 From 16d566b1139f56686fac969d3bc362be162bad28 Mon Sep 17 00:00:00 2001