Bug 13630 - ldb: LMDB disabled on 32-bit systems
Summary: ldb: LMDB disabled on 32-bit systems
Status: NEW
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: AD: LDB/DSDB/SAMDB (show other bugs)
Version: 4.9.0
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Samba QA Contact
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-09-23 08:38 UTC by Mathieu Parent
Modified: 2019-02-28 05:52 UTC (History)
2 users (show)

See Also:


Attachments
2GB-on-32bit -> failure (3.63 KB, patch)
2018-10-03 19:58 UTC, Mathieu Parent
no flags Details
256MB-on-32bit -> success (3.64 KB, patch)
2018-10-03 20:07 UTC, Mathieu Parent
no flags Details
Skip LMDB tests when not HAVE_LMDB (6.51 KB, patch)
2018-10-08 12:12 UTC, Mathieu Parent
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mathieu Parent 2018-09-23 08:38:17 UTC
On 32-bit system, ./configure gives:

  Checking for a 64-bit host to support lmdb : not found 
  --without-ldb-lmdb implied as this host is not 64-bit

Then make test fails:

  make test called, but ldb was built --without-ldb-lmdb

LMDB works on 32-bit system. So why ?

Complete build log:
https://buildd.debian.org/status/fetch.php?pkg=ldb&arch=i386&ver=2%3A1.4.2-2&stamp=1537337211&raw=0
Comment 1 Mathieu Parent 2018-09-24 11:01:11 UTC
There reason is probably (lib/ldb/ldb_mdb/ldb_mdb.c line 688):

 688         const size_t mmap_size = 8LL * GIGABYTE;
 ...
 733         /*
 734          * Currently we set a 8Gb maximum database size
 735          * via the constant mmap_size above
 736          */
 737         ret = mdb_env_set_mapsize(*env, mmap_size);


https://git.samba.org/?p=samba.git;a=blob;f=lib/ldb/ldb_mdb/ldb_mdb.c;h=646a67c554c21dec17f78ee87e2457368a411115;hb=HEAD#l688

Couldn't we use only 2GB for mmap_size on 32bit?
Comment 2 Andrew Bartlett 2018-09-24 17:31:13 UTC
We would have to use a tiny map size, like 256MB or so, because we map multiple databases and because LMDB has not reopen call, we map them multiple times.

The only way we could possibly use LMDB on a 32 bit host is if the host OS provides the lmdb lib built in the slower VL32 mode.
Comment 3 Mathieu Parent 2018-10-03 05:22:27 UTC
This is sad.

How can we still allow "make test" to success?
Comment 4 Mathieu Parent 2018-10-03 19:58:05 UTC
Created attachment 14508 [details]
2GB-on-32bit -> failure

I can confirm that even with 256MB, make test fails with a lot of:
Failed to connect to 'mdb://apitest.ldb' with backend 'mdb': Could not open DB apitest.ldb: Cannot allocate memory
Comment 5 Mathieu Parent 2018-10-03 20:07:36 UTC
Created attachment 14509 [details]
256MB-on-32bit -> success

Oups, previous patch was 2GB.

Actually, 256MB works.
Comment 6 Mathieu Parent 2018-10-05 04:19:39 UTC
Andrew, could we skip only the lmdb tests instead? Current status with no coverage on 32bit will hide future regressions.
Comment 7 Mathieu Parent 2018-10-08 12:12:13 UTC
Created attachment 14516 [details]
Skip LMDB tests when not HAVE_LMDB

I took a different path: skip LMDB tests when support is not available.

Patch is against ldb only and should be ported to the whole samba tree.

Andrew, what do you think?
Comment 8 Andrew Bartlett 2018-10-10 09:01:42 UTC
(In reply to Mathieu Parent from comment #7)
I'm still nervous, because in broader Samba we have had actual tests silently vanish this way.

In particular, in waf env.FOO is [] if FOO was never set. 

So, I'll accept this if a C and python test is written such that if the HAVE_LMDB is wrong, it fails.  So if HAVE_LMDB is 0, then it tries to crate an lmdb and checks for the correct error code.
Comment 9 Mathieu Parent 2019-02-28 05:52:06 UTC
(In reply to Andrew Bartlett from comment #8)

Done, and push as Gitlab MR: https://gitlab.com/samba-team/samba/merge_requests/266