Bug 15251 - 32bit build needs O_LARGEFILE in order to create files larger than 4GB (at least on 32bit linux kernels)
Summary: 32bit build needs O_LARGEFILE in order to create files larger than 4GB (at le...
Status: NEW
Alias: None
Product: cwrap
Classification: Unclassified
Component: library (show other bugs)
Version: unspecified
Hardware: All All
: P5 normal
Target Milestone: ---
Assignee: Stefan Metzmacher
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-11-22 09:04 UTC by Stefan Metzmacher
Modified: 2022-11-24 12:06 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Metzmacher 2022-11-22 09:04:15 UTC
Trying to get make test working on a pure 32bit ubuntu 18.04, revealed that
we need to pass O_LARGEFILE to openat[2]() in order to get large file support.
Comment 1 Ralph Böhme 2022-11-22 10:35:03 UTC
Hm, it's been some time since I dealt with LFS support, but isn't the problem that we fail to set _FILE_OFFSET_BITS=64?

We have a check in buildtools/wafsamba/samba_conftests.py, but that's how it's supposed to work afair. You don't check the define, you set it and thereby request the 'Large file' compilation environment.
Comment 2 Ralph Böhme 2022-11-22 10:36:44 UTC
See also _FILE_OFFSET_BITS in https://man7.org/linux/man-pages/man7/feature_test_macros.7.html:

              Defining this macro with the value 64 automatically
              converts references to 32-bit functions and data types
              related to file I/O and filesystem operations into
              references to their 64-bit counterparts.  This is useful
              for performing I/O on large files (> 2 Gigabytes) on
              32-bit systems.  (Defining this macro permits correctly
              written programs to use large files with only a
              recompilation being required.)

              64-bit systems naturally permit file sizes greater than 2
              Gigabytes, and on those systems this macro has no effect.
Comment 3 Stefan Metzmacher 2022-11-22 12:17:59 UTC
(In reply to Ralph Böhme from comment #2)

We already do all other magic, see:

$ git grep _FILE_OFFSET_BITS buildtools/
buildtools/wafsamba/samba_conftests.py:                       cflags='-D_FILE_OFFSET_BITS=64',
buildtools/wafsamba/samba_conftests.py:                       msg='Checking for -D_FILE_OFFSET_BITS=64'):
buildtools/wafsamba/samba_conftests.py:        conf.DEFINE('_FILE_OFFSET_BITS', 64)
Comment 4 Ralph Böhme 2022-11-22 14:01:31 UTC
(In reply to Stefan Metzmacher from comment #3)
The logic is a bit backward I guess, but if this correctly adds -D_FILE_OFFSET_BITS=64 then LFS should work on 32 bit, no need for O_LARGEFILE.
Comment 5 Jeremy Allison 2022-11-22 17:18:13 UTC
Yes, we have never needed O_LARGEFILE to work with 64-bit sizes on 32-bit systems. If we do, then there's a bug in the largefile environment detection (I remember adding this for IRIX back in 1999 or so :-).
Comment 6 Stefan Metzmacher 2022-11-22 20:23:42 UTC
(In reply to Stefan Metzmacher from comment #3)

And for most cases linux forces O_LARGEFILE within the kernel:

SYSCALL_DEFINE3(open, const char __user *, filename, int, flags, umode_t, mode)
{
        if (force_o_largefile())
                flags |= O_LARGEFILE;
        return do_sys_open(AT_FDCWD, filename, flags, mode);
}

SYSCALL_DEFINE4(openat, int, dfd, const char __user *, filename, int, flags,
                umode_t, mode)
{
        if (force_o_largefile())
                flags |= O_LARGEFILE;
        return do_sys_open(dfd, filename, flags, mode);
}

But userspace need to pass it explicitly if the kernel is build with
CONFIG_ARCH_32BIT_OFF_T.

But it turned out to be a problem with socket_wrapper implementing openat(),
otherwise glibc injects O_LARGEFILE
Comment 7 Stefan Metzmacher 2022-11-22 20:31:55 UTC
Socket wrapper should something like this:

get this:

+#if _FILE_OFFSET_BITS == 64 && SIZE_MAX == 0xffffffffUL && defined(O_LARGEFILE)
+       flags |= O_LARGEFILE;
+#endif
Comment 8 Samba QA Contact 2022-11-24 12:06:36 UTC
This bug was referenced in samba master:

6dddb268df08fd91f8e0f189f948ad76e5805dca