Bug 12694 - socket_wrapper doesn't replace open() on 32-bit
Summary: socket_wrapper doesn't replace open() on 32-bit
Alias: None
Product: cwrap
Classification: Unclassified
Component: library (show other bugs)
Version: unspecified
Hardware: x86 Linux
: P5 normal
Target Milestone: ---
Assignee: Andreas Schneider
QA Contact:
Depends on:
Reported: 2017-03-15 21:34 UTC by Matt Rogers
Modified: 2017-04-06 14:44 UTC (History)
1 user (show)

See Also:

test program (1.95 KB, text/x-csrc)
2017-03-15 21:34 UTC, Matt Rogers
no flags Details
patch (6.69 KB, patch)
2017-03-29 08:06 UTC, Andreas Schneider
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Rogers 2017-03-15 21:34:24 UTC
Created attachment 13068 [details]
test program

When using socket_wrapper on a 32bit system, calls to open() do not get wrapped properly (or the loading of the wrapped function does not persist). Since wrapping open() is necessary for tracking stale wrapper fds, this can lead to failed writes in some situations. The test program attached simulates this with a scenario where socket() is followed up by an internal close() letting the fd leak. When the open() is not wrapped on 32bit, the following write() fails with ENOTCONN as open() returned the leaked fd. Comparing the relevant portion of the LD_DEBUG output between 32-bit and 64-bit: 

$ LD_PRELOAD=libsocket_wrapper.so SOCKET_WRAPPER_DIR=sockets SOCKET_WRAPPER_DEBUGLEVEL=4 LD_DEBUG=symbols ./test

On 32-bit:
      1525:     symbol=open;  lookup in file=./test [0]
      1525:     symbol=open;  lookup in file=/lib/libsocket_wrapper.so [0]
      1525:     symbol=open;  lookup in file=/lib/libdl.so.2 [0]
      1525:     symbol=open;  lookup in file=/lib/libc.so.6 [0]
open() returned 3
      1525:     symbol=strlen;  lookup in file=./test [0]

On 64-bit:
     21379:     symbol=open;  lookup in file=./test [0]
     21379:     symbol=open;  lookup in file=/lib64/libsocket_wrapper.so [0]
     21379:     binding file ./test [0] to /lib64/libsocket_wrapper.so [0]: normal symbol `open' [GLIBC_2.2.5]
     21379:     symbol=open;  lookup in file=/lib64/libc.so.6 [0]
     21379:     binding file /lib64/libc.so.6 [0] to /lib64/libc.so.6 [0]: normal symbol `open'
SWRAP_TRACE(21379) - _swrap_load_lib_function: Loaded open from libc
SWRAP_TRACE(21379) - swrap_remove_stale: remove stale wrapper for 3
open() returned 3

One thing maybe notable about the problem is that running it as "bash -c ./test" in the LD_DEBUG output on 32-bit there does appear to be a properly wrapped open() before control is passed to ./test. Any open() in ./test at that point on is not wrapped. However I suspect there may be some compile time optimization in socket_wrapper (or perhaps glibc) that is interfering, I'm just not sure which one it would be.

Linux test.example.com 4.8.6-300.fc25.i686+PAE #1 SMP Tue Nov 1 12:54:54 UTC 2016 i686 i686 i386 GNU/Linux

$ rpm -q --queryformat="%{OPTFLAGS}\n" socket_wrapper
-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables
$ rpm -q --queryformat="%{OPTFLAGS}\n" glibc
-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables
Comment 1 Florian Weimer 2017-03-16 06:57:47 UTC
The 32-bit version of libsocket_wrapper.so in Fedora does not define an open function, only an open64 function, so this may not be a symbol binding issue at all.
Comment 2 Matt Rogers 2017-03-17 18:04:41 UTC
Thanks for pointing that out! I found that undefining __USE_FILE_OFFSET64 before including <fcntl.h> in socket_wrapper.c made it export fopen() and open() instead and everything worked.

So for the 32-bit build I'm not sure if we should be wrapping open64() and fopen64() instead somehow, or just use the above hack.
Comment 3 Andreas Schneider 2017-03-17 18:12:26 UTC
I've tried to install F25 32bit in VirtualBox today but it doesn't the iso. I need to try qemu next week.
Comment 4 Andreas Schneider 2017-03-29 08:06:14 UTC
Thanks for the hint Florian. I have a patchset which fixes the issue. Will be released with socket_wrapper-1.2 soon.

We are adding support for threads.
Comment 5 Andreas Schneider 2017-03-29 08:06:57 UTC
Created attachment 13119 [details]
Comment 6 Andreas Schneider 2017-04-06 14:44:07 UTC
Fix has been pushed to master. Will be released with socket_wrapper 1.2.0 in the next days when we have threading support finished. I've also wrapped fopen64() ...