Bug 13847 - Regression: Samba 4.10 fails to install
Regression: Samba 4.10 fails to install
Status: RESOLVED FIXED
Product: Samba 4.1 and newer
Classification: Unclassified
Component: Build
4.10.0
All All
: P5 normal
: ---
Assigned To: Karolin Seeger
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2019-03-20 10:13 UTC by Andreas Schneider
Modified: 2019-03-28 08:13 UTC (History)
4 users (show)

See Also:


Attachments
patch for 4.10 (1.19 KB, patch)
2019-03-20 14:02 UTC, Andreas Schneider
ab: review+
Details
patch for 4.9 (1.19 KB, patch)
2019-03-20 14:02 UTC, Andreas Schneider
ab: review+
Details
patch for 4.8 (1.17 KB, patch)
2019-03-20 14:04 UTC, Andreas Schneider
ab: review+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andreas Schneider 2019-03-20 10:13:09 UTC
'make install' fails on Samba 4.10!

+ install /builddir/build/BUILDROOT/samba-4.10.0-4.fc30.x86_64/usr/lib64/python3.7/site-packages/samba/dcerpc/idmap.cpython-37m-x86_64-linux-gnu.so (from bin/default/source4/librpc/python-idmap.inst.cpython-37m-x86_64-linux-gnu.so)
+ install /builddir/build/BUILDROOT/samba-4.10.0-4.fc30.x86_64/usr/lib64/python3.7/site-packages/samba/dcerpc/idmap.cpython-37m-x86_64-linux-gnu.so (from bin/default/source4/librpc/python-dcerpc-idmap.inst.cpython-37m-x86_64-linux-gnu.so)
Waf: Leaving directory `/builddir/build/BUILD/samba-4.10.0/bin/default'
BUILDSTDERR: Build failed
BUILDSTDERR: Traceback (most recent call last):
BUILDSTDERR:   File "/builddir/build/BUILD/samba-4.10.0/third_party/waf/waflib/Build.py", line 1162, in do_install
BUILDSTDERR:     self.copy_fun(src, tgt)
BUILDSTDERR:   File "/builddir/build/BUILD/samba-4.10.0/third_party/waf/waflib/Build.py", line 1066, in copy_fun
BUILDSTDERR:     shutil.copy2(src, tgt)
BUILDSTDERR:   File "/usr/lib64/python3.7/shutil.py", line 264, in copy2
BUILDSTDERR:     copystat(src, dst, follow_symlinks=follow_symlinks)
BUILDSTDERR:   File "/usr/lib64/python3.7/shutil.py", line 205, in copystat
BUILDSTDERR:     follow_symlinks=follow)
BUILDSTDERR: FileNotFoundError: [Errno 2] No such file or directory
BUILDSTDERR: During handling of the above exception, another exception occurred:
BUILDSTDERR: Traceback (most recent call last):
BUILDSTDERR:   File "/builddir/build/BUILD/samba-4.10.0/third_party/waf/waflib/Task.py", line 320, in process
BUILDSTDERR:     ret = self.run()
BUILDSTDERR:   File "/builddir/build/BUILD/samba-4.10.0/third_party/waf/waflib/Build.py", line 1101, in run
BUILDSTDERR:     fun(x.abspath(), y.abspath(), x.path_from(launch_node))
BUILDSTDERR:   File "/builddir/build/BUILD/samba-4.10.0/third_party/waf/waflib/Build.py", line 1168, in do_install
BUILDSTDERR:     raise Errors.WafError('Could not install the file %r' % tgt, e)
BUILDSTDERR: waflib.Errors.WafError: Could not install the file '/builddir/build/BUILDROOT/samba-4.10.0-4.fc30.x86_64/usr/lib64/python3.7/site-packages/samba/dcerpc/idmap.cpython-37m-x86_64-linux-gnu.so'
BUILDSTDERR: make: *** [Makefile:10: install] Error 1
BUILDSTDERR: error: Bad exit status from /var/tmp/rpm-tmp.FBqF0W (%install)
BUILDSTDERR:     Bad exit status from /var/tmp/rpm-tmp.FBqF0W (%install)


Patch will follow.
Comment 1 Andreas Schneider 2019-03-20 14:02:22 UTC
Created attachment 14957 [details]
patch for 4.10
Comment 2 Andreas Schneider 2019-03-20 14:02:50 UTC
Created attachment 14958 [details]
patch for 4.9
Comment 3 Andreas Schneider 2019-03-20 14:04:20 UTC
Created attachment 14959 [details]
patch for 4.8
Comment 4 Andreas Schneider 2019-03-20 17:30:44 UTC
Karolin, please apply the patches to the relevant branches. Thanks!
Comment 5 Alexander Bokovoy 2019-03-20 20:21:27 UTC
For those interested in the root cause, the bug is due to a combination of factors:
- two different waf subsystems defined the same installable target
- WAF install method uses shutil.copy2() Python method. This method copies a file and then copies attributes of the file. The copying process is done in Python and may take time
- WAF 2.x paralleled execution of install methods

On x86_64 in Fedora build system we generally get faster machines with more CPU resources available. It looks like two of these parallel installation jobs overlapped -- while one copied the file, the other one started to override the file. As result, a code that tried to run os.utime() against the destination path appeared in a situation when previously copied file was being overwritten by the parallel job and looked as not-existing one to this job. As result, os.utime() threw FileNotFoundError exception and broke.
Comment 6 Andrew Bartlett 2019-03-20 21:50:16 UTC
(In reply to Alexander Bokovoy from comment #5)
Wow!  Nice debugging!
Comment 7 Stefan Metzmacher 2019-03-21 10:58:27 UTC
So, avoid passing '-j' to make install would also avoid the problem and
only produce a warning, correct?
Comment 8 Alexander Bokovoy 2019-03-21 11:03:50 UTC
(In reply to Stefan Metzmacher from comment #7)
> So, avoid passing '-j' to make install would also avoid the problem and
> only produce a warning, correct?

Yes, that would be a workaround, at an expense of a slower build.

I think a better solution would be to add a WAF check to make sure there are no overlapping output destination targets.
Comment 9 Karolin Seeger 2019-03-21 12:44:04 UTC
(In reply to Alexander Bokovoy from comment #8)
Pushed to autobuild-v4-{10,9,8}-test.
Comment 10 Karolin Seeger 2019-03-28 08:13:35 UTC
(In reply to Karolin Seeger from comment #9)
Pushed to all branches.
Closing out bug report.

Thanks!