Bug 13646 - File saving issues with vfs_fruit on samba >= 4.8.5
File saving issues with vfs_fruit on samba >= 4.8.5
Status: RESOLVED FIXED
Product: Samba 4.1 and newer
Classification: Unclassified
Component: VFS Modules
4.8.5
All Linux
: P5 major
: ---
Assigned To: Karolin Seeger
Samba QA Contact
:
: 13636 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-10-06 10:37 UTC by thomas.schwark
Modified: 2018-12-11 12:02 UTC (History)
5 users (show)

See Also:


Attachments
samba log (4.46 KB, text/plain)
2018-10-06 10:37 UTC, thomas.schwark
no flags Details
Proposed patch for master backported to 4.8 (178.73 KB, patch)
2018-10-26 14:35 UTC, Ralph Böhme
no flags Details
Proposed patch for master backported to 4.8 (118.13 KB, patch)
2018-10-27 14:27 UTC, Ralph Böhme
no flags Details
Proposed patch for master backported to 4.9 (118.13 KB, patch)
2018-10-27 14:27 UTC, Ralph Böhme
no flags Details
Patch for 4.8 backported from master (94.97 KB, patch)
2018-11-01 12:57 UTC, Ralph Böhme
jra: review-
Details
Patch for 4.9 cherry-picked from master (93.07 KB, patch)
2018-11-01 12:57 UTC, Ralph Böhme
jra: review-
Details
Patch for 4.8 backported from master (94.18 KB, patch)
2018-11-01 20:09 UTC, Ralph Böhme
no flags Details
Patch for 4.9 cherry-picked from master (93.07 KB, patch)
2018-11-01 20:09 UTC, Ralph Böhme
no flags Details
Patch for 4.8 cherry-picked from master (96.07 KB, patch)
2018-11-02 09:35 UTC, Ralph Böhme
jra: review+
Details
Patch for 4.9 cherry-picked from master (94.18 KB, patch)
2018-11-02 09:36 UTC, Ralph Böhme
jra: review+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description thomas.schwark 2018-10-06 10:37:19 UTC
Created attachment 14514 [details]
samba log

Hey,

after updating to samba >= 4.8.5 (incl. 4.9.1) there are multiple issues with file saving and locking with vfs_fruit. The locking issue seems to be fixed in 4.9.1 but the file saving issues persist.

I´m using archlinux with hardened-kernel and all packages up-to-date as samba server and macOS High Sierra and / or Mojave as clients.

As an example to replicate the issue try creating a clean new document with Microsoft Excel (v. 16.17 on macOS) and save the file (the first time) on the samba volume. Excel opens a popup which claims the file has been saved but Excel could not open the file again. It says please close the file and open it again. But if you do that it results in a 0 KB file which is broken and the complete content is lost.

Downgrading to samba 4.8.4 fixes the issue. I attached the logfile that shows what happens when trying to save the new Excel file for the first time. Please let me know if you need anything else.

Additionally I´m happy about any suggestions for improving my smb.conf.

Here is the result from testparm:

Load smb config files from /etc/samba/smb.conf
rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
Processing section "[office]"
Processing section "[extra]"
Processing section "[scripts]"
Loaded services file OK.
Server role: ROLE_STANDALONE

Press enter to see a dump of your service definitions

# Global parameters
[global]
	dns proxy = No
	log file = /var/log/samba/samba.log.%m
	max log size = 50
	security = USER
	server multi channel support = Yes
	server role = standalone server
	unix extensions = No
	workgroup = LINUXSERVER
	fruit:model = Xserve
	idmap config * : backend = tdb
	mangled names = no


[office]
	case sensitive = Yes
	comment = office
	create mask = 0770
	delete veto files = Yes
	directory mask = 0770
	ea support = Yes
	force create mode = 0770
	force directory mode = 0770
	force group = +officesmbuser
	inherit owner = windows and unix
	map archive = No
	nt acl support = No
	path = /data/office
	read only = No
	store dos attributes = Yes
	valid users = @officesmbuser
	veto files = /.DS_Store/._.DS_Store/
	vfs objects = catia fruit streams_xattr recycle
	recycle:exclude_dir = _trash,*.sb-*
	recycle:exclude = ?~$*,~$*,._*,.smbdelete*,*.*.sb-*,*~*.idlk
	recycle:maxsize = 0
	recycle:versions = yes
	recycle:touch = yes
	recycle:keeptree = yes
	recycle:subdir_mode = 0770
	recycle:directory_mode = 0770
	recycle:repository = _trash/%U
	fruit:veto_appledouble = no
	fruit:encoding = native

Thanks for fixing in advance.

Kind regards

Tom
Comment 1 Ralph Böhme 2018-10-23 15:00:36 UTC
*** Bug 13636 has been marked as a duplicate of this bug. ***
Comment 2 Ralph Böhme 2018-10-26 14:35:40 UTC
Created attachment 14544 [details]
Proposed patch for master backported to 4.8

Here's a backport of the patchset that ought to fix this. I've already submitted this to the mailing list for review.

I would appreciate if folks affected by this could this with this patch. Note that this also includes the patches for bugs 13642 and 13649 to make backporting easier.

Anyone?
Comment 3 thomas.schwark 2018-10-27 11:09:09 UTC
Thanks for all your hard work on this, very much appreciated.

Are all the latest vfs_fruit patches already included in the 4.9 branch?

As archlinux is always using the latest version I would be interested and happily helping in testing the 4.9 branch. 

Unfortunately there does not seem to be a "samba-latest-test" package in the archlinux aur. I don`t know how to download, build and install the latest test version manually.

If you could provide a PKGBUILD in the aur or a bash script to do that I`ll happily try, test and report back. I can then also try the 4.8 branch of course as it follows the same install process I guess.

Thanks in advance
Comment 4 Ralph Böhme 2018-10-27 14:27:19 UTC
Created attachment 14545 [details]
Proposed patch for master backported to 4.8
Comment 5 Ralph Böhme 2018-10-27 14:27:57 UTC
Created attachment 14546 [details]
Proposed patch for master backported to 4.9

Patchset for 4.9.
Comment 6 Ralph Böhme 2018-11-01 12:57:01 UTC
Created attachment 14556 [details]
Patch for 4.8 backported from master

Karolin, please apply the backport for following dependent bugs in this order:

bug 13649
bug 13642 
bug 13646 (this one)
Comment 7 Ralph Böhme 2018-11-01 12:57:45 UTC
Created attachment 14557 [details]
Patch for 4.9 cherry-picked from master

Karolin, please apply the backport for following dependent bugs in this order:

bug 13649
bug 13642 
bug 13646 (this one)
Comment 8 Jeremy Allison 2018-11-01 19:35:22 UTC
Comment on attachment 14557 [details]
Patch for 4.9 cherry-picked from master

Ralph, can you add in / squash master git ref 9b28d47b0d86570be5a7c5628e460e01207afb00 to this patchset please ?

Author: Volker Lendecke <vl@samba.org>
Date:   Thu Nov 1 12:09:14 2018 +0100

    torture: Fix the clang build
    
    It's used uninitialized if an early torture_assert fails
    
    Signed-off-by: Volker Lendecke <vl@samba.org>
    Reviewed-by: Ralph Boehme <slow@samba.org>
Comment 9 Jeremy Allison 2018-11-01 19:36:38 UTC
Comment on attachment 14556 [details]
Patch for 4.8 backported from master

Ralph, can you add in / squash master git ref 9b28d47b0d86570be5a7c5628e460e01207afb00 to this patchset please ?
Comment 10 Ralph Böhme 2018-11-01 20:09:28 UTC
Created attachment 14558 [details]
Patch for 4.8 backported from master
Comment 11 Ralph Böhme 2018-11-01 20:09:58 UTC
Created attachment 14559 [details]
Patch for 4.9 cherry-picked from master
Comment 12 Ralph Böhme 2018-11-02 09:35:41 UTC
Created attachment 14561 [details]
Patch for 4.8 cherry-picked from master
Comment 13 Ralph Böhme 2018-11-02 09:36:09 UTC
Created attachment 14562 [details]
Patch for 4.9 cherry-picked from master
Comment 14 Jeremy Allison 2018-11-02 17:10:21 UTC
Re-assigning to Karolin for inclusion in 4.8.next, 4.9.next.
Comment 15 Karolin Seeger 2018-11-06 08:10:35 UTC
(In reply to Jeremy Allison from comment #14)
Pushed to autobuild-v4-{9,8}-test.
Comment 16 Karolin Seeger 2018-11-07 07:45:47 UTC
(In reply to Karolin Seeger from comment #15)
Pushed to both branches.
Closing out bug report.

Thanks!
Comment 17 thomas.schwark 2018-11-07 11:57:05 UTC
Thank you all for your help. I will try the fixes as soon as samba 4.9.2 will be available in the archlinux repos.
Comment 18 thomas.schwark 2018-11-20 08:51:23 UTC
Thanks a lot. This is resolved in 4.9.2
Comment 19 ryan 2018-12-11 10:15:24 UTC
Hi all,

The patch appears to fail against 4.8.6 and 4.8.7 and it doesn't appear in the release notes for 4.8.6 or 4.8.7:

patch -p1 <smbpatch.diff
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 2044 (offset -257 lines).
Hunk #2 succeeded at 2101 (offset -257 lines).
Hunk #3 succeeded at 2117 (offset -257 lines).
patching file source3/modules/vfs_streams_xattr.c
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 1970 (offset -141 lines).
Hunk #2 FAILED at 2190.
Hunk #3 succeeded at 4490 (offset -257 lines).
1 out of 3 hunks FAILED -- saving rejects to file source4/torture/vfs/fruit.c.rej
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 3413 (offset -262 lines).
Hunk #2 succeeded at 3424 (offset -262 lines).
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 2791 (offset -262 lines).
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 2044 (offset -262 lines).
Hunk #2 succeeded at 2163 (offset -262 lines).
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 2870 (offset -262 lines).
Hunk #2 succeeded at 2887 (offset -262 lines).
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 2970 (offset -262 lines).
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 3917 (offset -262 lines).
Hunk #2 succeeded at 3969 (offset -262 lines).
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 3188 (offset -262 lines).
Hunk #2 succeeded at 3233 (offset -262 lines).
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 3829 (offset -262 lines).
Hunk #2 succeeded at 3869 (offset -262 lines).
patching file selftest/knownfail.d/samba3.vfs.fruit
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file selftest/knownfail.d/samba3.vfs.fruit.rej
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 1904 (offset -141 lines).
patching file selftest/knownfail.d/samba3.vfs.fruit
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file selftest/knownfail.d/samba3.vfs.fruit.rej
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 3312 (offset -263 lines).
Hunk #2 succeeded at 3401 (offset -263 lines).
patching file selftest/knownfail.d/samba3.vfs.fruit
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file selftest/knownfail.d/samba3.vfs.fruit.rej
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 4719 (offset -262 lines).
Hunk #2 succeeded at 4823 (offset -262 lines).
Hunk #3 succeeded at 4894 (offset -262 lines).
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 5019 (offset -263 lines).
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 5528 (offset -263 lines).
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 1669 (offset -141 lines).
Hunk #2 succeeded at 1790 (offset -141 lines).
Hunk #3 succeeded at 3679 (offset -262 lines).
Hunk #4 succeeded at 4648 (offset -262 lines).
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 5604 (offset -263 lines).
patching file selftest/knownfail.d/samba3.vfs.fruit
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file selftest/knownfail.d/samba3.vfs.fruit.rej
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 5315 (offset -263 lines).
Hunk #2 succeeded at 5366 (offset -263 lines).
Hunk #3 succeeded at 5637 (offset -263 lines).
patching file source4/torture/smb2/util.c
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 2820 (offset -262 lines).
patching file selftest/knownfail.d/samba3.vfs.fruit
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file selftest/knownfail.d/samba3.vfs.fruit.rej
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 2820 (offset -262 lines).
Hunk #2 succeeded at 2879 (offset -262 lines).
Hunk #3 succeeded at 4997 (offset -262 lines).
Hunk #4 FAILED at 5912.
1 out of 4 hunks FAILED -- saving rejects to file source4/torture/vfs/fruit.c.rej
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 2111 (offset -263 lines).
Hunk #2 succeeded at 2132 (offset -263 lines).
Hunk #3 succeeded at 4536 (offset -263 lines).
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 5453 (offset -263 lines).
Hunk #2 succeeded at 5500 (offset -263 lines).
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 471 (offset -43 lines).
Hunk #2 succeeded at 5600 (offset -263 lines).
Hunk #3 succeeded at 5651 (offset -263 lines).
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 474 (offset -43 lines).
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 4222 (offset -263 lines).
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 3931 (offset -263 lines).
Hunk #2 succeeded at 4030 (offset -263 lines).
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 4709 (offset -169 lines).
Hunk #2 succeeded at 4724 (offset -169 lines).
Hunk #3 succeeded at 4885 (offset -263 lines).
Hunk #4 succeeded at 5154 (offset -263 lines).
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 4143 (offset -263 lines).
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 5069 (offset -263 lines).
Hunk #2 succeeded at 5081 (offset -263 lines).
Hunk #3 succeeded at 5138 (offset -263 lines).
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 4281 (offset -263 lines).
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 5119 (offset -263 lines).
Hunk #2 succeeded at 5143 (offset -263 lines).
patching file selftest/knownfail.d/samba3.vfs.fruit
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file selftest/knownfail.d/samba3.vfs.fruit.rej
patching file source3/modules/vfs_fruit.c
Hunk #1 succeeded at 3140 (offset -263 lines).
Hunk #2 succeeded at 3210 (offset -263 lines).
Hunk #3 succeeded at 3254 (offset -263 lines).
Hunk #4 succeeded at 3278 (offset -263 lines).
patching file source4/torture/vfs/fruit.c
Hunk #1 succeeded at 5199 (offset -262 lines).
Comment 20 Karolin Seeger 2018-12-11 10:22:56 UTC
(In reply to ryan from comment #19)
The patch will be included in the next 4.8 bugfix release. It has been pushed long ago, but a security release crossed the road...
Will be available soon
Comment 21 ryan 2018-12-11 10:51:38 UTC
(In reply to Karolin Seeger from comment #20)
Great thanks. Is there a working patch I can apply in the mean time, or was this patch against 4.8.5?
Comment 22 ryan 2018-12-11 12:02:09 UTC
(In reply to ryan from comment #21)
Sorry just seen this, will patch in order and test again:

Karolin, please apply the backport for following dependent bugs in this order:

bug 13649
bug 13642 
bug 13646 (this one)