Bug 11717 - Renames not reflected on Mac OS X client using 10.11 (El Capitan)
Summary: Renames not reflected on Mac OS X client using 10.11 (El Capitan)
Status: RESOLVED FIXED
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: File services (show other bugs)
Version: 4.2.7
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Ralph Böhme
QA Contact: Samba QA Contact
URL:
Keywords:
: 12156 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-02-05 14:50 UTC by John Mulligan (dead mail address)
Modified: 2021-04-26 22:07 UTC (History)
5 users (show)

See Also:


Attachments
pcap with samba as server (39.69 KB, application/x-xz)
2016-02-05 14:50 UTC, John Mulligan (dead mail address)
no flags Details
pcap with windows 8 as server (24.12 KB, application/x-xz)
2016-02-05 14:50 UTC, John Mulligan (dead mail address)
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description John Mulligan (dead mail address) 2016-02-05 14:50:18 UTC
Created attachment 11809 [details]
pcap with samba as server

I think that this issue, or one very similar, has been reported on the mailing list as well, here: 
  https://lists.samba.org/archive/samba-technical/2016-January/111875.html
I felt that it also deserved a bug for tracking purposes.

When using Mac OS X clients via finder, those running 10.11 (El Capitan) do not appear to "see" file or directory renames performed by another client. Renames are reflected on 10.10 (Yosemite) clients, even when the rename is performed by an 10.11 based client. This problem does not occur when using Windows (Windows 8 in my testing) as a server.

I am attaching tcpdumps of a failing session between clients and samba and a working session with the Windows 8 client. Samba was _not_ using vfs_fruit in this test. This test was using Samba 4.2, however samba 4.3 had the same result.

Please let me know if there is any other information I can provide.
Comment 1 John Mulligan (dead mail address) 2016-02-05 14:50:55 UTC
Created attachment 11810 [details]
pcap with windows 8 as server
Comment 2 mvermaes 2016-02-22 09:58:40 UTC
We are seeing what seems to be the same issue against samba-4.2.3-11.el7_2.x86_64 provided by CentOS 7.2. These are our first deployments of 4.x, but 3.0 and 3.6 versions of Samba did not have this issue. We also don't see this behaviour against Win2K12 or a Synology NAS running Samba 4.1.

As John mentioned, it is only the graphical 'Finder' application that is affected. Navigating to the mounted folder using the OS X Terminal app shows that the El Capitan client can see the renamed folder there.

The only workaround we have found for this currently is to set 'server max protocol = NT1' on the server (or use cifs:// when connecting from the El Capitan client). This seems to explain why we did not see this problem in our Samba 3.x installs, where the default value of server max protocol was NT1.

If I get some time, I will have a go at compiling Samba 4.4 and see if it behaves differently, as suggested on the mailing list.
Comment 3 Ralph Böhme 2016-02-22 11:56:59 UTC
Looks like this is a client bug that affects only shares without support for named streams. Network trace clearly show that the client received change notifications.

I've reported this to Apple, radar #24717318.
Comment 4 John Mulligan (dead mail address) 2016-02-22 15:37:11 UTC
I've discussed this a bit with Ralph outside of the bug and there are two fairly low-hassle workarounds that help deal with the issue.

1. As Ralph says this affects shares without support for named streams, so you can try enabling named streams for your share by adding 'fruit streams_xattr' to the 'vfs objects' section of your share configuration.

You then need to disconnect your client from the share and reconnect and retest.


2. On the Mac client edit/create the file /etc/nsmb.conf and make it contain:
[default]
file_ids_off=yes

Then reboot your client and retest.
Comment 5 Christoph Olszowka 2016-06-30 12:16:49 UTC
We were bitten recently by this issue too, after upgrading from an Ubuntu 12/3.6 installation to 16.04 which ships with 4.3. 

On El Capitan, file and folder renames did not propagate on the 4.3, while they do perfectly well on 3.6.

I've set up a minimal example based on Docker that makes it easy to reproduce the issue: https://github.com/protonet/samba-osx-rename-issue-demo

I would strongly suggest to have an actual on-by-default fix for this inside Samba because this is provably a regression from older versions, and while this seems to play together with a change in OS X, since on 3.6 it is working fine as a user upgrading my Samba install I would not expect an update to break things client-side.

Furthermore, this issue is pretty subtle to detect if you are unaware of it, and incredibly hard to google for. As Ubuntu 12.04, which comes with the working 3.6 version, is going out of support in 2017, I'd expect more users to be moving towards the 4+ releases over the next year, potentially running into this issue and causing plenty of headscratching, so having a fix in Samba by default really would make sense imho.

Thanks!
Comment 6 Ralph Böhme 2016-06-30 12:28:53 UTC
Ah, with 3.6 the connection is probably over SMB1, right? And with 4.3 SMB2 or higher. That's another variable in the mix that could explain the difference in behaviour with a 3.6 based Samba against a 4.3.

I remember discussing a gross hack to disable file ids for OS X clients with a vendor, but we they lost interest in this by simply using vfs_fruit and vfs_streams_xattr.
Comment 7 Christoph Olszowka 2016-06-30 13:53:37 UTC
(In reply to Ralph Böhme from comment #6)

Maybe it'd make sense to make the workaround the default instead of opt-in only after bumping into this problem? I have no idea what the implications of that would be for existing installations though, i.e. if stuff would could break if you enable it by default
Comment 8 Ralph Böhme 2016-06-30 14:10:47 UTC
(In reply to Christoph Olszowka from comment #7)
> I have no idea what the implications of that would be for existing installations 
> though, i.e. if stuff would could break if you enable it by default

That's the problem. It can have unknown side effects. So as long as nobody invests some serious time into this area this won't change.
Comment 9 mvermaes 2016-07-05 08:44:51 UTC
Just wanted to add that, per comment 4, adding 'vfs objects = fruit streams_xattr' to our shares resolved the rename issue for us. Unfortunately, this led to another issue: When copying a directory from an El Capitan client to the Samba 4.2.10 server, the user's execute permission is removed, and they are unable to access it. Would that be an expected side-effect of this setting?

We also tested by removing vfs_fruitt/treams_xattr from the share, and adding '... file_ids_off=yes' to nsmb.conf. This also resolves the rename issue, and does not result in the removal of the user's execute permission.

I'm wondering if anyone knows if the above is expected behaviour? We'd like to have a bit better understanding of the implications of setting file_ids_off=yes and not enabling vfs_fruit/streams_xattr, before we consider making that our default setting.

I realize that there might be something else in our environment that is resulting in the permission issue, so let me know if it's better for me to open that as a separate issue, or if you need more details about our setup.

Thanks for any suggestions.
Comment 10 Christoph Olszowka 2016-07-19 07:59:04 UTC
I can somewhat confirm what Michael reported above. With the `vfs objects` workaround, creating directories on a Samba share via OS X resulted in wrong permissions set on the directory, even though we have the relevant directory mask settings configured. I did not try the file ids off workaround, we went back to Samba 3.6 for the time being, where all of these issues do not apply :/
Comment 11 Ralph Böhme 2016-08-19 14:36:59 UTC
*** Bug 12156 has been marked as a duplicate of this bug. ***
Comment 12 Marcel Bischoff 2016-08-19 15:07:25 UTC
Since my bug report was closed as a duplicate (and I can see why, didn't find it though), I'm asking here:

Is there currently any reliable way to have a Linux-based file server that works reliably with Mac clients including working Spotlight indexing (or anything other than abysmally slow searching), providing decent performance? The more I look around the more problems I find. There is apparently still no way to do this with SMB (despite Apple having deprecated the protocol), which means for now, the only solution would be to use AFP exclusively via Netatalk, correct?

It confounds me to think that it's 2016 and we apparently still have no protocol to successfully operate a user-facing file server in a stable and reliable manner across platforms.
Comment 13 Ralph Böhme 2016-08-19 15:20:11 UTC
There is: Samba with vfs_fruit (and vfs_streams_xattr, cf man vfs_fruit). :) It even supports Spotlight by leveraging Gnome Tracker.
Comment 14 Marcel Bischoff 2016-08-20 17:27:55 UTC
Thanks Ralph, I'd read up on that as well but did not know about the Tracker integration.

Tried my best following the guide at https://wiki.samba.org/index.php/Spotlight but couldn't get it to work. The SMB share just won't respond to searches.
Comment 15 Ralph Böhme 2017-08-25 12:30:37 UTC
(In reply to Ralph Böhme from comment #3)
Fwiw, a while ago I got a followup from Apple stating they believe the issue would be fixed in 10.12.
Comment 16 Björn Jacke 2021-03-15 13:57:54 UTC
so this was a client bug, right? Is the client fixed in the meantime?
In any case this samba bug report can be closed?
Comment 17 Björn Jacke 2021-04-26 22:07:08 UTC
no answer, to my question, closing accordingly now.