Bug 10683 - Files save from OSX Photoshop seems to concatenate resource and data forks
Files save from OSX Photoshop seems to concatenate resource and data forks
Status: NEW
Product: Samba 4.1 and newer
Classification: Unclassified
Component: File services
4.1.6
x64 FreeBSD
: P5 major
: ---
Assigned To: Samba QA Contact
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-07-01 16:29 UTC by Scott Call
Modified: 2014-11-19 13:06 UTC (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Scott Call 2014-07-01 16:29:50 UTC
We recently switched from a proprietary SMB server to one based on Samba (4.1.6 specifically, with 4.1.9 coming soon).

IT received a report of corrupted PSD (Photoshop) files on the server.

I examined both the correct files (saved to the old server) and the corrupted ones (saved to the new) and found that the corrupted files contained a JFIF image (icon) concatenated with the proper data from the file.

It appears that the resource fork and the data fork (in Mac nomenclature) were appended together.

There are several known issues in the Apple (non-samba) implementation of the SMB2 protocol, especially, it appears, in concert with Adobe's Creative Suite,  and this may be completely on their side, but I wanted to report it in case Samba can be made to place nice with Apple.

As a side note, forcing the clients to "NT1" as the max protocol version seems, in limited testing, to avoid this issue, but we also give up a lot of the features of SMB2.

Thanks
Comment 1 Ralph Böhme 2014-07-03 09:34:01 UTC
Hi Scott

I'm sure Samba can do better and play quite ok with OS X if configured correctly, but there's of course always room for improvement. :)

Much depends on how Samba is configured wrt to supporting alternate data streams (ADS) or not, which means it depends on the exact VFS modules you configured in smb.conf.

The default Samba setup ootb will result in not supporting ADS, which is currently still the most sensible configuration for dealing with OS X SMB clients. I've written a new VFS module from scratch in the past months which will hopefully make it into Samba 4.2, this should help in this regard as it will be the only VFS module supporting ADS in a way that works with OS X clients.

(In reply to comment #0)
> We recently switched from a proprietary SMB server to one based on Samba (4.1.6
> specifically, with 4.1.9 coming soon).
> 
> IT received a report of corrupted PSD (Photoshop) files on the server.
> 
> I examined both the correct files (saved to the old server) and the corrupted
> ones (saved to the new) and found that the corrupted files contained a JFIF
> image (icon) concatenated with the proper data from the file.
>
> It appears that the resource fork and the data fork (in Mac nomenclature) were
> appended together.

In one file? I've seen many issues wrt to corrupted Mac metadata and resource forks, but this one is new.

> There are several known issues in the Apple (non-samba) implementation of the
> SMB2 protocol, especially, it appears, in concert with Adobe's Creative Suite, 
> and this may be completely on their side, but I wanted to report it in case
> Samba can be made to place nice with Apple.

As mentioned, we're working on it. :)
<https://lists.samba.org/archive/samba-technical/2014-June/100715.html>

> As a side note, forcing the clients to "NT1" as the max protocol version seems,
> in limited testing, to avoid this issue, but we also give up a lot of the
> features of SMB2.

Hm, that's somewhat strange. What VFS modules are you using? Can you show us your smb.conf?

Cheerio!
-r
Comment 2 Scott Call 2014-07-07 17:28:30 UTC
Unfortunately I did not keep the various iterations of the config file.   We are running Freenas, so most of the config is generated from them.

I did have some optimizations (socket options, with buffer sizes, etc) that I had found recommended online, and have removed as part of the debugging process.

The only other changes that we did were the min/max protocol NT1.

We also pushed out a nsmb.conf file for the mac clients with:
[default]
streams=no
smb_neg=smb1_only

This is the smb config that seems to be stable:
[global]
    server min protocol = NT1
    server max protocol = NT1
    encrypt passwords = yes
    dns proxy = no
    strict locking = no
    oplocks = yes
    deadtime = 15
    max log size = 51200
    max open files = 11070
    load printers = no
    printing = bsd
    printcap name = /dev/null
    disable spoolss = yes
    getwd cache = yes
    guest account = nobody
    map to guest = Bad User
    obey pam restrictions = Yes
    directory name cache size = 0
    kernel change notify = no
    panic action = /usr/local/libexec/samba/samba-backtrace
    server string = FreeNAS Server
    ea support = yes
    store dos attributes = yes
    map archive = no
    map readonly = no
    map hidden = no
    map system = no
    unix extensions = no
    time server = yes
    acl allow execute always = false
    server role = member server
    netbios name = JUPITER
    workgroup = DOMAIN
    realm = AD.DOMAIN.COM
    security = ADS
    client use spnego = yes
    cache directory = /var/tmp/.cache/.samba
    local master = no
    domain master = no
    preferred master = no
    acl check permissions = true
    acl map full control = true
    dos filemode = yes
    idmap config *:backend = tdb
    idmap config *:range = 90000000-100000000
    winbind cache time = 7200
    winbind offline logon = yes
    winbind enum users = yes
    winbind enum groups = yes
    winbind nested groups = yes
    winbind use default domain = no
    winbind refresh tickets = yes
    winbind nss info = rfc2307
    idmap config DOMAIN: backend = ad
    idmap config DOMAIN: schema_mode = rfc2307
    idmap config DOMAIN: range = 10000-90000000
    allow trusted domains = no
    template shell = /bin/sh
    template homedir = /home/%D/%U
    pid directory = /var/run/samba
    smb passwd file = /var/etc/private/smbpasswd
    private dir = /var/etc/private
    create mask = 0664
    directory mask = 0775
    client ntlmv2 auth = yes
    dos charset = CP437
    unix charset = UTF-8
    log level = 1
    idmap config DOMAIN: range = 1000-90000000
    reset on zero vc = yes
    force create mode = 0664
    force directory mode = 0775


[departments]
    path = /mnt/vol1/departments/Departments
    printable = no
    veto files = /.snap/.windows/.zfs/
    writeable = yes
    browseable = yes
    inherit owner = no
    inherit permissions = no
    recycle:repository = .recycle/%U
    recycle:keeptree = yes
    recycle:versions = yes
    recycle:touch = yes
    recycle:directory_mode = 0777
    recycle:subdir_mode = 0700
    shadow:snapdir = .zfs/snapshot
    shadow:sort = desc
    shadow:localtime = yes
    shadow:format = auto-%Y%m%d.%H%M-1w
    vfs objects = shadow_copy2 zfsacl streams_xattr aio_pthread
    hide dot files = yes
    guest ok = no
    inherit acls = yes
    nfs4:mode = special
    nfs4:acedup = merge
    nfs4:chown = yes
    zfsacl:acesort = dontcare
    veto oplock files = /*.doc/*.DOC/*.xls/*.XLS/*.xlsx/*.XLSX/*.docx/*.DOCX
Comment 3 Ralph Böhme 2014-07-08 10:55:38 UTC
(In reply to comment #2)
> Unfortunately I did not keep the various iterations of the config file.   We
> are running Freenas, so most of the config is generated from them.
> 
> I did have some optimizations (socket options, with buffer sizes, etc) that I
> had found recommended online, and have removed as part of the debugging
> process.
> 
> The only other changes that we did were the min/max protocol NT1.
> 
> We also pushed out a nsmb.conf file for the mac clients with:
> [default]
> streams=no

so you're trying to force disabled streams support. That means the OS X SMB client would use special named AppleDouble files for maintaining OS X metadata and resource fork. These files use the name of the basefile prefixed by "._".

These ._ AppleDouble files (as well as the plain basefile) are maintained, read and written by the client, the server doesn't perform any special things with them.

When you get the corrupted files, do you have a ._FILE in the same directory too? Can you upload a copy of both a FILE and it's ._FILE counterpart, I'd be most curious to see their content.

Forcing streams=no on the client is only necessary if you load VFS modules on the server that enable stream support like...

> [departments]
>     ....
>     vfs objects = shadow_copy2 zfsacl streams_xattr aio_pthread

... "streams_xattr". Unfortunately streams_xattr is not suitable as streams module for OS X SMB clients, because filesystem xattrs on FreeBSD are (iirc) limited in size and the OS X resource fork stream may be of arbitrary size. If this is the default FreeNAS configuration for OS X clients, I'd consider that a possible bug in FreeNAS. FreeNAS needs a VFS module like vfs_fruit for proper streams support for OS X systems, alternatively they could use vfs_depot, but that may have other subtle ramifications.