Bug 12160 - Samba client/server cannot manage special characters
Summary: Samba client/server cannot manage special characters
Status: RESOLVED DUPLICATE of bug 12830
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: File services (show other bugs)
Version: 4.4.5
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Samba QA Contact
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-08-18 00:34 UTC by JamesIsIn
Modified: 2017-11-11 03:29 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description JamesIsIn 2016-08-18 00:34:14 UTC
I was asked to file an up-stream bug based on testing with the latest build.  Here is my Ubuntu bug report:

https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1612000

A little background... I have a media server running Ubuntu 10.04 with a media share containing FLAC files in a pretty standard hierarchy. This server (in service since 2010) has been accessed by machines running from 10.04 to 14.04 without any issues.

I have recently built/upgraded two client machines to from 14.04 to 16.04 and these machines are having trouble with that Samba-served share. If a file or folder contains a question mark, that file or folder will not be accessible--though it will appear in Nautilus and in the Terminal.

In the case of a file (say 01 - Who Are You?.flac) if you double-click the file (or otherwise launch it) you will get a file not found error; this is true in the Terminal as well (using, say, ffplay). In the case of a folder it will appear to be empty and if you double-click the folder you will get a folder not found error. In either case, the question mark is clearly shown in both Nautilus and bash.

If I create a share on one of my Ubu 16.04 machines (with similar parameters) those files and folders which contain question marks do not appear on client machines (regardless of version) when that share is mounted. Interestingly, these 16.04 servers are also not able to share files or folders which contain colons.

This is all with mangled file names disabled. If I enable it, I get mangled file names which share as expected.

(Other characters may be effected, but these are the only two I have tested.)

In short, files and folders containing question marks and colons should appear in shares and be launchable. With 16.04 this is not the case.

From the 10.04 machine:

samba:
  Installed: 2:3.4.7~dfsg-1ubuntu3.15
  Candidate: 2:3.4.7~dfsg-1ubuntu3.15
  Version table:
 *** 2:3.4.7~dfsg-1ubuntu3.15 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid-updates/main Packages
        500 http://security.ubuntu.com/ubuntu/ lucid-security/main Packages
        100 /var/lib/dpkg/status
     2:3.4.7~dfsg-1ubuntu3 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages

From the 16.04 machine:

samba:
  Installed: 2:4.3.9+dfsg-0ubuntu0.16.04.2
  Candidate: 2:4.3.9+dfsg-0ubuntu0.16.04.2
  Version table:
 *** 2:4.3.9+dfsg-0ubuntu0.16.04.2 500
        500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 
Packages
        500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
        100 /var/lib/dpkg/status
     2:4.3.8+dfsg-0ubuntu1 500
        500 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 Packages

Subsequently I downloaded, compiled, and tested Samba 4.4.5 on a newly built Ubu 16.04 test machine.  The behaviors did not change.

I am happy to provide additional details or tests as requested.
Comment 1 JamesIsIn 2016-08-18 02:29:52 UTC
This is my results from first testing a new build and then from testing with the latest Samba (4.4.5).

I didn't want to mess with an existing machine so I built a new machine with Ubu 16.04 and updated it. I found I was able to mount the (10.04) server share through Nautilus without issue and play files containing question marks. However, when I added the share to fstab I was not able to interact with files/folders containing question marks via that mount point. Here is my line from fstab:

//server1/share2 /media/MountPoint2 cifs guest,iocharset=utf8 0 0

It's pretty basic. It's the same as I'm using in my 10.04 machines as well. I confirmed that the same is true on both of my running 16.04 machines (and not just this newly built test machine).

This has no impact on the same problem as reported from a 16.04 server to any client (obviously).

Is there something wrong with that line?

I added the latest upstream Samba to my test 16.04 machine using these instructions:

http://askubuntu.com/questions/813171/latest-upstream-samba-on-ubuntu-16-04-how

This appears to have had no appreciable effect on the performance of this machine to read files with question marks in their paths.

(The "connect to server" method through Nautilus from my previous post also still works on this machine.)
Comment 2 JamesIsIn 2016-10-03 18:27:58 UTC
Is there anything more I can do to help move this along?
Comment 3 George 2017-11-07 01:49:51 UTC
I also experienced this issue and found the problem described in the ArchLinux wiki. I'm pasting it here in case it helps someone else find a solution if the wiki changes ( https://wiki.archlinux.org/index.php/samba#Mapping_reserved_Windows_characters )

Mapping reserved Windows characters

Starting with kernel 3.18, the cifs module uses the "mapposix" option by default. When mounting a share using unix extensions and a default Samba configuration, files and directories containing one of the seven reserved Windows characters : \ * < > ? are listed but cannot be accessed.

Possible solutions are:

    Use the undocumented nomapposix mount option for cifs

 # mount.cifs //server/share /mnt/point -o nomapposix


I've also been told the catia solution is better but haven't taken the time to investigate: https://www.mankier.com/8/vfs_catia
Comment 4 George 2017-11-07 01:51:25 UTC
Here is the actual commit that caused the compatibility break and the discussion around it.

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2baa2682531ff02928e2d3904800696d9e7193db
Comment 5 Björn Jacke 2017-11-09 13:35:23 UTC
not a bug but a matter of correcting the configuration

*** This bug has been marked as a duplicate of bug 12830 ***
Comment 6 JamesIsIn 2017-11-10 07:41:19 UTC
(In reply to Björn Jacke from comment #5)

This is not a duplicate of 12380.  That bug (and its actual duplicate) concern the double quotation (").  Nowhere in this bug report is the double quotation mentioned and I have now tested the double quotation against my test machines and there is no related issue: Samba serves files with double quotation marks to clients using cifs (without nomapposix) without issue.

This bug concerns the question mark (?) and the colon (:) which are very common characters in mundane file shares the world over.  Samba serves the file paths, but it fails to serve the file contents when those characters are encountered.

The information provided by George surely points in the right direction toward locating and correcting the bug.

(Using nomapposix in the mount command, including via fstab, does allow the files to function on the client.  It will be useful as a workaround until this bug can be patched.)
Comment 7 Jeremy Allison 2017-11-10 18:37:47 UTC
This is a cifs client bug, not a server one. The server does not modify server stored paths IN ANY WAY when serving files out via the unix extensions.
Comment 8 JamesIsIn 2017-11-11 02:42:56 UTC
(In reply to Jeremy Allison from comment #7)

Using "mangled names" certainly does.
Comment 9 Jeremy Allison 2017-11-11 03:29:10 UTC
"mangled names" are not used when using unix extensions.