Bug 13107 - Missing directories with smbv2: with smbv1, they show up
Missing directories with smbv2: with smbv1, they show up
Status: NEW
Product: CifsVFS
Classification: Unclassified
Component: kernel fs
3.x
All All
: P5 normal
: ---
Assigned To: Steve French
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-10-27 13:31 UTC by Andreas Hasenack
Modified: 2018-04-12 06:01 UTC (History)
3 users (show)

See Also:


Attachments
possible fix - needs more testing (2.19 KB, patch)
2018-04-12 05:59 UTC, Steve French
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andreas Hasenack 2017-10-27 13:31:05 UTC
An ubuntu user filed a bug (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1572132) showing that when he uses the SMB2 protocol to mount a share exported by a windows 7 system, a couple of directories do not show up on the linux side.

I reproduced the issue using a windows 7 vm as the server, and with SMB2 two directories don't show up in the share. If I mount with version SMB1 (can't remember now if I had to explicitly enable smb1 in the windows 7 server), then the missing content shows up just fine.

Here is my test:
Curious indeed. I can reproduce this on xenial mounting a share from a windows 7 enterprise VM:

With vers 2.1 (same with 2.0):
root@nsn7:~# l /c
total 3,0G
drwxr-xr-x 2 andreas andreas 4,0K Jun 12 16:58 .
drwxr-xr-x 33 root root 4,0K Jun 27 15:56 ..
-rwxr-xr-x 1 andreas andreas 24 Jun 10 2009 autoexec.bat
-rwxr-xr-x 1 andreas andreas 10 Jun 10 2009 config.sys
drwxr-xr-x 2 andreas andreas 4,0K Out 23 2013 Documents and Settings
-rwxr-xr-x 1 andreas andreas 3,0G Jun 27 15:45 pagefile.sys
drwxr-xr-x 2 andreas andreas 0 Jul 13 2009 PerfLogs
drwxr-xr-x 2 andreas andreas 0 Jun 12 16:10 ProgramData
dr-xr-xr-x 2 andreas andreas 0 Jun 12 21:05 Program Files
drwxr-xr-x 2 andreas andreas 0 Out 23 2013 Recovery
drwxr-xr-x 2 andreas andreas 0 Nov 26 2014 System Volume Information
dr-xr-xr-x 2 andreas andreas 0 Out 23 2013 Users
drwxr-xr-x 2 andreas andreas 0 Out 23 2013 Wallpaper
drwxr-xr-x 2 andreas andreas 0 Jun 12 16:53 Windows
root@nsn7:~# mount|grep cifs
//10.0.7.128/c on /c type cifs (rw,relatime,vers=2.1,sec=ntlmssp,cache=strict,username=IEUser,domain=WORKGROUP,uid=1000,forceuid,gid=1000,forcegid,addr=10.0.7.128,file_mode=0755,dir_mode=0755,nounix,mapposix,rsize=1048576,wsize=1048576,actimeo=1)

Without the vers= parameter we get two extra directories: "Arquivos de Programas RFB" and "$Recycle.Bin":
root@nsn7:~# l /c
total 3,0G
drwxr-xr-x 2 andreas andreas 4,0K Jun 12 16:58 .
drwxr-xr-x 33 root root 4,0K Jun 27 15:56 ..
drwxr-xr-x 2 andreas andreas 0 Jun 12 20:52 Arquivos de Programas RFB
-rwxr-xr-x 1 andreas andreas 24 Jun 10 2009 autoexec.bat
-rwxr-xr-x 1 andreas andreas 10 Jun 10 2009 config.sys
drwxr-xr-x 2 andreas andreas 0 Jul 14 2009 Documents and Settings
-rwxr-xr-x 1 andreas andreas 3,0G Jun 27 15:45 pagefile.sys
drwxr-xr-x 2 andreas andreas 0 Jul 13 2009 PerfLogs
drwxr-xr-x 2 andreas andreas 0 Jun 12 16:10 ProgramData
dr-xr-xr-x 2 andreas andreas 0 Jun 12 21:05 Program Files
drwxr-xr-x 2 andreas andreas 0 Out 23 2013 Recovery
drwxr-xr-x 2 andreas andreas 0 Out 23 2013 $Recycle.Bin
drwxr-xr-x 2 andreas andreas 0 Nov 26 2014 System Volume Information
dr-xr-xr-x 2 andreas andreas 0 Out 23 2013 Users
drwxr-xr-x 2 andreas andreas 0 Out 23 2013 Wallpaper
drwxr-xr-x 2 andreas andreas 0 Jun 12 16:53 Windows
# mount|grep cifs
//10.0.7.128/c on /c type cifs (rw,relatime,vers=1.0,cache=strict,username=IEUser,domain=WORKGROUP,uid=1000,forceuid,gid=1000,forcegid,addr=10.0.7.128,file_mode=0755,dir_mode=0755,nounix,serverino,mapposix,rsize=61440,wsize=65536,actimeo=1)

I also enabled debugging, like this:
echo 1 > /proc/fs/cifs/traceSMB

That showed that the missing files/directories are sent to the client when "ls -l /mountpoint" is issued.


Finally, the moment I created another entry on C:\, one of the missing directories showed up on the linux side.

cifs-utils 6.4

Users reported this still happening with kernel 4.12.0
Comment 1 Andreas Hasenack 2017-10-27 13:31:42 UTC
Also, smbclient sees all files and directories without problems.
Comment 2 Gerald Hopf 2018-03-21 12:52:25 UTC
This problem is 100% reproducible for me on multiple systems. It is also very EASY to reproduce with any freshly installed Windows 10 system acting as Samba server, see my Test 3 description. It happens EVERY TIME. Could not be more reproducible. 

Also, I think this bug has been present for at least a year, since I first stumbled on this quite some time ago and thought "well, this is weird, but I guess such a prominent bug will be fixed soon so nothing I need to do here".

Test 1: Connecting from a Ubuntu 16.04 system (Linux 4.13.0-37-generic x86_64) to a Windows 10 machine which shares Drive D as the share "D" (I'm not using the D$ share here, D is a manually created share). The alphabetically first directory "Data001" is missing, Data002 and onward are there. Recycle bin also missing. When forcing vers=1.0 instead of "vers=default", the missing directories both appear. The funny thing is: I can enter Data001 using "cd Data001" and then list the contents of Data001 even though the directory Data001 does not appear to be there - so it's invisible but present when mounted using the newer samba protocol!

Test 2: Connecting from the same Ubuntu 16.04 system to a Windows Server 2012 (which is technologically very similar to the old Windows 8.0). When comparing what is available in Linux when mounted with vers=2.1 and vers=1.0, in the share "D$" the alphabetically first directory "Daten" is missing, as is $RECYCLE.BIN. Forcing vers=1.0 makes those 2 directories appear.

Test 3: Freshly installed Windows 10 Pro v1709 with one local (=offline) user and password created during installation. Turned on File and Printer Sharing and shared the root of Drive C: as "C" with permissions for everyone. Then connected from Linux, in this case a Gentoo Linux with Kernel 4.5.11 (latest stable vanilla kernel as of today) and mounted the share with
### mount -t cifs -o username=myuser,password="mypass" //192.168.7.153/C/ /mnt/test/ ###
Result: The alphabetically first directory, "Documents and Settings" is missing, as well as $Recycle.Bin.
Then enabled SMBv1 in Windows Features and settings (off by default for new installations these days!), rebooted Windows and then connected using
### mount -t cifs -o username=myuser,password="mypass",vers=1.0 //192.168.7.153/C/ /mnt/test/ ###
and of course those 2 directories are there when forcing vers=1.0!

From what I can gather, if the shared folder is not the root folder the bug is not present. If it is the windows root of a Drive, the alphabetically first folder will ALWAYS be missing on your linux mount unless you force vers=1.0!

I would have said this is a very prominent and serious bug which can result in data loss if you for example rely on rsyncing a D$ share to make backups... But I guess very few people share their root folders? Can't explain any other way that this has not gotten more prominence.
Comment 3 Aurélien Aptel 2018-04-06 07:49:02 UTC
Was able to reproduce this easily on win10. Just create a share for C:\, like so:

    net share C=C:\

then just mount it in linux with vers set to 2.0 or above. The problem seems to be related to the handling of . and .. (C:\ being the root of the disk, it doesn't return . and .. when listing it). We are looking into it.
Comment 4 Steve French 2018-04-10 02:57:26 UTC
I was able to reproduce this as well - the directory on the root of exported C:\ on Windows 2016 was missing the first two files - they do appear to be returned in the directory listing but skipped as they are processed or soon after.
Comment 5 Steve French 2018-04-11 19:29:12 UTC
We skip entries one and two since they are . and .. which are handled specially ... but in the case of the root directory windows does not return . and .. at the beginning.   Look at this function in fs/cifs/readdir.c

/*
 * Find the corresponding entry in the search. Note that the SMB server returns
 * search entries for . and .. which complicates logic here if we choose to
 * parse for them and we do not assume that they are located in the findfirst
 * return buffer. We start counting in the buffer with entry 2 and increment for
 * every entry (do not increment for . or .. entry).
 */
static int
find_cifs_entry(const unsigned int xid, struct cifs_tcon *tcon, loff_t pos,
                struct file *file, char **current_entry, int *num_to_ret)
...
Comment 6 Steve French 2018-04-11 22:08:11 UTC
Since the server is missing the first two entries in the readdir output ie the files . and ..      the call to kernel helper dir_emit_dots ends up skipping the first two files not . and .. 

static inline bool dir_emit_dots(struct file *file, struct dir_context *ctx)
{
        if (ctx->pos == 0) {
                if (!dir_emit_dot(file, ctx))
                        return false;
                ctx->pos = 1;
        }
        if (ctx->pos == 1) {
                if (!dir_emit_dotdot(file, ctx))
                        return false;
                ctx->pos = 2;
        }
        return true;
}
Comment 7 Steve French 2018-04-12 05:59:37 UTC
Created attachment 14125 [details]
possible fix - needs more testing
Comment 8 Steve French 2018-04-12 06:01:02 UTC
Has not been tested with cifs - just to Samba and Windows root share.  Needs testing and want second opinion if someone sees a better way to fix this.   Problem is that ctx->pos gets incremented for . and .. so isn't a reliable predictor of where we are in the buffer as we go through the query dir results.