Bug 10468 - Copying files results in incomplete set of files/directories
Summary: Copying files results in incomplete set of files/directories
Status: RESOLVED FIXED
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: File services (show other bugs)
Version: 4.12.10
Hardware: All All
: P5 major (vote)
Target Milestone: ---
Assignee: Samba QA Contact
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-02-24 20:04 UTC by Chip Schweiss
Modified: 2020-11-17 20:33 UTC (History)
2 users (show)

See Also:


Attachments
Small ISO which not all files will not copy. (3.55 MB, application/octet-stream)
2014-02-24 20:07 UTC, Chip Schweiss
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Chip Schweiss 2014-02-24 20:04:55 UTC
I have a test server setup with Samba 4.1.4.   All file interactions test work except when copying files from a CD or DVD to the samba share on Windows 7.

I have a CD with a folder that contains 324 files and 22 sub-folders.   Consistently if I drag/drop or copy/paste the folder in Windows Explorer directly from the CD to the samba share only 53 files and 4 folders get copied.  No error dialogs appear in Windows.  Sometimes the copy dialog appears other times it does not.

If copy the same folder from the CD to the local machine all files/folders copy.   If I copy again from the local machine to the samba share, again all files/folders copy. 

I've increase logging verbosity to 6 and it appears Windows is only trying the small subset of files/folders.   There are entries of looking up each file that is missing, but never copying them.

The attached ISO reproduces the problem when mounted in Windows 7. The problem has been verified on several other CD/DVDs.   This one was included in the ticket because it is small and contains no protected information.

Here is the relevant log entries for some files and a directory that did not copy:

[2014/02/19 15:14:13.823511,  5] ../source3/smbd/dir.c:493(dptr_create)
  dptr_create dir=Chip/FromCD/myTempHPF/ScAnVP6.2.1w/templates/Brain_Atlas
[2014/02/19 15:14:13.823555,  3] ../source3/smbd/dir.c:665(dptr_create)
  creating new dirptr 0 for path Chip/FromCD/myTempHPF/ScAnVP6.2.1w/templates/Brain_Atlas, expect_close = 0
[2014/02/19 15:14:13.823598,  6] ../source3/smbd/dir.c:1025(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry: dirptr 0x8092888 now at offset 0
[2014/02/19 15:14:13.823664,  3] ../source3/smbd/dir.c:1108(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry mask=[*] found Chip/FromCD/myTempHPF/ScAnVP6.2.1w/templates/Brain_Atlas/. fname=. (.)
[2014/02/19 15:14:13.823712,  6] ../source3/smbd/dir.c:1025(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry: dirptr 0x8092888 now at offset -2147483648
[2014/02/19 15:14:13.823771,  3] ../source3/smbd/dir.c:1108(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry mask=[*] found Chip/FromCD/myTempHPF/ScAnVP6.2.1w/templates/Brain_Atlas/.. fname=.. (..)
[2014/02/19 15:14:13.823858,  6] ../source3/smbd/dir.c:1025(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry: dirptr 0x8092888 now at offset 314393961
[2014/02/19 15:14:13.823913,  3] ../source3/smbd/dir.c:1108(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry mask=[*] found Chip/FromCD/myTempHPF/ScAnVP6.2.1w/templates/Brain_Atlas/ROIAtlasNotes.txt fname=ROIAtlasNotes.txt (ROIAtlas
Notes.txt)
[2014/02/19 15:14:13.823968,  6] ../source3/smbd/dir.c:1025(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry: dirptr 0x8092888 now at offset 331697583
[2014/02/19 15:14:13.824017,  3] ../source3/smbd/dir.c:1108(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry mask=[*] found Chip/FromCD/myTempHPF/ScAnVP6.2.1w/templates/Brain_Atlas/AtlasREADME.txt fname=AtlasREADME.txt (AtlasREADME.
txt)
[2014/02/19 15:14:13.824069,  6] ../source3/smbd/dir.c:1025(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry: dirptr 0x8092888 now at offset 395314558
[2014/02/19 15:14:13.824118,  3] ../source3/smbd/dir.c:1108(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry mask=[*] found Chip/FromCD/myTempHPF/ScAnVP6.2.1w/templates/Brain_Atlas/ROIAtlas.hdr fname=ROIAtlas.hdr (ROIAtlas.hdr)
[2014/02/19 15:14:13.824177,  6] ../source3/smbd/dir.c:1025(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry: dirptr 0x8092888 now at offset 430847674
[2014/02/19 15:14:13.824227,  3] ../source3/smbd/dir.c:1108(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry mask=[*] found Chip/FromCD/myTempHPF/ScAnVP6.2.1w/templates/Brain_Atlas/ROIAtlasLabs.txt fname=ROIAtlasLabs.txt (ROIAtlasLa
bs.txt)
[2014/02/19 15:14:13.824287,  6] ../source3/smbd/dir.c:1025(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry: dirptr 0x8092888 now at offset 461004237
[2014/02/19 15:14:13.824337,  3] ../source3/smbd/dir.c:1108(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry mask=[*] found Chip/FromCD/myTempHPF/ScAnVP6.2.1w/templates/Brain_Atlas/ROIAtlas.img fname=ROIAtlas.img (ROIAtlas.img)
[2014/02/19 15:14:13.824387,  6] ../source3/smbd/dir.c:1025(smbd_dirptr_get_entry)
  smbd_dirptr_get_entry: dirptr 0x8092888 now at offset -1

The only error I see in the logs is:

2014/02/19 14:55:06.542523,  2] ../source3/smbd/open.c:3060(open_directory)
  open_directory: unable to create Chip/FromCD/myTempHPF/ScAnVP6.2.1w/documentation. Error was NT_STATUS_OBJECT_NAME_COLLISION

If I launch a Samba 3.6.20 with the same smb.conf it works as expected.   

This is running on OmniOS with Samba 4.1.4 compiled from source.

The smb.conf is:

[global]

    log level = 6
    workgroup = NEUROIMAGE
    server string = ZFS Test Server - Samba Version %v
    netbios name = NILtest1

    winbind enum groups = yes
    winbind enum users = yes

    idmap backend = tdb
    idmap uid = 100000-10000000
    idmap gid = 100000-10000000
    idmap config NEUROIMAGE : backend  = nss
    idmap config NEUROIMAGE : range = 500-99999

    interfaces = lo e1000g0
    log file = /var/opt/csw/samba/log/%m.log
    max log size = 5000

    security = ads
    realm = NEUROIMAGE.WUSTL.EDU
    password server = nildc01 nildc02 nildc03
    socket options = TCP_NODELAY
    encrypt passwords = yes
    dns proxy = no
    mangled names = no
    preserve case = yes
    short preserve case = yes
   load printers = no

[scratch]
   comment = Public Stuff
   path = /testpool/scratch
   writable = yes
   browsable = yes


[scratch1]
   comment = Public Stuff 1
   path = /testpool/scratch1
   writable = yes
   browsable = yes

[scratch2]
   comment = Public Stuff 2
   path = /testpool/scratch2
   writable = yes
   browsable = yes
Comment 1 Chip Schweiss 2014-02-24 20:07:42 UTC
Created attachment 9718 [details]
Small ISO which not all files will not copy.
Comment 2 Chip Schweiss 2014-03-03 21:29:20 UTC
I have confirmed this problem exists in 4.1.4, 4.1.5 and 4.0.15.

It is not present in 3.6.20.
Comment 3 Björn Jacke 2020-11-17 19:54:11 UTC
this is still a problem, just tested with 4.12.10 and W10 as client with your attached iso image files. The copy process hangs always in "get_file.m", which *interestingly* is one if the very few filenames, that exist multiple times in different directories:

./myTempHPF/ScAnVP6.2.1w/vpm/Browser/NewBrowser/get_file.m
./myTempHPF/ScAnVP6.2.1w/vpm/Browser/OldBrowser/get_file.m
./myTempHPF/ScAnVP6.2.1w/vpm/get_file.m

Copying those files onto a *Windows* volume works without a problem, also via SMB3.

Copying it from there on a Samba share triggers the hang. Actually copying the vpm folder alone is enough to trigger the problem. SetInfo SMB2_FILE_ENDOFFILE_INFO comes from the client, the server replyies with a successful response, and then the client is unhappy:

Error 0x8007003B: An unexpected network error occurred

Something that smbd sends back seems to be bad here.
Comment 4 Björn Jacke 2020-11-17 20:33:45 UTC
I just realize, that the error described in comment #3 is acutally quite different from the initial report, the problem from the initial report don't pop up here any more. I'll close this bug as fixed and file a new bug for the new errors ...