Bug 3626 - trans2 bug, files missing in directory listing
trans2 bug, files missing in directory listing
Status: ASSIGNED
Product: Samba 3.0
Classification: Unclassified
Component: File Services
3.0.21c
PPC AIX
: P3 major
: none
Assigned To: Samba Bugzilla Account
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2006-03-23 05:06 UTC by Stefan Beck
Modified: 2008-03-19 12:43 UTC (History)
1 user (show)

See Also:


Attachments
this is a trace with a samba version 3.0.9 that does not have that bug (648.48 KB, application/octet-stream)
2006-03-23 05:10 UTC, Stefan Beck
no flags Details
this is a trace with a samba version 3.0.21c that has that bug (20.75 KB, application/octet-stream)
2006-03-23 05:10 UTC, Stefan Beck
no flags Details
this is an ethereal trace with a samba version 3.0.21c that has that bug, client: 3.0.14a-Ubuntu (21.87 KB, application/octet-stream)
2006-04-27 06:51 UTC, Stefan Beck
no flags Details
samba debug logs, smb.conf and the directory which causes the problem (85.58 KB, application/gzip)
2006-04-27 06:53 UTC, Stefan Beck
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Beck 2006-03-23 05:06:35 UTC
We have a directory with 1955 catia files. 

3.0.9 debian sarge x86: ok
3.0.14 debian sarge x86: fails (only 67 show up)
3.0.21c aix5.3 ppx: fails (only 67 show up)

I know that at least one earlier version of 3.0.x (x<9) had the same bug.
I looks like that the trans2 changes in 3.0.10 reintroduced the bug.
Comment 1 Stefan Beck 2006-03-23 05:10:17 UTC
Created attachment 1820 [details]
this is a trace with a samba version 3.0.9 that does not have that bug
Comment 2 Stefan Beck 2006-03-23 05:10:56 UTC
Created attachment 1821 [details]
this is a trace with a samba version 3.0.21c that has that bug
Comment 3 Jeremy Allison 2006-03-23 10:40:38 UTC
What I need is the debug level 10 log from the smbd serving the client showing the problem. Also I need to know exactly what client you're using. If you can attach a tarball containing the problem directory (I just need the filenames, not the contents of the files) that would also help.
Jeremy.
Comment 4 Radek Hladik 2006-04-01 10:54:41 UTC
I am expiriencing maybe the same problem. We have 6 separete filesystems merged by using UnionFS together. When sharing the resulting directory via samba-3.0.14a-2 (out-of-the box Fedora Core 4), then directory listing contains only about half of directories and files. But if I try to create a "merged directory" by using softlinks instead of unionfs I am able to see all the files and directories.
There are cca 340 directories in total, dir * shows only 280. But dir *s* shows cca 234 directories containing s and this listing contains all directories so it seems that there is some problem if there is too many of files to list. 
I can not attach real files and dirs (as they are private to my company :) ) but I have created a testcase:

I have created three directories: test ,test1 and test2. Directory test is shared via samba to windows.
1) In directory tets I execute 
for i in `seq 1 999` ; do mkdir dir$i ; done ; 
2)dir * shows all directories (output is in file dir1)

3) I've deleted all dirs in test and created dirs in test1 and test2

test1:for i in `seq 501 999` ; do mkdir dir$i ; done ;
test2:for i in `seq 1 500` ; do mkdir dir$i ; done ;

and merged them via unionfs into test:
mount -t unionfs -o dirs=/data/test/test1:/data/test/test2 none /data/test/test

4) ls dir* shows all files (dir.2)
5) dir * in windows show only 759 directories (dir.3)
6) running dir * for second times shows exactly the same files as in step 5

Comment 5 Jeremy Allison 2006-04-02 14:19:29 UTC
For comment #4. If you get a full listing when not using unionfs, but a truncated listing when using unionfs then it's a unionfs bug, not a Samba one.

For comment #2 from Stefan Beck, looking at the ethereal trace I see it end the search after being asked for the last file returned in the findfirst - I need to see a debug level 10 log of the smbd, plus a listing of all the files in the directory please.

Jeremy.
Comment 6 Radek Hladik 2006-04-02 19:52:47 UTC
(In reply to comment #5)
> For comment #4. If you get a full listing when not using unionfs, but a
> truncated listing when using unionfs then it's a unionfs bug, not a Samba one.
> 

I'm not quite sure that this is purely unionfs bug as samba is the only service which provides uncomplete directory listing. Many other services/programs are working correctly with unionfs: ls, mc, vsftp, sftp even nfsd.

I've just tried samba 3.0.8-pre1 (RPM package from not updated Fedora Core 3) and it works as expected. On the other hand samba 3.0.21b-2 (from Fedore Core 5) produces incomplete listing too. 

Radek
Comment 7 Jeremy Allison 2006-04-02 20:16:21 UTC
Unless you can reproduce this without unionfs I'm not going to spend any time on it. Sorry. We changed the way Samba used opendir/readdir etc. around that time but this change works perfectly on all POSIX complient systems. It may be we're using unionfs in a way that other clients aren't but it's not our job to debug unionfs.
Jeremy.
Comment 8 Stefan Beck 2006-04-27 06:51:46 UTC
Created attachment 1878 [details]
this is an ethereal trace with a samba version 3.0.21c that has that bug, client: 3.0.14a-Ubuntu

This is the ethereal trace which corresponds to the upcoming samba debug log
Comment 9 Stefan Beck 2006-04-27 06:53:35 UTC
Created attachment 1879 [details]
samba debug logs, smb.conf and the directory which causes the problem

This is a tgz file which contains the logs and the directory whith the files (truncated to 0 Bytes)
Comment 10 Jeremy Allison 2006-04-30 23:21:07 UTC
I have extracted the directory and listed the file using dir from a cmd.exe window on Windowx XP against the latest 3.0.23 pre-release code. I don't see a problem (I see all 1958 files). Please test with 3.0.23 on a standard filesystem (not unionfs ) and let me know if this works for you.
Jeremy.
Comment 11 Stefan Beck 2006-05-03 06:15:52 UTC
Hello Jeremy,       

'hide special files = Yes' was the key, changing that flag triggers the symptom ... (on actual and older releases).

but
- I thought this should only filter socket/pipe/dev files (i.e. I think this is a bug)
- the behaviour of that flag has obviously changed during the 3.0.x series
- if you think the behaviour is correct, please update smb.conf(5)

Cheers
Stefan
Comment 12 Alex Prestin 2008-03-19 12:43:16 UTC
we are seeing the same issue in 3.0.28.  a directory with about 60 thousand files is truncated to 63 files.