Bug 3626 - trans2 bug, files missing in directory listing
Summary: trans2 bug, files missing in directory listing
Status: RESOLVED WORKSFORME
Alias: None
Product: Samba 3.0
Classification: Unclassified
Component: File Services (show other bugs)
Version: 3.0.21c
Hardware: PPC AIX
: P3 major
Target Milestone: none
Assignee: Samba Bugzilla Account
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-03-23 05:06 UTC by Stefan Beck (dead mail address)
Modified: 2023-06-13 12:10 UTC (History)
1 user (show)

See Also:


Attachments
this is a trace with a samba version 3.0.9 that does not have that bug (648.48 KB, application/octet-stream)
2006-03-23 05:10 UTC, Stefan Beck (dead mail address)
no flags Details
this is a trace with a samba version 3.0.21c that has that bug (20.75 KB, application/octet-stream)
2006-03-23 05:10 UTC, Stefan Beck (dead mail address)
no flags Details
this is an ethereal trace with a samba version 3.0.21c that has that bug, client: 3.0.14a-Ubuntu (21.87 KB, application/octet-stream)
2006-04-27 06:51 UTC, Stefan Beck (dead mail address)
no flags Details
samba debug logs, smb.conf and the directory which causes the problem (85.58 KB, application/gzip)
2006-04-27 06:53 UTC, Stefan Beck (dead mail address)
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Beck (dead mail address) 2006-03-23 05:06:35 UTC
We have a directory with 1955 catia files. 

3.0.9 debian sarge x86: ok
3.0.14 debian sarge x86: fails (only 67 show up)
3.0.21c aix5.3 ppx: fails (only 67 show up)

I know that at least one earlier version of 3.0.x (x<9) had the same bug.
I looks like that the trans2 changes in 3.0.10 reintroduced the bug.
Comment 1 Stefan Beck (dead mail address) 2006-03-23 05:10:17 UTC
Created attachment 1820 [details]
this is a trace with a samba version 3.0.9 that does not have that bug
Comment 2 Stefan Beck (dead mail address) 2006-03-23 05:10:56 UTC
Created attachment 1821 [details]
this is a trace with a samba version 3.0.21c that has that bug
Comment 3 Jeremy Allison 2006-03-23 10:40:38 UTC
What I need is the debug level 10 log from the smbd serving the client showing the problem. Also I need to know exactly what client you're using. If you can attach a tarball containing the problem directory (I just need the filenames, not the contents of the files) that would also help.
Jeremy.
Comment 4 Radek Hladik 2006-04-01 10:54:41 UTC
I am expiriencing maybe the same problem. We have 6 separete filesystems merged by using UnionFS together. When sharing the resulting directory via samba-3.0.14a-2 (out-of-the box Fedora Core 4), then directory listing contains only about half of directories and files. But if I try to create a "merged directory" by using softlinks instead of unionfs I am able to see all the files and directories.
There are cca 340 directories in total, dir * shows only 280. But dir *s* shows cca 234 directories containing s and this listing contains all directories so it seems that there is some problem if there is too many of files to list. 
I can not attach real files and dirs (as they are private to my company :) ) but I have created a testcase:

I have created three directories: test ,test1 and test2. Directory test is shared via samba to windows.
1) In directory tets I execute 
for i in `seq 1 999` ; do mkdir dir$i ; done ; 
2)dir * shows all directories (output is in file dir1)

3) I've deleted all dirs in test and created dirs in test1 and test2

test1:for i in `seq 501 999` ; do mkdir dir$i ; done ;
test2:for i in `seq 1 500` ; do mkdir dir$i ; done ;

and merged them via unionfs into test:
mount -t unionfs -o dirs=/data/test/test1:/data/test/test2 none /data/test/test

4) ls dir* shows all files (dir.2)
5) dir * in windows show only 759 directories (dir.3)
6) running dir * for second times shows exactly the same files as in step 5

Comment 5 Jeremy Allison 2006-04-02 14:19:29 UTC
For comment #4. If you get a full listing when not using unionfs, but a truncated listing when using unionfs then it's a unionfs bug, not a Samba one.

For comment #2 from Stefan Beck, looking at the ethereal trace I see it end the search after being asked for the last file returned in the findfirst - I need to see a debug level 10 log of the smbd, plus a listing of all the files in the directory please.

Jeremy.
Comment 6 Radek Hladik 2006-04-02 19:52:47 UTC
(In reply to comment #5)
> For comment #4. If you get a full listing when not using unionfs, but a
> truncated listing when using unionfs then it's a unionfs bug, not a Samba one.
> 

I'm not quite sure that this is purely unionfs bug as samba is the only service which provides uncomplete directory listing. Many other services/programs are working correctly with unionfs: ls, mc, vsftp, sftp even nfsd.

I've just tried samba 3.0.8-pre1 (RPM package from not updated Fedora Core 3) and it works as expected. On the other hand samba 3.0.21b-2 (from Fedore Core 5) produces incomplete listing too. 

Radek
Comment 7 Jeremy Allison 2006-04-02 20:16:21 UTC
Unless you can reproduce this without unionfs I'm not going to spend any time on it. Sorry. We changed the way Samba used opendir/readdir etc. around that time but this change works perfectly on all POSIX complient systems. It may be we're using unionfs in a way that other clients aren't but it's not our job to debug unionfs.
Jeremy.
Comment 8 Stefan Beck (dead mail address) 2006-04-27 06:51:46 UTC
Created attachment 1878 [details]
this is an ethereal trace with a samba version 3.0.21c that has that bug, client: 3.0.14a-Ubuntu

This is the ethereal trace which corresponds to the upcoming samba debug log
Comment 9 Stefan Beck (dead mail address) 2006-04-27 06:53:35 UTC
Created attachment 1879 [details]
samba debug logs, smb.conf and the directory which causes the problem

This is a tgz file which contains the logs and the directory whith the files (truncated to 0 Bytes)
Comment 10 Jeremy Allison 2006-04-30 23:21:07 UTC
I have extracted the directory and listed the file using dir from a cmd.exe window on Windowx XP against the latest 3.0.23 pre-release code. I don't see a problem (I see all 1958 files). Please test with 3.0.23 on a standard filesystem (not unionfs ) and let me know if this works for you.
Jeremy.
Comment 11 Stefan Beck (dead mail address) 2006-05-03 06:15:52 UTC
Hello Jeremy,       

'hide special files = Yes' was the key, changing that flag triggers the symptom ... (on actual and older releases).

but
- I thought this should only filter socket/pipe/dev files (i.e. I think this is a bug)
- the behaviour of that flag has obviously changed during the 3.0.x series
- if you think the behaviour is correct, please update smb.conf(5)

Cheers
Stefan
Comment 12 Alex Prestin 2008-03-19 12:43:16 UTC
we are seeing the same issue in 3.0.28.  a directory with about 60 thousand files is truncated to 63 files.
Comment 13 Björn Jacke 2023-06-13 12:10:05 UTC
(In reply to Stefan Beck from comment #9)

those files are incorrectly encoded and not valid in the default unix charset (utf-8). If you convert the files to utf-8 with convmv all the files are listed correctly