I have noticed (as well as many of my colleagues) that when we are viewing a
directory with many files (in the hundreds or above) in konqueror, konqueror
will display an amount close to, but rarely exact, to the amount of files there
really are in the directory. Refreshing the directory will change that amount,
but usually not to the correct number.
Sorry. More Information:
I had a Dual processing Windows2000 machine mounted at startup and was viewing
the files from there. My colleagues and I are running SuSE-based systems (NLD9,
SuSE 9.3, 9.2, etc...)
This should be fixed in 3.0.20.
I am now running SuSE 10.0, and the same thing seems to happen (the konqueror behavior) when viewing a remote directory with lots of files in them.
I also noticed that when I did a listing ('ls') of a remote directory using a bash shell, then compared it to a local listing of the same directory copied over (making sure that EVERY file was copied) a few things occured:
1. The remote listing would randomly miss only 1 file in the output, even though the file was there.
2. The remote listing would randomly miss a couple files at the start of the output, then would consistently miss the 39th file. I have seen something similiar before in Samba 3.0.13 where copying a huge amount of files from a directory would result in missing the 41st file and every 39th file thereafter.
3. And lastly, the listing would come out just fine.... about 1 out of 4 tries for me.
The directory in question had 951 xml files and amounted to about 9.7MB.
My system used to run Samba 3.0.20b but was patched to 3.0.20 with a SUSE patch released.
I am now running kde 3.5.2
The remote machine is a dual processing Windows 2000 workstation which is mounted at startup.
One more thing. I just tried to delete all the files in the folder with 951 xml files, and it left behind the 21st file, the 32nd, and starting with the 125th, every 39th file thereafter. Coincidence? I think not....
This has got to be libsmbclient and not smbclient.
Lars, since this is from a novell.com address, could you follow up?
I have just installed the newest samba binaries (ver. 3.0.22) for SUSE 10.0 and the same problems exist; including the copying of files, not just removing and listing. Konqueror and nautilus and JEdit all have the file browsing problem with not showing the correct amount of files
This looks like a libsmbclient bug. Please log this at the novell bugzilla. We'll need to track it there.
Are you saying that this bug might be a SUSE bug as well? I am trying to add it at the bugzilla here, but there is no samba product to log it under so I will do so under SuSE Linux 10.0.
SuSE uses this component of Samba (libsmbclient). So if it's a Samba.org bug it's also a SuSE bug.
Ok the bug is up at bugzilla.novell.com. It is bug #164326.
The following information will help greatly in tracking down this problem:
- What version(s) of Windows is konqueror trying to enumerate files on (or delete files from)? Win98? XP Pro? etc.
- It sounds like you can reproduce this problem reasonably easily. Please attach to this bug network traces of both a successful and an unsucessful attempt to enumerate files in the same folder. (Preferably one trace file showing a success and a different trace file showing a failure.) We need raw network traces, not printed (ascii) versions. To create the network traces, you can use the following command:
tcpdump -s 0 -w outputfile.pcap
When you provide these traces, please let us know what file(s) are missing in the failed attempt that exist in the successful attempt.
Created attachment 1849 [details]
I had to remove the copy?.pcap files to get it down to size
Created attachment 1850 [details]
tcp dumbs from the copy tests
I couldn't include the 24 MB dump file for copy1.pcap
Ok. here goes....
To begin, I am remotely connected to a Windows 2000 server that has two partitions. I have each partition mounted. My tests involve a directory on each of these partitions, and a local directory. They are as follows. There really isn't any sensitive information as all the files I have been copying are made public anyways.
The first 2 are remote, 3rd one local.
/xeon/InfoRoot/Tids/InfoDocument/ original directory containing the 951 xml files. I will call it directory 'a' from now on.
/xeonx/davetemp/ directory on other partition that I performed my tests on (so to not harm the orig. data). I will call it directory 'b' from now on.
~/temp/temp/ local directory I copied files to. I will call it directory 'c' from now on.
Ok, attached are all the tcpdumps, except the copy1.pcap file, it is 24 MB, but is still described below; I am not sure how to give it to you if you want it. All of the tests were done in bash except for the konqueror ones, and all were done remotely.
Bash commands used: 'cp', 'rm *', 'ls * > ~/temp/list1.txt', etc.
I will explain each test I performed with the associated files:
copy1.pcap try to copy 951 files from 'a' to 'b', 933 copied
remove1.pcap tried to remove those 933 above from 'b', 922 removed
copy2.pcap try to copy 951 from 'a' to 'c', 943 copied
copy3.pcap* try to copy 951 from 'a' to 'c', 951 copied; however, 'c' already had 946 of the original previously copied
remove2.pcap try to remove 951 from 'b', 938 removed
remove3.pcap try to remove 951 from 'b', 941 removed
list1.pcap listing of 951 files in 'b', 946 listed; found in list1.txt
list2.pcap listing of 951 files in 'b', 942 listed; found in list2.txt
list3.pcap listing of 951 files in 'b', 935 listed; found in list3.txt
list4.pcap listing of 951 files in 'b', 937 listed; found in list4.txt
konqview1.pcap first viewing of directory 'a', 931 (of 951) shown
konqview2.pcap 5 refreshes of 'a' showing 936, 933, 935, 936, and 934 files respectively
konqview3.pcap second viewing of directory 'a', 934 files shown
konqview4.pcap 5 refreshes of 'a' showing 928, 931, 933, 930, and 936 files respectively
konqview5.pcap* to get the directory 'b' to show all the files, I did this: I started in 'b'; moved up one directory and navigated into a different one; clicked back and then selected 'b' again. It showed 951. I did a refresh and it showed 927. I am sorry all of this is in one dump
Today wasn't a good testing day:
I tried close to a hundred times to get a correct listing of 'b', and I didn't succeed once. Nor did I succeed in a complete removal of 'b'. I hope what I have sent is useful anyways.
I am glad you are taking the time to consider this problem seriously, it has been an hinderance to our team for some time now.
> To begin, I am remotely connected to a Windows 2000 server that has two
> partitions. I have each partition mounted. My tests involve a directory on
> each of these partitions, and a local directory. They are as follows. There
> really isn't any sensitive information as all the files I have been copying
> are made public anyways.
> The first 2 are remote, 3rd one local.
> /xeon/InfoRoot/Tids/InfoDocument/ original directory containing the 951 xml
> files. I will call it directory 'a' from now on.
> /xeonx/davetemp/ directory on other partition that I
> performed my tests on (so to not harm the orig. data). I will call it
> directory 'b' from now on.
> ~/temp/temp/ local directory I copied files to. I
> will call it directory 'c' from now on.
Ok, I need a bit of clarification on this. This looks like you have mounted the remote file systems using either smbfs or cifs. If you are using smbfs, I don't believe that it is supported any more, and it is known to have bugs. In that case, you should switch to using cifs instead, which is currently supported.
If you are already using cifs, then I'm the wrong person to look at this so I'll redirect this to the person who maintains cifs.
If these are not actually (remotely) mounted file systems, then I need to understand how you are accessing them as "/xeon/..." and "/xeonx/...". Are those the paths you are typing into konqueror or the paths that it is presenting?
Let's figure out what portion of the code this is in, and then we'll try to get it resolved.
Hmmmm... They are mounted as smbfs. I had no idea that was superceded by cifs. Perhaps that will make all the difference in the world. And yes, they are remotely mounted filesystem, one at /xeon, the other at /xeonx (sorry for the redundancy in my mounting abilities)
I got the large tcpdump to upload at novell. bug #164326
Ok, this is *not* a Samba bug - this is a kernel problem with smbfs it looks like. A little background - what confused us is that konqueror (and nautilus in gnome) have I/O helpers which can access remote filesystems via userspace calls into libsmbclient. If you're just accessing mounted filesystems then it's not going through any Samba code at all, but kernel module code. smbfs is unsupported in the current kernel, cifsfs is currently supported via Steve French at IBM. If you can reproduce the problem using CIFSFS you'll need to log a bug and the kernel team will have to fix it.
ok thank you, cifs is working just fine. I will stop writing about the bug here and continue only at novell.