Bug 2452 - Last entry missing/dropped from directory listing
Summary: Last entry missing/dropped from directory listing
Status: CLOSED FIXED
Alias: None
Product: Samba 3.0
Classification: Unclassified
Component: File Services (show other bugs)
Version: 3.0.14a
Hardware: SGI IRIX
: P3 normal
Target Milestone: none
Assignee: Samba Bugzilla Account
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-03-15 12:17 UTC by Joe Adams
Modified: 2005-08-24 10:18 UTC (History)
2 users (show)

See Also:


Attachments
Patch (480 bytes, patch)
2005-04-21 00:59 UTC, Jeremy Allison
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Joe Adams 2005-03-15 12:17:29 UTC
Note - This info is being reported for version 3.0.12rc1. I have not been
successful compiling 3.0.11 to confirm whether the problem is in that stable
version.

PROBLEM

When the contents of a directory is shown in Windows Explorer or through
smbclient, one entry/item is missing. The missing entry appears to always be the
 last directory entry, not the last entry alphabetically.


EVIDENCE

In the example below, notice that the entry "geowall" appears in the ls command
from the IRIX prompt but does not appear in the listing from smbclient.

bash-2.04# cd /mnt6
bash-2.04# ls -lU
total 8
drwxrwxrwx  19 djhester nmd          4096 Mar  1 13:00 due
drwxrwxrwx   2 root     nmd             9 Oct 20 11:53 tmp
drwxrwxr-x  28 djhester nmd          4096 Mar 10 14:58 etap
drwxrwsr-x   2 csmladin nmd             9 Oct 20 11:53 redriver
drwxr-xr-x   4 root     sys            34 May 15  2002 library
drwxr-xr-x   2 root     sys             9 Feb 24  2003 download
drwxrwxrwx   4 root     sys            43 Oct 20 11:53 geowall
bash-2.04#

smb: \mnt6\> ls
  .                                   D       0  Sun Mar 13 03:08:42 2005
  ..                                  D       0  Fri Mar 11 17:19:33 2005
  due                                 D       0  Tue Mar  1 13:00:39 2005
  tmp                                 D       0  Wed Oct 20 11:53:56 2004
  etap                                D       0  Thu Mar 10 14:58:35 2005
  redriver                            D       0  Wed Oct 20 11:53:56 2004
  library                             D       0  Wed May 15 10:24:22 2002
  download                            D       0  Mon Feb 24 13:52:21 2003

                33692 blocks of size 524288. 15607 blocks available


ENVIRONMENT

IRIX version = 6.5.24f

GCC = 3.3
gmake = 3.80

configure options = --with-included-popt --with-ads

(I've also tried with and without the option --with-acl-support)

Sorry in advance if this problem has been induced by me or if there is already
related bug open. I searched for awhile and could not find anything related.

Thanks for maintaining an amazing OpenSource product.

Joe Adams
US Geological Survey
Denver, Colorado


More information dumps:

smb.conf:

[global]
   server string = SGI Origin 2100 - Samba %v
   #netbios name = library
   workgroup = rocky
   #workgroup = rmmc
   guest account = daemon
   #guest account = nobody
   # guest account = ewadmin
   username map = /usr/local/samba/lib/usermap
   browseable = yes
   max log size = 50
   locking = yes
   log file = /usr/local/samba/var/logs/log.%m
   lock directory = /usr/local/samba/var/locks
   kernel oplocks = no
#   share modes = yes
#   socket options = TCP_NODELAY
#   socket options = SO_SNDBUF=1048576 SO_RCVBUF=1048576
#   level2 oplocks = yes
#   host msdfs = yes
#   nt acl support = no
#   max xmit = 65535

   # Restrict access to RMMC 100 subnet.
   hosts allow = 136.177.100.0/255.255.252.0 136.177.96.0/255.255.255.0
136.177.98.0/255.255.255.224

   #interfaces = 136.177.100.159
   #bind interfaces only = yes

   #wins server = 136.177.101.13
   wins server = 136.177.103.175 136.177.103.45
   name resolve order = wins host lmhosts bcast

   password server = 136.177.103.175 136.177.103.45
   security = ads
   #security = domain
   #password server = 136.177.101.13
   realm = rocky.local
#   security = user
   encrypt passwords = yes

# I switched to forced mask to ellimated a problem when a user
# copied over files from the CDROM on a PC; the permissions would
# be read-only. This forces everything to be writable.

#   create mask = 0775
#   directory mask = 0775

   force create mode = 0775
   force directory mode = 2775

   os level = 65
   preferred master = no
   domain master = no
   local master = no
   wins support = no

   preserve case = yes
   short preserve case = yes

   load printers = no
   printing = sysv
;   printcap = /usr/local/samba/lib/printer-list
;   print command = /usr/bin/lp -c -d%p %s ; /usr/bin/rm %s
   path = /tmp

#[printers]
#  comment = Printers server by this system
#  public = yes
#  printable = yes
#  browseable = no

[writable]
  comment = Login and Password required
  path = /usr/local/samba/samba_links
  read only = no
  public = no
  #write cache size = 1048576

[readonly]
  comment = Read-only access to data
  path = /usr/local/samba/ro_samba_links
  read only = yes
  public = yes
  guest only = yes



Output of testparm -v:

bash-2.04# ../bin/testparm -v
Load smb config files from /usr/local/samba/lib/smb.conf
Processing section "[writable]"
Processing section "[readonly]"
Loaded services file OK.
Server role: ROLE_DOMAIN_MEMBER
Press enter to see a dump of your service definitions

# Global parameters
[global]
        dos charset = ASCII
        unix charset = UTF8
        display charset = LOCALE
        workgroup = ROCKY
        realm = ROCKY.LOCAL
        netbios name = CRSTL0
        netbios aliases =
        netbios scope =
        server string = SGI Origin 2100 - Samba %v
        interfaces =
        bind interfaces only = No
        security = ADS
        auth methods =
        encrypt passwords = Yes
        update encrypted = No
        client schannel = Auto
        server schannel = Auto
        allow trusted domains = Yes
        hosts equiv =
        min password length = 5
        map to guest = Never
        null passwords = No
        obey pam restrictions = No
        password server = 136.177.103.175 136.177.103.45
        smb passwd file = /usr/local/samba/private/smbpasswd
        private dir = /usr/local/samba/private
        passdb backend = smbpasswd
        algorithmic rid base = 1000
        root directory =
        guest account = daemon
        enable privileges = No
        pam password change = No
        passwd program =
        passwd chat = *new*password* %n\n *new*password* %n\n *changed*
        passwd chat debug = No
        passwd chat timeout = 2
        check password script =
        username map = /usr/local/samba/lib/usermap
        password level = 0
        username level = 0
        unix password sync = No
        restrict anonymous = 0
        lanman auth = Yes
        ntlm auth = Yes
        client NTLMv2 auth = No
        client lanman auth = Yes
        client plaintext auth = Yes
        preload modules =
        use kerberos keytab = No
        log level = 0
        syslog = 1
        syslog only = No
        log file = /usr/local/samba/var/logs/log.%m
        max log size = 50
        debug timestamp = Yes
        debug hires timestamp = No
        debug pid = No
        debug uid = No
        smb ports = 445 139
        large readwrite = Yes
        max protocol = NT1
        min protocol = CORE
        read bmpx = No
        read raw = Yes
        write raw = Yes
        disable netbios = No
        acl compatibility =
        defer sharing violations = Yes
        nt pipe support = Yes
        nt status support = Yes
        announce version = 4.9
        announce as = NT
        max mux = 50
        max xmit = 16644
        name resolve order = wins host lmhosts bcast
        max ttl = 259200
        max wins ttl = 518400
        min wins ttl = 21600
        time server = No
        unix extensions = Yes
        use spnego = Yes
        client signing = auto
        server signing = No
        client use spnego = Yes
        change notify timeout = 60
        deadtime = 0
        getwd cache = Yes
        keepalive = 300
        kernel change notify = Yes
        lpq cache time = 30
        max smbd processes = 0
        paranoid server security = Yes
        max disk size = 0
        max open files = 10000
        socket options = TCP_NODELAY
        use mmap = Yes
        hostname lookups = No
        name cache timeout = 660
        load printers = No
        printcap cache time = 0
        printcap name =
        cups server =
        disable spoolss = No
        enumports command =
        addprinter command =
        deleteprinter command =
        show add printer wizard = Yes
        os2 driver map =
        mangling method = hash2
        mangle prefix = 1
        stat cache = Yes
        machine password timeout = 604800
        add user script =
        delete user script =
        add group script =
        delete group script =
        add user to group script =
        delete user from group script =
        set primary group script =
        add machine script =
        shutdown script =
        abort shutdown script =
        logon script =
        logon path = \\%N\%U\profile
        logon drive =
        logon home = \\%N\%U
        domain logons = No
        os level = 65
        lm announce = Auto
        lm interval = 60
        preferred master = No
        local master = No
        domain master = No
        browse list = Yes
        enhanced browsing = Yes
        dns proxy = Yes
        wins proxy = No
        wins server = 136.177.103.175, 136.177.103.45
        wins support = No
        wins hook =
        wins partners =
        kernel oplocks = No
        lock spin count = 3
        lock spin time = 10
        oplock break wait time = 0
        ldap admin dn =
        ldap delete dn = No
        ldap filter = (uid=%u)
        ldap group suffix =
        ldap idmap suffix =
        ldap machine suffix =
        ldap passwd sync = no
        ldap replication sleep = 1000
        ldap suffix =
        ldap ssl =
        ldap timeout = 15
        ldap user suffix =
        add share command =
        change share command =
        delete share command =
        config file =
        preload =
        lock directory = /usr/local/samba/var/locks
        pid directory = /usr/local/samba/var/locks
        utmp directory =
        wtmp directory =
        utmp = No
        default service =
        message command =
        dfree command =
        get quota command =
        set quota command =
        remote announce =
        remote browse sync =
        socket address = 0.0.0.0
        homedir map =
        afs username map =
        afs token lifetime = 604800
        log nt token command =
        time offset = 0
        NIS homedir = No
        panic action =
        host msdfs = No
        enable rid algorithm = Yes
        idmap backend =
        idmap uid =
        idmap gid =
        template primary group = nobody
        template homedir = /home/%D/%U
        template shell = /bin/false
        winbind separator = \
        winbind cache time = 300
        winbind enable local accounts = No
        winbind enum users = Yes
        winbind enum groups = Yes
        winbind use default domain = No
        winbind trusted domains only = No
        winbind nested groups = No
        comment =
        path = /tmp
        username =
        invalid users =
        valid users =
        admin users =
        read list =
        write list =
        printer admin =
        force user =
        force group =
        read only = Yes
        create mask = 0744
        force create mode = 0775
        security mask = 0777
        force security mode = 00
        directory mask = 0755
        force directory mode = 02775
        directory security mask = 0777
        force directory security mode = 00
        force unknown acl user = No
        inherit permissions = No
        inherit acls = No
        guest only = No
        guest ok = No
        only user = No
        hosts allow = 136.177.100.0/255.255.252.0, 136.177.96.0/255.255.255.0,
136.177.98.0/255.255.255.224
        hosts deny =
        allocation roundup size = 1048576
        ea support = No
        nt acl support = Yes
        profile acls = No
        map acl inherit = No
        afs share = No
        block size = 1024
        max connections = 0
        min print space = 0
        strict allocate = No
        strict sync = No
        sync always = No
        use sendfile = No
        write cache size = 0
        max reported print jobs = 0
        max print jobs = 1000
        printable = No
        printing = sysv
        cups options =
        print command = lp -c -d%p %s; rm %s
        lpq command = lpstat -o%p
        lprm command = cancel %p-%j
        lppause command = lp -i %p-%j -H hold
        lpresume command = lp -i %p-%j -H resume
        queuepause command = disable %p
        queueresume command = enable %p
        printer name =
        use client driver = No
        default devmode = No
        force printername = No
        default case = lower
        case sensitive = Auto
        preserve case = Yes
        short preserve case = Yes
        mangling char = ~
        hide dot files = Yes
        hide special files = No
        hide unreadable = No
        hide unwriteable files = No
        delete veto files = No
        veto files =
        hide files =
        veto oplock files =
        map system = No
        map hidden = No
        map archive = Yes
        mangled names = Yes
        mangled map =
        store dos attributes = No
        browseable = Yes
        blocking locks = Yes
        csc policy = manual
        fake oplocks = No
        locking = Yes
        oplocks = Yes
        level2 oplocks = Yes
        oplock contention limit = 2
        posix locking = Yes
        strict locking = Yes
        share modes = Yes
        copy =
        include =
        preexec =
        preexec close = No
        postexec =
        root preexec =
        root preexec close = No
        root postexec =
        available = Yes
        volume =
        fstype = NTFS
        set directory = No
        wide links = Yes
        follow symlinks = Yes
        dont descend =
        magic script =
        magic output =
        delete readonly = No
        dos filemode = No
        dos filetimes = No
        dos filetime resolution = No
        fake directory create times = No
        vfs objects =
        msdfs root = No
        msdfs proxy =

[writable]
        comment = Login and Password required
        path = /usr/local/samba/samba_links
        read only = No

[readonly]
        comment = Read-only access to data
        path = /usr/local/samba/ro_samba_links
        guest only = Yes
        guest ok = Yes
bash-2.04#
Comment 1 Joe Adams 2005-03-15 12:29:20 UTC
This problem does not appear in my installation of 3.0.12.rc1 under Solaris 8
and 9 with GCC 3.4.2.
Comment 2 Joe Adams 2005-03-21 06:14:12 UTC
This missing directory entry symptom is in 3.0.12 also.

Thanks for your help with this.
Comment 3 Gerald (Jerry) Carter (dead mail address) 2005-03-21 06:59:42 UTC
Just to clarify, is this an irix only bug ?  If so, what filesystem are you using ?
Comment 4 Joe Adams 2005-03-21 09:31:06 UTC
Thanks for looking at this Jerry.

I have one IRIX box and several Solaris 8 & 9 boxes in my environment. I'm only
seeing this symptom on the IRIX box.

Before trying to compile this latest version of Samba on this IRIX box, we were
running a pre-compiled version of Samba 2.2.8. We did not see this sympton under
2.2.8. (We upgraded to be able to talk Active Directory)

All filesystems on the IRIX box are XFS.
Comment 5 Gerald (Jerry) Carter (dead mail address) 2005-03-23 09:46:20 UTC
jeremy, I think this bug might be fixed by your recent changes.
Do you agree ?

Joe.  You might want to test out the latest SAMBA_3_0 svn tree
or give 3.0.13 a test when it is released tomorrow.
Comment 6 Joe Adams 2005-03-23 10:23:44 UTC
Thanks Jerry.

I will try 3.0.13 tomorrow.

Comment 7 Joe Adams 2005-03-24 09:29:59 UTC
Samba 3.0.13 still has this same symptom.

Thanks for continuing to keep this on the list of things to like about.
Comment 8 Mac 2005-03-31 09:16:46 UTC
(In reply to comment #4)
I'm running 3.0.6 in IRIX 6.5.5 with no 'missing file' problems.

3.0.12 and 3.0.13 do have the problem.
Comment 9 Mac 2005-04-04 02:39:20 UTC
(In reply to comment #8)
> (In reply to comment #4)
> I'm running 3.0.6 in IRIX 6.5.5 with no 'missing file' problems.
> 
> 3.0.12 and 3.0.13 do have the problem.

Much testing has now been done. by me and a colleague.

The state of play is thus:-

IRIX 6.5.7m

			local-xfs	NFS-xfs(IRIX)	NFS-ufs(Solaris)

Samba 3.0.13	Not OK	Not OK		OK
Samba 3.0.12	Not OK	Not OK		OK
Samba 3.0.11	OK		OK			OK
Samba 3.0.10	OK		OK			OK


level 10 debug and corresponding tcpdump captures have been made of 3.0.11
and 3.0.12 performing the same actions.  I sent a link to these to
jra@samba.org
Comment 10 Jeremy Allison 2005-04-04 10:19:07 UTC
Ok, the problem is definately the large directory code changes that went into
Samba in 3.0.12. However the problem with it only occurring on IRIX local XFS
means this will be difficult to track down (I don't have access to an IRIX box
locally). I'm going to ask James Peach of SGI if he can have a look at this bug
as he is the SGI Samba maintainer.
Jeremy.
Comment 11 James Peach 2005-04-04 16:56:41 UTC
(In reply to comment #10)
> Ok, the problem is definately the large directory code changes that went into
> Samba in 3.0.12. However the problem with it only occurring on IRIX local XFS
> means this will be difficult to track down (I don't have access to an IRIX box
> locally). I'm going to ask James Peach of SGI if he can have a look at this bug
> as he is the SGI Samba maintainer.

Can I also have the logs and traces?

Also .. 6.5.7 and 6.5.5 are ancient and unsupported releases. I strongly
recommend upgrading to something recent like 6.5.24. There have been a lot
of improvements to the I/O paths and network stack, not to mention bug fixes.

Comment 12 James Peach 2005-04-10 17:28:59 UTC
I just ran a simple test with Samba 3.0.13 (SVN HEAD) and an IRIX
6.5.24 server and was not able to reproduce the problem. Using a
pretty basic smb.conf:

oops 43# cat /usr/samba/lib/smb.conf
[global]
        security = user
        passwd program = /usr/bin/passwd %u
        passwd chat = *ew*password:* %n\n *e-enter*new*password:* %n\n
        os level = 0
        preferred master = No
        local master = No
        domain master = No
        comment = %h (Samba %v)
        printer admin = lp
        print command = /usr/samba/bin/sambalp %p %s %U %m
[tmp]
        comment = Temporary file space
        path = /usr/tmp
        read only = No
        guest ok = Yes

oops 44# ls -lf /usr/tmp/t
.
..
1
2
3
7
8
9
10
4
5
6

oops 45# df -h /usr/tmp/t
Filesystem             Type  Size   use  avail  %use  Mounted on
/dev/root               xfs  6.4G  2.8G   3.6G   44%  /

smb: \t\> dir
  .                                   D       0  Mon Apr 11 10:17:09 2005
  ..                                  D       0  Mon Apr 11 10:21:20 2005
  1                                   D       0  Mon Apr 11 10:16:48 2005
  2                                   D       0  Mon Apr 11 10:16:48 2005
  3                                   D       0  Mon Apr 11 10:16:48 2005
  7                                   D       0  Mon Apr 11 10:16:48 2005
  8                                   D       0  Mon Apr 11 10:16:48 2005
  9                                   D       0  Mon Apr 11 10:16:48 2005
  10                                  D       0  Mon Apr 11 10:16:48 2005
  4                                   D       0  Mon Apr 11 10:17:09 2005
  5                                   D       0  Mon Apr 11 10:17:09 2005
  6                                   D       0  Mon Apr 11 10:17:09 2005

                52762 blocks of size 131072. 29551 blocks available

AFAICT, the directory code is working fine.
Comment 13 James Peach 2005-04-12 17:36:41 UTC
Date: Tue, 12 Apr 2005 15:28:11 +1000
From: James Peach <jpeach@sgi.com>
To: Mac <mac@nibsc.ac.uk>
Cc: jpeach@sgi.com, jra@samba.org
Subject: Re: [Samba] last file missing from wildcard searches (3.0.13)

On Mon, Apr 11, 2005 at 05:50:30PM +0100, Mac wrote:
> Hi James,
> 
>       Just noticed two more things.
> 
> Firstly the original reporter of the problem (Joe Adams) states he is
> using 6.5.24f.
> 
> Secondly he saw the problem with directories (not plain files).

I still can't reproduce this using plain files or mixtures of
directories and files.

> So I'm now unsure what to suggest.  I'll continue to try and replicate
> the problem we saw onto 6.5.24 here at NIBSC, but perhaps we should talk
> to Joe Adams and see what light he can shed on this issue.

So I dug into your logs and came up with something interesting:

~ [21] grep get_lanman2_dir_entry log.3.0.12
  get_lanman2_dir_entry:readdir on dirptr 0x10437480 now at offset 49
  get_lanman2_dir_entry found fred/. fname=.
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x10437480 now at offset 50
  get_lanman2_dir_entry found fred/1 fname=1
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x10437480 now at offset 51
  get_lanman2_dir_entry found fred/2 fname=2
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x10437480 now at offset 55
  get_lanman2_dir_entry found fred/3 fname=3
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x10437480 now at offset 56
  get_lanman2_dir_entry found fred/7 fname=7
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x10437480 now at offset 57
  get_lanman2_dir_entry found fred/8 fname=8
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x10437480 now at offset 5934
  get_lanman2_dir_entry found fred/9 fname=9
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x10437480 now at offset 6320
  get_lanman2_dir_entry found fred/.. fname=..
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x10437480 now at offset 6320
  get_lanman2_dir_entry:readdir on dirptr 0x10499e38 now at offset 49
  get_lanman2_dir_entry found fred/. fname=.

Now in the HEAD code, the offsets in this message are the return value
of telldir. AFAICT XFS never stores directory entries at sequential
offsets. For example:

# ls -lf /usr/tmp/t
.
.. 
5
6 
7 
8 
9 
1 
2 
3 
4 
# /tmp/readdir /var/tmp/t
/var/tmp/t: . at offset 4
/var/tmp/t: .. at offset 6
/var/tmp/t: 5 at offset 16
/var/tmp/t: 6 at offset 18
/var/tmp/t: 7 at offset 20
/var/tmp/t: 8 at offset 22
/var/tmp/t: 9 at offset 24
/var/tmp/t: 1 at offset 26
/var/tmp/t: 2 at offset 28
/var/tmp/t: 3 at offset 30
/var/tmp/t: 4 at offset 32

My smbd log of this directory listing concurs:
grep get_lanman2_dir_entry log.smbd
  get_lanman2_dir_entry:readdir on dirptr 0x104da320 now at offset 4
  get_lanman2_dir_entry found t/. fname=. 
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x104da320 now at offset 6
  get_lanman2_dir_entry found t/.. fname=..
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x104da320 now at offset 16
  get_lanman2_dir_entry found t/5 fname=5
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x104da320 now at offset 18
  get_lanman2_dir_entry found t/6 fname=6
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x104da320 now at offset 20
  get_lanman2_dir_entry found t/7 fname=7
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x104da320 now at offset 22
  get_lanman2_dir_entry found t/8 fname=8
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x104da320 now at offset 24
  get_lanman2_dir_entry found t/9 fname=9
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x104da320 now at offset 26
  get_lanman2_dir_entry found t/1 fname=1
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x104da320 now at offset 28
  get_lanman2_dir_entry found t/2 fname=2
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x104da320 now at offset 30
  get_lanman2_dir_entry found t/3 fname=3
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x104da320 now at offset 32
  get_lanman2_dir_entry found t/4 fname=4
  get_lanman2_dir_entry: SMB_FIND_FILE_BOTH_DIRECTORY_INFO
  get_lanman2_dir_entry:readdir on dirptr 0x104da320 now at offset 32
So, I theorise that sometime between 3.0.12 and today's HEAD, the
directory code was fixed to use real telldir offsets instead of virtual
offsets. I had a brief archaeological excursion into svn but wasn't able
to quickly prove this. Jeremy can prolly comment usefully however.

In summary, my guess is that this problem is not in 3.0.13 or 3.0.14.
Sorry I can't be more helpful atm ..
Comment 14 Jeremy Allison 2005-04-12 17:45:41 UTC
The code was rewritten to use telldir() directory offsets. I'm closing this one
out unless it can be reproduced with the latest released code.
Jeremy.
Comment 15 Mac 2005-04-15 07:11:19 UTC
(In reply to comment #14)
> The code was rewritten to use telldir() directory offsets. I'm closing this 
one
> out unless it can be reproduced with the latest released code.
> Jeremy.
> 

We've tried 3_0r (using "svn co svn://svnanon.samba.org/samba/branches/
SAMBA_3_0_RELEASE samba-3_0_release" as of 2005-04-15 at around 08:15 GMT)

and the problem IS still there.  As suspected, the code introduced in 3.0.12 
that causes this trouble has not (yet) been repaired.


We will try 3.0.14a and 3.0.15rc1 when formally released, but I don't hold out 
much hope.
Comment 16 Jeremy Allison 2005-04-15 08:32:34 UTC
The code isn't going to change. You need to work with SGI support on this one -
there's definately something going wrong with an interaction between your
particular version of IRIX and the new directory code. As James Peach of SGI
says, it's not possible to reproduce this on any system he has access to. You
need to bring in vendor support, it doesn't look like a Samba problem.
Jeremy.
Comment 17 Jeremy Allison 2005-04-21 00:59:22 UTC
Created attachment 1167 [details]
Patch
Comment 18 Joe Adams 2005-04-21 09:51:55 UTC
Ya!!! The patch worked.

I just tried the smbd/dir.c patch in 3.0.14a. All directory entries are now 
appearing.

Thank you Jerry, James, and "mac" for all your support on this. I have a bunch 
of happy users again.

___________________________________________


bash-2.04# pwd
/mnt6
bash-2.04# /usr/freeware/bin/ls -Ul
total 8
drwxrwxrwx  20 djhester nmd          4096 Apr 20 09:30 due
drwxrwxrwx   4 root     nmd            42 Apr 19 10:26 tmp
drwxrwxr-x  28 djhester nmd          4096 Apr 21 09:50 etap
drwxr-xr-x   4 root     sys            34 May 15  2002 library
drwxr-xr-x   2 root     sys             9 Feb 24  2003 download
drwxrwxrwx   4 root     sys            43 Oct 20  2004 geowall
bash-2.04#

X:\mnt6>dir
 Volume in drive X is writable
 Volume Serial Number is 022B-2147

 Directory of X:\mnt6

04/20/2005  06:38 AM    <DIR>          .
04/17/2005  03:00 AM    <DIR>          ..
04/20/2005  09:30 AM    <DIR>          due
04/19/2005  10:26 AM    <DIR>          tmp
04/21/2005  09:50 AM    <DIR>          etap
05/15/2002  10:24 AM    <DIR>          library
02/24/2003  01:52 PM    <DIR>          download
10/20/2004  11:53 AM    <DIR>          geowall
               0 File(s)              0 bytes
               8 Dir(s)   7,615,504,384 bytes free

X:\mnt6>



Comment 19 Mac 2005-04-22 07:48:10 UTC
(In reply to comment #18)
> Ya!!! The patch worked.
> 
> I just tried the smbd/dir.c patch in 3.0.14a. All directory entries are now 
> appearing.

We've just tried the Cale patch, and then Jeremy patch and both seem to exhibit 
the same behaviour.

In summary:

          IT WORKS!

Hooray.  Thank you all for the efforts.  


One difference between 3.0.11 (the last version that worked) and
3.0.14a+this-patch is the ordering of the files returned in the CMD's "dir" 
command is now the natural order of the files on the disk, not the sorted order.
(this actually only showed up for the '..' entry.)

I'm sure this makes no practical difference to anything though.
Comment 20 James Peach 2005-04-25 18:03:49 UTC
Capturing some info that flew past the list ..

This problem is specific to XFS filesystems using version 1 naming (ie.
directories). In this case, the last offset returned by telldir is
4294967295, which gets clobbered to -1 when the Samba code casts the 64
bit off_t to a 32 bit long.

Version 2 naming example:

oops 42# xfs_growfs -n /mnt/alternate/
meta-data=/mnt/alternate/        isize=256    agcount=8, agsize=211200 blks
data     =                       bsize=4096   blocks=1689600, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 2              bsize=4096   mixed-case=Y
log      =internal               bsize=4096   blocks=1200
realtime =none                   extsz=65536  blocks=0, rtextents=0
oops 45# /tmp/readdir /mnt/alternate/tmp/t
/mnt/alternate/tmp/t: . at telldir offset 4, d_off offset 4
/mnt/alternate/tmp/t: .. at telldir offset 6, d_off offset 6
/mnt/alternate/tmp/t: 1 at telldir offset 8, d_off offset 8
/mnt/alternate/tmp/t: 2 at telldir offset 10, d_off offset 10
/mnt/alternate/tmp/t: 3 at telldir offset 12, d_off offset 12
/mnt/alternate/tmp/t: 4 at telldir offset 14, d_off offset 14
/mnt/alternate/tmp/t: 5 at telldir offset 16, d_off offset 16
/mnt/alternate/tmp/t: 6 at telldir offset 18, d_off offset 18
/mnt/alternate/tmp/t: 7 at telldir offset 20, d_off offset 20
/mnt/alternate/tmp/t: 8 at telldir offset 22, d_off offset 22
/mnt/alternate/tmp/t: 9 at telldir offset 24, d_off offset 24
/mnt/alternate/tmp/t: 10 at telldir offset 26, d_off offset 26

Version 1 naming example:

oops 46# xfs_growfs -n /mnt/jpeach/
meta-data=/mnt/jpeach/           isize=256    agcount=18, agsize=262144 blks
data     =                       bsize=4096   blocks=4495025, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 1              bsize=4096   mixed-case=Y
log      =internal               bsize=4096   blocks=1136
realtime =none                   extsz=65536  blocks=0, rtextents=0
oops 47# /tmp/readdir /mnt/jpeach/t
/mnt/jpeach/t: . at telldir offset 49, d_off offset 49
/mnt/jpeach/t: 1 at telldir offset 50, d_off offset 50
/mnt/jpeach/t: 2 at telldir offset 51, d_off offset 51
/mnt/jpeach/t: 3 at telldir offset 52, d_off offset 52
/mnt/jpeach/t: 4 at telldir offset 53, d_off offset 53
/mnt/jpeach/t: 5 at telldir offset 54, d_off offset 54
/mnt/jpeach/t: 6 at telldir offset 55, d_off offset 55
/mnt/jpeach/t: 7 at telldir offset 56, d_off offset 56
/mnt/jpeach/t: 8 at telldir offset 57, d_off offset 57
/mnt/jpeach/t: 9 at telldir offset 5934, d_off offset 5934
/mnt/jpeach/t: .. at telldir offset 6320, d_off offset 6320
/mnt/jpeach/t: 10 at telldir offset 4294967295, d_off offset 4294967295

You should not be using v1 directories. There are a number of known
problems with them, ranging from being slow, to sometime returning
incorrect results. Fortunately for Linux folks, v1 naming was never
part of the Linux port.

Comment 21 Joe Adams 2005-04-26 08:46:08 UTC
Thanks for that info James. This is good stuff to know.

Our SGI Origin 2100 and storage have been around for a few years. On the page 
http://oss.sgi.com/projects/xfs/faq.html, I found that version 2 directory 
naming became the default at IRIX 6.5.5. It's very possible these XFS 
filesystems were created before 6.5.5 and I did verify with the xfs_growfs -n 
command that our file systems are version 1.

Comment 22 Gerald (Jerry) Carter (dead mail address) 2005-08-24 10:18:50 UTC
sorry for the same, cleaning up the database to prevent unecessary reopens of bugs.