Bug 3255 - The smb.conf option "store dos attributes = yes" does not always create the EA "user.DOSATTRIB"
The smb.conf option "store dos attributes = yes" does not always create the E...
Status: RESOLVED FIXED
Product: Samba 3.0
Classification: Unclassified
Component: File Services
3.0.21
All All
: P3 normal
: none
Assigned To: Jeremy Allison
Samba QA Contact
:
: 3256 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2005-11-11 18:31 UTC by Guenter Kukkukk
Modified: 2005-11-15 20:30 UTC (History)
1 user (show)

See Also:


Attachments
Patch (695 bytes, patch)
2005-11-13 23:27 UTC, Jeremy Allison
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Guenter Kukkukk 2005-11-11 18:31:15 UTC
Hello Jeremy,

the following is related to samba3 svn rev. 11685.

I know, that some days ago the semantics of "store dos attributes" has
chandeg slightly.

Beside other changes, samba3 now _first_ tries to store dos attributes into
the EA "user.DOSATTRIB". If that fails, a fallback to the former used file 
system bits is done.

I cannot say, whether the latest code changes result in the following glitch,
cause I never did test this before!

SYMPTOM:
When the following is set in smb.conf in the [global] section
  store dos attributes = yes
  ea support = yes
  (and _no_ "map" entry is used)
and the used file system does support xattr.

When copying a new file from OS/2 to a samba3 share (so the file is newly 
created on samba3), linux getfattr shows, that "user.DOSATTRIB" is not created!
The former used file system execute bits and the inverted user 'w' are still
in use. ('chown u-x file.txt' will now remove the Archive bit again).

The source file had only the archive (or readonly) bit set.
(When the System or Hidden bits were already present in the source, then
the EA is always created!)

I was wondering, why this does not happen, when a windows client is used.
Some additional DEBUG() statements in smbd/dosmodes.c and 2 ethereal traces
are showing the following.
In smbd/dosmode.c, function file_set_dosmode() line 372

        if (dos_mode(conn,fname,st) == dosmode)      !!!!!
                return(0);                           !!!!!

        /* Store the DOS attributes in an EA by preference. */
        if (set_ea_dos_attribute(conn, fname, st, dosmode)) {
                return 0;
        }

if (dos_mode(...) == dosmode) was always true with OS/2, but not with windows,
so the following EA setting function was not called!
The interesting thing is, that when the archive bit 0x20 should be set,
OS/2 is passing dosmode == 0x20 and windows (mostly) is passing 0xa0 !!!
So the Bit FILE_ATTRIBUTE_NORMAL (0x80) is additionally set on windows,
it was not defined in the OS/2 days - and should be masked off here also
in samba.
So, if we add the following to dosmode.c: 
        dosmode &= SAMBA_ATTRIBUTES_MASK;     /* !!!!!!! */
        if (dos_mode(conn,fname,st) == dosmode)
                return(0);

OS/2 and windows should behave the same here - and of course, now the
bug can _also_ be seen when using windows!

Jeremy, I leave it up to you, to find the location, which needs some
modification.

Best wishes.
Guenter Kukkukk

PS - my irc talks yesterday where not intended to make you feel unhappy! :-)
I hope, this is helping a bit...
-----
btw - OS/2 is also not masking out "invalid file bits" - here FileCommander/2
was falling into that pitfall:
http://bugs.silk.apana.org.au/show_bug.cgi?id=95
Samba3 is setting FILE_ATTRIBUTE_NORMAL on line 2920 in trans2.c
        if (!mode)
                mode = FILE_ATTRIBUTE_NORMAL;
whithout looking for the used SMB dialect version.
I agree, OS/2 should also mask off the upper bits!
FC/2 was taking the samba3 passed file attributes and passed them over to
DosOpen() for the dest. file - "access denied". Also not bad :-)
Comment 1 Guenter Kukkukk 2005-11-11 18:58:54 UTC
*** Bug 3256 has been marked as a duplicate of this bug. ***
Comment 2 Jeremy Allison 2005-11-11 22:21:26 UTC
You are still not understanding me. I still need a coherent bug report and this
is not it. I will try and explain again.

Just because "store dos attributes = yes" IT IS NOT A BUG if user.DOSATTRIB is
not created. If the client sets a certain set of attributes and when it does a
getattr it gets the same attributes back then that is not a bug. It is
*IRRELEVENT* whether the user.DOSATTRIB is set or not. Is that clear. Not
setting a user.DOSATTRIB EA IS NOT A BUG SO LONG AS THE CLIENT SEES THE CORRECT
VALUE !!!!!!

Now, if you have a case where you are setting dos attributes using a file create
or a ATTR command in OS/2, and the attribute bits you are getting back are
different from the ones you set, then THAT IS A BUG.

What I want you to do is to demonstrate this case to me. Simply not adding a
user.DOSATTRIB is NOT A BUG if the client gets the correct result back. Is the
client getting the correct result back ? If not when is this occuring. How do
you reproduce that ?

That is the bug report I have asked for. This is not it. I will fix the adding
of the FILE_ATTRIBUTE_NORMAL setting if the protocol level is < NT1, that is a
valid bug.

Jeremy.
Comment 3 Guenter Kukkukk 2005-11-12 20:58:21 UTC
Hello Jeremy,

some strange noise here - and on irc - about me ... 
I am not a bastard on a beautiful island laying at the beach, to distract you 
from
your great samba work!

Yeah - I am interested in "meta data" being stored onto *nix file systems, like
OS/2 EAs or windows alternate data streams...
And samba3 and samba4 are the ones (which I am aware of), who really use those 
features.

I do not boot my OS/2 boxes every day anymore - but I have some programming 
experience with that anchient operating system - and a still working development
environment.
My intention is, to help a bit, so still active OS/2 users can reliably use 
samba.

Oh well - on that way, sometimes there are pitfalls and misunderstandings.
-----
And this one seems to hit all possible ugly pitfalls at one time:
  - Some days ago I was preparing my post to samba-technical
    "Is the sky actually falling? (NTFS streams) - some additional notes"  
    To do a _fair_ comparison between samba3 and samba4 regarding the max.
    available xattr space, I tried to _force_ samba3 to use it's own
    internal xattr space (i.e. "user.DOSATTRIB") - so it would be subtracted
    from the remaining user xattr space.
    That first tries failed for me! I was not able to force samba3 to do so!
    (And I was using an OS/2 applet to check the max. EA size)

  - I had followed your very initial irc talks with abartlet about some new
    semantics with "map readonly", where you said something like
    "... you can only do that right, when using EAs..."
    Oh well - that one did start MY misunderstanding!  

From that moment on, I thought, "EAs _must_ always be used to do it right..."
I AM REALLY SORRY - MY FAULT!

And - windows was ALWAYS creating an EA, when a file was copied over to samba!
(OS/2 did not).
I got completely confused, when you said on irc "that is an optimisation" -
 "that
is done for efficiency"

11/11/05  4:34:46 <wanon> I take it you're talking about :
11/11/05  4:34:56 <wanon> if (dos_mode(conn,fname,st) == dosmode) { return 0; }
11/11/05  4:35:03 <wanon> This is intentional.
11/11/05  4:35:33 <wanon> If the dos mode we want to set is identical to what 
is already seen there (either via ea entries or by looking at the sbuf)
11/11/05  4:35:38 <wanon> then we don't set a new one.
11/11/05  4:35:45 <wanon> This is an efficiency.  !!!!!!!!!!!!!!!!!!!!!!!!
11/11/05  4:36:06 <wanon> Every file doesn't need to have an EA if the mode 
bits would already evaluate to the correct value.
11/11/05  4:36:22 <wanon> Why do you need an EA if the correct value is 
returned ?

What an "efficiency", when _the_ samba client (windows) was _always_ _creating_ 
an EA?
That lead to my proposal to add
        dosmode &= SAMBA_ATTRIBUTES_MASK;     /* !!!!!!! */
        if (dos_mode(conn,fname,st) == dosmode)
                return(0);
So then, that comparison would also been hit true, when windows was including 
the
FILE_ATTRIBUTE_NORMAL (0x80) bit (which has no meaning here) - and which is 
_lateron_ 
masked off in
   set_ea_dos_attribute()
      ...
      snprintf(attrstr, sizeof(attrstr)-1, "0x%x", dosmode & 
SAMBA_ATTRIBUTES_MASK);
      ...
(I still think - a bit too late regarding efficiency...)
-------------------
CONLUSION:
I definitely went the _wrong_ way - was hunting the "efficiency" way on the
source code side - not what the samba client is seeing.
So - this one is NO bugreport at all!
Jeremy - what I have seen so far - the clients _always_ get the right 
attributes back! :-)
-------------------

But - there is another possible pitfall - when one is talking to other users, 
too:

There was a strange time coincidence on 10th November 2005 - which got me 
walking
my wrong way some steps further:
----------
On 10th November 2005 on irc channel #samba-os2-technical:
btw - nick "nefud" (Alex Masterov) is a _very_ skilled OS/2 and samba user!
(samba bugzilla also reflects that)

[08:59:09] <nefud> I can not set any DOS attribute on SVN version of samba...  
smbclient:
smb: \2\> dir  
    1.txt          A        0  Wed Nov  9 17:11:54 2005    
smb: \2\> setmode 1.txt +h    
perm set 2 0  
smb: \2\> dir
    .              D        0  Wed Nov  9 17:12:28 2005    
    ..             D        0  Wed Oct 26 08:17:04 2005
    1.txt          A        0  Wed Nov  9 17:11:54 2005
[08:59:21] <nefud> Can you check it?
---------------

I'm sure, Alex will report on this!
Cause he is often testing against FreeBSD, this one could be
a completely different story.

Jeremy - again, sorry for all the confusion!

Best wishes - Guenter
Comment 4 Guenter Kukkukk 2005-11-13 12:37:59 UTC
Jeremy,

i definitely agree, that setting the dos attributes could be
expensive regarding syscall overhead.
So any optimisation seems to be the right way.

Both OS/2 and windows at least use 2 calls regarding dos
attr settings, when 1 file is copied.

Assuming a source file has only the ARCHIVE Bit set:
                      dosmode  dosmode
                       OS/2    Windows
--------------------+--------+--------+
1. During Create       0x20     0x20
2. SET_FILE_INFO       0x20     0xa0

I had added some DEBUG() output to watch the programm flow:

#if defined(GK_OPTIMISE)
       dosmode &= SAMBA_ATTRIBUTES_MASK;
#endif
        if (dos_mode(conn,fname,st) == dosmode) {
                DEBUG(0,("GK0: EARLY RETURN from setting dos mode 0x%x on file %
s\n", dosmode, fname));
                return(0);
        }
        DEBUG(0,("GK1: TRY EA setting dos mode 0x%x on file %s\n", dosmode, 
fname));

        /* Store the DOS attributes in an EA by preference. */
        if (set_ea_dos_attribute(conn, fname, st, dosmode)) {
                return 0;
        }
....
Obviously, on OS/2 the "GK0: EARLY RETURN..." path was taken both times.
On Windows, only the 1st call did hit that, and the 2nd call always went
the "GK1: TRY EA setting...." path.

I might be wrong, but I guess, your intention was, that the "EARLY RETURN.."
path should have been always taken (here) - for efficiency.
The FILE_ATTRIBUTE_NORMAL Bit has no meaning here, so my proposal was to add
       dosmode &= SAMBA_ATTRIBUTES_MASK;

So even more efficiency is possible!? :-)

I guess, this whole stuff had better been discussed on samba-technical @

Cheers, Guenter
Comment 5 Alex Masterov 2005-11-13 21:08:58 UTC
Hello, All!

I found that some dos attributes can not be changed since samba 3.0.20b
It's new problem.

Samba 3.0.20b
OS Linux RedHat 7.1 
FS Ext2 w/o extattr

store dos attributes is not set.

[alex@~]$ smbclient //prosmb/alex -U alex
added interface ip=192.168.1.200 bcast=192.168.1.255 nmask=255.255.255.0
Got a positive name query response from 192.168.1.220 ( 192.168.1.220 )
Password:
Domain=[HELMI] OS=[Unix] Server=[Samba 3.0.20b]
smb: \> cd 2
smb: \2\> ls
  .                                   D        0  Mon Nov 14 09:38:31 2005
  ..                                  D        0  Mon Nov 14 09:38:21 2005
  1.txt                               A        0  Mon Nov 14 09:38:30 2005

                60037 blocks of size 65536. 28912 blocks available
smb: \2\> setmode 1.txt +h

perm set 2 0
smb: \2\> ls
  .                                   D        0  Mon Nov 14 09:38:31 2005
  ..                                  D        0  Mon Nov 14 09:38:21 2005
  1.txt                               A        0  Mon Nov 14 09:38:30 2005

                60037 blocks of size 65536. 28911 blocks available

i.e. dos attrib hidden is not set.
Comment 6 Alex Masterov 2005-11-13 22:02:13 UTC
It's interesting but I can set Archive and cannot set Hidden attribute on RedHat
7.1.

smb: \2\> ls
  .                                   D        0  Mon Nov 14 09:38:31 2005
  ..                                  D        0  Mon Nov 14 10:39:11 2005
  1.txt                                        0  Mon Nov 14 09:38:30 2005

                60037 blocks of size 65536. 28847 blocks available
smb: \2\> setmode 1.txt +a
smb: \2\> ls
  .                                   D        0  Mon Nov 14 09:38:31 2005
  ..                                  D        0  Mon Nov 14 10:39:11 2005
  1.txt                               A        0  Mon Nov 14 09:38:30 2005

                60037 blocks of size 65536. 28847 blocks available


Setting world executable flag to the file via chmod
-rwxr--r-x    1 alex     alex            0 &#1053;&#1086;&#1103; 14 09:38 1.txt

does not make this file hidden

  1.txt                               A        0  Mon Nov 14 09:38:30 2005
Comment 7 Alex Masterov 2005-11-13 22:06:17 UTC
On RH 7.1 I tried to set "store dos attributes" to yes without any effect.
Comment 8 Alex Masterov 2005-11-13 22:09:23 UTC
I have the same troubles on FreeBSD with samba 3.0.22pre1-SVN-build-11591.

But I can not set Hidden dos atribute with "store dos attribute" set to yes or no.
Setting extattr DOSATTRIB by hand has no effect. 
Comment 9 Jeremy Allison 2005-11-13 23:06:49 UTC
Just tried to reproduce this problem on a revision 11716 (Nov 13th Pacific time)
build of SAMBA_3_0 and cannot. With "store dos attributes = yes" set on a xfs
filesystem supporting EA's I can set any combination of EA's desired.
Jeremy.
Comment 10 Jeremy Allison 2005-11-13 23:27:39 UTC
Created attachment 1569 [details]
Patch

This should restore the optimisation for Windows as well as OS/2.
Jeremy.
Comment 11 Alex Masterov 2005-11-14 04:01:49 UTC
(In reply to comment #9)
> Just tried to reproduce this problem on a revision 11716 (Nov 13th Pacific time)
> build of SAMBA_3_0 and cannot. With "store dos attributes = yes" set on a xfs
> filesystem supporting EA's I can set any combination of EA's desired.
> Jeremy.

Jeremy, problem as I described on RedHat 7.1 appears only on filesystems that
does not support extattr at all or "store dos attributes = no". 
Comment 12 Jeremy Allison 2005-11-14 17:52:29 UTC
Ok, you know that by default "map hidden" and "map system" are set to "no" right ? And you also know that you have to set the "create mask = 0777" in order for the mapping onto permission bits to work on a filesystem with no EA's right... ?

I'm asking because when I set these correctly and try and reproduce your problem with "store dos attributes = no" I can't reproduce the problem....

I don't think you've got your smb.conf set up correctly to do this.

Jeremy.
Comment 13 Alex Masterov 2005-11-14 21:00:40 UTC
(In reply to comment #12)
> 
> I don't think you've got your smb.conf set up correctly to do this.

You are right, map hidden & create mask 0777 helps.

But problem with FreeBSD still exists.
In samba 3.0.20b and upper under FreeBSD option "store dos attributes" does not work, i.e. extattr DOSATTIB does not affect to file attributes that fileserver shows. Changing file attributes works but samba changes permission bits not DOSATTRIB extattr. 
OS/2 EAs work perfectly.
Comment 14 Jeremy Allison 2005-11-14 21:29:23 UTC
This looks like a FreeBSD bug not a Samba one. I believe this code works perfectly under Linux as Guenter Kukkukk has confirmed. You need to tell me exactly how you reproduce this problem under FreeBSD so I can confirm this myself.
Jeremy.
Comment 15 Alex Masterov 2005-11-14 21:36:03 UTC
(In reply to comment #14)
> This looks like a FreeBSD bug not a Samba one.

OK, I'll open new report.
Thanks!

Comment 16 Guenter Kukkukk 2005-11-15 20:25:43 UTC
Jeremy,

what a shame - what you now (simply) pass here - and put into
http://websvn.samba.org/cgi-bin/viewcvs.cgi/branches/SAMBA_3_0/source/smbd/dosmode.c?rev=11718&view=log
...
Filter stored DOS attributes by SAMBA_ATTRIBUTES_MASK (0x7f).
Jeremy
...
If you look at your former commit message(s), you put some
_more_ info there (...IRC talks ...)

Why did you NOT mention our sometimes ugly IRC talks there?
And, that it was not _your_ idea, to improve the code?
(tridge nearly always is doing so)!!!

You and abartlet were "whining" on irc ... frustrating guy...

Well - I'm looking forward for better times and understandings.
Guenter
Comment 17 Jeremy Allison 2005-11-15 20:30:44 UTC
Sorry, I usually do put comments into the commit that it was someone else's idea -  but I'm only human and sometimes I forget. Remember I've had to deal with you and Alex bugging me on this one for a long time, and most of the bug report is just wrong. Once I saw your matrix I understood what the problem was and applied it immediately. Not the world's best dealt with bug report but it got fixed in the end.
Jeremy.