I suspect the hidden flag is being persisted into the DOS attributes, stored as an xattr on the file.

As Andrew already said, this is likely an interaction between the stored DOS attribues and an attribute being set due to filename.

If a client writes the returned attributes back to the server, then the HIDDEN attribute may get persisted into the extended attribute that stores DOS attributes.

What isn't clear is how you are reproducing this.

Can you be more specific in how you reproduce ? ie. Don't just say:

"1. touch .file3 ( OK hidden DOS flag enabled )   
2. mv -v .file3 file3 ( OK hidden DOS flag disabled )"

Are you doing this on the Linux server ? Or are you doing this via a Windows or Linux client ? What is the contents of your smb.conf.

What would help is instructions to reproduce from first principles.

Be clear whether actions are being done on client or server. Be clear on what client/server you are using. Post full smb.conf.

Thanks !

There is only a server and two clients involved

Server:
Red Hat Enterprise Linux release 8.5 (Ootpa)
samba-4.14.5-7.el8_5.x86_64

[SHARE1]
    path = /SHARES/SHARE1
    valid users = +testgroup
    public = no
    writable = no
    write list = user1 user2 user3
    printable = no
    create mask = 0660
    directory mode = 0770
    force group = +testgroup
    sync always = yes


Linux Client:
samba-common-4.14.5-9.el8_5.noarch
samba-common-libs-4.14.5-9.el8_5.x86_64
samba-client-libs-4.14.5-9.el8_5.x86_64

Windows 10 client


All following commands are done on the linux samba client which has mounted a share from the server with credentials from user3 which is in the testgroup. The Windows 10 client also has the same share mounted and is purely used to ***observe*** the changes on files.

All following commands are executed on the Linux client

1. touch .file3 ( OK hidden DOS flag enabled )   
2. mv -v .file3 file3 ( OK hidden DOS flag disabled )
3. mv -v file3 .file3 ( OK hidden DOS flag enabled ) 
4. mv -v .file3 file3 ( FAIL hidden DOS flag enabled, should be disabled )

Another test

1. touch .file4.syncthing.tmp ( OK hidden DOS flag enabled )   
2. mv -v .file4.syncthing.tmp file4 ( OK hidden DOS flag disabled )
3. mv -v file4 .file4.syncthing.tmp ( OK hidden DOS flag enabled ) 
4. mv -v .file4.syncthing.tmp file4 ( FAIL hidden DOS flag enabled, should be disabled )

OK, this gives me enough info to try and reproduce. However, it's likely this is the client writing back the attribute onto the file, and once the hidden bit gets set (from the .filename) and returned to the client, the client is probably propagating this across file renames.

The addition of FILE_ATTRIBUTE_HIDDEN on a dot name is done via the function dos_mode_post(), which overlays HIDDEN onto a DOS mode read from the user.DosAttrib extended attribute. This attribute is *NOT* stored by the server inside the user.DosAttrib extended attribute when it comes from a dot-file name, but only added into the value returned to the client.

But once the client writes a DOS attribute back onto a file via SMB1/2/3, then if that attribute contains HIDDEN then it's going to be written back into the user.DosAttrib extended attribute, and at that point we no longer know if the attribute came from the name, or from the client application driving the SMB redirector (i.e. the client application *needs* the file to be marked as "HIDDEN", no matter what the name). So we can't remove the client requested HIDDEN attribute, because we don't know if the client is deliberately setting it, or if it is propagated from a previous name of the file.

What I'm saying is this is probably unfixable. "hide dot files = yes" is a hint, and to be honest you should probably not expect to depend on it being removed when the name is changed to remove the dot.