Bug 3815 - cifs kernel vfs reports incorrect file times
cifs kernel vfs reports incorrect file times
Status: RESOLVED FIXED
Product: CifsVFS
Classification: Unclassified
Component: kernel fs
2.6
x86 Linux
: P3 major
: ---
Assigned To: Steve French
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2006-05-31 09:34 UTC by Jan Hlavaty
Modified: 2009-10-30 16:16 UTC (History)
3 users (show)

See Also:


Attachments
Raw capture file of Windows XP doing dir on the file (3.62 KB, application/octet-stream)
2006-06-01 06:01 UTC, Jan Hlavaty
no flags Details
Raw capture of smbclient making dir on the file (1.25 KB, application/octet-stream)
2006-06-01 06:01 UTC, Jan Hlavaty
no flags Details
Raw capture file of cifs vfs doing stat on the file (1.47 KB, application/octet-stream)
2006-06-01 06:02 UTC, Jan Hlavaty
no flags Details
Ethereal dump of the cifs capture (37.51 KB, text/plain)
2006-06-01 06:03 UTC, Jan Hlavaty
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Hlavaty 2006-05-31 09:34:11 UTC
I have file on one linux system which is exported using samba server and I'm trying to access it from another linux system using cifs filesystem to back it up. Both systems are set to use Europe/Prague timezone (which has DST support - CET +0100 n winter, CEST +0200 summer) and their time is synchronized correctly. Now it is DST, file has date from non-DST time.

On local filesystem using stat, file has these dates:
Access: 2006-05-31 14:48:28.000000000 +0200
Modify: 2006-01-25 16:17:31.000000000 +0100
Change: 2006-01-25 16:17:31.000000000 +0100

When accessing the file from a Windows machine or from linux machine using smbclient, I can see correct times. I take it as a proof that samba server side works correctly.

However, when mouning the share using cifs filesystem, times seem to be incorrect (stat over the file in the mounted directory):

Access: 2006-05-31 14:48:28.000000000 +0200
Modify: 2006-01-25 15:17:31.000000000 +0100 <<< one less hour
Change: 2006-01-25 15:17:31.000000000 +0100 <<< one less hour

This is very annoying because it ruins incremental backups over the cifs.
Looks like it is only taking into account timezone offset but not the DST somewhere. Because all works well in the samba-windows & samba-smbclient cases it must be a problem in cifs vfs.
Comment 1 Steve French 2006-05-31 12:41:00 UTC
Could you get two ethereal traces (or tcpdump traces, saving the file as binary) of stat of the file.  One trace of stat over Linux cifs client and the other of smbclient or Windows client doing the equivalent.

Since Windows servers (at least since Windows NT 4), most SMB/CIFS appliances and Samba servers return the time in "DCE time" (100 nanoseconds since 1600) date conversion is not usually an issue - but since Windows clients do not support the CIFS Protocol Unix extensions, the Samba server is responding to a somewhat different request from Windows and Linux CIFS clients (so it could be a server bug).
Comment 2 Jan Hlavaty 2006-06-01 05:59:48 UTC
I have done the traces of windows XP client, smbvlient and cifs vfs all getting the file information. It looks that in all cases, times are sent correnctly on the wire in GMT:

Windows XP client:

  capture-windows.bin, frame 5 request, frame 6 response:

        QUERY_PATH_INFO Data
            Created: Jan 25, 2006 15:17:31.000000000
            Last Access: May 31, 2006 15:09:27.000000000
            Last Write: Jan 25, 2006 15:17:31.000000000
            Change: Jan 25, 2006 15:17:31.000000000

Smblient on linux:

  capture-smbclient.bin, frame 1 reuqest, frame 2 reposnse:
  
        FIND_FIRST2 Data
                Created: Jan 25, 2006 15:17:31.000000000
                Last Access: May 31, 2006 15:09:27.000000000
                Last Write: Jan 25, 2006 15:17:31.000000000
                Change: Jan 25, 2006 15:17:31.000000000 

cifs:

  capture-cifs.bin, request frame 6, response frame 7:
  
        QUERY_PATH_INFO Data
            Last status change: Jan 25, 2006 15:17:31.000000000
            Last access: May 31, 2006 15:09:27.000000000
            Last modification: Jan 25, 2006 15:17:31.000000000

So server is probably OK.

There must be a problem in the way the time is presented to the kernel.
Maybe you pass it in GMT and it is supposed to be in local time???
Comment 3 Jan Hlavaty 2006-06-01 06:01:10 UTC
Created attachment 1929 [details]
Raw capture file of Windows XP doing dir on the file
Comment 4 Jan Hlavaty 2006-06-01 06:01:53 UTC
Created attachment 1930 [details]
Raw capture of smbclient making dir on the file
Comment 5 Jan Hlavaty 2006-06-01 06:02:32 UTC
Created attachment 1931 [details]
Raw capture file of cifs vfs doing stat on the file
Comment 6 Jan Hlavaty 2006-06-01 06:03:19 UTC
Created attachment 1932 [details]
Ethereal dump of the cifs capture
Comment 7 Jan Hlavaty 2006-06-01 06:38:32 UTC
Looking at the way inode->i_atime is filled, these times seem to be in the servers local timezone (atime comes from current_fs_time() -> current_kernel_time() -> xtime).

So what we probably need to do is convert the time from GMT to servers timezone before filling the inode->i_*time ...
Comment 8 Jan Hlavaty 2006-06-01 09:25:41 UTC
But what intriques me is that there are files that have the time right.
The one hour difference manifests itself only for dates that do not fall within DST. For me, 1.1.2006 will have problem, 1.6.2006 will not...
Is it known what timezone should the data on the wire be in for the protocol?

I will try some more packet captures on file with DST...
Comment 9 Jan Hlavaty 2006-06-01 10:43:15 UTC
Looks like Samba is sending times not in GMT, but in samba server timezone wall clock, including DST offset. That is, for Europe/Prague timezone, it will send times in CET at winter and in CEST st summer - regardless of the time's value.
If you ignore the DST status, file dates will appear to move at DST change.

In contrast, Linux only uses the timezone offset internally and does not care about DST.

To compensate for this, cifs would have to know whether the server thinks it is DST or not, otherwise file dates will change with DST change...
Too bad kernel does not have full timezone information available ;(

Maybe server timezone could be set as one of the paramemters for mount?
Since mount.cifs is in userspace, it could eventually extract the required information (timezone offset and DST intervals) and pass it to kernel...
That way we would be able to somewhat reliably convert smb times to GMT and then to linux kernel time.
Comment 10 Steve French 2006-06-01 15:14:29 UTC
Interesting ... 
My guess is that the userspace smbclient is checking timezone and dst differently (perhaps in something in smb.conf) than userspace (stat etc.).
Do kde and/or gnome file manager display the same as in "stat -f?"
It sounds like a localization setting is wrong on the Linux client. Not sure if this conversion is done in libc or in the utility (stat or gnome or kde)

they (system time and dce time) are both UTC time.  As you can see from the cifs code, kernel does not know anything about timezone or dst (see extract from fs/cifs/netmisc.c)

889 struct timespec
890 cifs_NTtimeToUnix(u64 ntutc)
891 {
892 struct timespec ts;
893 /* BB what about the timezone? BB */
894
895 /* Subtract the NTFS time offset, then convert to 1s intervals. */
896 u64 t;
897
898 t = ntutc - NTFS_TIME_OFFSET;
899 ts.tv_nsec = do_div(t, 10000000) * 100;
900 ts.tv_sec = t;
901 return ts;
902 }
Comment 11 Steve French 2006-06-01 15:25:54 UTC
Presumably functions like ctime handle the conversion
http://en.wikipedia.org/wiki/Unix_timestamp
Comment 12 Steve French 2006-06-01 15:36:28 UTC
I am fairly sure we will find out the cause is the incorrect setting of timezone on one of the machines (or a bug in a userspace time conversion routine used by stat application)
Comment 13 Steve French 2006-06-01 15:36:44 UTC
I am fairly sure we will find out the cause is the incorrect setting of timezone on one of the machines (or a bug in a userspace time conversion routine used by stat application)
Comment 14 Jan Hlavaty 2006-06-02 04:36:42 UTC
Nope, I have checked that, of course.

Both machines are set for timezone "Europe/Prague", and system hardware clock is configured to be in UTC. /etc/localtime are binary identical on both systems.

Only possible difference is the uptime - client has booted now (in DST) and server is up almost for months now (booted in non-DST time).

But the unix timestamps as seen on stat() do not differ between the servers, they have same values for identical times.

How can i find out the exact values for the times from the packets?
Ethereal seems to decode the times into something that is in CEST timezone (DST). I dont know if it makes some timezone conversion or not.

As for the theory that kernel times (xtime, current_kernel_time()) are in UTC, I will try to verify that.
Comment 15 Jan Hlavaty 2006-06-02 05:20:38 UTC
I have verified that all linux timestamps are in UTC - all the way from stat() thru syscall to inode in vfs.
Comment 16 Jan Hlavaty 2006-06-02 05:55:23 UTC
Here is a summary of what is going on:

file:                    test1.txt                  test2.txt
linux local stat:        2006-06-01 00:00:00 +0200  2006-01-01 00:00:00 +0100
windows time (CEST):     2006-06-01 00:00           2006-01-01 00:00
windows time (CET):      2006-05-31 23:00           2005-12-31 23:00
local timespec:          1149112800                 1136070000
cifs timespec:           1149112800                 1136066400
windows timespec (CEST): 1149112800                 1136070000
windows timespec (CET):  1149109200                 1136066400

I have changed windows local time to january to get the CET values.
Comment 17 Jan Hlavaty 2006-06-02 06:19:24 UTC
in readable form:

local timespec:          2006-05-31 22:00:00 UTC    2005-12-31 23:00:00 UTC
cifs timespec:           2006-05-31 22:00:00 UTC    2005-12-31 22:00:00 UTC
windows timespec (CEST): 2006-05-31 22:00:00 UTC    2005-12-31 23:00:00 UTC
windows timespec (CET):  2006-05-31 21:00:00 UTC    2005-12-31 22:00:00 UTC
Comment 18 Steve French 2006-06-02 17:19:57 UTC
Also curious if smbfs (mount -t smbfs) sees the same problem as cifs
Comment 19 Jan Hlavaty 2006-06-05 05:44:51 UTC
Yes smbfs suffers the same problem. I mounted the share locally on the same server to avoid any undetected timezone setting problem.


One difference I have noticed that may cause some confusion is in the way Windows and Linux handle local time.

On Windows, timestamps are always displayed in the current DST offset - so all file dates dates appear to move one hour with the DST change (I have verified that for NTFS file times and samba mounted shares).

On Linux, if i see the output of stat or ls with --time-style=full-iso, I see that times are handled differently - they seem to display according to DST mode of the time itself - notice the +0100 for winter date and +0200 for summer date.

Since I have set the file dates through the Windows client, timestamp of the linux file backing the share is in fact incorrect - it reads 2006-01-01 00:00:00.000000000 +0100 while through Windows I have in fact set it to 2006-01-01 00:00 +0200 so it should read 2005-12-31 23:00 +0100!
Comment 20 Jan Hlavaty 2006-06-05 06:16:48 UTC
I tried to mount a real Windows share with cifs, and file dates seem to behave in a correct and expected way - that is, dates as seen in linux stat on the mounted share that are not in DST show as one less hour because linux shows them in CET istead of CEST as Windows does.

So as far as I can tell, real problem is not in cifs / smbfs but in Samba itself sending incorrect GMT times!

Someone probably saw the one hour difference between linux and windows times without understanding where it comes from and "fixed" it by subtracting one hour somewhere from the GMT times that are being sent over the wire... now to find that place in Samba ;)
Comment 21 Guenter Kukkukk 2006-10-06 18:13:34 UTC
Hi Jan,
does this problem still exist?

If so, please post more background information:
 - the Linux kernel versions used on client and server
   (plus the used Linux distros)
 - the version of samba you are running on the server
 - the Microsoft clients you used - WinXP Home or Professional
   (did you use other Windows clients during your tests?)

I stay in the same timezone and DST area as you (Berlin, Germany).
That should not matter anyway - but helps me a bit, when I look at
your ethereal sniffs, where wire timestamps are displayed in localtime!

I tried to re-produce your problem - but was not able to find _any_
glitch inside cifs or samba3 - when using files with timestamps outside
or inside DST.

Which does not mean, that your observations are wrong!

There are different timestamp related calls inside the SMB/CIFS
protocol.
In your very first post you said, that the *nix command 
'stat filename' resulted in a _different_ output, when issued 
 - on the local side on the server 
 - or on the mounted cifs share on the client
That still holds true?

Assuming your Linux server is named 'server' and your Linux
client is named 'client'.

What's the output of the following - done on your 'server':
  net time -S client
  net time zone -S client

and the corresponding ones on the Linux 'client' side:
  net time -S server
  net time zone -S server

Cheers, Guenter
Comment 22 Felix Miata 2006-11-29 20:22:54 UTC
Someone please tell me if https://bugzilla.novell.com/show_bug.cgi?id=224790 is essentially this bug, or if I should file a new one in this bugzilla to get cifs timestamps fixed.
Comment 23 Steve French 2009-10-30 16:16:34 UTC
times seen to be sent correctly.  Reopen if you see a problem in the cifs client.