Bug 13654 - Samba doesn't implement macOS SMB2 F_FULLFSYNC Extension
Summary: Samba doesn't implement macOS SMB2 F_FULLFSYNC Extension
Status: RESOLVED INVALID
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: File services (show other bugs)
Version: unspecified
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Ralph Böhme
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-10-13 07:32 UTC by Ralph Böhme
Modified: 2020-03-16 13:32 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ralph Böhme 2018-10-13 07:32:02 UTC
Summary says it all. This should be implemented to make Time Machine backups reliable in event of connection failures or other error conditions.

References:
https://developer.apple.com/library/archive/releasenotes/NetworkingInternetWeb/Time_Machine_SMB_Spec/#//apple_ref/doc/uid/TP40017496-CH1-SW2
https://lists.samba.org/archive/samba-technical/2017-January/118292.html
https://lists.samba.org/archive/samba-technical/2017-February/118779.html
Comment 1 Ralph Böhme 2018-10-15 14:51:21 UTC
...adjusting summary to make it more clear that is about a Apple specific SMB2 protocol   modification...
Comment 2 Ralph Böhme 2019-02-12 08:00:19 UTC
What's missing is something like a call to syncsf() on Linux or sync() on other OSes.
Comment 3 Jeremy Allison 2019-02-12 20:01:28 UTC
Should we add a VFS_SYNCFS(int fd) call - on other OS'es this just does an underlying sync() (no fd) ? Then the AAPL code could call it.
Comment 4 Ralph Böhme 2019-02-12 20:34:31 UTC
Either that or we somehow manage to funnel this through SMB_VFS_FSCTL. Do we a private FSCTL_XXX namespace that we can use and add a FSCTL_SAMBA_F_FULLSYNC?
Comment 5 Ralph Böhme 2020-01-06 15:04:26 UTC
I did some more research and I'm concluding that modern Linux fsync() behaves exactly like macOS fcntl(fd, ..., F_FULLSYNC).

Modern Linux fsync():

       fsync() transfers ("flushes") all modified in-core  data  of
       (i.e., modified buffer cache pages for) the file referred to
       by the file descriptor fd to the disk device (or other  per‐
       manent  storage  device) so that all changed information can
       be retrieved even if the  system  crashes  or  is  rebooted.
       This  includes  writing  through or flushing a disk cache if
       present.

Iow modern Linux fsync takes two steps:
1. Flush OS buffers to devices
2. Request device to flush device buffers to plattern

By contrast macOS fsync():

     Fsync() causes all modified data and attributes of fildes to
     be moved to a permanent storage device.  This normally
     results in all in-core modi-fied modified fied copies of
     buffers for the associated file to be written to a disk.

     Note that while fsync() will flush all data from the host to
     the drive (i.e. the "permanent storage device"), the drive
     itself may not physi-cally physically cally write the data
     to the platters for quite some time and it may be written in
     an out-of-order sequence.

This is only step 1 from above. The F_FULLSYNC fcntl() implements 1 and 2:

     F_FULLFSYNC Does the same thing as fsync(2) then asks the drive to
                 flush all buffered data to the permanent storage
                 device (arg is ignored).  This is currently
                 imple-mented implemented
                 mented on HFS, MS-DOS (FAT), and Universal Disk Format
                 (UDF) file systems.  The operation may take quite a
                 while to complete.  Certain FireWire drives have also
                 been known to ignore the request to flush their
                 buffered data.


On older Linux kernels fsync() didn't implements step 2, also there we're some filesystem bugs that got fixed, eg https://bugs.launchpad.net/ubuntu/+source/linux/+bug/504632.

On such systems that don't implement step 2 in fsync() which likely also includes FreeBSD, it might still be possible to workaround this by calling syncfs() or sync() when we receive the macOS SMB2 FULLSYNC request, given that those syscalls are likely requesting a device buffer flush. But on modern Linux there's nothing to do for us.
Comment 6 Jeremy Allison 2020-01-06 23:19:38 UTC
So we can just map the F_FULLSYNC on a handle to our regular VFS_FSYNC_SEND/RECV() calls and we're done, yeah ?
Comment 7 Ralph Böhme 2020-01-07 09:17:02 UTC
(In reply to Jeremy Allison from comment #6)
No, we already call VFS_FSYNC_SEND for an SMB2 FLUSH. No further action needed.