Bug 12380 - vfs_fruit support for Time Machine over SMB
vfs_fruit support for Time Machine over SMB
Status: ASSIGNED
Product: Samba 4.1 and newer
Classification: Unclassified
Component: VFS Modules
unspecified
All All
: P5 enhancement
: ---
Assigned To: Ralph Böhme
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2016-10-14 02:23 UTC by Kevin
Modified: 2016-11-10 23:16 UTC (History)
3 users (show)

See Also:


Attachments
Capture from a stock Samba 4.4.6 installation (4.52 MB, application/gzip)
2016-10-14 19:18 UTC, Kevin
no flags Details
Capture from a Samba version running the attached patch for capabilities (15.31 KB, application/gzip)
2016-10-14 19:19 UTC, Kevin
no flags Details
Capture from a working Mac OSX server providing Time capsule services (257.92 KB, application/gzip)
2016-10-14 19:20 UTC, Kevin
no flags Details
WIP: Samba capabilities patch (1.55 KB, patch)
2016-10-14 19:22 UTC, Kevin
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kevin 2016-10-14 02:23:38 UTC
Given that Apple has deprecated AFP for Time Machine backups in favor of SMB it would be beneficial to be able to use Samba to provide a network based backup destination.

Apple has published the required capabilities for supporting Time Machine backups over SMB and can be found here: https://developer.apple.com/library/content/releasenotes/NetworkingInternetWeb/Time_Machine_SMB_Spec/

Looking through the code I think this probably fits the best in vfs_fruit.c as that module handles a large amount of other integrations for Mac OSX.

I also have Wireshark captures of a working Time Machine backup to an OSX server and a non-working backup to a Samba server if that would be beneficial.

I started attempting to implement the required features but I would appreciate some help with some of the pieces. 

I was able to change vfs_fruit to advertise the capability correctly but I can not determine if it is possible to flush the data when recieving an SMB2 FLUSH within the VFS layer. Is that possible?

Based on what I can find in the captures I believe that all of the other features are implemented already including SMB2_CREATE_DURABLE_HANDLE_REQUEST_V2 and SMB2_CREATE_REQUEST_LEASE_V2. 

-Kevin
Comment 1 Stefan Metzmacher 2016-10-14 07:53:30 UTC
Please compress the pcap files with gzip --best and attach them to this bug report.

Attaching you work in progress patch would also help.

Thanks!
Comment 2 Ralph Böhme 2016-10-14 08:33:39 UTC
Here's some more stuff that nicely explains what F_FULLSYNC does on OS X:
<http://lists.apple.com/archives/darwin-dev/2005/Feb/msg00072.html>

I remember I simply faked this for Netatalk.
Comment 3 Jeremy Allison 2016-10-14 17:45:41 UTC
Ralph, please ping me about this requirement. I've been talking to Apple directly about this.

Jeremy.
Comment 4 Kevin 2016-10-14 19:18:58 UTC
Created attachment 12576 [details]
Capture from a stock Samba 4.4.6 installation
Comment 5 Kevin 2016-10-14 19:19:38 UTC
Created attachment 12577 [details]
Capture from a Samba version running the attached patch for capabilities
Comment 6 Kevin 2016-10-14 19:20:15 UTC
Created attachment 12578 [details]
Capture from a working Mac OSX server providing Time capsule services
Comment 7 Kevin 2016-10-14 19:22:31 UTC
Created attachment 12579 [details]
WIP: Samba capabilities patch

This is a patch to advertise F_FULLSYNC to the client, but it does nothing beyond that at this time.
Comment 8 Kevin 2016-10-16 02:02:44 UTC
I was able to get the patch that I provided to work successfully for Time Machine backups. The reason that it was not working after applying the patch was due to not having durable file handles configured. After adding the following to my smb.conf file I was able to start a Time Machine backup.

- durable handles = yes
- kernel oplocks = no
- kernel share modes = no
- posix locking = no

After the backup I will attempt to restore a file to verify functionality. After reading what Ralph mentioned regarding the F_FULLSYNC fnctrl, my personal opinion is that it is OK to fake it like I did in the patch but I would certainly like some concurrence from others with more knowledge on Samba internals.

Thanks!
Kevin
Comment 9 Kevin 2016-10-24 00:43:40 UTC
I did some more research on this and I believe that advertising the capability and taking no other action is an acceptable course of action. I went ahead and made it a configurable option, updated the man page, tested the code, and created a Github pull request for this issue. Here is the link to the PR: https://github.com/samba-team/samba/pull/64

Thanks!
Kevin
Comment 10 Ralph Böhme 2016-11-01 11:18:36 UTC
Thanks for the patches and please keep pushing me to get them integrated. I need to discuss a few technical things with Jeremy related to this before I can start to do a review. So please stay tuned... :)
Comment 11 Kevin 2016-11-03 01:48:56 UTC
Thanks Ralph! I'll ping you in a week through this bug (if you prefer a different interval or communication method just let me know) for a status check/update. In the meantime if you think of anything that can be added or changed just let me know.
Comment 12 Kevin 2016-11-10 23:16:41 UTC
Hi Ralph,
  Have you had a chance to discuss this patch with Jeremy yet?

Thanks,
Kevin