Using the fruit:time machine setting. On the macOS computer Time Machine sees the identity of the machine changing every time and thus rejects any previous backups there and will create new ones and not allow restoration. However macOS calculates the "identity" of the backup is apparently not compatible with Samba's fruit:time machine plugin, at least for now. Tested on Fedora 28. Samba v4.8.5 included with Fedora. Apparently, macOS's Time Machine changed with Sierra and this has been a problem since then for various people in several situations. Most of these are sporadic though (not always being invalidated). However, the Samba problem is quite persistent, even with backups attempted minutes apart. I can find one other person with this problem, also persistent, asked on Sep 19, 2018: https://apple.stackexchange.com/questions/336930/time-machine-over-samba-the-identity-of-the-backup-disk-has-changed They were using Arch Linux and it still occurred. This problem seems to be relatively new and not a problem since macOS Sierra.
I had this problem with with 4.8.x, it looked like the MachineID file would end up being empty after every backup that ran. The 'workaround' I used was to remove write permission for that file, that made the backup work but resulted in a verify after every backup. When using 4.9.1 + the solaris build patch I am no longer seeing the issue (file is now writable again and it gets update successfully) So there is potentially a fix in 4.9.x or master for this.
I tried to build and install Samba 4.9.1 without squashing the system one without success. The Fedora line will update to 4.9.1 in about 11 days which is why I didn't want to squash the system one so readily. Will report back around then.
Got around to testing this with 4.9.1 now that FC29 is out and it works as expected. Must have been fixed by something. Thanks!