rsync 3.0.9 on Mac OS X 10.6.8 As an example for working extended attributes: File labels are always properly synced: Both during their first sync when the file + attributes is copied and also after you change their states and sync again, the file remains untouched, but the attributes are properly updated! But Spotlight comments are not transferred, neither when a file is synced the first time nor when the comment is changed, and another sync is then run. For completeness, this is the command line I ran: rsync \ --protect-args \ \ --archive \ --crtimes \ --hard-links \ --acls \ --xattrs \ --fileflags \ --one-file-system \ --force-change \ \ --exclude=".*" \ --exclude="Icon*" \ \ --verbose \ --stats \ --progress \ --itemize-changes \ --human-readable \ \ --dry-run \ \ ~/Temporary/ \ /Volumes/ExternalDisk/All/
1) Could someone please answer how to successfully sync Spotlight comments? Thanks! 2) As of 3.0.9 this functionality seems not to be integrated in the main distribution, as my testing in my previous comment shows, but through patching it seems to be possible User Gnarlodious reports at: https://discussions.apple.com/message/12142363#12142363 Downloaded 3.0.7 and applied the patch "fileflags.diff" (among others) and then comments seem to get properly synced.
I downloaded 3.0.9 source + patches, patched in the 2 patches: patch -p1 <patches/fileflags.diff patch -p1 <patches/crtimes.diff And although running with the appropriate arguments (as in the original report), the comments still do not get synced! Help please!
Hmm, stock 3.0.7 with official fileflags & crtimes patches was working and 3.0.9 with those patches is not? That's weird. I don't have access to an OS X box at the moment, but should be able to look into this in the future. If anyone wants to dive in and debug, feel free.
I'd appreciate some help! I'm still unable to correctly rsync Finder / Spotlight comments.
(In reply to comment #3) > Hmm, stock 3.0.7 with official fileflags & crtimes patches was working and 3.0.9 with those patches is not? That's weird. I now also downloaded source 3.0.7, patched it with 3.0.7, and built it, and it doesn't work as well.
My own build of rsync preserves Spotlight comments just fine. Those comments are stored as extended attributes, so as long as you preserve xattrs, your Spotlight comments should be preserved. Before you make the conclusion that your Spotlight comments have *not* been preserved, verify whether the "com.apple.metadata:kMDItemFinderComment" exists on the destination file. Also, you should verify that Spotlight is enabled on the destination. mdutil -s /Volumes/Target [bombich:/Volumes/Source] touch test [bombich:/Volumes/Source] xattr --list test <add a Spotlight comment in Finder> [bombich:/Volumes/Source] xattr --list test com.apple.metadata:kMDItemFinderComment [bombich:/Volumes/Source] xattr --get com.apple.metadata:kMDItemFinderComment test com.apple.metadata:kMDItemFinderComment bplist00^This is a tes [bombich:/Volumes/Source] ls -la /Volumes/Target/test ls: /Volumes/Target/test: No such file or directory <rsync Source to Target, preserving xattrs -- I used 3.0.6 + some various bug fixes from 3.0.7-3.0.9> [bombich:/Volumes/Source] ls -la /Volumes/Target/test -rw-r--r--@ 1 bombich wheel 0 Mar 23 11:18 /Volumes/Target/test [bombich:/Volumes/Source] xattr --list /Volumes/Target/test com.apple.metadata:kMDItemFinderComment [bombich:/Volumes/Source] xattr --get com.apple.metadata:kMDItemFinderComment /Volumes/Target/test com.apple.metadata:kMDItemFinderComment bplist00^This is a tes [Note: I use a custom build of the "xattr" tool so the arguments may be different on your system] Lastly, you should note that Finder has some display problems with Spotlight comments. I don't know if the problem resides within Finder or Spotlight, but I've seen odd delays associated with the proper information appearing the the Spotlight comments field. For example, the Spotlight comments appear in the xattr on the destination file, but they do not appear immediately in the Finder. Additionally, if you delete the source file, then re-touch it and Get Info, the old Spotlight comments are still there, even though the xattr is empty. So don't use the Finder as a measure of whether these comments are preserved, use the xattr contents instead.
Dear Mike, thanks for your effort! Some questions remain, maybe you can help? I performed the tests as advised, and can now confirm: What I experience(d) all the time, is indeed a heavy Finder bug, rather than an rsync bug! xattr proofed, that after every rsync, all comments were properly synced! It is the buggy Finder, which hides the proper comment info: * Finder does not only conceal Finder comments of freshly synced files (0-10 seconds ago), but also from files, which were synced many months (!) and sessions ago! * Even if I select "Show Info" (CMD-I) for a destination file, which was recently or long ago rsynced, I get no info, even though xattr prooves, that it is there! * "Odd delay" is an understatement! This is a large scale bug. Is there any way to force Finder to update its comment display? (Or a Spotlight Update/Re-Indexing of the target volume?) The methods below did not accomplish comment display update: * Refresh a Finder window to force it to update: http://hints.macworld.com/article.php?story=2009091413423819 * Restart Finder * Log out & in again * Restart Mac
Interesting: I now performed a .DS_Store removal, according to: http://www.finetunedmac.com/forums/ubbthreads.php?ubb=showflat&Number=11266#Post11266 cd /Volumes/Destination/Folder osascript -e 'tell application "Finder" to quit' rm .DS_Store rm ../.DS_Store osascript -e 'tell application "Finder" to activate' After that ALL comments are gone (in the Finder view). Before that, at least those which I added at the destination were shown. xattr -rl prooves that all comments are still there, only now NONE shown in Finder.
mdutil -E /Volumes/Destination Re-indexing the destination volume did not help either. xattr -rl showed that all comment data is still there.
User V.K. posted a script here: https://discussions.apple.com/message/10549961#10549961 I ran V.K.'s script over the rsync backup'ed files and then the comments were restored! Also after disconnecting, un-powering, re-powering, re-connecting the external harddisk, the restored comments persisted. My post: https://discussions.apple.com/message/21704242#21704242 On my rsync backup destination the files had shown NO comments in the Finder but mdls -name kMDItemFinderComment <YourFile> had revealed that they were stored (at least somewhere). For test purposes I created the folder "_1" with comment "1" on the source with Mac OS X Finder 10.6.8, ran rsync 3.0.7 (compiled with xattrs and others), and on the destination again no comment was shown for folder "_1". After running the comment-restore-applescript over "_1" the comment was there! I created folder "_2" with comment "22" on the source, and copied it via Finder. The comment appeared on the destination as well. CONCLUSION: It seems rsync syncs the Finder comments to a deprecated storage place?! Or Finder has a bug, ignoring info on valid storage positions.
Apple Developer Bug Reporting Team replied today: Engineering has determined that this issue behaves as intended based on the following information: Comments are stored primarily in the .DS_Store files and mirrored to Spotlight for searching. xattrs are not the primary storage. This is the effect of attributes being stored externally, and command line tools not being aware of them. --- Now I realize why my rsync failed: I had used --exclude=".*" which excluded .DS_Store from being synced! Finder's metadata handling being only .DS_Store-aware and mostly ignoring xattrs poses the following problems: PROBLEM 1: It is neither fish nor fowl a) The labels ARE considered from xattrs, more precisely at least ALSO considered from xattrs. Proof: rsync xattr sync results immediately visible in Finder. b) Whereas the comments are considered SOLELY from .DS_store. Proof: Rsync results not visible in Finder. Workaround: See comment 10 PROBLEM 2: Partial file+metadata syncs are IMPOSSIBLE by design without .DS_Store merging capabilities Even if I would include .DS_Store files in my sync, synching a folder partially MUST fail by design. Why? If metadata would be per file, it is transported correctly on whatever selection of files I sync over. If metadata is per one .DS_Store per folder for multiple files, and you sync only a few files plus the .DS_Store of that folder, you OVERWRITE the .DS_Store on the destination. Then the metadata for the files being both in the source and destination are correct, but those only at the destination LOOSE their metadata. So a sync process must be .DS_Store aware and MERGE .DS_Store from source and destination, which practically EXCLUDES MOST 3rd PARTY apps from properly syncing metadata!!! What is my application: My entire projects are too large for my internal disk. Therefore I eventually keep them on an external disk. The actual projects are always on my internal disk, and I rsync them over additively (without the option "delete on destination") to my external disk every here and then. If eventually done, I sync a last time, and then erase on my internal disk. Therefore my source (internal HD) is a subset of my external disk. And syncing only parts over with a .DS_Store unaware tool would copy over only the synced files' metadata but ERASE the metadata of all other files on the destination (external HD). EXAMPLE: Source: 3, 4 -- all with metadata Destination: 1, 2, 3, 7, 8 -- all with metadata Outcome: If I sync over 3+4 with a DS_Store unaware sync tool, then 3 gets its metadata updated, 4 gets copied freshly, and all others (1, 2, 7, 8) loose their metadata!