When using the --link-dest= option and the link destination is the same as the source, rsync should skip the comparison step and just link all regular files and create other filetypes (directories, device nodes, etc).
Hi. I filed bug 8712 as I was not sure if my situation there is the same as you are describing here. Could you take a look at 8712 and see if that describes what you are referring to?