Using the following system:
Debian 11 with rsync 3.2.3
Fedora 38 with rsync 3.2.7

Creating the following environment:
mkdir -p src/x cmp/x dst/
setfattr -n user.foo -v bar cmp/x

Executing the following command:
rsync --recursive --xattrs --dry-run --compare-dest=../cmp/ src/ dst/

Results in the following error:
rsync: [generator] copy_xattrs: lsetxattr("/home/user/dst/x","user.foo") failed: No such file or directory (2)

The issue only seems to occur if:
x is a directory (inside another, not as a direct argument)
--recursive, --xattrs and --dry-run are set
--compare-dest, --copy-dest or --link-dest are set

The error seems to come from:
source file generator.c in function recv_generator (line 1489)
https://github.com/WayneD/rsync/blob/v3.2.7/generator.c#L1489

The error seems to be either:
recv_generator (in generator.c) calling copy_xattrs when it should not
or
copy_xattrs (in xattrs.c) / sys_lsetxattr (in lib/sysxattrs.c) not checking the dry-run flag similarly to the functions in syscall.c do