Bug 12153 - Possible malfunctions of rsync when destination is on fat32 filesystem
Summary: Possible malfunctions of rsync when destination is on fat32 filesystem
Status: RESOLVED WORKSFORME
Alias: None
Product: rsync
Classification: Unclassified
Component: core (show other bugs)
Version: 3.1.1
Hardware: x86 Linux
: P5 normal (vote)
Target Milestone: ---
Assignee: Wayne Davison
QA Contact: Rsync QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-08-15 10:18 UTC by Daniel
Modified: 2020-07-27 23:18 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel 2016-08-15 10:18:06 UTC
I use rsync to transfer a large number of files locally from one disk to another.

rsync -avv --progress /media/source/ /media/destination/

source is on an ext4 filesystem 
destination is on a fat32 filesystem

The destination disk was checked and found to be OK by the badblocks-command
The destination filesystem was freshly created by mkdosfs.

There are chinese characters in the filenames. Filename encoding is utf-8 on both source (defined by locales) and destination (reported by mount command).

Rsync ends up with an error message saying that a file from source is bigger than 4gb. So far, this is expected behaviour because of the file size limitation on a FAT32 filesystem.

Possible malfunction 1: Some other files smaller than 4gb are missing in the destination as well.

Possible malfunction 2: When repeating the rsync command (to resume the transfer), files are transferred that do already exist in the dest directory. The files match in creation time and in comparison by diff command. fsck does not report any errors. But nevertheless, some files are transferred again. I can see big files being transferred again using the --progress option.
Comment 1 Daniel 2016-08-15 11:43:05 UTC
Possible malfunction 2 might be somehow related to case sensitivity in filenames.
Comment 2 Daniel 2016-08-15 12:43:17 UTC
Possible malfunction 2: Files are missing in destination. These files have been shown to be transferred with the --progress option at a speed that exceeds the maximum write speed of the disk on which the destination filesystem lays on.
Comment 3 Daniel 2016-08-15 14:01:53 UTC
Have been picking on the problem.

I think that the problem with files which already exist in destination but are transferred again is not caused by rsync but by a time comparison mismatch which can occur when timestamp accuracy differs between source and destination filesystem.

It would be nice to add something to the manpage, section "--modify-window", that this option makes sense when timestamp accuracy of source and destination filesystem differs.

The reported problem remains that files are missing in the destination (and reported to have been transfered at a speed greater than possible)
Comment 4 Daniel 2016-08-15 14:40:29 UTC
This report might be a duplicate of 10675
Comment 5 Wayne Davison 2020-07-27 23:18:04 UTC
If rsync aborts with an error then there might be more files yet to transfer than rsync didn't get around to.

The extra transferring might be the off-by-one timestamps since some FAT filesystems cannot store an odd timestamp.  You should also check to make sure that you don't have files that differ by upper/lower case that are confusing rsync.