Bug 10545 - Rsync first time deletes files, second time it synchronates files with same name
Summary: Rsync first time deletes files, second time it synchronates files with same name
Status: NEEDINFO
Alias: None
Product: rsync
Classification: Unclassified
Component: core (show other bugs)
Version: 3.0.9
Hardware: x64 Linux
: P5 normal (vote)
Target Milestone: ---
Assignee: Wayne Davison
QA Contact: Rsync QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-04-09 20:03 UTC by Erik
Modified: 2020-07-26 07:48 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Erik 2014-04-09 20:03:09 UTC
I have debian 7 X64 latest installed on my box.
Running rsync 3.0.9.
On both boxes the same versions.
I have an box with ip 192.168.0.6 on
which i make a daily backup

Daily backups are made in the directory /var/disk/disk-20/backup/daily
The mounting point of the disk is disk-20
Disk-20 is a 3 Tb Harddisk
It has a directory backup with 2 sub-dirs; one is daily, one is month.
Every first Monday of the month the fullbackup from Sunday is moved from daily to month.

The backup is copied to another host with rsync.
I use the command:

su - rsync -c "/usr/bin/rsync -avh --delete --progress /var/disk/disk-20/ --exclude={lost+found} rsync:192.168.0.5:/var/disk-01 >/$logpath/rsync.log"

(disk-01 is a partition 2 of 146 Gb on a 200Gb Harddisk; partition 1 is Debian)

I noticid that synchronizing on Monday was just a few minutes,
but on thuesday it took about 26 minutes.
I found out that on thuesday it took so long because the file in month was synchronised. This should be done on monday.
So on Monday after move of the full backup from daily to month the file in daily was deleted.
And on thuesday month was synchronized.

Has this something to do with the fact that the names of the files are the same.
(BACKUP-x-full-2014-04-06.tgz


If there is any information you need, please let me know.

Erik
Comment 1 Kevin Korb 2014-04-09 20:11:58 UTC
I am really not sure I understand what you are describing here so I will give general backup with rsync advice...

1.  Don't rsync -c unless you really really have to.  It is horribly inefficient and should only be used in a few unusual use cases.

2.  Forget this daily/monthly moving stuff.  Use rsync with either --link-dest or --backup-dir to manage your multiple backups.  Note that you can get backup systems that are based on this such as rsnapshot, dirvish, backuppc if you don't want to do your own scripting.
Comment 2 Erik 2014-04-10 15:08:31 UTC
Thanx for your respons.


I dont understand your remark at dont rsync -c. I think i'm doing the following:


What i do with this command is:

su - rsync -c "/usr/bin/rsync -avh  --delete --progress /var/disk/disk-20/ --exclude={/lost+found} rsync@192.168.0.5:/var/disk/disk-01 > $logpath/rsync.log"

Change user to user rsync 
and execute the next command (as user rsync) :

/usr/bin/rsync -avh  --delete --progress /var/disk/disk-20/ --exclude={/lost+found} rsync@192.168.0.5:/var/disk/disk-01


In short just copy all from disk-20 to host:disk-01
( rsync -avh  --delete  /var/disk/disk-20/ rsync@192.168.0.5:/var/disk/disk-01 )

But Also if its not efficient to move files.
Still i think its strange
Day 1: 1 mv file 1 from daily to month on host 1 (before running rsync)

On host 2; 
Day 1 : file is deleted from daily
Day 2 : file is rsynced/copied to month

why does this cost 2 days, and is it not done al the same. Just because i ask
rsync to synchronize one dir (disk-20) with one dir (disk-01)?

I want to understand..
Comment 3 Wayne Davison 2020-07-26 07:48:00 UTC
You'd need to provide more information on the layout of the files in your copy for us to try to help you.