Configure rsync to sync data between two Linux/Unix server (Backup data)

What is Rsync

Rsync (Remote Sync) is a  most commonly used command for copying and synchronizing files and directories remotely as well as locally in Linux/Unix Systems,

With the help of rsync command you can copy and synchronize your data remotely and locally across directories, across disks and networks, perform data backups and mirroring between two Linux machines.

Advantages and features of rsync command

1. It efficiently copies and sync files to or from a remote system.
2. Supports copying links, devices, owners, groups and permissions.
3. It’s faster than scp (Secure Copy) because rsync uses remote-update protocol which allows to transfer just the differences between two sets of files. First time, it copies the whole content of a file or a directory from source to destination but from next time, it copies only the changed blocks and bytes to the destination.
4. Rsync consumes less bandwidth as it uses compression and decompression method while sending and receiving data both ends.

syntax of rsync command

# rsync options source destination

common options used with rsync commands

1. -v : verbose
2. -r : copies data recursively (but don’t preserve timestamps and permission while transferring data
3. -a : archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps
4. -z : compress file data
5. -h : human-readable, output numbers in a human-readable format

1. Copy/Sync Files and Directory Locally

[root@gcdc-nms source]# rsync -azvh /opt/source/ /tmp/backup/

sending incremental file list

created directory /tmp/backup

./

1

2

3

4

file1

sent 255 bytes  received 110 bytes  730.00 bytes/sec

total size is 0  speedup is 0.00

[root@gcdc-nms source]#

2. Copy/Sync Files and Directory to or From a Remote Server

[root@gcdc-nms source]# rsync -azvh /opt/source/ root@10.200.236.60:/tmp/backup/

root@10.200.236.60's password:

sending incremental file list

created directory /tmp/backup

./

1

2

3

4

file1

sent 255 bytes  received 110 bytes  42.94 bytes/sec

total size is 0  speedup is 0.00

You have new mail in /var/spool/mail/root

[root@gcdc-nms source]#

3. Rsync Over SSH

[root@gcdc-nms source]# rsync -azvhP -e ssh /opt/source/ root@10.200.236.60:/tmp/backup
/root@10.200.236.60's password:sending incremental file listcreated directory /tmp/backup./1           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=4/6)2           0 100%    0.00kB/s    0:00:00 (xfer#2, to-check=3/6)3           0 100%    0.00kB/s    0:00:00 (xfer#3, to-check=2/6)4           0 100%    0.00kB/s    0:00:00 (xfer#4, to-check=1/6)file1           0 100%    0.00kB/s    0:00:00 (xfer#5, to-check=0/6)sent 255 bytes  received 110 bytes  42.94 bytes/sectotal size is 0  speedup is 0.00
[root@gcdc-nms source]#

4. Use of –include and –exclude Options

[root@gcdc-nms source]# ll

total 0

-rw-r--r-- 1 root root 0 Apr 19 09:58 file1.txt

-rw-r--r-- 1 root root 0 Apr 19 09:58 file2.txt

-rw-r--r-- 1 root root 0 Apr 19 09:58 file3.log

[root@gcdc-nms source]# rsync -azvP --include '*.txt' --exclude '*' -e ssh /opt/source/ root@10.200.236.60:/tmp/backup/

root@10.200.236.60's password:

sending incremental file list

created directory /tmp/backup

./

file1.txt

           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=1/3)

file2.txt

           0 100%    0.00kB/s    0:00:00 (xfer#2, to-check=0/3)

sent 130 bytes  received 53 bytes  21.53 bytes/sec

total size is 0  speedup is 0.00

[root@gcdc-nms source]#

5. Use of –delete Option

[root@gcdc-nms source]# ll

total 0

-rw-r--r-- 1 root root 0 Apr 19 09:58 file1.txt

-rw-r--r-- 1 root root 0 Apr 19 09:58 file2.txt

-rw-r--r-- 1 root root 0 Apr 19 09:58 file3.log

[root@gcdc-nms source]# ll ../destination/

total 0

-rw-r--r-- 1 root root 0 Apr 18 13:01 file1

[root@gcdc-nms source]# rsync -azvP --delete -e ssh /opt/source/ root@10.200.236.60:/opt/destination/

root@10.200.236.60's password:

sending incremental file list

./

deleting file1

file1.txt

           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=2/4)

file2.txt

           0 100%    0.00kB/s    0:00:00 (xfer#2, to-check=1/4)

file3.log

           0 100%    0.00kB/s    0:00:00 (xfer#3, to-check=0/4)

sent 181 bytes  received 72 bytes  29.76 bytes/sec

total size is 0  speedup is 0.00

You have new mail in /var/spool/mail/root

[root@gcdc-nms source]# ll ../destination/

total 0

-rw-r--r-- 1 root root 0 Apr 19 09:58 file1.txt

-rw-r--r-- 1 root root 0 Apr 19 09:58 file2.txt

-rw-r--r-- 1 root root 0 Apr 19 09:58 file3.log

[root@gcdc-nms source]#

6. Setup cron job for rsync

Generate the rsa file on client side (Destination). 

[root@gcdc-nms opt]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): /root/.ssh/rsync_key
Enter passphrase (empty for no passphrase): [Hit Enter]
Enter same passphrase again: [Hit Enter]
Your identification has been saved in /root/.ssh/rsync_key.
Your public key has been saved in /root/.ssh/rsync_key.pub.
The key fingerprint is:
SHA256:9hAlaN2gBCOmbIVv4QcGOiMSZp9yIHcVFgeWvHizo/4 root@gcdc-nms.localdomain
The key's randomart image is:
+---[RSA 2048]----+
|o=*.++OB+o.      |
|**+*.==o.o.      |
|B+++oo...        |
|+oo+..+  .       |
|  . .. oS        |
|      o. o       |
|     . .  .      |
|    .            |
|   ...E          |
+----[SHA256]-----+

Now copy the key to server (Source) from Client (destination).

[root@gcdc-nms ~]# cat /root/.ssh/rsync_key.pub | ssh root@10.200.236.60 "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >>  ~/.ssh/authorized_keys"

[root@gcdc-nms ~]# cp /root/.ssh/rsync_key /opt/rsa/

[root@gcdc-nms ~]# ls /opt/rsa/

key  key.pub  rsync_key

[root@gcdc-nms ~]# 

Now check rsync on server (source)

[root@gcdc-nms ~]# rsync -azvP --delete -e "ssh -i /opt/rsa/rsync_key" /opt/source/ root@10.200.236.60:/opt/destination/              sending incremental file list

sent 63 bytes  received 12 bytes  13.64 bytes/sec

total size is 0  speedup is 0.00

You have new mail in /var/spool/mail/root

[root@gcdc-nms ~]#

Setup a cron job

First create a script which contains the rsync commands.

[root@gcdc-nms ~]# vim /opt/backup

rsync -azvP --delete -e "ssh -i /opt/rsa/rsync_key" /opt/source/ root@10.200.236.60:/opt/destination/

[root@gcdc-nms ~]# chmod 775 /opt/backup

Now create a cron job.

[root@gcdc-nms ~]# crontab -e

* * * * * sh /opt/backup