What is Rsync
Rsync (Remote Sync) is a most commonly used command for copying and synchronizing files and directories remotely as well as locally in Linux/Unix Systems,
With the help of rsync command you can copy and synchronize your data remotely and locally across directories, across disks and networks, perform data backups and mirroring between two Linux machines.
Advantages and features of rsync command
- It efficiently copies and sync files to or from a remote system.
- Supports copying links, devices, owners, groups and permissions.
- It’s faster than scp (Secure Copy) because rsync uses remote-update protocol which allows to transfer just the differences between two sets of files. First time, it copies the whole content of a file or a directory from source to destination but from next time, it copies only the changed blocks and bytes to the destination.
- Rsync consumes less bandwidth as it uses compression and decompression method while sending and receiving data both ends.
syntax of rsync command
# rsync options source destination
common options used with rsync commands
- -v : verbose
- -r : copies data recursively (but don’t preserve timestamps and permission while transferring data
- -a : archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps
- -z : compress file data
- -h : human-readable, output numbers in a human-readable format
1. Copy/Sync Files and Directory Locally
[root@gcdc-nms source]# rsync -azvh /opt/source/ /tmp/backup/
sending incremental file list
created directory /tmp/backup
./
1
2
3
4
file1
sent 255 bytes received 110 bytes 730.00 bytes/sec
total size is 0 speedup is 0.00
[root@gcdc-nms source]#
2. Copy/Sync Files and Directory to or From a Remote Server
[root@gcdc-nms source]# rsync -azvh /opt/source/ root@10.200.236.60:/tmp/backup/
root@10.200.236.60's password:
sending incremental file list
created directory /tmp/backup
./
1
2
3
4
file1
sent 255 bytes received 110 bytes 42.94 bytes/sec
total size is 0 speedup is 0.00
You have new mail in /var/spool/mail/root
[root@gcdc-nms source]#
3. Rsync Over SSH
[root@gcdc-nms source]# rsync -azvhP -e ssh /opt/source/ root@10.200.236.60:/tmp/backup
/root@10.200.236.60's password:sending incremental file listcreated directory /tmp/backup./1 0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=4/6)2 0 100% 0.00kB/s 0:00:00 (xfer#2, to-check=3/6)3 0 100% 0.00kB/s 0:00:00 (xfer#3, to-check=2/6)4 0 100% 0.00kB/s 0:00:00 (xfer#4, to-check=1/6)file1 0 100% 0.00kB/s 0:00:00 (xfer#5, to-check=0/6)sent 255 bytes received 110 bytes 42.94 bytes/sectotal size is 0 speedup is 0.00
[root@gcdc-nms source]#
4. Use of –include and –exclude Options
[root@gcdc-nms source]# ll
total 0
-rw-r--r-- 1 root root 0 Apr 19 09:58 file1.txt
-rw-r--r-- 1 root root 0 Apr 19 09:58 file2.txt
-rw-r--r-- 1 root root 0 Apr 19 09:58 file3.log
[root@gcdc-nms source]# rsync -azvP --include '*.txt' --exclude '*' -e ssh /opt/source/ root@10.200.236.60:/tmp/backup/
root@10.200.236.60's password:
sending incremental file list
created directory /tmp/backup
./
file1.txt
0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=1/3)
file2.txt
0 100% 0.00kB/s 0:00:00 (xfer#2, to-check=0/3)
sent 130 bytes received 53 bytes 21.53 bytes/sec
total size is 0 speedup is 0.00
[root@gcdc-nms source]#
5. Use of –delete Option
[root@gcdc-nms source]# ll
total 0
-rw-r--r-- 1 root root 0 Apr 19 09:58 file1.txt
-rw-r--r-- 1 root root 0 Apr 19 09:58 file2.txt
-rw-r--r-- 1 root root 0 Apr 19 09:58 file3.log
[root@gcdc-nms source]# ll ../destination/
total 0
-rw-r--r-- 1 root root 0 Apr 18 13:01 file1
[root@gcdc-nms source]# rsync -azvP --delete -e ssh /opt/source/ root@10.200.236.60:/opt/destination/
root@10.200.236.60's password:
sending incremental file list
./
deleting file1
file1.txt
0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=2/4)
file2.txt
0 100% 0.00kB/s 0:00:00 (xfer#2, to-check=1/4)
file3.log
0 100% 0.00kB/s 0:00:00 (xfer#3, to-check=0/4)
sent 181 bytes received 72 bytes 29.76 bytes/sec
total size is 0 speedup is 0.00
You have new mail in /var/spool/mail/root
[root@gcdc-nms source]# ll ../destination/
total 0
-rw-r--r-- 1 root root 0 Apr 19 09:58 file1.txt
-rw-r--r-- 1 root root 0 Apr 19 09:58 file2.txt
-rw-r--r-- 1 root root 0 Apr 19 09:58 file3.log
[root@gcdc-nms source]#
6. Setup cron job for rsync
Generate the rsa file on client side (Destination).
[root@gcdc-nms opt]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): /root/.ssh/rsync_key
Enter passphrase (empty for no passphrase): [Hit Enter]
Enter same passphrase again: [Hit Enter]
Your identification has been saved in /root/.ssh/rsync_key.
Your public key has been saved in /root/.ssh/rsync_key.pub.
The key fingerprint is:
SHA256:9hAlaN2gBCOmbIVv4QcGOiMSZp9yIHcVFgeWvHizo/4 root@gcdc-nms.localdomain
The key's randomart image is:
+---[RSA 2048]----+
|o=*.++OB+o. |
|**+*.==o.o. |
|B+++oo... |
|+oo+..+ . |
| . .. oS |
| o. o |
| . . . |
| . |
| ...E |
+----[SHA256]-----+
Now copy the key to server (Source) from Client (destination).
[root@gcdc-nms ~]# cat /root/.ssh/rsync_key.pub | ssh root@10.200.236.60 "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys"
[root@gcdc-nms ~]# cp /root/.ssh/rsync_key /opt/rsa/
[root@gcdc-nms ~]# ls /opt/rsa/
key key.pub rsync_key
Now check rsync on server (source)
[root@gcdc-nms ~]# rsync -azvP --delete -e "ssh -i /opt/rsa/rsync_key" /opt/source/ root@10.200.236.60:/opt/destination/ sending incremental file list
sent 63 bytes received 12 bytes 13.64 bytes/sec
total size is 0 speedup is 0.00
You have new mail in /var/spool/mail/root
[root@gcdc-nms ~]#
Setup a cron job
First create a script which contains the rsync commands.
[root@gcdc-nms ~]# vim /opt/backup
rsync -azvP --delete -e "ssh -i /opt/rsa/rsync_key" /opt/source/ root@10.200.236.60:/opt/destination/
[root@gcdc-nms ~]# chmod 775 /opt/backup
Now create a cron job.
[root@gcdc-nms ~]# crontab -e
* * * * * sh /opt/backup
