Rsync is one of the best tools to transfer and synchronize files and directories between
Linux servers or virtual machines. It is an ideal syncing tool used by web hosting and
cloud companies to migrate the data among servers automatically.
Normally, we use the scp command to copy files from one server to another. The scp
command copies every file and folder each time it is executed. But the rsync command
works in a different way. If you make certain changes in a file, it will copy the changes
only saving the time and amount of data being copied. That’s the beauty of rsync and
that’s why system administrators prefer to use it to move data.
In this tutorial, we’ll learn how to rsync files between two Linux servers automatically.
Prerequisites
You must have the following prerequisites to work with rsync.
I. Two Linux servers with Ubuntu or CentOS installed.
II. Network or internet access.
III. SSH access between both servers.
IV. The rsync utility.
For this tutorial, I have two EC2 instances on AWS running in the same virtual private
cloud.
Rsync Installation
In most of the Linux distributions, rsync is installed by default. In case it's not, installing
rsync is a piece of cake.
On Ubuntu, you can install it with the following command.
apt install rsync
On a CentOS server, we can install it with yum.
yum install rsync
Run the rsync command and it will give you the version, usage and multiple other
options.
~# rsync
rsync version 3.1.2 protocol version 31
Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison,
and others.
Web site: [Link]
Configuring SSH Authentication between two Linux
servers
We’ll setup password-less login setup so that both servers can communicate without
password. It's a mandatory step for easy synchronization of the files among Linux
servers. We’ll use the SSH keygen utility for creating an SSH keypair which contains both
public and private keys.
Login to the source server and execute the following command to generate the keypair.
The above command will generate a key pair of private and public keys.
-rw------- 1 frehman domain users 1679 Aug 16 13:28 id_rsa
-rw-r--r-- 1 frehman domain users 404 Aug 16 13:28 id_rsa.pub
I have already created a .ssh directory on the destination server [Link]. Run the
following command to upload the public key into the authorized_keys file.
cat .ssh/id_rsa.pub | ssh frehman@[Link] 'cat >>
.ssh/authorized_keys'
frehman@[Link]'s password: [Enter Your Password Here]
Set the correct permissions on the destination server to avoid any further issues during
the synchronization.
ssh frehman@[Link] "chmod 700 .ssh; chmod 640
.ssh/authorized_keys"
frehman@[Link]'s password: [Put the password here]
We are all set, let’s test the connection.
frehman@ip-10-24-2-148:~# ssh frehman@[Link]
Welcome to Ubuntu 18.04.2 LTS (GNU/Linux 5.4.0-1051-aws x86_64)
* Documentation: [Link]
* Management: [Link]
* Support: [Link]
System information as of Mon Aug 16 14:01:38 UTC 2021
System load: 0.13
Usage of /: 51.1% of 55.19GB
Memory usage: 69%
Swap usage: 0%
Processes: 232
Users logged in: 0
IP address for eth0: [Link]
New release '20.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
Last login: Wed Aug 11 14:38:36 2021 from [Link]
It works! We are now ready to sync the files between these two servers.
Syncing files between two server Linux servers
The syntax of rsync is simple and similar to scp or cp commands in Linux where we
mention the source and destination of the file.
rsync [OPTION]... SRC [SRC]... DEST
The syntax for rsync to copy files to a remote server is:
rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST
Now let’s create a directory with some test files on the source server.
$ mkdir src
$ touch src/myfiles{1..10}
Run the ls command to see if the files have been created.
$ ls
myfiles1 myfiles10 myfiles2 myfiles3 myfiles4 myfiles5 myfiles6
myfiles7 myfiles8 myfiles9
Before creating another directory on the destination server, make sure you can SSH to
the remote machine. If you don’t have the SSH access, you will not be able to sync or
copy the files. Also, rsync should be installed on both Linux servers.
Create a directory on the remote server.
$ mkdir dest
The dest is the directory on the remote server where I’ll sync the files from the src
directory of the source server.
That’s how the rsync command should look like.
rsync -a ~/source_directory username@remote_host:destination_directory
I will change it with my username and IP of the remote machine with frehman as the
user and [Link] as the IP address respectively.
~# rsync -a -v ~/src frehman@[Link]:dest/
sending incremental file list
sent 196 bytes received 17 bytes 426.00 bytes/sec
total size is 0 speedup is 0.00
It will copy the whole directory with all the files in the dest directory on the remote
server.
Run the ls command to confirm the synchronization.
frehman@ip-10-24-2-193:dest/src$ ls
myfiles1 myfiles10 myfiles2 myfiles3 myfiles4 myfiles5 myfiles6
myfiles7 myfiles8 myfiles9
Let’s edit one of the files on the source server.
~/src# cat myfiles1
This is a change
I have modified myfiles1 file on the source server by adding a simple text.
I’ll run the rsync command again.
~/src# rsync -a -v ~/src frehman@[Link]:/dest/
sending incremental file list
src/
src/myfiles1
sent 270 bytes received 39 bytes 618.00 bytes/sec
total size is 17 speedup is 0.06
You can see src/myfiles1 in the output explicitly mentioning that this file was
changed. We’ll confirm it on the destination server.
frehman@ip-10-24-2-193:dest/src$ cat myfiles1
This is a change
Syncing files between two server Linux servers
Automatically
In order to sync files between two Linux servers on auto-pilot, we’ll write a simple bash
script for that.
Here is the script that syncs the files automatically when it’s executed.
~/src# cat rsync_files.sh
#!/bin/bash
SOURCE='/home/frehman/src/'
DESTINATION='/home/frehman/dest/'
REMOTE_SERVER='[Link]'
USER='frehman'
rsync -av --rsh=ssh $SOURCE $USER@$REMOTE_SERVER:$DESTINATION
Please change the source and destination paths accordingly and update the script before
executing.
Make the script executable by running the following command.
~/src# chmod +x [Link]
Let’s run the script to see if it works.
~/src# sh [Link]
sending incremental file list
./
myfiles1
myfiles10
myfiles2
myfiles3
myfiles4
myfiles5
myfiles6
myfiles7
myfiles8
myfiles9
[Link]
sent 860 bytes received 228 bytes 2,176.00 bytes/sec
total size is 188 speedup is 0.17
Boom! The script worked perfectly and synchronized the files on the destination server.
Again, I have made a change in the following file.
~/src# cat myfiles2
This is change no.2
Running the script again on the source server.
~/src# sh [Link]
sending incremental file list
./
myfiles2
sent 302 bytes received 38 bytes 226.67 bytes/sec
total size is 210 speedup is 0.62
We can see the file has been changed on the destination server.
frehman@ip-10-24-2-193:/home/frehman/dest$ cat myfiles2
This is change no.2
Finally, we’ll add the script in cron to run after every minute.
Edit the cron with the following command.
crontab -e
Add the file down below, save the file and exit.
* * * * * /home/frehman/[Link]
Conclusion
Rsync is a wonderful tool that saves a lot of time while copying files among the servers.
It’s syntax is easy to work with and understandable. Also, we saved more of our time by
writing a bash script and adding it into the cronjob. It’ll work on auto-pilot and each time
changes made on the source server will be automatically synced with the destination
server.