16 rsync Command in Linux with Examples

1 Shares

In this guide, we are going to be learning How to use the rsync command.

The rsync command allows you to transfer and sync files efficiently across different directories and machines.

rsync stands for Remote Synchronization. It’s an advanced alternative tool for commands like scp and sftp.

So if you ever want to set up a backup for your files or sync files across different machines then rsync would be a great way to do this.

It’s a lot more efficient than just copying files because rsync only copies what’s different.

For example, if you were backing up thousands of files/pictures, the copying all of these every time way too inefficient so if you did this with rsync then it would only need to sync what’s different since the last backup.

And also if you were copying data and lost a network connection or something like that then when you reran the rsync command it would know where it left off and wouldn’t have to start from scratch.

Hence this is an extremely beneficial command to know How to use properly.

So let’s go ahead and get started and look at some examples.

Now rsync might already be installed on your machine.

If you are on a Debian based Linux machine and don’t have rsync installed then you can install that with the help of the following command.

$ sudo apt-get install rsync

and If you are on an RPM based Linux machine like Redhat or CentOS then you can install this with the help of the following command.

# yum -y install rsync

Okay, so when you have rsync installed and ready to go let’s go ahead and look at a very simple example.

Table of Contents

Features of rsync Command:

Sync(Transfer) files locally and remotely
Delete source contents after successful data sync
Limit socket I/O bandwidth
Limit data transfer size
Perform Dry Run before you actually run the rsync command
Delete unimportant files from the destination directory
Include or exclude files from the transfer
Sync data over Network
Sync data over SSH

First of all, let’s focus on some of the most important options that we can use with the rsync.

Options	Explanation
-a, --archive	Archive mode; equals to -rlptgoD
-r, --recursive	Recurse into directories
-v, --verbose	Display verbose output
-q, --quiet	Suppress non-error messages
-z, --compress	Compress file data during the transfer
-n, --dry-run	Perform a Dry Run with no changes made
--exclude	exclude files matching Pattern
--include	Don't exclude files matching Pattern
--max-size & --min-size	Define how much data(Size) to transfer
--delete	Delete unimportant files from destination directory(Mirroring)
-P, --progress	Show progress during transfer
--version	Display version of rsync Command
--help	Display help page of rsync Command.

You must follow the syntax given below to use the rsync command.

rsync [OPTION]... SRC [SRC]... DEST

Where:

[SRC] – Source
[DEST] – Destination

So I have two directories that I created here on my desktop. The first directory is called data1 and the second one is named data2.

Desktop]# ls
data1  data2

Now the data1 has some files the data2 is currently empty.

# cd data1/

data1]# ls
file1.txt  file2.txt  file3.txt  file4.txt  file5.txt

1. How to sync files locally

So let’s run a rsync command to sync these two directories and copy the files from data1 to the data2 directory.

So in the terminal, I am just going to navigate to where these directories are located and that is on my desktop and now to use the rsync I will just type the following command.

We could use some options for rsync command and we will look at that in just a minute but for now, we are not going to give any options.

So now we just want to put the Source directory that we want to sync.

In this case, we want to sync the files in the data1 directory and since we are not using any options we actually have to specify the files and not just a directory.

# rsync data1/* data2/

# ls -l data2/
total 0
-rw-r--r--. 1 root root 0 Feb 18 21:50 file1.txt
-rw-r--r--. 1 root root 0 Feb 18 21:50 file2.txt
-rw-r--r--. 1 root root 0 Feb 18 21:50 file3.txt
-rw-r--r--. 1 root root 0 Feb 18 21:50 file4.txt
-rw-r--r--. 1 root root 0 Feb 18 21:50 file5.txt

Here I used the asterisk mark(*) because I had to copy all the files available inside the data1 directory.

As you can see that it synced all of the files from our data1 directory to our data2 directory.

Note: Now there is no harm in running this again as it won’t do anything since there are no files different or any files that need to be recent.

To copy a particular file type the following command.

# rsync data1/file1.txt data2/

2. How to sync files and directories locally

Now the way this is running right now, it won’t actually copy directories and that is why we needed to add the asterisk(*) to specify that we wanted all of the files within the data1 but a more common usage for rsync is to sync everything.

For example, if we were to create a subdirectory within data1 so I will create a new directory called mydata.

# mkdir data1/mydata

and Now if we rerun our previous rsync command then we can see in our output here that it said skipping directory mydata one that we just created.

# rsync data1/* data2/
skipping directory mydata

So if we want to sync the entire directory then we can use the -r option to say that we want to recurse into directories and copy the total contents of the directory and subdirectories.

Now we no longer need this asterisk(*) here.

# rsync -r data1/ data2/

If we come to our directories after running the above command, we can see that all the contents are copied to our data2 directory.

# ls data2/
file1.txt  file2.txt  file3.txt  file4.txt  file5.txt  mydata

Now when you do this way notice that I have the trailing slash(/) hereafter the data1 directory now that’s actually important.

So that means that we want to sync the contents of the data1 directory. If we leave it off then it will sync the directory(data1) itself.

Just let me show you the difference here.

So I will sync only the directory and I will leave the trailing slash.

Now if I run the command without that trailing slash if we look at our data2 directory then we can see that instead of just syncing the contents of the data1 directory it actually synced the data1 directory itself.

# rsync -r data1 data2/

# ls data2/
data1

Most of the time I intend to sync the contents of these directories so you want that trailing slash and when you include that and run the command the it will copy everything and will sync everything as it should.

3. Mapping Owner and Group of the destination file

You can map the owner and group of the destination file while syncing contents. For this, you have to use the --chown option with rsync.

The following command will make helpdesk the owner of the file named file1.txt and map linuxadmin as a group.

Note: The user or group you are mapping must exist in the system.

# rsync -a --chown=helpdesk:linuxadmin data1/file1.txt data2/

# ls  -l data2/file1.txt 
-rw-r--r--. 1 helpdesk linuxadmin 0 Feb 19 04:04 data2/file1.txt

4. Change permission of the destination file

Type the following command to change the permission of the destination file. For this, you have to use the --chmod option with rsync.

# rsync -a --chmod=777 data1/file1.txt data2/

# ls -l data2/file1.txt 
-rwxrwxrwx. 1 root root 0 Feb 19 04:04 data2/file1.txt

5. Remove source files after successful data transfer(Sync)

This is a convenient feature inside rsync. This is possible if you want to delete the source files after successfully transferring the data.

To do this, run the following command.

# rsync -a --remove-source-files data1/file1.txt data2/

6. Usage of –max-size and –min-size

With the rsync command, you can define how much data to transfer.

For example:

#1: Don’t transfer any file larger than 1 MB

# rsync -a --max-size='1m' data1/ data2/

#2: don’t transfer any file smaller than 1 MB

# rsync -a --min-size='1m' data1/ data2/

7. How to sync data in Archive Mode

I’d say the more common option for data sync is the -a option. This option is actually a combination of many different options.

It stands for archive and not only does it recurse into directories but it also copies symlinks, preserves permission, modification times, groups, owners, and things like that.

So I like to use the -a option most of the time to copy everything.

# rsync -a data1/ data2/

Typically rsync copies a file to a destination location with only its original name.

But if you want to save to the destination location with a new name run the following command.

# rsync -a data1/file1.txt data2/myfile.txt

8. Perform a Dry Run with rsync Command

Now if you are about to run a large job that syncs a lot of files and directories then you probably want to do a check on what files and folders are going to be synced.

So that you can make sure that it’s what you expect because checking what going to be synced before you actually run the command is a lot better than realizing afterward that you spent a lot of time moving gigabytes of data to the wrong place.

So to check what files are going to be synced without actually syncing the files we can use the --dry-run option.

So we can specify that we want to perform the dry run option with --dry-run or we can use -n for short.

Let’s say we wanted to run a dry run of syncing our data1 directory contents with our data2 directory. So to do that you can type the following command.

# rsync -a --dry-run data1/ data2/

After running the above command it doesn’t actually show us any output and if we look in the data2 directory then it didn’t actually sync those files or directories.

So it did do a dry run but the point of this is for us to know what files are going to be synced so that we can make sure that it looks correct.

So to display this we need to pair --dry-run with verbose output and to get the verbose output we can include the -v option.

# rsync -av --dry-run data1/ data2/
sending incremental file list
./
file1.txt
file2.txt
file3.txt
file4.txt
file5.txt
mydata/

Now we can see that it displays the files that would have copied to the data2 directory but since we had that dry run option as well it didn’t actually make those changes.

This allows us to make sure that everything looks correct and if it does look good then you can run the same command without the dry run included.

Okay so just to show you why rsync command is so much more beneficial than just copying files over let me delete a couple of these files from the data2 directory.

# cd data2/

# rm file2.txt file3.txt 
rm: remove regular empty file 'file2.txt'? y
rm: remove regular empty file 'file3.txt'? y

Now if we rerun that command with the --dry-run option then we can see that it knows that it only needs to copy these two files and not any of the others.

# rsync -av --dry-run data1/ data2/
sending incremental file list
./
file2.txt
file3.txt

sent 184 bytes  received 26 bytes  420.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)

And these two files that it says that it will copy over are the ones that we just deleted. So you can imagine that this would save an incredible amount of time if you had a lot of files or a lot of directories.

9. Delete unimportant files from destination directory(Mirroring)

What happens if our data2 directory contains some that our data1 directory doesn’t have.

For example, let me add a simple file to our data2 directory.

# touch data2/file7.txt

So now our data2 directory contains this file7.txt but our data1 directory does not. So let’s another dry run and see if this matters at all.

# rsync -av --dry-run data1/ data2/
sending incremental file list
./
file2.txt
file3.txt

sent 184 bytes  received 26 bytes  420.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)

So we can see that it still wants to sync those two files that we deleted from earlier but other than that it doesn’t seem to care that there are more files in the destination.

Now if we want our destination to mirror our source and get rid of any files that don’t exist from our source directory the we can add the --delete option to do this.

# rsync -av --delete --dry-run data1/ data2/
sending incremental file list
deleting file7.txt
./
file2.txt
file3.txt

sent 184 bytes  received 39 bytes  446.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)

As you can see that it still wants to sync the two files that exist in the source directory but it also says that it wants to delete file7.txt.

Now you have to be really careful with this because there are some horror stories online where people have accidentally set this delete flag on an empty directory which resulted in their entire backup being wiped out.

Because if their source directory is empty and you say you want to delete everything in your backup that doesn’t exist in the source directory then that would be everything.

So you do want to be extremely careful with that and I don’t use it a whole lot.

10. Usage of `--include` and `--exclude` with rsync Command

To use this feature of rsync, we can include wildcards.

For example, I want to copy all files whose extension is .txt and exclude all other contents. To do this, run the following command.

# rsync -a --include '*.txt' --exclude '*' data1/ data2/

11. Transfer(Sync) data from Local machine to Remote machine

Now let’s actually see what it looks like to sync files between a local and a remote machine.

I have a test machine that I have SSH access to and I have its IP Address(192.168.1.100).

Now if you are transferring data to a machine over the network then the rsync command does have the option to compress while transferring and to that, we can use the -z option.

And also a useful option for transferring over the network is the -P which will show the progress.

So to sync data over the network type the following command.

# rsync -zaP data1/ [email protected]:~/public
sending incremental file list
file1.txt
              0 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=5/7)
file2.txt
              0 100%    0.00kB/s    0:00:00 (xfr#2, to-chk=4/7)
file3.txt
              0 100%    0.00kB/s    0:00:00 (xfr#3, to-chk=3/7)
file4.txt
              0 100%    0.00kB/s    0:00:00 (xfr#4, to-chk=2/7)
file5.txt
              0 100%    0.00kB/s    0:00:00 (xfr#5, to-chk=1/7)
mydata/

Transferring data over the network — transferring data over the network

Note: I do have a key-based authentication setup for my test machine there so it didn’t ask me for a password.

Now if you don’t have that setup then it might ask you for a password to fill in and then after you insert that you should see output similar to this here.

So you can see that it shows us what is sent over and what the progress was.

So now let’s SSH into that remote machine and make sure that everything got there. To do that type the following command.

# ssh [email protected]

Now let me navigate to the directory where I synced those files.

# cd ~/public/

# ls
file1.txt  file2.txt  file3.txt  file4.txt  file5.txt  mydata

As you can see that the files were synced between our local machine and this remote machine and again this is better than just copying the file because it will only sync the differences.

12. Transfer(Sync) data from Remote machine to Local machine

Now we can do this in reverse also. You can transfer data from the remote machine to the local machine. To do this type the following command.

# rsync -zaP [email protected]:~/public data2/

13. Transfer content over SSH

SSH stands for Secure Shell and its default port is 22. You can sync data over SSH using rsync.

To do this, run the following command.

# rsync -avze ssh [email protected]:~/Desktop/data1/* data2/

If you manually set a port for SSH, you must specify that port. I have set the port number 1111 to explain to you.

# rsync -avze "ssh -p 1111" [email protected]:~/Desktop/data1/* data2/

14. How to limit socket I/O bandwidth

You can limit socket I/O bandwidth with the help of rsync.

The following command will set a bandwidth limit of 500KB per second for the rsync command.

# rsync --bwlimit=500 data1/ data2/

15. Check the version of rsync command

Check the rsync command version using the following command.

# rsync --version
rsync  version 3.1.3  protocol version 31

16. Help/Manual page access

Use the following commands to access the Manual Page/Help Page of rsync command.

# man rsync

# rsync --help

You can visit at following website to get more information on rsync.

useradd man page

Conclusion

I hope that now you have a good understanding of how the rsync command works and you have some ideas for how you can use this within your workflow.

If anyone does have any questions about what we covered in this guide then feel free to ask in the comment section below and I will do my best to answer those.

Pin1