{"id":4504,"date":"2015-12-20T03:05:26","date_gmt":"2015-12-20T08:05:26","guid":{"rendered":"http:\/\/cdnhost.net\/?p=4504"},"modified":"2022-09-19T10:42:49","modified_gmt":"2022-09-19T15:42:49","slug":"how-to-use-rsync-to-sync-local-and-remote-directories-on-a-vps","status":"publish","type":"post","link":"https:\/\/cdnhost.net\/content\/how-to-use-rsync-to-sync-local-and-remote-directories-on-a-vps\/","title":{"rendered":"How To Use Rsync to Sync Local and Remote Directories on a VPS"},"content":{"rendered":"<h3 id=\"introduction\">Introduction<\/h3>\n<p><em>Rsync<\/em>, which stands for &#8220;remote sync&#8221;, is a remote and local file synchronization tool. It uses an algorithm that minimizes the amount of data copied by only moving the portions of files that have changed.<\/p>\n<p>In this guide, we will cover the basic usage of this powerful utility. We will be using an Ubuntu 12.04 VPS in the examples, but you can use any modern Linux distribution to follow along.<\/p>\n<div data-unique=\"what-is-rsync\"><\/div>\n<h2 id=\"what-is-rsync\">What Is Rsync?<\/h2>\n<p>Rsync is a very flexible network-enabled syncing tool. It can <em>also<\/em> refer to the network protocol developed to utilize this tool.<\/p>\n<p>When we reference rsync in this guide, we are mainly referring to the utility, and not the protocol.<\/p>\n<p>Due to its ubiquity on Linux and Unix-like systems and its popularity as a tool for system scripts, it is included on most Linux distributions by default.<\/p>\n<div data-unique=\"basic-syntax\"><\/div>\n<h2 id=\"basic-syntax\">Basic Syntax<\/h2>\n<p>The basic syntax of rsync is very straight forward, and operates in a way that is similar to ssh, scp, and cp.<\/p>\n<p>We will create two test directories and some test files with the following commands:<\/p>\n<pre class=\"code-pre \"><code>cd ~\nmkdir dir1\nmkdir dir2\ntouch dir1\/file{1..100}\n<\/code><\/pre>\n<p>We now have a directory called <code>dir1<\/code> with 100 empty files in it.<\/p>\n<pre class=\"code-pre \"><code>ls dir1\n<\/code><\/pre>\n<pre class=\"code-pre \"><code>file1    file18  file27  file36  file45  file54  file63  file72  file81  file90\nfile10   file19  file28  file37  file46  file55  file64  file73  file82  file91\nfile100  file2   file29  file38  file47  file56  file65  file74  file83  file92\nfile11   file20  file3   file39  file48  file57  file66  file75  file84  file93\nfile12   file21  file30  file4   file49  file58  file67  file76  file85  file94\nfile13   file22  file31  file40  file5   file59  file68  file77  file86  file95\nfile14   file23  file32  file41  file50  file6   file69  file78  file87  file96\nfile15   file24  file33  file42  file51  file60  file7   file79  file88  file97\nfile16   file25  file34  file43  file52  file61  file70  file8   file89  file98\nfile17   file26  file35  file44  file53  file62  file71  file80  file9   file99\n<\/code><\/pre>\n<p>We also have an empty directory called <code>dir2<\/code>.<\/p>\n<p>To sync the contents of <code>dir1<\/code> to <code>dir2<\/code> on the same system, type:<\/p>\n<pre class=\"code-pre \"><code>rsync -r dir1\/ dir2\n<\/code><\/pre>\n<p>The <strong>-r<\/strong> option means recursive, which is necessary for directory syncing.<\/p>\n<p>We could also use the <strong>-a<\/strong> flag instead:<\/p>\n<pre class=\"code-pre \"><code>rsync -a dir1\/ dir2\n<\/code><\/pre>\n<p>The <strong>-a<\/strong> option is a combination flag.<\/p>\n<p>It stands for &#8220;archive&#8221; and syncs recursively and preserves symbolic links, special and device files, modification times, group, owner, and permissions.<\/p>\n<p>It is more commonly used than <strong>-r<\/strong> and is usually what you want to use.<\/p>\n<h3 id=\"an-important-note\">An Important Note<\/h3>\n<p>You may have noticed that there is a trailing slash (\/) at the end of the first argument in the above commands:<\/p>\n<pre>rsync -a dir1<span class=\"highlight\">\/<\/span> dir2\n<\/pre>\n<p>This is necessary to mean &#8220;the contents of <code>dir1<\/code>&#8220;.<\/p>\n<p>The alternative, without the trailing slash, would place <code>dir1<\/code>, including the directory, within <code>dir2<\/code>. This would create a hierarchy that looks like:<\/p>\n<pre class=\"code-pre \"><code>~\/dir2\/dir1\/[files]\n<\/code><\/pre>\n<p>Always double-check your arguments before executing an rsync command.<\/p>\n<p>Rsync provides a method for doing this by passing the <strong>-n<\/strong> or <strong>&#8211;dry-run<\/strong> options. The <strong>-v<\/strong> flag (for verbose) is also necessary to get the appropriate output:<\/p>\n<pre class=\"code-pre \"><code>rsync -anv dir1\/ dir2\n<\/code><\/pre>\n<pre class=\"code-pre \"><code>sending incremental file list\n.\/\nfile1\nfile10\nfile100\nfile11\nfile12\nfile13\nfile14\nfile15\nfile16\nfile17\nfile18\n. . .\n<\/code><\/pre>\n<p>Compare this output to the output we get when we remove the trailing slash:<\/p>\n<pre class=\"code-pre \"><code>rsync -anv dir1 dir2\n<\/code><\/pre>\n<pre class=\"code-pre \"><code>sending incremental file list\ndir1\/\ndir1\/file1\ndir1\/file10\ndir1\/file100\ndir1\/file11\ndir1\/file12\ndir1\/file13\ndir1\/file14\ndir1\/file15\ndir1\/file16\ndir1\/file17\ndir1\/file18\n. . .\n<\/code><\/pre>\n<p>You can see here that the directory itself is transfered.<\/p>\n<div data-unique=\"how-to-use-rsync-to-sync-with-a-remote-system\"><\/div>\n<h2 id=\"how-to-use-rsync-to-sync-with-a-remote-system\">How To Use Rsync to Sync with a Remote System<\/h2>\n<p>Syncing to a remote system is trivial if you have SSH access to the remote machine and rsync installed on both sides. If you need to <a href=\"https:\/\/www.digitalocean.com\/community\/articles\/how-to-set-up-ssh-keys--2\">set up SSH keys<\/a>, click here.<\/p>\n<p>Once you have SSH access verified on between the two machines, you can sync the <code>dir1<\/code> folder from earlier to a remote computer by using this syntax (note that we <em>want<\/em> to transfer the actual directory in this case, so we omit the trailing slash):<\/p>\n<pre>rsync -a ~\/dir1 <span class=\"highlight\">username<\/span>@<span class=\"highlight\">remote_host<\/span>:<span class=\"highlight\">destination_directory<\/span>\n<\/pre>\n<p>This is called a &#8220;push&#8221; operation because it pushes a directory from the local system to a remote system.<\/p>\n<p>The opposite operation is &#8220;pull&#8221;. It is used to sync a remote directory to the local system. If the <code>dir1<\/code> were on the remote system instead of our local system, the syntax would be:<\/p>\n<pre>rsync -a <span class=\"highlight\">username<\/span>@<span class=\"highlight\">remote_host<\/span>:<span class=\"highlight\">\/home\/username\/dir1<\/span> <span class=\"highlight\">place_to_sync_on_local_machine<\/span>\n<\/pre>\n<p>Like &#8220;cp&#8221; and similar tools, the source is always the first argument, and the destination is always the second.<\/p>\n<div data-unique=\"useful-options-for-rsync\"><\/div>\n<h2 id=\"useful-options-for-rsync\">Useful Options for Rsync<\/h2>\n<p>Rsync provides many options for altering the default behavior of the utility. We have already discussed some of the more necessary flags.<\/p>\n<p>If you are transferring files that have not already been compressed, like text files, you can reduce the network transfer by adding compression with the <strong>-z<\/strong> option:<\/p>\n<pre>rsync -az <span class=\"highlight\">source<\/span> <span class=\"highlight\">destination<\/span>\n<\/pre>\n<p>The <strong>-P<\/strong> flag is very helpful. It combines the flags <strong>&#8211;progress<\/strong> and <strong>&#8211;partial<\/strong>. The first of these gives you a progress bar for the transfers and the second allows you to resume interrupted transfers:<\/p>\n<pre>rsync -azP <span class=\"highlight\">source<\/span> <span class=\"highlight\">destination<\/span>\n<\/pre>\n<pre class=\"code-pre \"><code>sending incremental file list\n.\/\nfile1\n           0 100%    0.00kB\/s    0:00:00 (xfer#1, to-check=99\/101)\nfile10\n           0 100%    0.00kB\/s    0:00:00 (xfer#2, to-check=98\/101)\nfile100\n           0 100%    0.00kB\/s    0:00:00 (xfer#3, to-check=97\/101)\nfile11\n           0 100%    0.00kB\/s    0:00:00 (xfer#4, to-check=96\/101)\n. . .\n<\/code><\/pre>\n<p>If we run the command again, we will get a shorter output, because no changes have been made.<\/p>\n<p>This illustrates rsync&#8217;s ability to use modification times to determine if changes have been made.<\/p>\n<pre>rsync -azP <span class=\"highlight\">source<\/span> <span class=\"highlight\">destination<\/span>\n<\/pre>\n<pre class=\"code-pre \"><code>sending incremental file list\n\nsent 818 bytes received 12 bytes 1660.00 bytes\/sec\ntotal size is 0 speedup is 0.00\n<\/code><\/pre>\n<p>We can update the modification time on some of the files and see that rsync intelligently re-copies only the changed files:<\/p>\n<pre>touch dir1\/file{1..10}\nrsync -azP <span class=\"highlight\">source<\/span> <span class=\"highlight\">destination<\/span>\n<\/pre>\n<pre class=\"code-pre \"><code>sending incremental file list\nfile1\n           0 100%    0.00kB\/s    0:00:00 (xfer#1, to-check=99\/101)\nfile10\n           0 100%    0.00kB\/s    0:00:00 (xfer#2, to-check=98\/101)\nfile2\n           0 100%    0.00kB\/s    0:00:00 (xfer#3, to-check=87\/101)\nfile3\n           0 100%    0.00kB\/s    0:00:00 (xfer#4, to-check=76\/101)\n. . .\n<\/code><\/pre>\n<p>In order to keep two directories truly in sync, it is necessary to delete files from the destination directory if they are removed from the source. By default, rsync does not delete anything from the destination directory.<\/p>\n<p>We can change this behavior with the <strong>&#8211;delete<\/strong> option. Before using this option, use the <strong>&#8211;dry-run<\/strong> option and do testing to prevent data loss:<\/p>\n<pre>rsync -a --delete <span class=\"highlight\">source<\/span> <span class=\"highlight\">destination<\/span>\n<\/pre>\n<p>If you wish to exclude certain files or directories located inside a directory you are syncing, you can do so by specifying them in a comma-separated list following the <strong>&#8211;exclude=<\/strong> option:<\/p>\n<pre>rsync -a --exclude=<span class=\"highlight\">pattern_to_exclude<\/span> <span class=\"highlight\">source<\/span> <span class=\"highlight\">destination<\/span>\n<\/pre>\n<p>If we have specified a pattern to exclude, we can override that exclusion for files that match a different pattern by using the <strong>&#8211;include=<\/strong> option.<\/p>\n<pre>rsync -a --exclude=<span class=\"highlight\">pattern_to_exclude<\/span> --include=<span class=\"highlight\">pattern_to_include<\/span> <span class=\"highlight\">source<\/span> <span class=\"highlight\">destination<\/span><\/pre>\n<!--themify_builder_content-->\n<div id=\"themify_builder_content-4504\" data-postid=\"4504\" class=\"themify_builder_content themify_builder_content-4504 themify_builder tf_clear\">\n    <\/div>\n<!--\/themify_builder_content-->\n","protected":false},"excerpt":{"rendered":"<p>Introduction Rsync, which stands for &#8220;remote sync&#8221;, is a remote and local file synchronization tool. It uses an algorithm that minimizes the amount of data copied by only moving the portions of files that have changed. In this guide, we will cover the basic usage of this powerful utility. We will be using an Ubuntu [&hellip;]<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-4504","post","type-post","status-publish","format-standard","hentry","category-blog","has-post-title","has-post-date","has-post-category","has-post-tag","has-post-comment","has-post-author",""],"builder_content":"","_links":{"self":[{"href":"https:\/\/cdnhost.net\/content\/wp-json\/wp\/v2\/posts\/4504","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cdnhost.net\/content\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cdnhost.net\/content\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cdnhost.net\/content\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/cdnhost.net\/content\/wp-json\/wp\/v2\/comments?post=4504"}],"version-history":[{"count":0,"href":"https:\/\/cdnhost.net\/content\/wp-json\/wp\/v2\/posts\/4504\/revisions"}],"wp:attachment":[{"href":"https:\/\/cdnhost.net\/content\/wp-json\/wp\/v2\/media?parent=4504"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cdnhost.net\/content\/wp-json\/wp\/v2\/categories?post=4504"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cdnhost.net\/content\/wp-json\/wp\/v2\/tags?post=4504"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}