Two friends of rsync: `rsync+inotify-tools` & `rsync+sersync`

Using rsync for remote file real-time synchronization actually utilizes the daemon mode of rsync. This mode requires the installation of the application on the source server: rsync + inotify-tools or rsync + inotify-tools, and only rsync needs to be installed on the target server.

`rsync` Configuration Steps#

Server Information#

Target Server: 172.16.12.141
Source Server: 172.16.12.142

Target Server Configuration (Server-side Configuration)#

Create Configuration File: Modify `/etc/rsync.conf` to operate in daemon mode#

RedHat7 and below CentOS7 do not have this by default, and you need to create it yourself.

Configuration File Explanation#

#rsyncd.conf configuration file explanation:

log file = /var/log/rsyncd.log    # Location of the log file, this file is automatically generated after starting rsync, no need to create it in advance
pidfile = /var/run/rsyncd.pid     # Location to store the pid file
lock file = /var/run/rsync.lock   # Lock file supporting max connections parameter
secrets file = /etc/rsync.pass    # User authentication configuration file, which stores usernames and passwords, this file must be created manually

[etc_from_client]     # Custom synchronization name
path = /tmp/          # Path where rsync target server data is stored, data from the source server will be synchronized to this directory
comment = sync etc from client
uid = root        # Set rsync running permissions to root
gid = root        # Set rsync running permissions to root
port = 873        # Default port
ignore errors     # Indicates to ignore errors if they occur
use chroot = no       # Default is true, change to no to increase backup of directory file soft links
read only = no    # Set rsync source server to read-write permissions
list = no     # Do not display rsync source server resource list
max connections = 200     # Maximum number of connections
timeout = 600     # Set timeout duration
auth users = admin        # Username for executing data synchronization, can set multiple, separated by commas in English
hosts allow = 192.168.110.12   # Allowed source server IP addresses for data synchronization, can set multiple, separated by commas in English
hosts deny = 192.168.110.11      # Prohibited source server IP addresses for data synchronization, can set multiple, separated by commas in English

Note: `host allow` and `host deny` Parameters#

Both parameters absent -- All users can access freely;
Only allow present -- Only users in the whitelist can access the module;
Only deny present -- Only users in the blacklist are prohibited from accessing the module;
Both parameters present -- Whitelist is checked first
- If matched successfully, access is allowed;
- If matching fails, check the blacklist; if matched successfully, access is denied;
- If neither matches successfully, access is allowed.

Configuration File as Follows#

Note: The fake super = yes option was not present in previous versions; in the new version, if this parameter is not added, a permission error will be reported: rsync:chgrp ".hosts.G6sZha” (in backup) failed: Operation not permitted (1)

#rsyncd.conf configuration file:

uid = rsync                # User for remote command using rsync to access shared directory
gid = rsync                # User group
use chroot = no            # Security related
max connections = 200      # Maximum number of connections
timeout = 300              # Timeout duration (how long to disconnect without backup)
pid file = /var/run/rsyncd.pid             # Process corresponding process ID file (stores service running process ID)
lock file = /var/run/rsync.lock            # Lock file
log file = /var/log/rsyncd.log             # Log file, displays error information
fake super = yes 		   # Must add this in the new version
# Module information:
[backup]                 # Custom module name
comment = "backup"
path = /backup/web       # Path corresponding to the module
ignore errors            # Ignore error programs
read only = false        # Whether it is read-only (here it is false, indicating it can be written)
list = false             # Whether it can list*
hosts allow = 172.16.12.0/24        # Allowed range of clients accessing the rsync server (whitelist)
#hosts deny = 0.0.0.0/32            # Prohibited range of clients accessing the rsync server (blacklist)
auth users = rsync_backup          # Non-existent user, used only for authentication
# Set the key file for connection authentication:
secrets file = /etc/rsync.password    # Key file for authentication

Create Password File, Change Permissions to `600`#

# Write password content to file
  echo "rsync_backup:123456" > /etc/rsync.password
  # For security, change the password file permissions for the authenticated user to 600
  chmod 600 /etc/rsync.password

Create System User#

# Create a virtual user for rsync, used only by the program
useradd rsync -s /sbin/nolgin -M

Create Directory Corresponding to Module and Change Ownership and Group to System User#

mkdir /backup/web
chown -R rsync.rsync /bakcup/

Start `rsync` Daemon and Check if Started#

rsync --daemon  # Start the daemon
ps -ef | grep rsync    # Check if the process has started
netstat -lntup | grep rsync # Check if listening on port 873

# Check if it has started normally and is listening on port 873
[root@fzfcdb ~]# ps -ef | grep rsync
root     19489     1  0 09:42 ?        00:00:00 rsync --daemon
root     27296 27115  0 15:09 pts/6    00:00:00 grep --color=auto rsync
[root@fzfcdb ~]# netstat -lntup | grep rsync
tcp        0      0 0.0.0.0:873             0.0.0.0:*               LISTEN      19489/rsync         
tcp6       0      0 :::873                  :::*                    LISTEN      19489/rsync         
[root@fzfcdb ~]#

Source Server Configuration#

Install `rsync` Software, Just Install, No Need to Start or Configure#

yum -y install rsync
# Confirm if rsync is installed
rpm -qa | grep rsync
rsync-3.1.2-10.el7.x86_64

Create Password File#

The client password file only needs the password, and the password file permissions are 600

echo "123456" > /etc/rsync.password
# Set file owner to have read and write permissions
chmod 600 /etc/rsync.password

Source Server Upload to Server Test#

/www/ will upload all files from the local www folder to the server; if it is /www, it will back up the entire www folder;
rsync_backup is the auth users parameter in the server-side configuration file;
::backup is the module parameter and the [backup] in the configuration file, followed by the path to the configuration file's password file, allowing transfer without interaction, no password input required.

rsync -avz /www/ [email protected]::backup --password-file=/etc/rsync.password

Source Server Download Test#

Download files from the server's backup module configured path to the local www directory.

 rsync -avz [email protected]::backup --password-file=/etc/rsync.password /www

At this point, the rsync configuration is complete.

Use `inotify-tools` for Real-time Synchronization#

Install `inotify-tools`, Trigger `rsync` for Synchronization in Real-time#

# Check if the server kernel supports inotify
# If there are these three max-prefixed files, it indicates that the server kernel supports inotify
[root@localhost ~]# ll /proc/sys/fs/inotify/
total 0
-rw-r--r--. 1 root root 0 May 11 16:15 max_queued_events
-rw-r--r--. 1 root root 0 May 11 16:15 max_user_instances
-rw-r--r--. 1 root root 0 May 11 16:15 max_user_watches

# Install inotify-tools
yum -y install inotify-tools
# Check if installed
rpm -qa | grep inotify-tools
inotify-tools-3.21.9.6-1.16.el7.x86_64

Write Synchronization Script#

[root@localhost ~]# mkdir /scripts
[root@localhost ~]# touch /scripts/inotify.sh
[root@localhost ~]# chmod 755 /scripts/inotify.sh
[root@localhost ~]# ll /scripts/inotify.sh
-rwxr-xr-x 1 root root 0 Aug 10 13:02 /scripts/inotify.sh
[root@localhost ~]# vim /scripts/inotify.sh
host=172.16.2.4           # IP of the target server (backup server)
src=/www                       # Backup directory to monitor on the source server (can customize, but must exist)
des=backup             # Custom module name, must match the synchronization name defined on the target server
password=/etc/rsync.password        # Password file for executing data synchronization
user=rsync                      # Username for executing data synchronization
inotifywait=/usr/bin/inotifywait

$inotifywait -mrq --timefmt '%Y%m%d %H:%M' --format '%T %w%f%e' -e modify,delete,create,attrib $src \
| while read files;do
    rsync -avzP --delete  --timeout=100 --password-file=${password} $src $user@$host::$des
    echo "${files} was rsynced" >>/tmp/rsync.log 2>&1
done

Start Script#

# & indicates running in the background
nohup bash /scripts/inotify.sh &

Test: Generate a New File on the Source Server#

Check `inotify` Generated Logs#

Use `rsync+sersync` for Real-time Synchronization#

Introduction to `sersync`#

sersync is developed based on inotify, similar to inotify-tools. Therefore, the kernel must first support inotify to set it up.
sersync can record changes (including additions, deletions, modifications) to specific files or directories in the monitored directory, and then use rsync to synchronize only the changed files or directories.
Compared to inotify-tools, it traverses faster and has advantages when synchronizing large amounts of data, and it is faster to set up without the need to write additional scripts.

Install `sersync`#

sersync was developed by a talented individual in China and cannot be installed using yum; you need to find it directly on github or google code and install it yourself.
Google code address: https://code.google.com/archive/p/sersync/
GitHub address: https://github.com/wsgzao/sersync

Source Server Configuration#

Download `sersync`#

wget https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/sersync/sersync2.5.4_64bit_binary_stable_final.tar.gz

Unzip to Get Folder and Rename#

tar -zxvf sersync2.5.4_64bit_binary_stable_final.tar.gz 
# Move to /usr/local and rename to sersync
mv GUN-Linux-x86 /usr/local/sersync

Enter Directory and Modify `xml` Configuration File#

cd  /usr/local/sersync 				# Enter sersync installation directory
cp confxml.xml confxml.xml-bak 		# Backup original file
vim confxml.xml  					# Edit and modify the following code

Modify several places in the configuration content: relevant information can be seen in your rsync server configuration file /etc/rsyncd.con.
Original configuration file:

# Server IP, file path, and module name settings
<localpath watch="/opt/tongbu">						
# Fill in the path of the folder to be synchronized on the NFS storage server (source server).

<remote ip="127.0.0.1" name="tongbu1"/>				
# Fill in the IP address and module name of the rsync backup server (target server), multiple servers can be configured
<!--<remote ip="192.168.8.39" name="tongbu"/>-->
<!--<remote ip="192.168.8.40" name="tongbu"/>-->
</localpath>

# Authentication part (rsync password authentication)
<rsync>
<auth start="false" users="root" passwordfile="/etc/rsync.pas"/>    
# Enable password authentication, configure auth users + password file path, authentication information for the rsync backup server.
</rsync>

# Modify the location of the synchronization failure log, and re-synchronize failed logs every 60 minutes (optional configuration, not required)
<failLog path="/tmp/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->

Modified configuration file:

<sersync>
	<localpath watch="/www">
	    <remote ip="172.16.12.141" name="backup"/>
	</localpath>
	<rsync>
	    <!--<commonParams params="-artuz"/> -->
      <!-- If you need to back up the original files, you need to add the parameter b suffix: set the suffix for the backup of the original file, backup-dir: directory for backing up original components -->
        <commonParams params="-artucb --suffix=_bak_`date +%Y%m%d%H%M%S --backup-dir=`date +%Y%m%d`"/>
	    <auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/>
	    <userDefinedPort start="false" port="874"/><!-- port=874 -->
	    <timeout start="false" time="100"/><!-- timeout=100 -->
	    <ssh start="false"/>
	</rsync>
	<failLog path="/var/log/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once-->

Note: If you need to retain files deleted on the source side on the server side, you need to modify the following configuration.

<inotify>
        <!--<delete start="true"/>    Change true to false-->
        <delete start="false"/>
        <createFolder start="true"/>
        <createFile start="false"/>
        <closeWrite start="true"/>
        <moveFrom start="true"/>
        <moveTo start="true"/>
        <attrib start="false"/>
        <modify start="false"/>
    </inotify>

Start `sersync` Daemon to Synchronize Data and Test Program#

Start `sersync` Daemon#

/usr/local/sersync/sersync2  -d -r -o /usr/local/sersync/confxml.xml

`sersync` Parameter Usage#

-d : Enable daemon mode
-r : Before monitoring, push the monitored directory to the remote host using the rsync command
-n : Specify the number of daemon threads to start, default is 10
-o : Specify the configuration file, default uses confxml.xml

Successfully Started, Not Successful Stuck#

[root@localhost sersync]# /usr/local/sersync/sersync2  -d -r -o /usr/local/sersync/confxml.xml 
set the system param
execute：echo 50000000 > /proc/sys/fs/inotify/max_user_watches
execute：echo 327679 > /proc/sys/fs/inotify/max_queued_events
parse the command param
option: -d 	run as a daemon
option: -r 	rsync all the local files to the remote servers before the sersync work
option: -o 	config xml name：  /usr/local/sersync/confxml.xml
daemon thread num: 10
parse xml config file
host ip : localhost	host port: 8008
will ignore the inotify delete event
daemon start，sersync run behind the console 
use rsync password-file :
user is	rsync_backup
passwordfile is 	/etc/rsync.password
config xml parse success
please set /etc/rsyncd.conf max connections=0 Manually
sersync working thread 12  = 1(primary thread) + 1(fail retry thread) + 10(daemon sub threads) 
Max threads numbers is: 22 = 12(Thread pool nums) + 10(Sub threads)
please according your cpu ，use -n param to adjust the cpu rate
------------------------------------------
rsync the directory recursivly to the remote servers once
working please wait...
execute command: cd /root/rsync_test/log && rsync -artuzb --suffix=._bak_`date +%Y%m%d%H%M%S` --backup-dir=`date +%Y%m%d` -R  ./ [email protected]::log_backup --password-file=/etc/rsync.password >/dev/null 2>&1 
run the sersync: 
watch path is: /root/rsync_test/log

十一