Keepalived+rsync+ZK实现SFTP高可用方案

前言

想要搭建一个高可用的SFTP服务，并通过rsync实现数据有序同步，同时利用ZK（Zookeeper）管理同步状态，确保数据传输的有序性和高可用性。这是一个结合多种技术的优化方案，既能保证数据的安全性，又能实现负载均衡和自动化同步。

一、配置SFTP服务

1.2 配置SFTP用户

groupadd sftp
useradd -g sftp -s /sbin/nologin -M sftpuser
passwd sftpuser

创建专属目录：

mkdir -p /data/sftp/sftpuser
chown root:sftp /data/sftp/
chown sftpuser:sftp /data/sftp/sftpuser

1.3 配置SFTP服务器

编辑/etc/ssh/sshd_config：

Subsystem sftp /usr/libexec/openssh/sftp-server
Match Group sftp
ChrootDirectory /data/sftp
ForceCommand internal-sftp
AllowTcpForwarding no
X11Forwarding no

启动服务：

systemctl restart sshd

在另一台服务器执行相同配置并重启服务。

1.6 测试连接

sftp sftpuser@192.168.5.5

二、Keepalived配置

2.1 安装

yum install -y keepalived

2.2 配置备份节点

编辑keepalived.conf：

global_defs {
    notification_email { xxx@xxx.com }
    notification_email_from xxx@xxx.com
    smtp_server smtp.xxx.com
    smtp_connect_timeout 30
    router_id LVS_DEVEL
    vrrp_skip_check_adv_addr
    vrrp_garp_interval 0
    vrrp_gna_interval 0
}
vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 53
    priority 100
    nopreempt
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.5.16
    }
}

启动服务：

systemctl start keepalived
systemctl enable keepalived

三、rsync配置

3.1 rsync配置

编辑rsyncd.conf：

uid = sftpuser
gid = sftp
use chroot = yes
max connections = 4
pid file = /var/run/rsyncd.pid
exclude = lost+found
transfer logging = yes
timeout = 900
ignore nonreadable = yes
dont compress = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2

3.2 启动服务

systemctl start rsyncd
systemctl enable rsyncd

3.4 测试同步

rsync -auz /data/sftp/sftpuser/* sftpuser@192.168.5.5::webgis --password-file=/root/sftppassword/sftppassword

四、单向同步脚本

结合ZK实现有序同步，确保数据传输的高可用性和有序性。以下为两个备份节点的脚本配置。

4.1 backup2脚本

#!/bin/bash
curNode=backup2
anoNode=backup1
zkClient=/opt/apache-zookeeper-3.6.2-bin/bin/zkCli.sh
vip=192.168.5.16
user=sftpuser
srcPath=/data/sftp/sftpuser
destIp=192.168.5.5
module=webgis
pwdFile=/root/sftppassword/sftppassword
signalPath=/keepalived/rsync_signal
nodePath=/keepalived/$curNode
getCurrentSyncTimes() {
    prefixNum=1373736
    startNum=1373736000
    maxNum=$(( $startNum + 10 ))
    rsyncTimes=$(zkClient get $nodePath | grep $prefixNum )
    if [[ $rsyncTimes == "" ]]; then
        zkClient set $nodePath $startNum > /dev/null 2>&1
    elif [[ $rsyncTimes -ge $maxNum ]]; then
        zkClient set $nodePath $startNum > /dev/null 2>&1
    else
        rsyncTimes=$(( rsyncTimes + 1 ))
        zkClient set $nodePath $rsyncTimes > /dev/null 2>&1
    fi
    totalTimes=$(( $rsyncTimes - $startNum ))
    echo $totalTimes
}
if [[ $services2user -eq 0 ]]; then
    if [[ $signal =~ $curNode || $signal =~ $anoNode\_node\_stop ]]; then
        echo "停止前最后一次发送数据，$signal"
        rsync -auz $srcPath $user@$destIp::$module --password-file=$pwdFile
        zkClient set $signalPath "$curNode\_node\_stop" > /dev/null 2>&1
    fi
else
    echo "正在发送数据，$signal"
    if [[ $signal =~ $curNode\_node\_sync\_finished || $signal =~ $anoNode\_node\_stop ]]; then
        zkClient set $signalPath "$curNode\_node\_synchronizing" > /dev/null 2>&1
        rsync -auz $srcPath $user@$destIp::$module --password-file=$pwdFile
        curTotalTimes=$(getCurrentSyncTimes)
        echo "第$curTotalTimes次发送"
        if [[ $curTotalTimes -ge $maxTimes ]]; then
            echo "让另一个节点执行一次"
            zkClient set $signalPath "$curNode\_node\_stop" > /dev/null 2>&1
        else
            zkClient set $signalPath "$curNode\_node\_sync\_finished" > /dev/null 2>&1
        fi
    fi
fi

4.2 backup1脚本

#!/bin/bash
curNode=backup1
anoNode=backup2
zkClient=/opt/apache-zookeeper-3.6.2-bin/bin/zkCli.sh
vip=192.168.5.16
user=sftpuser
srcPath=/data/sftp/sftpuser
destIp=192.168.5.3
module=webgis
pwdFile=/root/sftppassword/sftppassword
signalPath=/keepalived/rsync_signal
nodePath=/keepalived/$curNode
getCurrentSyncTimes() {
    prefixNum=1373736
    startNum=1373736000
    maxNum=$(( $startNum + 10 ))
    rsyncTimes=$(zkClient get $nodePath | grep $prefixNum )
    if [[ $rsyncTimes == "" ]]; then
        zkClient set $nodePath $startNum > /dev/null 2>&1
    elif [[ $rsyncTimes -ge $maxNum ]]; then
        zkClient set $nodePath $startNum > /dev/null 2>&1
    else
        rsyncTimes=$(( rsyncTimes + 1 ))
        zkClient set $nodePath $rsyncTimes > /dev/null 2>&1
    fi
    totalTimes=$(( $rsyncTimes - $startNum ))
    echo $totalTimes
}
if [[ $services2user -eq 0 ]]; then
    if [[ $signal =~ $curNode || $signal =~ $anoNode\_node\_stop ]]; then
        echo "停止前最后一次发送数据，$signal"
        rsync -auz $srcPath $user@$destIp::$module --password-file=$pwdFile
        zkClient set $signalPath "$curNode\_node\_stop" > /dev/null 2>&1
    fi
else
    echo "正在发送数据，$signal"
    if [[ $signal =~ $curNode\_node\_sync\_finished || $signal =~ $anoNode\_node\_stop ]]; then
        zkClient set $signalPath "$curNode\_node\_synchronizing" > /dev/null 2>&1
        rsync -auz $srcPath $user@$destIp::$module --password-file=$pwdFile
        curTotalTimes=$(getCurrentSyncTimes)
        echo "第$curTotalTimes次发送"
        if [[ $curTotalTimes -ge $maxTimes ]]; then
            echo "让另一个节点执行一次"
            zkClient set $signalPath "$curNode\_node\_stop" > /dev/null 2>&1
        else
            zkClient set $signalPath "$curNode\_node\_sync\_finished" > /dev/null 2>&1
        fi
    fi
fi

4.3 定时调用

在每个节点上添加 cron 表达式：

crontab -e
* * * * * sh /sftp_data_rsync.sh > /tmp/synclog.log

五、日志查看

查看同步日志：

tail -n 400 /tmp/synclog.log | grep backup

转载地址：http://mvqdz.baihongyu.com/

你可能感兴趣的文章