Nginx + keepalived高可用
目录
- 实验简介
- 实验目标
使用keepalived实现Nginx服务的高可用
1)master节点的Nginx宕机,VIP自动切换到backup
保证用户可以正常访问web页面
2)master节点的Nginx宕机,自动发送邮件
3)master节点的Nginx恢复也不抢占VIP,除非backup故障
- 实验使用的软件包
Linux:CentOS-7.6-x86_64-DVD-1810
Nginx: Nginx 1.16.1
- 实验拓扑
Master: node1 10.86.24.2
Slave: Node2 10.86.24.3- 关闭防火墙&配置Base和EPEL源
- 关闭防火墙&配置Base和EPEL源
- 关闭防火墙
systemctl stop firewalld.service
systemctl disable firewalld.service
/usr/sbin/setenforce 0
echo “/usr/sbin/setenforce 0” >> /etc/rc.local
- 配置Base和EPEL源
1)配置Base源
备份系统base源
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
更新系统base源为清华源
sudo sed -e ‘s|^mirrorlist=|#mirrorlist=|g’ \
-e ‘s|^#baseurl=http://mirror.centos.org|baseurl=https://mirrors.tuna.tsinghua.edu.cn|g’ \
-i.bak \
/etc/yum.repos.d/CentOS-Base.repo
最后,更新软件包缓存
sudo yum makecache
详见https://mirrors.tuna.tsinghua.edu.cn/help/centos/
2)配置EPEL源
安装epel-release
yum install epel-release
配置使用清华的epel-release,可以用如下命令自动替换:
sed -e ‘s!^metalink=!#metalink=!g’ \
-e ‘s!^#baseurl=!baseurl=!g’ \
-e ‘s!//download\.fedoraproject\.org/pub!//mirrors.tuna.tsinghua.edu.cn!g’ \
-e ‘s!http://mirrors\.tuna!https://mirrors.tuna!g’ \
-i /etc/yum.repos.d/epel.repo /etc/yum.repos.d/epel-testing.repo
3)查看当前配置的YUM源
yum repolist
报错信息:No package Keepalived available Error: Nothing to do
报错总结:未配置Base和EPEL源或配置不正确
无法解析Base和EPEL源的地址—正确配置DNS
无法访问Base和EPEL源的地址—更换国内源
- 安装Nginx
- 安装Nginx
- Master–Node1
安装Nginx
yum install nginx -y
创建测试页面
echo “this is 10.86.24.2 node1” > /usr/share/nginx/html//index.html
启动Nginx
nginx
- Backup–Node2
安装Nginx
yum install nginx -y
创建测试页面
echo “this is 10.86.24.3 node2” > /usr/share/nginx/html//index.html
启动Nginx
nginx
- 安装配置keepalived
- 安装配置keepalived
- Master–Node1
1)安装keepalived
yum install keepalived -y
2)配置keepalived的Master节点
备份keepalived.conf文件
mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.20210213.bak
配置keepalived.conf文件
vi /etc/keepalived/keepalived.conf
### 配置keepalived服务:
# master服务器配置:
! Configuration File for keepalived
global_defs {
#指定机器的ID,一般设置为主机名,但不是必须
router_id node1
# 严格遵守VRRP协议,如果没有配置VIP,keepalived会无法正常工作vrrp_strict
}
vrrp_instance VI_1 {
state MASTER
# 应该是ens160,是执行ifconfigh获取的接口名称,而不是接口文件名ifcfg-ens160
interface ens160
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.86.24.226
}
}
3)启动keepalived
systemctl start keepalived
验证
cat /var/log/messages 查看日志
ip addr show
从其他服务器ping vip10.86.24.216
- Backup–Node2
1)安装keepalived
yum install keepalived -y
2)配置keepalived
备份keepalived.conf文件
mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.20210213.bak
配置keepalived.conf文件
vi /etc/keepalived/keepalived.conf
### 配置keepalived服务:
# backup服务器配置:
! Configuration File for keepalived
global_defs {
#指定机器的ID,一般设置为主机名,但不是必须
router_id node2
}
vrrp_instance VI_1 {
state BACKUP
# 应该是ens160,是执行ifconfigh获取的接口名称,而不是接口文件名ifcfg-ens160
interface ens160
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.86.24.226
}
}
3)启动keepalived
systemctl start keepalived
验证
cat /var/log/messages 查看日志
ip addr show
- 测试高可用
1)默认情况下
VIP http://10.86.24.226页面由node1提供
2)停止node1的keepalived,页面由node2提供
Node1停掉 keepalived ,systemctl stop keepalived
Node2使用ip addr show查看VIP是否转移
访问http://10.86.24.226页面由node2提供
- keepalived自动切换&切换告警
- keepalived自动切换&切换告警
- Nginx宕机keepalived自动切换
1)master节点—node1
Keepalived配置文件
vi /etc/keepalived/keepalived.conf
红色部分是为Nginx宕机keepalived自动切换增加的脚本内容
### 配置keepalived服务:
# master服务器配置:
! Configuration File for keepalived
global_defs {
#指定机器的ID,一般设置为主机名,但不是必须
router_id node1
}
vrrp_script check_nginx {
script “/data/sh/check_nginx.sh”
interval 1
weight -20
}
vrrp_instance VI_1 {
state MASTER
# 应该是ens160,是执行ifconfigh获取的接口名称,而不是接口文件名ifcfg-ens160
interface ens160
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.86.24.226
}
track_script {
check_nginx
}
}
创建Nginx检测脚本
mkdir -p /data/sh/
vim /data/sh/check_nginx.sh
chmod +x /data/sh/check_nginx.sh
#!/bin/bash
#############################
killall -0 nginx &>/dev/null
if [ $? -ne 0 ];then
exit 1
fi
2)backup节点—node2
# 配置keepalived backup服务器配置:
! Configuration File for keepalived
global_defs {
#指定机器的ID,一般设置为主机名,但不是必须
router_id node2
}
vrrp_script check_nginx {
script “/data/sh/check_nginx.sh”
interval 1
weight -20
}
vrrp_instance VI_1 {
state BACKUP
# 应该是ens160,是执行ifconfigh获取的接口名称,而不是接口文件名ifcfg-ens160
interface ens160
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.86.24.226
}
track_script {
check_nginx
}
}
3)测试自动切换
重启keepalived使新的配置生效
systemctl restart keepalived
手动关闭Nginx,模拟Nginx故障
nginx -s stop
查看日志及VIP转移
cat /var/log/message
备注:可使用tcpdump抓包来分析
安装抓包软件
yum install tcpdump -y
抓取vrrp数据包
tcpdump -i ens160 vrrp -nn
- keepalived切换实时邮件告警
1)master节点—node1
Keepalived配置文件
vi /etc/keepalived/keepalived.conf
红色部分是为实现keepalived切换实时邮件告警增加的内容
### 配置keepalived服务:
# master服务器配置:
! Configuration File for keepalived
global_defs {
#指定机器的ID,一般设置为主机名,但不是必须
router_id node1
}
vrrp_script check_nginx {
script “/data/sh/check_nginx.sh”
interval 1
weight -20
}
vrrp_instance VI_1 {
state MASTER
# 应该是ens160,是执行ifconfigh获取的接口名称,而不是接口文件名ifcfg-ens160
interface ens160
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.86.24.226
}
track_script {
check_nginx
}
notify_backup “/data/sh/notify.sh backup”
notify_master “/data/sh/notify.sh master”
}
安装mailx实现linux 使用外部邮箱实现发送邮件功能
- 安装mailx
# systemctl stop postfix
# systemctl disable postfix
# yum -y install mailx
postfix为本地邮箱推送方式需安装的软件,若同时配置本地和外部邮箱两种发送方式,系统会优先采用外部邮箱方式,所以该步骤不是必须但最好关闭这个服务。
- 编辑配置文件
vim /etc/mail.rc
set from=xxx@163.com
set smtp=smtp.163.com
set smtp-auth-user=xxx@163.com
set smtp-auth-password=xxxxxx
set smtp-auth=login
发送测试邮件
echo “test mail …” |mail -s “test” xxx@163.com
创建邮件告警脚本
vim /data/sh/notify.sh
### 配置keepalived服务:
# master服务器配置:
#!/bin/bash
#############################
SERVICE_NAME=”nginx+keepalived”
if [ $1 = “backup” ];then
echo “
时间=`date +%F-%H:%M:%S`
内容=`hostname` 的 $SERVICE_NAME 服务故障,目前切换为备用服务
器!” | mailx -s “`hostname` $SERVICE_NAME down” xxx@163.com
else
echo “
时间=`date +%F-%H:%M:%S`
内容=`hostname` 的 $SERVICE_NAME 服务恢复,目前切换为主
服务器!” | mailx -s “`hostname` $SERVICE_NAME up” xxx@163.com
fi
执行以下脚本,测试是否可以收到邮件
bash /data/sh/notify.sh master
bash /data/sh/notify.sh backup
停止Nginx测试是否可以收到邮件
重启keepalived使更新的配置生效
systemctl restart keepalived
手动停止、启动Nginx查收邮件
nginx -s stop 停止Nginx
nginx 启动Nginx
备注:backup节点的邮件告警设置可以参考master节点
- 配置Keepalived非抢占模式
在上面的配置中,master的nginx服务宕机,VIP会自动切换到backup服务
器上。但是在master端的nginx服务器恢复后,VIP会自动切换到master端。如果有需要,可以实现即使master恢复,VIP也不切换回去,只有在backup端宕机后,再进行切换。
配置要点:master和backup的state均设置为BACKUP,其次是master端配置nopreempt,backup不用配置。
- 配置Keepalived非抢占模式
- Master–Node1
### 配置keepalived服务:
# master服务器配置:
! Configuration File for keepalived
global_defs {
#指定机器的ID,一般设置为主机名,但不是必须
router_id node1
}
vrrp_script check_nginx {
script “/data/sh/check_nginx.sh”
interval 1
weight -20
}
vrrp_instance VI_1 {
#不抢占
Nopreempt
# 原来的state 为MASTER
state BACKUP
# 应该是ens160,是执行ifconfigh获取的接口名称,而不是接口文件名ifcfg-ens160
interface ens160
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.86.24.226
}
track_script {
check_nginx
}
}
- Backup–Node2
# 配置keepalived backup服务器配置:
! Configuration File for keepalived
global_defs {
#指定机器的ID,一般设置为主机名,但不是必须
router_id node2
}
vrrp_script check_nginx {
script “/data/sh/check_nginx.sh”
interval 1
weight -20
}
vrrp_instance VI_1 {
state BACKUP
# 应该是ens160,是执行ifconfigh获取的接口名称,而不是接口文件名ifcfg-ens160
interface ens160
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.86.24.226
}
track_script {
check_nginx
}
}