- A+
所属分类：linux技术
Lvs
目录
Lvs简介
LVS（Linux Virtual Server）即Linux虚拟服务器，是由章文嵩博士主导的开源负载均衡项目，目前LVS已经被集成到Linux内核模块中。在Linux内核中实现了基于IP的数据请求负载均衡调度方案，终端互联网用户从外部访问公司的负载均衡服务器，终端用户的Web请求会发送给LVS调度器，调度器根据自己预设的算法决定将该请求发送给后端的某台Web服务器，比如，轮询算法可以将外部的请求平均分发给后端的所有服务器，终端用户访问LVS调度器虽然会被转发到后端真实的服务器，但如果真实服务器连接的是相同的存储，提供的服务也是相同的服务，最终用户不管是访问哪台真实服务器，得到的服务内容都是一样的，整个集群对用户而言都是透明的。最后根据LVS工作模式的不同，真实服务器会选择不同的方式将用户需要的数据发送到终端用户，LVS工作模式分为NAT模式、TUN模式、以及DR模式。
VIP RIP
官方站点
体系结构
使用LVS架设的服务器集群系统有三个部分组成：最前端的负载均衡层（Loader Balancer），中间的服务器群组层，用Server Array表示，最底层的数据共享存储层，用Shared Storage表示。在用户看来所有的应用都是透明的，用户只是在使用一个虚拟服务器提供的高性能服务。
Load Balancer层：位于整个集群系统的最前端，有一台或者多台负载调度器（Director Server）组成，LVS模块就安装在Director Server上，而Director的主要作用类似于一个路由器，它含有完成LVS功能所设定的路由表，通过这些路由表把用户的请求分发给Server Array层的应用服务器（Real Server）上。同时，在Director Server上还要安装对Real Server服务的监控模块Ldirectord，此模块用于监测各个Real Server服务的健康状况。在Real Server不可用时把它从LVS路由表中剔除，恢复时重新加入。
Server Array层：由一组实际运行应用服务的机器组成，Real Server可以是WEB服务器、MAIL服务器、FTP服务器、DNS服务器、视频服务器中的一个或者多个，每个Real Server之间通过高速的LAN或分布在各地的WAN相连接。在实际的应用中，Director Server也可以同时兼任Real Server的角色。
Shared Storage层：是为所有Real Server提供共享存储空间和内容一致性的存储区域，在物理上，一般有磁盘阵列设备组成，为了提供内容的一致性，一般可以通过NFS网络文件系统共享数据，但是NFS在繁忙的业务系统中，性能并不是很好，此时可以采用集群文件系统，例如Red hat的GFS文件系统，oracle提供的OCFS2文件系统等。
从整个LVS结构可以看出，Director Server是整个LVS的核心，目前，用于Director Server的操作系统只能是Linux和FreeBSD，linux2.6内核不用任何设置就可以支持LVS功能，而FreeBSD作为 Director Server的应用还不是很多，性能也不是很好。对于Real Server，几乎可以是所有的系统平台，Linux、windows、Solaris、AIX、BSD系列都能很好的支持。
LVS管理工具
ipvs：内核中的协议栈上实现
ipvs是LVS软件核心，是允许在LB（负载均衡层）上，这是个基于ip层的负载均衡
ipvs的总体结构主要有ip包处理，负载均衡算法，系统配置和管理三个模块以及虚拟服务器与真实服务器链表组成。
ipvs管理集群服务管理服务上的RS(real server)。
一个ipvs主机可以同时定义多个cluster server，但可能会影响调度性格。
一个ipvs服务至少应该有一个RS。
配置 lvs-nat 模式的 httpd 负载集群---http
环境说明：
| 主机名称 网卡信息 | （ens160 为 nat、ens192 为仅主机） | 安装应用 | 系统 |
|---|---|---|---|
| Client 客户端 | 192.168.222.250（ens33） | 无 | RHEL8 |
| DR | DIP:192.168.222.137(ens33)---VIP:192.168.17.10(ens37) | ipvsadm | RHEL8 |
| RS1 | RIP:192.168.222.138(ens33)---gw:192.168.222.137 | httpd | RHEL8 |
| RS2 | RIP:192.168.222.139(ens33)---gw:192.168.222.137 | httpd | RHEL8 |
DR、RS1、RS2 三台主机都关闭防火墙和 selinux
[root@DR ~]# systemctl stop firewalld.service [root@DR ~]# vim /etc/selinux/config SELINUX=disabled [root@DR ~]# setenforce 0 [root@DR ~]# systemctl disable --now firewalld.service Removed /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. [root@RS1 ~]# systemctl stop firewalld.service [root@RS1 ~]# vim /etc/selinux/config SELINUX=disabled [root@RS1 ~]# setenforce 0 [root@RS1 ~]# systemctl disable --now firewalld.service Removed /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. [root@RS2 ~]# systemctl stop firewalld.service [root@RS2 ~]# vim /etc/selinux/config SELINUX=disabled [root@RS2 ~]# setenforce 0 [root@RS2 ~]# systemctl disable --now firewalld.service Removed /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. 配置 ip 信息
DR：
添加ens192网卡信息仅主机
[root@DR ~]# ifconfig //查看刚刚添加的网卡 ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.222.137 netmask 255.255.255.0 broadcast 192.168.222.255 inet6 fe80::20c:29ff:fec7:eb2 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:c7:0e:b2 txqueuelen 1000 (Ethernet) RX packets 122996 bytes 47438695 (45.2 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 159300 bytes 28724758 (27.3 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens37: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.17.134 netmask 255.255.255.0 broadcast 192.168.17.255 inet6 fe80::318:a5d1:6043:25f prefixlen 64 scopeid 0x20<link> ether 00:0c:29:c7:0e:bc txqueuelen 1000 (Ethernet) RX packets 6 bytes 1022 (1022.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 12 bytes 1468 (1.4 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 查看刚刚添加网卡的网段
[root@DR ~]# cd /etc/sysconfig/network-scripts/ [root@DR network-scripts]# ls ifcfg-ens33 [root@DR network-scripts]# vim ifcfg-ens33 [root@DR network-scripts]# cat ifcfg-ens33 TYPE=Ethernet BOOTPROTO=static NAME=ens33 DEVICE=ens33 ONBOOT=yes IPADDR=192.168.222.137 NETMASK=255.255.255.0 GATEWAY=192.168.222.2 DNS1=192.168.222.2 [root@DR network-scripts]# systemctl restart NetworkManager [root@DR network-scripts]# nmcli connection up ens36 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3) [root@DR network-scripts]# cp ifcfg-ens33 ifcfg-ens37 [root@DR network-scripts]# ls ifcfg-ens33 ifcfg-ens37 [root@DR network-scripts]# vim ifcfg-ens37 [root@DR network-scripts]# cat ifcfg-ens37 TYPE=Ethernet BOOTPROTO=static NAME=ens37 DEVICE=ens37 ONBOOT=yes IPADDR=192.168.17.10 NETMASK=255.255.255.0 GATEWAY=192.168.222.2 DNS1=192.168.222.2 [root@DR network-scripts]# systemctl restart NetworkManager [root@DR network-scripts]# nmcli connection up ens37 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3) [root@DR network-scripts]# nmcli connection show NAME UUID TYPE DEVICE ens33 c96bc909-188e-ec64-3a96-6a90982b08ad ethernet ens33 ens37 4a5516a4-dfa4-24af-b1c4-e843e312e2fd ethernet ens37 Wired connection 1 2fcbb410-1e4b-3138-a0de-629a7e701de6 ethernet -- RS1:
[root@RS1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33 [root@RS1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33 TYPE=Ethernet BOOTPROTO=static NAME=ens33 DEVICE=ens33 ONBOOT=yes IPADDR=192.168.222.138 NETMASK=255.255.255.0 GATEWAY=192.168.222.137 DNS1=192.168.222.2 [root@RS1 ~]# systemctl restart NetworkManager [root@RS1 ~]# nmcli connection up ens33 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3) RS2:
[root@RS2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33 [root@RS2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33 TYPE=Ethernet BOOTPROTO=static NAME=ens33 DEVICE=ens33 ONBOOT=yes IPADDR=192.168.222.139 NETMASK=255.255.255.0 GATEWAY=192.168.222.137 DNS1=192.168.222.2 [root@RS2 ~]# systemctl restart NetworkManager [root@RS2 ~]# nmcli connection up ens33 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/2) 后端RS1和RS2部署WEB服务器
RS1:
[root@RS1 ~]# cd /etc/yum.repos.d/ [root@RS1 yum.repos.d]# ls CentOS-Base.repo epel-next-testing.repo epel-playground.repo epel-testing.repo epel-modular.repo epel-next.repo epel-testing-modular.repo epel.repo [root@RS1 yum.repos.d]# rm -rf * [root@RS1 yum.repos.d]# ls [root@RS1 yum.repos.d]# vim lnh.repo //配置本地的yum仓库 [root@RS1 yum.repos.d]# cat lnh.repo [BaseOS] name=111 baseurl=file:///mnt/BaseOS gpgcheck=0 enabled=1 [AppStream] name=222 baseurl=file:///mnt/AppStream gpgcheck=0 enabled=1 [root@RS1 yum.repos.d]# cd [root@RS1 ~]# mount /dev/cdrom /mnt/ //挂载镜像 mount: /mnt: WARNING: device write-protected, mounted read-only. [root@RS1 ~]# dnf -y install httpd [root@RS1 ~]# echo RS1 > /var/www/html/index.html //创建一个网站 [root@RS1 ~]# systemctl enable --now httpd //设置开机自启 Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. [root@RS1 ~]# curl http://192.168.222.138 //可以访问自己 RS1 RS2:
[root@RS2 ~]# cd /etc/yum.repos.d/ [root@RS2 yum.repos.d]# ls CentOS-Base.repo epel-next-testing.repo epel-playground.repo epel-testing.repo epel-modular.repo epel-next.repo epel-testing-modular.repo epel.repo [root@RS2 yum.repos.d]# rm -rf * [root@RS2 yum.repos.d]# vim lnh.repo //配置本地yum仓库 [root@RS2 yum.repos.d]# cat lnh.repo [BaseOS] name=111 baseurl=file:///mnt/BaseOS gpgcheck=0 enabled=1 [AppStream] name=222 baseurl=file:///mnt/AppStream gpgcheck=0 enabled=1 [root@RS2 ~]# mount /dev/cdrom /mnt/ //挂载镜像 mount: /mnt: WARNING: device write-protected, mounted read-only. [root@RS2 ~]# dnf -y install httpd [root@RS2 ~]# echo RS2 > /var/www/html/index.html //创建一个网站 [root@RS2 ~]# systemctl enable --now httpd //设置开机自启 Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. [root@RS2 ~]# curl http://192.168.222.139 //可以访问自己 RS2 配置DR
开启IP转发功能
[root@DR ~]# vim /etc/sysctl.conf net.ipv4.ip_forward = 1 //添加(等于号前后都要空格) [root@DR ~]# sysctl -p net.ipv4.ip_forward = 1 安装ipvsadm并添加规则
[root@DR ~]# cd /etc/yum.repos.d/ [root@DR yum.repos.d]# ls CentOS-Base.repo epel-next-testing.repo epel-playground.repo epel-testing.repo epel-modular.repo epel-next.repo epel-testing-modular.repo epel.repo [root@DR yum.repos.d]# rm -rf * [root@DR yum.repos.d]# ls [root@DR yum.repos.d]# vim lnh.repo //配置本地yum仓库 [root@DR yum.repos.d]# cat lnh.repo [BaseOS] name=111 baseurl=file:///mnt/BaseOS gpgcheck=0 enabled=1 [AppStream] name=222 baseurl=file:///mnt/AppStream gpgcheck=0 enabled=1 [root@DR yum.repos.d]# cd [root@DR ~]# mount /dev/cdrom /mnt/ //挂载镜像 mount: /mnt: WARNING: device write-protected, mounted read-only. [root@DR ~]# dnf -y install ipvsadm [root@DR ~]# ipvsadm -A -t 192.168.17.10:80 -s rr //这里是vip的地址 [root@DR ~]# ipvsadm -a -t 192.168.17.10:80 -r 192.168.222.138:80 -m //vip加RS1的地址 [root@DR ~]# ipvsadm -a -t 192.168.17.10:80 -r 192.168.222.139:80 -m //vip加RS2的地址 [root@DR ~]# ipvsadm -Ln //列出添加的ip IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.17.10:80 rr -> 192.168.222.138:80 Masq 1 0 0 -> 192.168.222.139:80 Masq 1 0 0 [root@DR ~]# ipvsadm -Sn > /etc/sysconfig/ipvsadm //因为开机自启后这些就会消失，所以将这些导入文件里面存放 [root@DR ~]# cat /etc/sysconfig/ipvsadm //查看 -A -t 192.168.17.10:80 -s rr -a -t 192.168.17.10:80 -r 192.168.222.138:80 -m -w 1 -a -t 192.168.17.10:80 -r 192.168.222.139:80 -m -w 1 [root@DR ~]# systemctl enable --now ipvsadm.service //设置开机自启 Created symlink /etc/systemd/system/multi- user.target.wants/ipvsadm.service → /usr/lib/systemd/system/ipvsadm.service. 客户端测试
配置好ip信息
测试：
[root@Client ~]# curl http://192.168.17.10 RS2 [root@Client ~]# curl http://192.168.17.10 RS1 [root@Client ~]# curl http://192.168.17.10 RS2 [root@Client ~]# curl http://192.168.17.10 RS1 配置lvs-nat模式的httpd负载集群---https
在DR中生成一对密钥
[root@DR ~]# mkdir -p /etc/pki/CA/private [root@DR ~]# cd /etc/pki/CA/ [root@DR CA]# ls private [root@DR CA]# (umask 077;openssl genrsa -out private/cakey.pem 2048) Generating RSA private key, 2048 bit long modulus (2 primes) ............................................................................................+++++ ......................................................+++++ e is 65537 (0x010001) [root@DR CA]# openssl rsa -in private/cakey.pem -pubout writing RSA key -----BEGIN PUBLIC KEY----- MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAlc/ybW176QXUucAGvPei 39qN2tibEGL6FdllTJ1lyYTOSsyKZVqhshcuCA74W34NtQ7C6d7SE9y7u4WKN+I9 TR07+JgUXdfDXZxPl1cV19b2z8xJttObxkTtm09OyzUDepKjCqt8m2HQqI2imi5f ooBW44sHCkPRnTx11CR1DTwVqQmEh6cfuj2QbhUkkin68omgabJvvveW08XnXmHZ rXYsjaq0ekWdfVobCG45nJMVHG3LRVfEUaI53TBswyhTHvyRDlSDHCQkCo+LuBPW Z94CR9C2GSUCTo9O6lnAkiqbUkrPs/tUE9RhxmlsEoANNbHBKv172Qdtjjx2tg2X ZwIDAQAB -----END PUBLIC KEY----- [root@DR CA]# openssl req -new -x509 -key private/cakey.pem -out cacert.pem -days 1024 You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:CN State or Province Name (full name) []:HB Locality Name (eg, city) [Default City]:WH Organization Name (eg, company) [Default Company Ltd]:TU^H^C [root@DR CA]# openssl req -new -x509 -key private/cakey.pem -out cacert.pem -days 1024 You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:CN //国家 State or Province Name (full name) []:HB //省份 Locality Name (eg, city) [Default City]:WH //城市 Organization Name (eg, company) [Default Company Ltd]:tushanbu //组织 Organizational Unit Name (eg, section) []:linux //部门 Common Name (eg, your name or your server's hostname) []:lnh //名字 Email Address []:lnh@example.com //邮箱地址 [root@DR CA]# touch index.txt && echo 01 > serial //提前创建好数据存放的文件 在RS1中生成证书签署请求，并发送给CA
[root@RS1 ~]# dnf -y install mod_ssl [root@RS1 ~]# mkdir /etc/httpd/ssl [root@RS1 ~]# cd /etc/httpd/ssl/ [root@RS1 ssl]# (umask 077;openssl genrsa -out httpd.key 2048) Generating RSA private key, 2048 bit long modulus (2 primes) ................+++++ .............................................+++++ e is 65537 (0x010001) [root@RS1 ssl]# openssl req -new -key httpd.key -days 1024 -out httpd.csr Ignoring -days; not generating a certificate You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:CN //国家 State or Province Name (full name) []:HB //省份 Locality Name (eg, city) [Default City]:WH //城市 Organization Name (eg, company) [Default Company Ltd]:tushanbu //组织 Organizational Unit Name (eg, section) []:linux //部门 Common Name (eg, your name or your server's hostname) []:lnh //名字 Email Address []:lnh@example.com //邮箱地址 Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: //直接回车 An optional company name []: //直接回车 [root@RS1 ssl]# ls httpd.csr httpd.key [root@RS1 ssl]# scp httpd.csr root@192.168.222.137:/root/ //发送给DR The authenticity of host '192.168.222.137 (192.168.222.137)' can't be established. ECDSA key fingerprint is SHA256:jJ7HFCOrVQKPjfacavF08vxsn4hSKTG3q9SV78ApryQ. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '192.168.222.137' (ECDSA) to the list of known hosts. root@192.168.222.137's password: httpd.csr 100% 1025 603.0KB/s 00:00 在DR中查看
[root@DR ~]# ls anaconda-ks.cfg httpd.csr CA签署证书并发给RS1
[root@DR ~]# mkdir /etc/pki/CA/newcerts [root@DR ~]# openssl ca -in httpd.csr -out httpd.crt -days 1024 Using configuration from /etc/pki/tls/openssl.cnf Check that the request matches the signature Signature ok Certificate Details: Serial Number: 1 (0x1) Validity Not Before: Sep 27 08:49:07 2022 GMT Not After : Jul 17 08:49:07 2025 GMT Subject: countryName = CN stateOrProvinceName = HB organizationName = tushanbu organizationalUnitName = linux commonName = lnh emailAddress = lnh@example.com X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: 57:37:7D:6C:96:E5:E4:7A:86:12:46:2A:5C:65:6A:B6:AC:88:47:02 X509v3 Authority Key Identifier: keyid:E9:0F:F6:03:3D:0B:EA:6A:09:BE:98:B3:B4:74:BE:7A:16:0C:6A:22 Certificate is to be certified until Jul 17 08:49:07 2025 GMT (1024 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated [root@DR ~]# ls anaconda-ks.cfg httpd.crt httpd.csr 将CA签署的证书httpd.crt和服务器的证书cacert.pem发送给RS1
[root@DR ~]# scp httpd.crt root@192.168.222.138:/etc/httpd/ssl The authenticity of host '192.168.222.138 (192.168.222.138)' can't be established. ECDSA key fingerprint is SHA256:hbtTaGj2vCLMFuGhq4w/WkcSJKQKqlkFRFyigFAZGZY. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '192.168.222.138' (ECDSA) to the list of known hosts. root@192.168.222.138's password: httpd.crt 100% 4545 4.2MB/s 00:00 [root@DR ~]# scp /etc/pki/CA/cacert.pem root@192.168.222.138:/etc/httpd/ssl root@192.168.222.138's password: cacert.pem 100% 1383 436.3KB/s 00:00 RS2配置https
[root@RS2 ~]# dnf -y install mod_ssl [root@RS2 ~]# mkdir /etc/httpd/ssl ** RS1中把RS1的证书和密钥发送给RS2**
[root@RS1 ssl]# scp cacert.pem httpd.crt httpd.key root@192.168.222.139:/etc/httpd/ssl The authenticity of host '192.168.222.139 (192.168.222.139)' can't be established. ECDSA key fingerprint is SHA256:anVVbTlEIzA1E8rB7IbLzaf7t9oQjB0qFP6Dd/ijnJI. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '192.168.222.139' (ECDSA) to the list of known hosts. root@192.168.222.139's password: cacert.pem 100% 1383 1.5MB/s 00:00 httpd.crt 100% 4545 2.6MB/s 00:00 httpd.key 100% 1679 1.2MB/s 00:00 在RS1中修改https的配置文件
[root@RS1 ssl]# vim /etc/httpd/conf.d/ssl.conf SSLCertificateFile /etc/httpd/ssl/httpd.crt //修改 SSLCertificateKeyFile /etc/httpd/ssl/httpd.key //修改 # huge file containing all of them (file must be PEM encoded) #SSLCACertificateFile /etc/pki/tls/certs/ca-bundle.crt SSLCACertificateFile /etc/httpd/ssl/cacert.pem //添加 [root@RS1 ssl]# systemctl enable --now httpd [root@RS1 ssl]# ss -tnl |grep 443 LISTEN 0 128 *:443 *:* [root@RS1 ssl]# ss -antl State Recv-Q Send-Q Local Address:Port Peer Address:Port Process LISTEN 0 128 0.0.0.0:111 0.0.0.0:* LISTEN 0 128 0.0.0.0:22 0.0.0.0:* LISTEN 0 128 *:443 *:* LISTEN 0 128 [::]:111 [::]:* LISTEN 0 128 *:80 *:* LISTEN 0 128 [::]:22 [::]:* 在RS2中修改https的配置文件
[root@RS2 ~]# cd /etc/httpd/ssl/ [root@RS2 ssl]# ls cacert.pem httpd.crt httpd.key [root@RS2 ssl]# pwd /etc/httpd/ssl [root@RS2 ssl]# vim /etc/httpd/conf.d/ssl.conf SSLCertificateFile /etc/httpd/ssl/httpd.crt //修改 SSLCertificateKeyFile /etc/httpd/ssl/httpd.key //修改 # certificate for convenience. #SSLCertificateChainFile /etc/pki/tls/certs/server-chain.crt SSLCACertificateFile /etc/httpd/ssl/cacert.pem //添加 [root@RS2 ssl]# systemctl enable --now httpd [root@RS2 ssl]# ss -antl State Recv-Q Send-Q Local Address:Port Peer Address:Port Process LISTEN 0 128 0.0.0.0:22 0.0.0.0:* LISTEN 0 128 [::]:22 [::]:* LISTEN 0 128 *:443 *:* LISTEN 0 128 *:80 *:* 在DR中添加规则
[root@DR ~]# ipvsadm -A -t 192.168.17.10:443 -s rr [root@DR ~]# ipvsadm -a -t 192.168.17.10:443 -r 192.168.222.138 -m [root@DR ~]# ipvsadm -a -t 192.168.17.10:443 -r 192.168.222.139 -m [root@DR ~]# ipvsadm -Ln //列出刚刚添加的 IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.17.10:80 rr -> 192.168.222.138:80 Masq 1 0 0 -> 192.168.222.139:80 Masq 1 0 0 TCP 192.168.17.10:443 rr -> 192.168.222.138:443 Masq 1 0 0 -> 192.168.222.139:443 Masq 1 0 0 [root@DR ~]# ipvsadm -Sn > /etc/sysconfig/ipvsadm //因为开机后这些就会消失，所以将这些导入文件中 [root@DR ~]# cat /etc/sysconfig/ipvsadm -A -t 192.168.17.10:80 -s rr -a -t 192.168.17.10:80 -r 192.168.222.138:80 -m -w 1 -a -t 192.168.17.10:80 -r 192.168.222.139:80 -m -w 1 -A -t 192.168.17.10:443 -s rr -a -t 192.168.17.10:443 -r 192.168.222.138:443 -m -w 1 -a -t 192.168.17.10:443 -r 192.168.222.139:443 -m -w 1 客户端测试
[root@Client ~]# curl -k https://192.168.17.10:443 RS2 [root@Client ~]# curl -k https://192.168.17.10:443 RS1 [root@Client ~]# curl -k https://192.168.17.10:443 RS2 [root@Client ~]# curl -k https://192.168.17.10:443 RS1 配置lvs-dr模式的httpd负载集群--https协议
| Lvs服务器（DR） | DIP 192.168.222.137 | VIP 192.168.222.100 |
|---|---|---|
| Apache服务器（RS1） | RIP 192.168.222.138 | VIP 192.168.222.100 |
| Apache服务器（RS2） | RIP 192.168.222.139 | VIP 192.168.222.100 |
| client客户端 | IP 192.168.222.250 | 无 |
配置httpd
RS1：
关闭防火墙和selinux
[root@RS1 ~]# systemctl stop firewalld.service [root@RS1 ~]# vim /etc/selinux/config SELINUX=disabled root@RS1 ~]# setenforce 0 [root@RS1 ~]# systemctl disable --now firewalld.service Removed /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. 安装httpd
[root@RS1 ~]# dnf -y install httpd [root@RS1 ~]# echo "RS1" > /var/www/html/index.html [root@RS1 ~]# systemctl enable --now httpd Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. RS2：
关闭防火墙和selinux
[root@RS2 ~]# systemctl stop firewalld.service [root@RS2 ~]# vim /etc/selinux/config SELINUX=disabled [root@RS2 ~]# setenforce 0 [root@RS2 ~]# systemctl disable --now firewalld.service Removed /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. 安装httpd
[root@RS2 ~]# dnf -y install httpd [root@RS2 ~]# echo "RS2" > /var/www/html/index.html [root@RS2 ~]# systemctl enable --now httpd Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. DR上配置ip
DR：
关闭防火墙和selinux
[root@DR ~]# systemctl stop firewalld.service [root@DR ~]# vim /etc/selinux/config SELINUX=disabled [root@DR ~]# setenforce 0 [root@DR ~]# systemctl disable --now firewalld.service Removed /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. 添加ip
[root@DR ~]# dnf -y install net-tools root@DR ~]# ifconfig lo 192.168.222.100/32 broadcast 192.168.222.100 netmask 255.255.255.255 up [root@DR ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 192.168.222.100/0 brd 192.168.222.100 scope global lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 00:0c:29:c7:0e:b2 brd ff:ff:ff:ff:ff:ff inet 192.168.222.137/24 brd 192.168.222.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fec7:eb2/64 scope link valid_lft forever preferred_lft forever RS上配置arp内核参数
RS1和RS2上都需要操作
RS1:
[root@RS1 ~]# vim /etc/sysctl.conf #将对应网卡设置为只回应目标IP为自身接口地址的ARP请求 net.ipv4.conf.all.arp_ignore = 1 #将ARP请求的源IP设置为所有接口的IP，也就是RIP net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.lo.arp_ignore = 1 net.ipv4.conf.lo.arp_announce = 2 //等于号前后都要空格 [root@RS1 ~]# sysctl -p net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.lo.arp_ignore = 1 net.ipv4.conf.lo.arp_announce = 2 RS2:
[root@RS2 ~]# vim /etc/sysctl.conf #将对应网卡设置为只回应目标IP为自身接口地址的ARP请求 net.ipv4.conf.all.arp_ignore = 1 #将ARP请求的源IP设置为所有接口的IP，也就是RIP net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.lo.arp_ignore = 1 net.ipv4.conf.lo.arp_announce = 2 //等于号前后都要空格 [root@RS2 ~]# sysctl -p net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.lo.arp_ignore = 1 net.ipv4.conf.lo.arp_announce = 2 RS上配置VIP
一定要先配置好内核参数，再配置VIP，如果先配置VIP，VIP配置好后会立即通告给所有人，而修改内核参数就是为了不通告。
DR服务器的ens33网卡的ip：192.168.222.100作为VIP
两台RS都要做
RS1:
[root@RS1 ~]# dnf -y install net-tools [root@RS1 ~]# ifconfig lo 192.168.222.100/32 broadcast 192.168.222.100 netmask 255.255.255.255 up RS2:
[root@RS2 ~]# dnf -y install net-tools [root@RS2 ~]# ifconfig lo 192.168.222.100/32 broadcast 192.168.222.100 netmask 255.255.255.255 up 添加路由信息
RS1:
[root@RS1 ~]# route add -host 192.168.222.100/32 dev lo RS2:
[root@RS2 ~]# route add -host 192.168.222.100/32 dev lo 添加并保存规则
[root@DR ~]# dnf -y install ipvsadm [root@DR ~]# ipvsadm -A -t 192.168.222.100:80 -s rr [root@DR ~]# ipvsadm -a -t 192.168.222.100:80 -r 192.168.222.138:80 -g [root@DR ~]# ipvsadm -a -t 192.168.222.100:80 -r 192.168.222.139:80 -g [root@DR ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.222.100:80 rr -> 192.168.222.138:80 Route 1 0 0 -> 192.168.222.139:80 Route 1 0 0 root@DR ~]# ipvsadm -Sn > /etc/sysconfig/ipvsadm [root@DR ~]# cat /etc/sysconfig/ipvsadm -A -t 192.168.222.100:80 -s rr -a -t 192.168.222.100:80 -r 192.168.222.138:80 -g -w 1 -a -t 192.168.222.100:80 -r 192.168.222.139:80 -g -w 1 [root@DR ~]# systemctl enable --now ipvsadm.service Created symlink /etc/systemd/system/multi-user.target.wants/ipvsadm.service → /usr/lib/systemd/system/ipvsadm.service. 客户端验证
[root@client ~]# curl http://192.168.222.100 RS2 [root@client ~]# curl http://192.168.222.100 RS1 [root@client ~]# curl http://192.168.222.100 RS2 [root@client ~]# curl http://192.168.222.100 RS1 配置lvs-tun模式的负载均衡集群--http协议
DR：
关闭防火墙和selinux
修改内核参数，开启IP转发
[root@DR ~]# systemctl stop firewalld.service [root@DR ~]# vim /etc/selinux/config SELINUX=disabled [root@DR ~]# setenforce 0 [root@DR ~]# systemctl disable --now firewalld.service Removed /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. [root@DR ~]# dnf -y install ipvsadm [root@DR ~]# dnf -y install net-tools [root@DR ~]# vim /etc/sysctl.conf net.ipv4.ip_forward = 1 //等于号前后都要空格 [root@DR ~]# sysctl -p net.ipv4.ip_forward = 1 [root@DR ~]# ifconfig tunl0 192.168.222.55 broadcast 192.168.222.55 netmask 255.255.255.255 up RS1和RS2（两台主机操作一样）
关闭防火墙和selinux，部署httpd
RS1:
[root@RS1 ~]# systemctl stop firewalld.service [root@RS1 ~]# vim /etc/selinux/config SELINUX=disabled [root@RS1 ~]# setenforce 0 [root@RS1 ~]# systemctl disable --now firewalld.service Removed /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. [root@RS1 ~]# dnf -y install httpd [root@RS1 ~]# echo "RS1" > /var/www/html/index.html [root@RS1 ~]# cat /var/www/html/index.html RS1 [root@RS1 ~]# systemctl enable --now httpd Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. RS2:
root@RS2 ~]# systemctl stop firewalld.service [root@RS2 ~]# vim /etc/selinux/config SELINUX=disabled [root@RS2 ~]# setenforce 0 [root@RS2 ~]# systemctl disable --now firewalld.service Removed /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. [root@RS2 ~]# dnf -y install httpd [root@RS2 ~]# echo "RS2" > /var/www/html/index.html [root@RS2 ~]# cat /var/www/html/index.html RS2 [root@RS2 ~]# systemctl enable --now httpd Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. 启用ipip模块
RS1：
[root@RS1 ~]# dnf -y install net-tools [root@RS1 ~]# modprobe ipip [root@RS1 ~]# ifconfig tunl0 192.168.222.55 broadcast 192.168.222.55 netmask 255.255.255.255 up RS2：
[root@RS2 ~]# dnf -y install net-tools [root@RS2 ~]# modprobe ipip [root@RS2 ~]# ifconfig tunl0 192.168.222.55 broadcast 192.168.222.55 netmask 255.255.255.255 up 修改内核参数为
RS1:
[root@RS1 ~]# vim /etc/sysctl.conf net.ipv4.conf.tunl0.arp_ignore = 1 net.ipv4.conf.tunl0.arp_announce = 2 net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.tunl0.rp_filter = 0 net.ipv4.conf.all.rp_filter = 0 // 等于号前后都要空格 [root@RS1 ~]# sysctl -p net.ipv4.conf.tunl0.arp_ignore = 1 net.ipv4.conf.tunl0.arp_announce = 2 net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.tunl0.rp_filter = 0 net.ipv4.conf.all.rp_filter = 0 RS2:
[root@RS2 ~]# vim /etc/sysctl.conf net.ipv4.conf.tunl0.arp_ignore = 1 net.ipv4.conf.tunl0.arp_announce = 2 net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.tunl0.rp_filter = 0 net.ipv4.conf.all.rp_filter = 0 // 等于号前后都要空格 [root@RS2 ~]# sysctl -p net.ipv4.conf.tunl0.arp_ignore = 1 net.ipv4.conf.tunl0.arp_announce = 2 net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 net.ipv4.conf.tunl0.rp_filter = 0 net.ipv4.conf.all.rp_filter = 0 DR上添加规则
[root@DR ~]# ipvsadm -A -t 192.168.222.55:80 -s rr [root@DR ~]# ipvsadm -a -t 192.168.222.55:80 -r 192.168.222.138 -i [root@DR ~]# ipvsadm -a -t 192.168.222.55:80 -r 192.168.222.139 -i [root@DR ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.222.55:80 rr -> 192.168.222.138:80 Tunnel 1 0 0 -> 192.168.222.139:80 Tunnel 1 0 0 [root@DR ~]# ipvsadm -Sn > /etc/sysconfig/ipvsadm [root@DR ~]# systemctl enable --now ipvsadm.service Created symlink /etc/systemd/system/multi-user.target.wants/ipvsadm.service → /usr/lib/systemd/system/ipvsadm.service. 客户端验证
[root@client ~]# curl http://192.168.222.55 RS2 [root@client ~]# curl http://192.168.222.55 RS1 [root@client ~]# curl http://192.168.222.55 RS2 [root@client ~]# curl http://192.168.222.55 RS1 
