I Working principle of LVS-DR
1. Packet flow analysis
(1) The client sends a request to the Director Server, and the requested data message (the source IP is CIP and the target IP is VIP) reaches the kernel space.
(2) The director server and Real Server are in the same network, and the data is transmitted through the two-layer data link layer.
(3) The kernel space judges that the target IP of the packet is the local VIP. At this time, IPVS (IP virtual server) compares whether the service requested by the packet is a cluster service. If it is a cluster service, the packet will be re encapsulated. Modify the source MAC address to the MAC address of Director Server and the target MAC address to the MAC address of Real Server. The source IP address and the target IP address have not changed, and then send the packet to Real Server.
(4) If the MAC address of the request message arriving at the Real Server is its own MAC address, this message will be received. The data packet re encapsulates the message (the source IP address is VIP and the target IP is CIP), transmits the response message to the physical network card through the lo interface, and then sends it out.
(5) The real server directly transmits the response message to the client.
2. Characteristics of Dr mode
(1)Director Server and Real Server must be in the same physical network.
(2)Real Server can use private address or public address. If the public network address is used, RIP can be accessed directly through the Internet.
(3) The director server serves as an access portal to the cluster, but not as a gateway.
(4) All request messages pass through the Director Server, but the reply response message cannot pass through the Director Server.
(5) The gateway of the Real Server is not allowed to point to the Director Server IP, that is, the packets sent by the Real Server are not allowed to pass through the Director Server.
(6) The lo interface on the real server configures the IP address of the VIP.
3. ARP problem in lvs-dr
Problem 1: in LVS-DR load balancing cluster, the load balancer and node server should be configured with the same VIP address and the same IP address in LAN, which is bound to cause the disorder of ARP communication of servers
Solution: when the ARP broadcast is sent to the LVS-DR cluster, it is assumed that both the load balancer and the node server are connected to the same network, and they will receive the ARP broadcast, but only the front-end load balancer responds, and other node servers should not respond to the ARP broadcast.
Solution: use the virtual interface lo: 0 to host the VIP address;
Set kernel parameter arp_ignore=1: the system only responds to ARP requests whose destination IP is local IP.
Question 2: when the message returned by RealServer (the source IP is VIP) is forwarded by the router and re encapsulated, it is necessary to obtain the MAC address of the router first. When sending ARP request, Linux defaults to using the source IP address of the IP packet (i.e. VIP) as the IP address in the ARP request packet instead of the IP address of the sending interface. The routing table is based on the ARP table item, The new request message will be forwarded to RealServer, resulting in the failure of the Director's VIP
Solution: process the node server and set the kernel parameter ARP_ Announcement = 2: the system does not use the source address of the IP packet to set the source address of the ARP request, but selects the IP address of the sending interface.
II LVS-DR load balancing cluster deployment
DR server (load scheduler): 192.168.121.11
Web server 1 (node server 1): 192.168.121.33
Web server 2 (node server 2): 192.168.121.44
NFS server: 192.168.121.55
VIP: 192.168.121.100
Client: 192.168.121.200
1. Configure load scheduler (192.168.121.11)
systemctl stop firewalld.service setenforce 0 modprobe ip_vs cat /proc/net/ip_vs yum -y install ipvsadm
1. Configure virtual IP address (VIP: 192.168.121.100)
cd /etc/sysconfig/network-scripts/ cp ifcfg-ens33 ifcfg-ens33:0 #If in tunnel mode, copy to ifcfg-tunl0 vim ifcfg-ens33:0 DEVICE=ens33:0 ONBOOT=yes IPADDR=192.168.121.100 NETMASK=255.255.255.255 ifup ens33:0 ifconfig ens33:0 route add -host 192.168.121.100 dev ens33:0
2. Adjust proc response parameters
Since the LVS load scheduler and all nodes need to share VIP addresses, the redirection parameter response of the Linux kernel should be turned off.
vim /etc/sysctl.conf net.ipv4.ip_forward = 1 net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.default.send_redirects = 0 net.ipv4.conf.ens33.send_redirects = 0 sysctl -p
3. Configure load distribution strategy
ipvsadm-save > /etc/sysconfig/ipvsadm systemctl start ipvsadm ipvsadm -C ipvsadm -A -t 192.168.121.100:80 -s rr ipvsadm -a -t 192.168.121.100:80 -r 192.168.121.33:80 -g #If tunnel mode, - g is replaced by - i ipvsadm -a -t 192.168.121.100:80 -r 192.168.121.44:80 -g ipvsadm ipvsadm -ln #Check the node status. Route represents DR mode
2. Deploy shared storage (NFS server: 192.168.121.55)
systemctl stop firewalld.service setenforce 0 yum -y install nfs-utils rpcbind mkdir /opt/kgc /opt/benet chmod 777 /opt/kgc /opt/benet
vim /etc/exports /opt/kgc 192.168.121.0/24(rw,sync) /opt/benet 192.168.121.0/24(rw,sync) exportfs -rv systemctl start nfs.service systemctl start rpcbind.service
3. Configure node servers (192.168.121.33, 192.168.121.44)
The configuration of 192.168.121.33 is the same as that of 192.168.121.44, and different parts are configured separately.
systemctl stop firewalld.service setenforce 0
1. Configure virtual IP address (VIP: 192.168.121.100)
cd /etc/sysconfig/network-scripts/ cp ifcfg-ens33 ifcfg-lo:0 vim ifcfg-lo:0 DEVICE=lo:0 ONBOOT=yes IPADDR=192.168.121.100 NETMASK=255.255.255.255 #Note: the subnet mask must be all 1 ifup lo:0 ifconfig lo:0 route add -host 192.168.121.100 dev lo:0 #Add a VIP local access route to limit the data accessing the VIP locally to avoid communication disorder
2. Adjust proc response parameters
vim /etc/sysctl.conf ...... net.ipv4.conf.lo.arp_ignore = 1 #The system only responds to ARP requests whose destination IP is local IP net.ipv4.conf.lo.arp_announce = 2 #The system does not use the source address of the IP packet to set the source address of the ARP request, but selects the IP address of the sending interface net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2 sysctl -p
yum -y install nfs-utils rpcbind httpd systemctl start rpcbind systemctl start httpd
–192.168.121.33—
mount 192.168.121.55:/opt/kgc /var/www/html echo 'Three thousand guests drunk with flowers in the hall' > /var/www/html/index.html
–192.168.121.44—
mount 192.168.121.55:/opt/benet /var/www/html echo 'A sword frosts fourteen states' > /var/www/html/index.html
4. Test LVS Cluster
Use browser access on the client http://192.168.121.100/ The default gateway points to 192.168.121.100
