nginx load balancing -- principle of keepalived and implementation of dual vip

Posted by djopie on Tue, 04 Jan 2022 20:19:38 +0100

1, What is kept alive?

keepalived is a routing software written in C. The main goal of the project is to provide simple and robust load balancing and high availability facilities for Linux system and Linux infrastructure. The load balancing framework relies on the well-known and widely used Linux virtual server (IPVS) kernel module to provide layer 4 load balancing. Keepalive implements a set of checkers to dynamically and adaptively maintain and manage the load balancing server pool according to its health status.

On the other hand, VRRP protocol achieves high availability. VRRP is the basic brick of router failover. In addition, Keepalived also implements a set of hooks to VRRP finite state machines to provide low-level and high-speed protocol interaction. In order to provide the fastest network fault detection, Keepalive implements BFD protocol. BFD prompt can be considered for VRRP state transition to promote fast state transition. The retention framework can be used independently or all together to provide a resilient infrastructure.

Load balancing and high availability are two concepts

Load balancing: distribute many requests to many servers at the back end -- a software to relieve pressure
High availability: two or more servers (people) do the same thing and back up each other.

load balancing

The load balancing function of kept is realized by lvs software
The load balancing framework relies on the well-known and widely used Linux virtual server (IPVS) kernel module that provides layer 4 load balancing.

2, VRRP protocol

1.vrrp protocol format

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |Version| Type  | Virtual Rtr ID|   Priority    | Count IP Addrs|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Auth Type   |   Adver Int   |          Checksum             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         IP Address (1)                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                            .                                  |
   |                            .                                  |
   |                            .                                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         IP Address (n)                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Authentication Data (1)                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Authentication Data (2)                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

2. Related terms

Virtual Router: it is composed of one Master router and multiple Backup routers. The host uses the virtual router as the default gateway.
VRID: the identification of the virtual router. A group of routers with the same VRID constitute a virtual router.
Master router: the router that undertakes the task of message forwarding in the virtual router.
Backup router: a router that can replace the Master router when the Master router fails.
Virtual IP address: the IP address of the virtual router. A virtual router can have one or more IP addresses.
IP address owner: a router whose interface IP address is the same as the virtual IP address is called an IP address owner.
Virtual MAC address: a virtual router has a virtual MAC address. The format of the virtual MAC address is 00-00-5E-00-01-{VRID}. Usually, the virtual router responds to ARP requests using the virtual MAC address. Only when the virtual router is specially configured, it responds to the real MAC address of the interface.
Priority: VRRP determines the status of each router in the virtual router according to the priority.
Non preemptive mode: if the Backup router works in non preemptive mode, as long as the Master router does not fail, the Backup router will not become the Master router even if it is configured with a higher priority.
Preemption mode: if the Backup router works in preemption mode, it will compare its priority with the priority in the notification message after receiving the VRRP message. If your priority is higher than that of the current Master router, you will take the initiative to seize the Master router; Otherwise, the Backup state will be maintained.

3. Working principle of VRRP protocol

What is the VRRP protocol?
VRRP protocol is designed to eliminate the network failure caused by the single point failure of the default router in the static default routing environment, so that the internal and external data communication can not be affected and the network parameters of the internal network do not need to be modified when the device function switching occurs. VRRP protocol needs to have the functions of IP address backup, priority routing and reducing unnecessary communication between routers.

Working principle of VRRP protocol
VRRP protocol virtualizes two or more router devices into one device and provides virtual router IP (one or more) externally. Within the router group, if the router actually owns the external IP works normally, it is the MASTER, or it is generated through algorithm selection. The MASTER realizes various network functions for virtual router IP, such as ARP request, ICMP, And data forwarding; Other devices do not own the IP and the status is BACKUP. They do not perform external network functions except receiving VRRP status notification information from MASTER. When the host fails, BACKUP will take over the network function of the original MASTER.

When configuring VRRP protocol, you need to configure the virtual router ID(VRID) and priority weight of each router. The routers are grouped by VRID. The routers with the same VRID value are the same group, and the VRID is a positive integer from 0 to 255; Routers in the same group elect MASTER by using priority weight. The higher priority is MASTER, and the priority is also a positive integer from 0 to 255.

VRRP notice
VRRP protocol uses multicast data to transmit VRRP data. VRRP data uses a special virtual source MAC address to send data instead of the MAC address of its own network card. During VRRP operation, only the MASTER router sends VRRP notification information regularly, indicating that the MASTER works normally and the virtual router IP (Group). BACKUP only receives VRRP data and does not send data, If the notification information of the MASTER is not received within a certain period of time, each BACKUP will declare itself to be the MASTER, send the notification information and resume the MASTER election status.

VRRP advertisement is encapsulated by IP multicast packets. The source address is the local address and the destination address is 224.0 0.18, only routers running VRRP protocol will accept multicast addresses.

4.master election

4.1 priority of routers in backup group

VRRP determines the role of each router in the Backup group (Master router or Backup router) according to the priority. The higher the priority, the more likely it is to become a Master router.

When two routers with the same priority compete for the Master at the same time, compare the size of the interface IP address. The greater the interface address, the Master is selected.

The value range of VRRP priority is 0 to 255 (the higher the value, the higher the priority). The configurable range is 1 to 254. Priority 0 is reserved for the router to actively abandon the Master location, and 255 is reserved for the IP address owner. If the IP address of the VRRP router is the same as the interface IP address of the virtual router, the VRRP router is called the owner of the IP address; the IP address owner automatically Has the highest priority: 255. Therefore, when there is an IP address owner in the backup group, it is the Master router as long as it works normally.

4.2 working mode of router in backup group

Non preemptive mode: if the router in the Backup group works in non preemptive mode, the Backup router will not become the Master router even if it is configured with a higher priority as long as the Master router does not fail.

Preemption mode: if the router in the Backup group works in preemption mode, it will send VRRP notification message once it finds that its priority is higher than that of the current Master router. As a result, the routers in the Backup group re elect the Master router and eventually replace the original Master router. Accordingly, the original Master router will become a Backup router. After receiving the VRRP notification message, the Backup router will only compare its priority with the priority in the notification message, and will not compare the IP address. Only when its priority is greater than the priority in the notification message will it preempt and become the Master.

The initially created router works in the Backup state, and the priority of other members in the virtual router is known through the interaction of VRRP messages:

If the priority of the Master router in the VRRP message is higher than its own priority, the router remains in the Backup state;

If the priority of the Master router in the VRRP message is lower than its own priority, the router in the preemptive mode will preempt into the Master state, send VRRP messages periodically, and the router in the non preemptive mode will still maintain the Backup state;

If the VRRP message is not received within a certain period of time, the router switches to the Master state.

3, vip drift

vip - virtual ip
The vip will drift. When the master hangs up, the vip will automatically drift to the backup. After the master is suspended and restarted, the priority of the master is higher, and the vip will drift back to the master to achieve high availability.

How do I know if the master is dead
```
 zabbix monitor
```

When the load balancer returns data to users, does it use vip or its own real ip?

 External use is vip，This machine is used internally to achieve high availability IP

Does the client know when vip drifts?

 There are no changes on the browser, arp The cache table changes. vip of mac Address is master Server mac Address, when vip When drifting, it will turn into current master Server mac address

4, Cerebral fissure phenomenon

If there are vip addresses on two or more load balancers, brain crack will occur

What reason can appear brain crack

The firewall blocks the notification of kept vrrp messages
virtual_ router_ When the ID is different. The priority and password are different, and there will be no brain crack.

The brain crack phenomenon does not affect the user's access or the back-end server, but can play a role in load balancing
The user will select a vip machine. The user's choice and operation and maintenance are uncontrollable, which leaves a potential safety hazard. > The load balancer accessed by the user is beyond the control of the operation and maintenance personnel, which is unacceptable as a manager.
Users of the external network will always access one of the load balancing servers, and the other will not. However, this decision is not made by the operation and maintenance personnel, but by the router at the exit. In this way, the master and backup selected in our internal network have no meaning

If it is only a single vip, the master will work and the backup server will be idle. How to make backup not idle? What methods do you have?

Deliberately create a brain crack
dns domain name resolution to achieve double vip

5, Double vip experiment

1. Install nginx

Install nginx on both load balancers
The author has compiled and installed nginx in LB1. Just transfer the installation script to LB2 to install nginx

[root@LB1 ~]# scp onekey_install_shediao_v12.sh  192.168.0.111:/root
The authenticity of host '192.168.0.111 (192.168.0.111)' can't be established.
ECDSA key fingerprint is SHA256:iVMGjapIaq/B9GUPAueatX3CTeUTM+tM3QlHqYYFTy0.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.0.111' (ECDSA) to the list of known hosts.
root@192.168.0.111's password: 
onekey_install_shediao_v12.sh 

[root@LB1 ~]bash onekey_install_shediao_v12.sh  #Execute one click installation script

2. Configure nginx load balancing function to copy directly from LB1 to LB2

[root@LB conf]# mv nginx.conf nginx.conf.bak  #Back up the original nginx Conf file
[root@LB conf]# scp nginx.conf 192.168.0.111:/usr/local/scxx99/conf
root@192.168.0.111's password: 
nginx.conf

3. Start nginx service

Both load balancers need to start the nginx service

[root@LB1 ~]# systemctl start nginx
[root@LB1 ~]# ps aux | grep nginx  #Check whether nginx is started

4. Install the keepalived file

keepalived needs to be installed on both load balancers

[root@LB1 ~]# yum install keepalived -y

5. Configure keepalived Conf file, add vip and related information

 [root@LB1 ~]# vim /etc/keepalived/keepalived.conf



vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 105
    priority 120
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 11112222
    }
    virtual_ipaddress {
        192.168.0.198
    }
}
vrrp_instance VI_2 {
    state BACKUP
    interface ens33
    virtual_router_id 110
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 11112222
    }
    virtual_ipaddress {
        192.168.0.199
    }
}

[root@LB2 ~]# vim /etc/keepalived/keepalived.conf



vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 105
    priority 120
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 11112222
    }
    virtual_ipaddress {
        192.168.0.198
    }
}
vrrp_instance VI_2 {
    state MASTER
    interface ens33
    virtual_router_id 110
    priority 180
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 11112222
    }
    virtual_ipaddress {
        192.168.0.199
    }

6. Restart keepalived

Restart keepalived on both load balancers

[root@LB1 ~]#systemctl restart keepalived

7. Check whether the ip verification is successful

LB1

[root@LB1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:31:00:af brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.112/24 brd 192.168.0.255 scope global dynamic noprefixroute ens33
       valid_lft 5501sec preferred_lft 5501sec
    inet 192.168.0.198/32 scope global ens33
       valid_lft forever preferred_lft forever
[root@LB1 ~]#

LB2

[root@LB2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:bb:a0:b0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.111/24 brd 192.168.0.255 scope global dynamic noprefixroute ens33
       valid_lft 5379sec preferred_lft 5379sec
    inet 192.168.0.199/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::9fea:369d:550e:e3c6/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
[root@LB2 ~]#

Programmer Think