Nginx + Tomcat load balancing cluster

Posted by veveu on Fri, 15 Oct 2021 19:30:31 +0200


Generally, a Tomcat site can not be used in the production environment alone because it may have a single point of failure and can not cope with the complex and diverse requests of too many customers. Therefore, a more reliable solution is needed to improve the Web site architecture.

1, Case overview

  • Nginx is a very excellent http server software. It can support the response of up to 50000 concurrent connections, has strong static resource processing capacity, runs very stably, and consumes very low system resources such as memory and CPU.

  • At present, many large websites use Nginx server as the reverse proxy and load balancer of back-end website programs to improve the load concurrency of the whole site

  • The case uses Nginx as the load balancer and Tomcat as the load cluster setting method of the application server. The architecture diagram is as follows

2, Environment deployment

  • The case environment is as follows
hostoperating systemIP addressMain software
Nginx serverCentOS 7.4 x86_64192.168.8.140nginx-1.12.2.tar.gz
Tomcat server 1CentOS 7.4 x86_64192.168.8.133①apache-tomcat-9.0.16.tar.gz / ②jdk-8u201-linux-x64.rpm
Tomcat server 2CentOS 7.4 x86_64192.168.8.134①apache-tomcat-9.0.16.tar.gz / ② jdk-8u201-linux-x64.rpm
  • Turn off the firewall and turn off the automatic startup
[root@ng133 ~]#systemctl stop firewalld.service            #Turn off firewall and start-up function
[root@ng133 ~]#systemctl status firewalld.service
[root@ng133 ~]#setenforce 0								#Turn off the security enhancement system
[root@ng133 ~]#setenforce: SELinux is disabled

3, Nginx host installation

  • To install the Nginx service, use the one click deployment script here
iptables -F
yum -y install epel-release && yum clean all && yum makecache
yum -y install pcre-devel zlib-devel gcc gcc-c++ make wget
useradd -M -s /sbin/nologin nginx
wget -P /opt
tar zxvf /opt/nginx-1.12.2.tar.gz -C /opt
cd /opt/nginx-1.12.2

./configure \
--prefix=/usr/local/nginx \
--user=nginx \
--group=nginx \

cd /opt/nginx-1.12.2
make -j 4 && make install
ln -s /usr/local/nginx/sbin/nginx /usr/local/sbin/

cat > /usr/lib/systemd/system/nginx.service <<EOF
PIDFile =/usr/local/nginx/logs/
ExecrReload=/bin/kill -s HUP $MAINPID
ExecrStop=/bin/kill -s QUIT $MAINPID

chmod 754 /usr/lib/systemd/system/nginx.service
systemctl daemon-reload && systemctl start nginx.service && systemctl enable nginx.service

echo " "
pgrep "nginx" &> /dev/null
if [ $? -eq 0 ];then
        echo -e "\033[32mnginx The service is running normally and can be curl see\033[0m"
        echo -e "\033[31mnginx The service is running abnormally. Please check\033[0m"
  • View results after installation
[root@ng140 /opt/nginx-1.12.2]#curl -I
HTTP/1.1 200 OK
Server: nginx/1.12.2
Date: Fri, 15 Oct 2021 01:46:17 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Fri, 15 Oct 2021 01:45:34 GMT
Connection: keep-alive
ETag: "6168dd3e-264"
Accept-Ranges: bytes

4, Tomcat installation and configuration

1. Install Tomcat

The configuration methods of Tomcat server 1 and Tomcat server 2 are basically the same

  • Install JDK and configure environment
[root@tm1133 ~]#cd /opt/									   #Upload the installation package to the / opt directory
[root@tm1133 /opt]#ls
apache-tomcat-9.0.16.tar.gz  jdk-8u201-linux-x64.rpm  
[root@tm1133 /opt]#rpm -ivh jdk-8u201-linux-x64.rpm			   #install
[root@tomcat /opt]#vim /etc/profile.d/				   #/etc/profile.d / environment variable script directory	

export JAVA_HOME=/usr/java/jdk1.8.0_201-amd64
export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar		
export PATH=$JAVA_HOME/bin:$PATH

[root@tm1133 /opt]#source /etc/profile.d/ 			   #Import the script into the environment variable for it to take effect
[root@tm1133 /opt]#java -version							   #View version
java version "1.8.0_201"
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
  • Unpack
[root@tm1133 /opt]#tar zxvf apache-tomcat-9.0.16.tar.gz  	    #Unpack
[root@tm1133 /opt]#mv apache-tomcat-9.0.16 /usr/local/tomcat	#Transfer package location and rename
  • Start and optimize the management of Tomcat (create a soft connection and optimize the start command)
[root@tm1133 /opt]#ln -s /usr/local/tomcat/bin/ /usr/local/bin/
[root@tm1133 /opt]#ln -s /usr/local/tomcat/bin/ /usr/local/bin/
[root@tm1133 /opt] 								   #open
Using CATALINA_BASE:   /usr/local/tomcat
Using CATALINA_HOME:   /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME:        /usr/java/jdk1.8.0_201-amd64
Using CLASSPATH:       /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Tomcat started.
[root@tm1133 /opt]#netstat -antp |grep 8080					  #Check whether it is opened successfully
tcp6       0      0 :::8080                 :::*                    LISTEN      2520/java  

2. Tomcat server 1 configuration

[root@tm1133 ~]#mkdir /usr/local/tomcat/webapps/test          			 #Create a test directory
[root@tm1133 ~]#vim /usr/local/tomcat/webapps/test/index.jsp

<%@ page language="java" import="java.util.*" pageEncoding="UTF-8"%>     #Configuration of dynamic pages
<title>JSP test1 page </title>
<% out.println("Dynamic page 1,");%>

#Add the following content at line 151														
[root@tm1133 ~]#vim /usr/local/tomcat/conf/server.xml 					#Add virtual host configuration
   <Context docBase="/usr/local/tomcat/webapps/test" path="" reloadable="true" />

[root@tm1133 ~] 									
[root@tm1133 ~] 												#Restart service

3. Tomcat server 2 configuration

[root@tm2134 /opt]#mkdir /usr/local/tomcat/webapps/test					#Create a test directory
[root@tm2134 /opt]#vim /usr/local/tomcat/webapps/test/index.jsp			#Configuration of dynamic pages
<%@ page language="java" import="java.util.*" pageEncoding="UTF-8"%>
<title>JSP test2 page </title>
<% out.println("Dynamic page 2,");%>

#Edit the tomcat main configuration file and add the following classes at line 151
[root@tm2134 /opt]#vim /usr/local/tomcat/conf/server.xml			   #Add virtual host configuration
<Context docBase="/usr/local/tomcat/webapps/test" path=" " reloadable="true" />

[root@tm2134 /opt]		
[root@tm2134 /opt]			 								#Restart service

5, Nginx server configuration

  • Static page configuration
[root@ng140 ~]#echo '<html><body><h1>this is static</h1></body></html>' > /usr/local/nginx/html/index.html
[root@ng140 ~]#cat /usr/local/nginx/html/index.html 
<html><body><h1>this is static</h1></body></html>

[root@ng140 /usr/local/nginx/html]#mkdir /usr/local/nginx/html/picture		#Upload pictures
[root@ng140 /usr/local/nginx/html]#cd picture/
[root@ng140 /usr/local/nginx/html/picture]#rz -E
rz waiting to receive.
[root@ng140 /usr/local/nginx/html/picture]#ls
#Add the following configuration parameters under line 57 of the main configuration file to enable the page to load pictures
[root@ng140 /usr/local/nginx/html/picture]#vim /usr/local/nginx/conf/nginx.conf
location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|css)$ {
root /usr/local/nginx/html/picture;
expires 10d;
[root@ng140 /usr/local/nginx/html/picture]#nginx -t
[root@ng140 /usr/local/nginx/html/picture]#systemctl restart nginx.service 
  • Configure nginx master profile
[root@ng140 ~]#vim /usr/local/nginx/conf/nginx.conf
#Configure the list of servers for load balancing. The weight parameter indicates the weight. The higher the weight, the greater the probability of being assigned

#gzip  on;													#Add the following after line 33
upstream tomcat_server {
server weight=1;
server weight=1;

															#Add the following configuration parameters under line 45
 45			#access_log  logs/host.access.log  main;

 46         location ~ .*\.jsp$ {
 47         proxy_pass http://tomcat_server; 
 48         proxy_set_header HOST $host;
 49         proxy_set_header X-Real-IP $remote_addr;                
 50         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 51        }            
  • Add parameter resolution
location ~ .*\.jsp$ {			
#Assign the ip address of the client received by nginx to the source ip in the jump to tomcat request, identify the real ip of the client, and assign and jump
proxy_pass http://tomcat_server; 

proxy_set_header HOST $host;	
#Set the host name (domain name or ip, port) of the request received by the back-end web server. The default host value is proxy_ Host name of pass direct connection settings

proxy_set_header X-Real-IP $remote_addr;		
#Put $remote_addr is copied to x-real-ip (customized) and goes back and forth to the source IP

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;		
#When nginx is the most proxy server, the IP list set will record the passing IP, proxy and IP
  • Check the main configuration file syntax and restart the service
[root@ng140 ~]#nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
[root@ng140 ~]#systemctl restart nginx.service

6, Verification results

  • Test the effect of static pages with Firefox browser

  • Test whether load balancing is effective. When refreshing the web page, you can see switching back and forth between dynamic pages 1 and 2


  • Two or more Tomcat servers can be put into the upstream of Nginx to form a load balancing cluster, and then through proxy_pass is a Web proxy. Set the cluster site in location, and then set the weight of Tomcat server through the weight value.
  • In the production environment, the hardware configuration of Tomcat server may be different. You can modify the weight value of the corresponding server to allocate and control the access requests of servers with higher or lower configuration.

Topics: Web Server Operation & Maintenance Load Balance Nginx Tomcat