An article from principle to practice teaches you to use Nginx_lua implements WAF

Posted by mdinowitz on Wed, 16 Feb 2022 03:46:51 +0100

Part1Nginx_lua implements waf

Background of WAF

In the past, enterprises usually used firewalls as the first line of defense for security; At that time, the firewall only effectively blocked some data packets in the third layer (network layer); As the functions of web applications become more and more abundant, web server has become the main target of attack (layer 7 application layer) because of its powerful computing power, processing performance and high value. However, the traditional firewall has no way to prevent attacks using application vulnerabilities; In this context, WAF(Web Application Firewall) came into being.

2 what is WAF

WAF Web application firewall is designed to protect Web applications from various application layer attacks, such as cross site scripting (XSS), SQL injection, cookie poisoning and other applications. It is the gateway to your important data, Therefore, attacks against applications have become the main cause of vulnerabilities. With WAF, we can intercept a series of attacks that attempt to leak data through intrusion system.

3 working principle

The user sends a Web page request to the Web server through the browser
Before the user's request reaches the Web server, WAF filters the user's request
WAF gets the user's HTTP request parameters and compares them with the rules defined in the configuration file. If they match, it returns 403 reject, otherwise it is released
The WEB server responds to the user's request and returns the page data to the user.

4WAF action

waf is a product that protects Web applications by implementing a series of security policies for HTTP/HTTPS

Difference between 5WAF and traditional firewall

Traditional Firewalls Work in the network layer (layer 3) and transport layer (layer 4)
WAF works in the application layer (layer 7)
Traditional firewalls filter more IP and ports
WAF is used to filter HTTP requests, including URL, IP, user agent and so on.

6WAF and DDos

The full name of DDos is Distributed Denial of service. It mainly relies on a group of computers to initiate requests to a single target system, resulting in the target system running out of resources and rejecting normal requests

According to the OSI network model, there are three types of DDos: layer 3 (network layer) DDos, layer 4 (transport layer) DDos and layer 7 (application layer) DDos

WAF mainly deals with layer 7 DDos attacks. It will be more efficient than other protection methods when dealing with layer 7 DDos attacks. WAF will make a detailed analysis of HTTP traffic, so that WAF can model normal access requests, and then use these models to distinguish between normal requests and requests triggered by attackers using robots or scripts.

7Nginx WAF function

Support IP whitelist and blacklist functions, and directly deny IP access to the blacklist (the new cdip function supports IP segment)
Support URL whitelist and define URLs that do not need to be filtered
Support user agent filtering, match entries in custom rules, and then process them
CC attack protection is supported. The number of visits of a single URL at the specified time exceeds the set value (new for different domain names)
Support Cookie filtering, match entries in custom rules, and then process them
Support URL filtering to match entries in custom rules. If the URL requested by the user contains these entries
Support URL parameter filtering. The principle is the same as above
Support logging, and record all rejected operations in the log
New support for black cache (600 seconds by default)

8Nginx Waf protection process

if whiteip() then
elseif blockip() then
elseif denycc() then
elseif ngx.var.http_Acunetix_Aspect then
    ngx.exit(444)
elseif ngx.var.http_X_Scan_Memo then
    ngx.exit(444)
elseif whiteurl() then
elseif ua() then
elseif url() then
elseif args() then
elseif cookie() then
elseif PostCheck then

Check the IP white list, and if it passes, it will not be detected;
Check the IP blacklist and reject it if it fails;
Check CC attack and reject if it matches
Check http_ Acunetix_ Whether aspect scanning is on
Check http_ X_ Scan_ Is memo scan on
Check whitelist URL;
Check UA and reject if UA fails;
Check URL parameters;
Check cookie s;
Check post;

9 implementation of WAF based on Nginx

Install dependent packages

yum -y install gcc gcc-c++ autoconf automake make unzip
yum -y install zlib zlib-devel openssl openssl-devel pcre pcre-devel

Install luajit2 0

LuaJIT is Lua's real-time compiler. In short, LuaJIT is an efficient Lua virtual machine

# Enter directory
cd /usr/local/src

# Download luajit2 one
wget https://github.com/openresty/luajit2/archive/refs/heads/v2.1-agentzh.zip

# decompression
unzip v2.1-agentzh.zip && cd luajit2-2.1-agentzh

# compile
make -j 3

# install
make install PREFIX=/usr/local/lj2

# Establish soft connection
ln -sf /usr/local/lj2/lib/libluajit-5.1.so.2 /lib64/libluajit-5.1.so.2

# Add environment variable
export LUAJIT_LIB=/usr/local/lj2/lib
export LUAJIT_INC=/usr/local/lj2/include/luajit-2.1

Install ngx_devel_kit

kit module is a module to expand the core functions of nginx server. The third-party module development can be realized quickly based on it

# Enter directory
cd /usr/local/src

# download
wget https://github.com/vision5/ngx_devel_kit/archive/refs/tags/v0.3.1.tar.gz -O ngx_devel_kit.tar.gz

# decompression
tar zxxf ngx_devel_kit.tar.gz

Install Lua nginx module

ngx_lua_module is an nginx http module, which embeds the Lua parser into nginx to parse and execute the web page background script written in Lua language.

ngx_ Principle of lua module

Each worker (work process) creates a Lua VM, and all processes in the worker share the VM;
Encapsulate the Nginx I/O primitive and inject it into Lua VM to allow direct access to Lua code;
Each external request is processed by a Lua cooperation process, and the data is isolated between the cooperation processes;
When Lua code calls asynchronous interfaces such as I/O operations, it will suspend the current collaboration (and protect the context data) without blocking the worker;
When I/O and other asynchronous operations are completed, restore the relevant process context data and continue to run.

# Enter directory
cd /usr/local/src

# download
wget https://github.com/openresty/lua-nginx-module/archive/refs/tags/v0.10.14.zip

# decompression
unzip v0.10.14.zip

Installing nginx

# Enter directory
cd /usr/local/src

# download
wget http://nginx.org/download/nginx-1.21.0.tar.gz

# decompression
tar zxvf nginx-1.21.0.tar.gz

# Add user
groupadd nginx
useradd -g nginx nginx -s /sbin/nologin

# compile
./configure --prefix=/usr/local/nginx \
    --with-http_ssl_module \
    --with-http_flv_module \
    --with-http_stub_status_module \
    --with-http_gzip_static_module \
    --with-http_realip_module \
    --with-pcre \
    --add-module=/usr/local/src/lua-nginx-module-0.10.14 \
    --add-module=/usr/local/src/ngx_devel_kit-0.3.1 \
    --with-stream \
    --user=nginx \
    --group=nginx \
    --pid-path=/var/run/nginx.pid \
    --lock-path=/var/run/nginx.lock \
    --error-log-path=/var/log/nginx/error.log \
    --http-log-path=/var/log/nginx/access.log \
    --with-pcre \
    --with-file-aio \
    --without-http_scgi_module \
    --without-http_uwsgi_module \
    --without-http_fastcgi_module

# install
make -j 3 && make install

# Modify configuration (add configuration to server block)
vim /usr/local/nginx/conf/nginx.conf
location /lua {
    default_type 'text/plain';
 
    content_by_lua 'ngx.say("hello, lua")';
}

# Configure boot
cat > /usr/lib/systemd/system/nginx.service <<- 'EOF'
[Unit]
Description=The nginx HTTP and reverse proxy server
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
PIDFile=/var/run/nginx.pid
# Nginx will fail to start if /run/nginx.pid already exists but has the wrong
# SELinux context. This might happen when running `nginx -t` from the cmdline.
# https://bugzilla.redhat.com/show_bug.cgi?id=1268621
ExecStartPre=/usr/bin/rm -f /var/run/nginx.pid
ExecStartPre=/usr/local/nginx/sbin/nginx -t
ExecStart=/usr/local/nginx/sbin/nginx
ExecReload=/bin/kill -s HUP $MAINPID
KillSignal=SIGQUIT
TimeoutStopSec=5
KillMode=process
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF
# start nginx 
systemctl start nginx

Installing nginx_lua_waf

# Enter directory
cd /usr/local/src

# download
wget https://github.com/loveshell/ngx_lua_waf/archive/refs/heads/master.zip

# decompression
unzip master.zip -d /usr/local/nginx/conf

# Change directory name
mv /usr/local/nginx/conf/ngx_lua_waf-master /usr/local/nginx/conf/waf

# Create attack log directory
mkdir -pv /usr/local/nginx/logs/hack
chown nginx /usr/local/nginx/logs/hack

# nginx http segment add configuration
lua_package_path "/usr/local/nginx/conf/waf/?.lua";
lua_shared_dict limit 10m;
init_by_lua_file  /usr/local/nginx/conf/waf/init.lua;
access_by_lua_file /usr/local/nginx/conf/waf/waf.lua;

# View waf configuration
cat /usr/local/nginx/conf/waf/config.lua
# Rule storage path
RulePath = "/usr/local/nginx/conf/waf/wafconf/"
# logdir needs to be configured to enable attack information recording
attacklog = "on"
# log storage directory, which needs to be created by the user, and requires the writable permission of nginx user
logdir = "/usr/local/nginx/logs/hack/"
# Block url access
UrlDeny="on"
# Redirect after blocking
Redirect="on"
# Block cookie attacks
CookieMatch="on"
# Whether to intercept post attack
postMatch="on"
# Open URL whitelist
whiteModule="on"
# Fill in the suffix type of files that are not allowed to be uploaded
black_fileExt={"php","jsp"}
# ip whitelist. Multiple IPS are separated by commas
ipWhitelist={"127.0.0.1"}
# ip blacklist. Multiple IPS are separated by commas
ipBlocklist={"192.168.10.1"}
# Whether to start intercepting cc attacks (add lua_shared_dict limit 10m to the http section of nginx.conf)
CCDeny="off"
# Set cc attack frequency in seconds
# By default, the same IP can only request the same address 100 times in one minute
CCrate="100/60"
# Alarm content
html= [[Please go away~~]]

# Reload nginx
systemctl reload nginx

Test waf firewall

curl -v 10.155.14.125/?id=<script

10nginx combined with lua to realize the interface

# Configuration interface
cat > /usr/local/nginx/conf/conf.d/luatest.conf <<- 'EOF'
server {
    listen       80;
    server_name  _;
    access_log /usr/local/nginx/logs/luatest.log;
    error_log /usr/local/nginx/logs/luatest_error.log;

    location /luatest {
        default_type "text/html";
        content_by_lua_file /usr/local/nginx/lua/luatest.lua;

    }

    location /luamysql {
        default_type "text/html";
        content_by_lua_file /usr/local/nginx/lua/luamysql.lua;

    }

}
EOF
# Write lua script
cat > /usr/local/nginx/lua/luatest.lua <<- 'EOF'
local request_uri = ngx.var.request_uri;
local args = ngx.req.get_uri_args()
ngx.say("request_uri: ", request_uri);
ngx.say("decode request_uri: ",ngx.unescape_uri(request_uri))
ngx.say("ngx.md5 : ", ngx.md5("123"))
ngx.say("ngx.http_time : ", ngx.http_time(ngx.time()))
ngx.say("ngx.id: ",args.id)
EOF

cat > /usr/local/nginx/lua/luamysql.lua <<- 'EOF'
a = 5
local b =5
function joke()
    c = 5
    local d = 6
end
joke()
ngx.say(c,d)
EOF

Programmer Think