Using OpenResty to build a simple file server

Posted by juline on Tue, 04 Jan 2022 00:03:02 +0100

preface

Use nginx + nginx a few days ago_ upload_ Module + python (callback handler) builds a simple file server. Many people on the Internet recommend using Lua to extend the functions of nginx, so they ponder how to use Lua language to expand the functions of nginx. After consulting many materials on the Internet, they find that the environment construction is still troublesome. LuaJIT needs to be installed, and nginx needs to compile NGX together_ devel_ Kit, Lua nginx module, and then found openresty.

I knew about openresty when I knew about the kong gateway before, but I didn't study it. This opportunity just let me know and learn.

About openresty

Official website,Official blog,Official forum

OpenResty ® Is based on Nginx With Lua's high-performance Web platform, it integrates a large number of sophisticated Lua libraries, third-party modules and most dependencies. It is used to easily build dynamic Web applications, Web services and dynamic gateways that can handle ultra-high concurrency and high scalability.

OpenResty ® By bringing together a variety of well-designed Nginx Module (mainly independently developed by OpenResty team), so as to Nginx Effectively become a powerful general Web application platform. In this way, Web developers and system engineers can use Lua scripting language Nginx Support various C and Lua modules to quickly construct a high-performance Web application system capable of 10K or even 1000K single machine concurrent connection.

OpenResty ® The goal is to make your Web services run directly in Nginx Internal service, make full use of Nginx The non blocking I/O model provides consistent high-performance responses not only to HTTP client requests, but also to remote backend such as MySQL, PostgreSQL, Memcached and Redis.

-- from the official website

Start building

Install OpenResty

The official document on the installation method of OpenResty has a very detailed description and is very comprehensive. It may be related to Zhang Yichun's character. It is very simple and fast.

  • Install directly using the installation tool (recommended)

    • Official course
    • centos example
    • # add the yum repo:
      wget https://openresty.org/package/centos/openresty.repo
      sudo mv openresty.repo /etc/yum.repos.d/ 
      # update the yum index:
      sudo yum check-update
      sudo yum install -y openresty

      I installed it using yum. The default installation location is / usr/local/openresty

    • [root@cloudfile openresty]# pwd
      /usr/local/openresty
      [root@cloudfile openresty]# ls
      bin  conf  COPYRIGHT  luajit  lualib  nginx  openssl  pcre  site  zlib

      Write Lua script for file processing

    • File location: / usr / local / openresty / nginx / conf / Lua / update lua

    • -- upload.lua
      --==========================================
      -- File upload
      --==========================================
      -- Import module
      local upload = require "resty.upload"
      local cjson = require "cjson"
      -- Define the structure of the reply
      -- Basic structure
      local function response(status,msg,data)
          local res = {}
          res["status"] = status
          res["msg"] = msg
          res["data"] = data
          local jsonData = cjson.encode(res)
          return jsonData
      end
      -- Default successful response
      local function success()
          return response(0,"success",nil)
      end
      -- Default with data response
      local function successWithData(data)
          return response(0,"success",data)
      end
      -- aborted response
      local function failed( msg )
          return response(-1,msg,nil)
      end
      -- end
      local chunk_size = 4096
      -- Get requested form
      local form, err = upload:new(chunk_size)
      if not form then
          ngx.log(ngx.ERR, "failed to new upload: ", err)
          ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) 
          ngx.say(failed("No files were obtained"))
      end
      form:set_timeout(1000)
      -- Define string split Split attribute
      string.split = function(s, p)
          local rt= {}
          string.gsub(s, '[^'..p..']+', function(w) table.insert(rt, w) end )
          return rt
      end
      -- Before and after defining support strings trim attribute
      string.trim = function(s)
          return (s:gsub("^%s*(.-)%s*$", "%1"))
      end
      -- The root path where the file is saved
      local saveRootPath = ngx.var.store_dir
      -- Saved file object
      local fileToSave
      -- Identifies whether the file was saved successfully
      local ret_save = false
      -- Actual received file name
      local rawFileName
      -- Actual saved file name
      local filename
      -- Start processing data
      while true do
          -- Read data
          local typ, res, err = form:read()
          if not typ then
              ngx.say(failed(err))
              return
          end
          -- Start reading http header
          if typ == "header" then
              -- Resolve the file name uploaded this time
              local key = res[1]
              local value = res[2]
              if key == "Content-Disposition" then
                  -- Resolve the file name uploaded this time
                  -- form-data; name="keyName"; filename="xxx.xx"
                  local kvlist = string.split(value, ';')
                  for _, kv in ipairs(kvlist) do
                      local seg = string.trim(kv)
                      if seg:find("filename") then
                          local kvfile = string.split(seg, "=")
                          -- Actual file name
                          rawFileName = string.sub(kvfile[2], 2, -2)
                          -- Rename file name
                          filename = os.time().."-"..rawFileName
                          if filename then
                              -- open(establish)file
                              fileToSave = io.open(saveRootPath .."/" .. filename, "w+")
                              if not fileToSave then
                                  -- ngx.say("failed to open file ", filename)
                                  ngx.say(failed("fail to open file"))
                                  return
                              end
                              break
                          end
                      end
                  end
              end
          elseif typ == "body" then
              -- Start reading http body
              if fileToSave then
                  -- Write file contents
                  fileToSave:write(res)
              end
          elseif typ == "part_end" then
              -- File write finished, close the file
              if fileToSave then
                  fileToSave:close()
                  fileToSave = nil
              end
               
              ret_save = true
              -- End of file reading
          elseif typ == "eof" then 
              break
          else
              ngx.log(ngx.INFO, "do other things")
          end
      end
      if ret_save then
          local uploadData = {}
          uploadData["file"] = rawFileName
          uploadData["url"] = "http://xxx.com/download/"..filename
          ngx.say(successWithData(uploadData))
      else
          ngx.say(failed("System exception"))
      end
      

Writing nginx configuration files

File location: / usr / local / openresty / nginx / conf / nginx conf

user root;
worker_processes  20;
error_log  logs/error.log notice;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    server {
        listen       80;
        server_name  localhost;
        # Maximum file size allowed to upload
        client_max_body_size 300m; 
        set $store_dir "/var/www/download"; # File storage path
        # File upload interface: http://xxx.com/uploadfile
        location /uploadfile {
            # Realize the logic of file upload
          content_by_lua_file conf/lua/update.lua; 
        }
        # File download portal: http://xxx.com/download
        location /download {
            alias /var/www/download;
            autoindex on;
            autoindex_localtime on; 
        }
        # redirect server error pages to the static page /50x.html
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

 

Start test

The way to start nginx here is the same as that of ordinary nginx:

  • Start: / usr/local/openresty/nginx/sbin/nginx
  • Close: / usr/local/openresty/nginx/sbin/nginx -s stop
  • Reload configuration: / usr/local/openresty/nginx/sbin/nginx -s reload

Upload file

POST /uploadfile

Download File

Just access GET /download

 

 

 

 

Topics: Operation & Maintenance Nginx server lua