Project start

In August 2021, the idle teacher found some old friends and joined the whole department in a new job. The work content is relatively easy. The idle teacher is the front-end identity in the team.
In January 2022, one month before the new year, the idle teacher basically completed the work of the whole year and was relatively idle. At the same time, the company bought a new server. The idle teacher spent time building a micro cluster system on the new server to establish an architectural foundation for the future system expansion.

Construction process

1. Install Docker Foundation

Docker is an open source application container engine, which allows developers to package their applications and dependent packages into a portable container, and then publish them to any popular Linux or Windows operating system machine. It can also realize virtualization. The container completely uses the sandbox mechanism, and there will be no interface between them.

Use the official instructions to install docker. The general CentOS system executes the following commands

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install docker-ce docker-ce-cli containerd.io

Note: it is best to set the docker command to have no sudo prefix for ease of use
First, establish the docker user group sudo groupadd docker
Add the current user to the docker user group sudo usermod -aG docker current user
Finally, relink the server

Portal: Installation documentation,Set no sudo operation

2. Initialize Swarm cluster

Swarm is a cluster management tool officially provided by Docker. Its main function is to abstract several Docker hosts into a whole and uniformly manage various Docker resources on these Docker hosts through one portal.

Execute the initialization command: docker swarm init -- advertisement addr = broadcast IP address
After execution, return the token required to join the cluster in the work node mode: docker swarm join token worker
After execution, return the token required to join the cluster in the management node mode: docker swarm join token manager
After execution, join the node to the existing cluster: docker swarm join --token your TOKEN management node broadcasts the IP address

Prepare unified network: docker network create --driver overlay app
Prepare working directory: MKDIR / data & CD / data

Portal: Cluster document

3. Deploy the portal service

Container is a GUI project used to manage containers

Download the Chinese package curl - SL in advance https://github.com/eysp/public/archive/public.tar.gz | tar xz
Create a new container stack file VIM container agent stack yml

version: '3.7'
services:
  agent:
    image: portainer/agent:2.11.0
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock # docker pipeline agent
      - /var/lib/docker/volumes:/var/lib/docker/volumes # docker data broker
    networks:
      - app
    deploy:
      mode: global
      placement:
        constraints: [node.platform.os == linux] # Deploy to linux environment only
  portainer:
    image: portainer/portainer-ce:2.11.0
    command: -H tcp://tasks.agent:9001 --tlsskipverify
    ports:
      - "8080:9000" # Map page port to 8080
    volumes:
      - /data/public:/public # Mapping Chinese folders
      - portainer:/data
    networks:
      - app
    deploy:
      mode: replicated
      replicas: 1 # Deploy only one
      placement:
        constraints: [node.role == manager] # Deploy to management node only
networks:
  app:
    external: true # Use the created network
volumes:
  portainer: # Use a data folder named portal

Deploy service stack docker stack deploy - C portal-agent-stack yml portainer
Open the portal management interface http: / / cluster management server ip:8080
Create a new administrator account and password. For security, please pay attention to the complexity of the account password

Portal: stack,Portainer,Sinicization

4. Deploy Kong gateway stack

Kong is a highly available and extensible API Gateway project based on OpenResty (Nginx + Lua module), which is open source by Mashape company. Kong is built based on NGINX and Apache Cassandra or PostgreSQL. It can provide easy-to-use RESTful API to operate and configure the API management system. Therefore, it can horizontally expand multiple Kong servers and evenly distribute requests to each Server through pre load balancing configuration to deal with a large number of network requests.

Open the container address, select the management node, enter the storage volume management, and create a new global volume: acme and postgres
Open the port address, select the management node, enter stack management, and create a new stack as needed: gateway

version: "3.7"

networks:
  app:
    external: true
volumes:
  acme:
    external: true
  postgres: 
    external: true
services:
  postgres: # kong Data Service
    image: postgres:9.6
    restart: always
    networks:
      - app
    volumes:
      - postgres:/var/lib/postgresql/data
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_DB=postgres
      - POSTGRES_PASSWORD=postgres
  migration: # kong data table initialization service
    image: kong:2.7.0-alpine
    command: "kong migrations bootstrap"
    networks:
      - app
    restart: on-failure
    environment:
      - KONG_DATABASE=postgres
      - KONG_PG_HOST=postgres
      - KONG_PG_DATABASE=postgres
      - KONG_PG_USER=postgres
      - KONG_PG_PASSWORD=postgres
    depends_on:
      - postgres # Start after database startup
  core: # kong core services
    image: kong:2.7.0-alpine
    restart: always
    networks:
      - app
    environment:
      - KONG_DATABASE=postgres
      - KONG_PG_HOST=postgres
      - KONG_PG_DATABASE=postgres
      - KONG_PG_USER=postgres
      - KONG_PG_PASSWORD=postgres
      - KONG_PROXY_ACCESS_LOG=/dev/stdout
      - KONG_ADMIN_ACCESS_LOG=/dev/stdout
      - KONG_PROXY_ERROR_LOG=/dev/stderr
      - KONG_ADMIN_ERROR_LOG=/dev/stderr
      - KONG_ADMIN_LISTEN=0.0.0.0:8001
      - LUA_SSL_TRUSTED_CERTIFICATE=system
    ports:
      - "80:8000"
      - "443:8443"
    depends_on:
      - postgres # Start after database startup
  admin: # konga management interface
    image: pantsel/konga:0.14.9
    restart: always
    networks:
      - app
    environment:
      - NODE_ENV=development
      - DB_ADAPTER=postgres #mongo,mysql,postgres
      - DB_HOST=xxxxxxxx
      - DB_PORT=xxxx
      - DB_USER=xxxxxx
      - DB_PASSWORD=xxxxxxxx
      - DB_DATABASE=xxxxxx
    ports:
      - "8081:1337"
  acme: # Used to generate SSL certificates
    image: neilpang/acme.sh
    networks:
     - app
    volumes:
      - acme:/acme.sh
    command: daemon
    environment:
      - KONG_URL=http://core:8001
      - Ali_Key=xxxxxxxx
      - Ali_Secret=xxxxxxxx

Open the konga management interface http: / / cluster management server ip:8081
Create a new administrator account and password. For security, please pay attention to the complexity of the account password

Portal: Kong,Konga,acme.sh

5. Set management interface route

Open Konga address, enter SERVICES, and create new SERVICES
Note: this corresponds to the services to be accessed in the portal

1. Name: container management service, Url:http://portainer:9000
2. Name: gateway management service, Url:http://gateway_admin:1337

Select the corresponding service, enter the Routes sub item, and create a new external access route (ADD ROUTE)
Note: this corresponds to the specific domain name paths accessed by the Internet. Dev.abc Com needs to be changed to your own domain name

1. Name: container management service, hosts: container dev.abc. com
2. Name: gateway management service, hosts: Konga dev.abc. com

Generate SSL certificate and use HTTPS to access

acme service will automatically renew the generated certificate, so we only need to manually generate the certificate once.

• Open container management in the container and find acme SH is the only container to access the server console.
• Run the command to generate the certificate acme sh --issue --dns dns_ ali -d dev.abc. com -d *. dev.abc. com
• Deploy certificate to kongacme.com sh --deploy -d dev.abc. com --deploy-hook kong
• Enter the Konga management page, select CERTIFICATES and find the certificate dev.abc COM, ADD SNI * dev.abc.com is used to automatically fuzzy match CERTIFICATES
• After two months, check whether the certificate is updated normally

Note: idle teachers use Alibaba cloud domain names. Please refer to acme for the generation methods of other domain names SH's Wiki. Portal
Note: if you need to force HTTPS, modify the HTTP redirect status code of the routing setting in Konga to 301, 302, 307 or 308, and delete HTTP in Protocols to make it only HTTPS

Close external port

In order to make the service more secure, after adopting forced SSL, it is better to selectively close the external ports 8080 and 8081 to make the service more secure
After shutdown, the corresponding services will only be accessed through the domain name, and the SSL certificate will be forcibly loaded

Deployment complete

So far, we have deployed a basic swarm cluster with container management tools and gateways.
All business contents can be deployed through services. Due to space reasons, this chapter will not be explained.
The next chapter will explain how to deploy an automatically built project environment. Welcome to pay attention.
You can also add the QQ group of idle teachers to learn from each other: 1033245535