k8s core concept progress

Posted by rushdot on Thu, 20 Jan 2022 10:47:43 +0100

1. What is progress

Generally, the IP addresses of service and pod can only be accessed within the cluster. Requests outside the cluster need to be forwarded to the NodePort exposed by the service on the Node through load balancing, and then forwarded to the relevant pod by Kube proxy.

Ingress is a collection of routing rules for requests entering the cluster, as shown in the following figure

  internet
        |
   [ Ingress ]
   --|-----|--
   [ Services ]

Ingress can provide service s with URL s, load balancing, SSL termination, HTTP routing, etc. for external access to the cluster. In order to configure these progress rules, the cluster administrator needs to deploy one Ingress controller , it monitors the changes of progress and service, configures load balancing according to rules, and provides access entry.

An Ingress controller is software that can integrate a specific load balancer with Kubernetes. Open source NGINX and NGINX Plus have corresponding Ingress controller and GitHub warehouse( https://github.com/nginxinc/kubernetes-ingress ). I understand that Ingress Controller actually integrates nginx and kubernetes, so that nginx can get the service information of kubernetes. After we start an Ingress Controller, we can write the corresponding rule s to realize forwarding and load balancing.

It can be understood that ingress is a proxy for a service. We do not need to know the port information exposed by the service to pods. The port changes exposed by pods restart service are not sensitive to the application. The application only deals with ingress.

2. Relationship with pod

Ingress and pod are associated through service. As a unified entry, ingress is associated with a group of pods by service.

3. Progress workflow

As a unified portal, ingress forwards requests to different back-end services according to different request paths.

4. Use ingress:

1. Deploy ingress controller

2. Create ingress rules

5. Deployment testing

Select the official maintenance nginx controller to realize deployment. The simple implementation is to forward to different service s based on different domain names. The effects are as follows:

1. Create nginx application and use service to expose the port

kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80 --type=NodePort

View svc and related information:

[root@k8smaster1 ~]# kubectl get svc -o wide
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE   SELECTOR
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP        8d    <none>
nginx        NodePort    10.109.175.64   <none>        80:32503/TCP   32m   app=nginx

Next, you can access nginx from windows through port 32503 of any k8s node.

2. Create a tomcat service and expose its port with the service

[root@k8smaster1 ~]# kubectl create deployment tomcat --image=tomcat:8.0.15-jre8
deployment.apps/tomcat created
[root@k8smaster1 ~]# kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
nginx-f89759699-vkf7d    1/1     Running   1          59m
tomcat-58767d5b5-f5qwj   1/1     Running   0          8s
[root@k8smaster1 ~]# kubectl expose deployment tomcat --port=8080 --type=NodePort
service/tomcat exposed
[root@k8smaster1 ~]# kubectl get svc -o wide
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE   SELECTOR
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP          8d    <none>
nginx        NodePort    10.109.175.64   <none>        80:32503/TCP     59m   app=nginx
tomcat       NodePort    10.100.178.25   <none>        8080:30158/TCP   6s    app=tomcat

 3. Finally, the effect of service exposure is as follows: the specified service can be accessed from the outside through k8s any node IP + port

 

 4. Creating an ingress controller

(1) Create ingress controller yaml

apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      hostNetwork: true
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeSelector:
        kubernetes.io/os: linux
      containers:
        - name: nginx-ingress-controller
          image: lizhenliang/nginx-ingress-controller:0.30.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 101
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

---

apiVersion: v1
kind: LimitRange
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  limits:
  - min:
      memory: 90Mi
      cpu: 100m
    type: Container

(2) Create related resources

[root@k8smaster1 ~]# kubectl apply -f ingress-controller.yaml 
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
limitrange/ingress-nginx created
[root@k8smaster1 ~]# kubectl get all -o wide -n ingress-nginx
NAME                                           READY   STATUS              RESTARTS   AGE   IP               NODE       NOMINATED NODE   READINESS GATES
pod/nginx-ingress-controller-766fb9f77-tksqp   0/1     ContainerCreating   0          36s   192.168.13.105   k8snode2   <none>           <none>

NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS                 IMAGES                                        SELECTOR
deployment.apps/nginx-ingress-controller   0/1     1            0           36s   nginx-ingress-controller   lizhenliang/nginx-ingress-controller:0.30.0   app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx

NAME                                                 DESIRED   CURRENT   READY   AGE   CONTAINERS                 IMAGES                                        SELECTOR
replicaset.apps/nginx-ingress-controller-766fb9f77   1         1         0       36s   nginx-ingress-controller   lizhenliang/nginx-ingress-controller:0.30.0   app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx,pod-template-hash=766fb9f77

(3) Go to the k8snode2 node to check the listening port of nginx. The nginx controller actually starts an nginx

[root@k8snode2 ~]# netstat -nltp | grep 80
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      5852/nginx: master  
tcp6       0      0 :::80                   :::*                    LISTEN      5852/nginx: master  
[root@k8snode2 ~]# netstat -nltp | grep 443
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      5852/nginx: master  
tcp6       0      0 :::443                  :::*                    LISTEN      5852/nginx: master 

5. Write ingress rules

(1) Write ingress rule yaml

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: example-ingress
spec:
  rules:
  - host: nginx.ingredemo.com
    http:
      paths:
      - path: /
        backend:
          serviceName: nginx
          servicePort: 80
  - host: tomcat.ingredemo.com
    http:
      paths:
      - path: /
        backend:
          serviceName: tomcat
          servicePort: 8080          

(2) Execute create and view

[root@k8smaster1 ~]# kubectl apply -f ingress-rule.yaml 
ingress.networking.k8s.io/example-ingress configured
[root@k8smaster1 ~]# kubectl get ing -o wide
NAME              CLASS    HOSTS                                      ADDRESS   PORTS   AGE
example-ingress   <none>   nginx.ingredemo.com,tomcat.ingredemo.com             80      15h
[root@k8smaster1 ~]# kubectl describe ing example-ingress
Name:             example-ingress
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host                  Path  Backends
  ----                  ----  --------
  nginx.ingredemo.com   
                        /   nginx:80 (10.244.1.89:80)
  tomcat.ingredemo.com  
                        /        tomcat:8080 (10.244.1.92:8080)
Annotations:            Events:  <none>

6. Test:

(1) Configure the virtual domain name under Windows:

C: Add the following configuration to the \ windows \ system32 \ drivers \ etc \ hosts file. Note that the IP is the node where your nginx controller is located

192.168.13.105    nginx.ingredemo.com
192.168.13.105    tomcat.ingredemo.com

(2) Access test: you can access different services according to different domain names

Add: ingress can also be assigned to different service s according to different paths

For example:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - path: /foo
        backend:
          serviceName: s1
          servicePort: 80
      - path: /bar
        backend:
          serviceName: s2
          servicePort: 80

The following results are shown above:

foo.bar.com -> 178.91.123.132 -> / foo    s1:80
                                 / bar    s2:80

 

Topics: Kubernetes