traefik Ingress https configuration

Posted by MiCR0 on Wed, 31 Jul 2019 18:03:36 +0200

Environmental Science

. kubernetes 1.14.3

. traefik V1.7.12

.IP 192.168.30.35

.kubectl label nodes ingress ingress=yes

https certificate application

It is recommended to apply for free certificate with acme.sh. The specific method is not described in detail.
Use self-visa

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=*.mddgame.com"

traefik configuration

Add traefik.toml file:

1,http,https At the same time, providing services to the outside world
defaultEntryPoints = ["http","https"]
[entryPoints]
  [entryPoints.http]
  address = ":80"
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]
      [[entryPoints.https.tls.certificates]]
      certFile = "/certs/tls.crt"
      keyFile = "/certs/tls.key"

2,http Forced jump https
defaultEntryPoints = ["http","https"]
[kubernetes]
[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
      entryPoint = "https"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]
      [[entryPoints.https.tls.certificates]]
      CertFile = "/certs/tls.crt"
      KeyFile = "/certs/tls.key"

Among them, tls.crt and tls.key are certificate files. Note that the certificate file name must be fixed. The file will be read when mounted in the container.

Create certificate secret s for easy hanging
kubectl -n kube-system create secret generic mddgame-tls-cert --from-file=tls.key --from-file=tls.crt
 Note: Since secret cannot span namespaces, if the application is deployed in the default namespace or other namespaces, then you need to create a secret in the default namespace and modify the name of - n kube-system at the end of the above for other namespaces.
Create configmap
kubectl create configmap traefik-conf --from-file=traefik.toml -n kube-system

traefik deployment configuration

1,traefik-rbac.yaml

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik
  namespace: kube-system
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
    - extensions
    resources:
    - ingresses/status
    verbs:
    - update
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik
subjects:
- kind: ServiceAccount
  name: traefik
  namespace: kube-system

2 Create traefik users

vi traefik-serviceaccount.yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik
  namespace: kube-system

3. deployment of yaml

vi traefik-deployment-https.yaml
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik
  namespace: kube-system
  labels:
    k8s-app: traefik
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik
  template:
    metadata:
      labels:
        k8s-app: traefik
        name: traefik
    spec:
      serviceAccountName: traefik
      terminationGracePeriodSeconds: 60
      volumes:
      - name: ssl
        secret:
          secretName: mddgame-tls-cert
      - name: config
        configMap:
          name: traefik-conf
          defaultMode: 0644
          items:
          - key: traefik.toml
            path: traefik.toml 
      hostNetwork: true       # If you don't configure hostPort 443 port to map to host opportunity without using hostNetwork, you will not be able to access k8s api 10.64.0.1:443 port. 
      dnsPolicy: ClusterFirstWithHostNet      
      containers:
      - image: traefik
        name: traefik
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - mountPath: /certs         
          name: "ssl"
        - mountPath: /etc/traefik.toml
          subPath: traefik.toml
          name: "config"
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: https
          containerPort: 443
          hostPort: 443
        - name: admin
          containerPort: 8080
        args:
        - --api
        - --web
        - --api.dashboard
        - --logLevel=INFO
        - --web.metrics
        - --metrics.prometheus
        - --web.metrics.prometheus
        - --kubernetes
        - --traefiklog
        - --traefiklog.format=json
        - --accesslog
        - --accesslog.format=json
        - --accessLog.fields.headers.defaultMode=redact
        - --insecureskipverify=true
        - --configFile=/etc/traefik.toml       
      nodeSelector:
        ingress: "yes"
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/ingress
        operator: Equal

4. daemonset deployment

vi traefik-daemonset-https.yaml
---
kind: DaemonSet
apiVersion: extensions/v1beta1
metadata:
  name: traefik
  namespace: kube-system
  labels:
    k8s-app: traefik
spec:
  selector:
    matchLabels:
      k8s-app: traefik
  template:
    metadata:
      labels:
        k8s-app: traefik
        name: traefik
    spec:
      serviceAccountName: traefik
      terminationGracePeriodSeconds: 60
      volumes:
      - name: ssl
        secret:
          secretName: mddgame-tls-cert
      - name: config
        configMap:
          name: traefik-conf
          defaultMode: 0644
          items:
          - key: traefik.toml
            path: traefik.toml
      hostNetwork: true #If you don't configure hostPort 443 port to map to host opportunity without using hostNetwork, you will not be able to access k8s api 10.64.0.1:443 port. 
      dnsPolicy: ClusterFirstWithHostNet      
      containers:
      - image: traefik
        name: traefik
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - mountPath: /certs         
          name: "ssl"
        - mountPath: /etc/traefik.toml
          subPath: traefik.toml
          name: "config"
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: https
          containerPort: 443
          hostPort: 443
        - name: admin
          containerPort: 8080
        securityContext:
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        args:
        - --api
        - --web
        - --api.dashboard
        - --logLevel=INFO
        - --web.metrics
        - --metrics.prometheus
        - --web.metrics.prometheus
        - --kubernetes
        - --traefiklog
        - --traefiklog.format=json
        - --accesslog
        - --accesslog.format=json
        - --accessLog.fields.headers.defaultMode=redact
        - --insecureskipverify=true
        - --configFile=/etc/traefik.toml       
      nodeSelector:
        ingress: "yes"
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/ingress
        operator: Equal

5. Create traefik Service

vi traefik-service.yaml
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: traefik
  name: traefik
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik
  clusterIP: None
  ports:
    - protocol: TCP
      port: 80
      name: http
    - protocol: TCP
      port: 443
      name: https
    - protocol: TCP
      port: 8080
      name: admin

6. Create traefik Service Monitor

vi traefik-serviceMonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    k8s-app: traefik
  name: traefik
  namespace: monitoring
spec:
  endpoints:
  - honorLabels: true
    interval: 15s
    port: admin
  jobLabel: k8s-app
  namespaceSelector:
    matchNames:
    - kube-system
  selector:
    matchLabels:
      k8s-app: traefik

7. Create traefik dashboard Ingress

vi traefik-dashboard.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-dashboard
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.ingress.kubernetes.io/frontend-entry-points: http,https
spec:
  rules:
  - host: traefik.mddgame.com
    http:
      paths:
        - backend:
            serviceName: traefik
            servicePort: 8080
  tls:
   - secretName: mddgame-tls-cert

Execute yaml to create traefik service

kubectl apply -f .

Verify the traefik service

kubectl get all -n kube-system | grep traefik
root@Qist:/mnt/e/work/k8s-game# kubectl get all -n kube-system | grep traefik

pod/traefik-2d5k8                          1/1     Running   0          16h

service/traefik                ClusterIP   None            <none>        80/TCP,443/TCP,8080/TCP   16h

daemonset.apps/traefik       1         1         1       1            1           ingress=yes     16h
//Write hosts
192.168.30.35 traefik.mddgame.com
//Visit traefik.mddgame.com

http access

https access

Create tests

vi nginx.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      k8s-app: nginx
  template:
    metadata:
      labels:
        k8s-app: nginx
    spec:
      containers:
        - name: nginx
          image: docker.mddgame.com/library/nginx
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 80
              name: web
              protocol: TCP
            - containerPort: 8080
              name: vts
              protocol: TCP
          readinessProbe:
            failureThreshold: 10
            httpGet:
              path: /healthz
              port: vts
              scheme: HTTP
            initialDelaySeconds: 3
            periodSeconds: 5
            successThreshold: 1
            timeoutSeconds: 3
          resources:
            requests:
              cpu: 200m
              memory: 200Mi
        - name: nginx-vts-exporter
          image: docker.mddgame.com/library/nginx-vts-exporter
          imagePullPolicy: IfNotPresent
          args:
          - "-nginx.scrape_uri=http://localhost:8080/format/json"
          ports:
            - containerPort: 9913
              name: http-metrics
              protocol: TCP
          resources:
            requests:
              memory: 30Mi
              cpu: 102m
            limits:
              memory: 50Mi
              cpu: 250m
---
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: nginx
  name: nginx
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.ingress.kubernetes.io/affinity: "true" # Back-end Cookie-based session
    traefik.ingress.kubernetes.io/load-balancer-method: drr   #Modify load mode
spec:
  sessionAffinity: ClientIP
  sessionAffinityConfig:
    clientIP:
      timeoutSeconds: 10800
  selector:
    k8s-app: nginx
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 8080
      name: vts
    - protocol: TCP
      port: 9913
      name: http-metrics
  type: ClusterIP
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.frontend.rule.type: PathPrefixStrip
    traefik.ingress.kubernetes.io/frontend-entry-points: http,https
spec:
  rules:
  - host: nginx.mddgame.com
    http:
      paths:
        - path: /
          backend:
            serviceName: nginx
            servicePort: 80
  tls:
   - secretName: mddgame-tls-cert
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    k8s-app: nginx
  name: nginx
spec:
  endpoints:
  - honorLabels: true
    interval: 15s
    port: http-metrics
  jobLabel: k8s-app
  selector:
    matchLabels:
      k8s-app: nginx

//Creating nginx services
kubectl apply -f .
hosts Write in
192.168.30.35 nginx.mddgame.com 


Open normally

External business provides services using traefik

with Confluence take as an example
vi wiki.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: wiki
  name: wiki
  namespace: default
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.ingress.kubernetes.io/affinity: "true"
    traefik.ingress.kubernetes.io/load-balancer-method: drr
spec:
  clusterIP: None
  ports:
  - name: http
    port: 8080
    protocol: TCP
    targetPort: 8080
  sessionAffinity: None
  type: ClusterIP
---
apiVersion: v1
kind: Endpoints
metadata:
  labels:
    k8s-app: wiki
  name: wiki
  namespace: default
subsets:
- addresses:
  - ip: 192.168.30.11 # Multiple IPS can be added directly to the next line - ip:192.168.30.22 
  ports:
  - name: http
    port: 8080
    protocol: TCP
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: wiki
  namespace: default
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.frontend.rule.type: PathPrefixStrip
    traefik.ingress.kubernetes.io/frontend-entry-points: http,https
    traefik.ingress.kubernetes.io/redirect-entry-point: https # Force adjustment to https
spec:
  rules:
  - host: wiki.mddgame.com
    http:
      paths:
        - path: /
          backend:
            serviceName: wiki
            servicePort: 8080
  tls:
   - secretName: mddgame-tls-cert

Binding hosts
192.168.30.35 wiki.mddgame.com
Verify that it can be opened

Can open normally

Topics: Linux Nginx Kubernetes JSON SSL