Installing Jenkins on k8s and common problems

Posted by indigo2k on Tue, 05 Nov 2019 08:22:34 +0100

Continuous integration and deployment is an important part of DevOps. Jenkins is a very popular tool for continuous integration and deployment. Recently, I tested Jenkins and found that it is the most complex tool I have used for some time. One possible reason is that it needs to be integrated with various other tools to complete the task, and the integration methods are different. Among these tools, Docker is the simplest, which is really very easy to use. K8s is complex, and it will take some time to get familiar with it at first, but its overall design is very reasonable. Once the core concepts are clear and the context is mastered, it will be very smooth. Its command format is both standardized and unified, so that some commands can be guessed by themselves, which is the benefit of good design.. But the feeling Jenkins gives is that they didn't design well at the beginning, and they kept patching later, which led to several different ways of doing something, and they were at a loss for unfamiliar people. There is no unified style, and there are accidents everywhere, which makes the whole system look complicated and disorganized. Of course, it also has to do with the long time it took to come out. Although it may not be the best, it's free, so it can't be too demanding.

For various reasons, my Jenkins installation encountered various problems, so I checked a lot of materials. Unfortunately, everyone has different ways to install Jenkins. It's hard to find an article that can solve all the problems. In my opinion, the installation of Jenkins has two or three key points, which are very error prone. You must understand thoroughly to succeed.

This article is divided into two parts. The first part talks about the normal installation steps. If everything goes well, you don't need to see the second part. I can only say congratulations. Your luck is so good. The second part is about various problems and solutions, which should be the most valuable part of this article.

Part I: deploying Jenkins on k8s

1. Where is it installed?

Containerization is the general trend. It includes not only the containerization of applications, but also the containerization of related tools. When Jenkins is deployed on K8s, the master node of Jenkins will automatically generate child nodes (new containers) to complete the task according to the situation, and the child nodes will be automatically destroyed after the task is completed.

I first deployed VirtulBox virtual machine on Windows, and used Vagrant to manage the virtual machine, and then deployed k8s on the virtual machine. And the network sharing between the virtual machine and the host computer is set through Vagrant, so that Jenkins on k8s can be directly accessed on the host computer by the tourist. In addition, the hard disk of the host computer should be attached to Jenkins, so that the physical storage of Jenkins is still on the host computer. Even if there is a problem with the virtual machine, all configuration and data will not be lost.

2. Select image file

This doesn't seem to be a problem, but it's easy to make mistakes if you don't pay attention. Because I chose the wrong image, I installed it many times and finally succeeded. I will explain it in detail in the second part of this article. The final image file I used was "Jenkins CI / Jenkins: 2.154 slim". Later, I found that this is an older version, and the new image is "jenkins/jenkins:lts". But because it has been successfully installed, it has not been replaced. Jenkins is really tricky. There are three images: "Jenkins", "jenkinsci/jenkins", "jenkins/jenkins", and the right one is "jenkins/jenkins".

After selecting the image, you can run the following command to download the Jenkins image file to the local (virtual machine).

vagrant@ubuntu-xenial:~$ docker pull jenkinsci/jenkins:2.154-slim

3. Install Jenkins image:

Before installation, you need to mount the Jenkins installation directory of the host computer to the virtual machine, so that you can directly operate Jenkins locally.
The following is the setting in Vagrant's configuration file, which mounts the app directory of the host to "/ home/vagrant/app" of the virtual machine. Jenkins is installed in the app directory.

config.vm.synced_folder "app/", "/home/vagrant/app", id: "app"

Here is the Jenkins directory installed on the host computer

The installation of Jenkins image is divided into four parts: create service account, install persistent volume, install deployment and install services in sequence. The key is to create a service account, which is necessary. Without it, it will not succeed. I don't know why some articles on the Internet don't mention it.

Service account profile (service account. Yaml):

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: default
  name: service-reader
rules:
  - apiGroups: [""] # "" indicates the core API group
    resources: ["services"]
    verbs: ["get", "watch", "list"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get","list","watch"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get"]

Here, a "ClusterRole" named "service reader" is created, and specific permissions (such as ["get", "watch", "list") are assigned to specific resources (such as ["services").

Run the following command to create a cluster role binding named "service reader pod". Its "clusterrole" is "service reader", and its name is "default:default". The first "default" is the namespace, the second "default" is the name of the service account, which will be referred to in the subsequent deployment configuration file. Here, because I didn't create a separate namespace for Jenkins, it uses the default namespace ("default").

kubectl create clusterrolebinding service-reader-pod --clusterrole=service-reader  --serviceaccount=default:default

For the definition of permissions for the service account, see "Kubernetes plugin for Jenkins" .

Persistent volume profile (Jenkins column. Yaml):

apiVersion: v1
kind: PersistentVolume
metadata:
  name: k8sdemo-jenkins-pv
  labels:
    app: k8sdemo-jenkins
spec:
  capacity:
    storage: 1Gi
  # volumeMode field requires BlockVolume Alpha feature gate to be enabled.
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  storageClassName: standard
  local:
    path: /home/vagrant/app/jenkins
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - minikube
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: k8sdemo-jenkins-pvclaim
  labels:
    app: k8sdemo-jenkins
spec:
  accessModes:
    - ReadWriteOnce
  # storageClassName: local-storage
  resources:
    requests:
      storage: 1Gi #1 GB

Deployment profile (jenkins-deployment.yaml):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: k8sdemo-jenkins-deployment
  labels:
    app: k8sdemo-jenkins
spec:
  selector:
    matchLabels:
      app: k8sdemo-jenkins
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: k8sdemo-jenkins
    spec:
      serviceAccountName: default # The name of the service account is default
      containers:
        - image: jenkinsci/jenkins:2.154-slim
          name: k8sdemo-jenkins-container
          imagePullPolicy: Never
          ports:
            - containerPort: 8080
            - containerPort: 50000
          volumeMounts:
            - name: k8sdemo-jenkins-persistentstorage
              mountPath: /var/jenkins_home
      volumes:
        - name: k8sdemo-jenkins-persistentstorage
          persistentVolumeClaim:
            claimName: k8sdemo-jenkins-pvclaim

Note that the service account "default" (serviceAccountName: default) is referenced here.

Service configuration file (jenkins-service.yaml):

apiVersion: v1
kind: Service
metadata:
  name: k8sdemo-jenkins-service
  labels:
    app: k8sdemo-jenkins
spec:
  type: NodePort
  selector:
    app: k8sdemo-jenkins
  ports:
    - port: 8080
      name: http
      protocol : TCP
      nodePort: 30080
      targetPort: 8080
    - port: 50000
      name: agent
      protocol: TCP
      targetPort: 50000

One of the key points is that deployment and service expose two container ports, one is 8080, the other is 50000. "8080" is the external access port to Jenkins, and "50000" is the mutual communication port between Jenkins internal clusters. The Jenkins cluster here does not need to be built by you, but is generated automatically by Jenkins as required. Therefore, these two ports must be configured. The configuration commands here are all standard k8s configurations, so there is no detailed explanation.

If you want to know more about the k8s command (including the Vagrant configuration), see "Master the important concepts of k8s (Kubernetes) by building MySQL (I): network and persistent volume".

Run the following command to create Jenkins:

kubectl apply -f jenkins-volume.yaml
kubectl apply -f jenkins-deployment.yaml
kubectl apply -f jenkins-service.yaml

Verify installation:

Get Jenkins' Pod name

vagrant@ubuntu-xenial:~$ kubectl get pod
NAME                                           READY   STATUS    RESTARTS   AGE
envar-demo                                     1/1     Running   15         27d
k8sdemo-backend-deployment-6b99dc6b8c-tbl4v    1/1     Running   7          11d
k8sdemo-database-deployment-578fc88c88-mm6x8   1/1     Running   9          16d
k8sdemo-jenkins-deployment-675dd574cb-bt7rx    1/1     Running   2          24h

View Jenkins logs

vagrant@ubuntu-xenial:~$ kubectl logs k8sdemo-jenkins-deployment-675dd574cb-bt7rx

Running from: /usr/share/jenkins/jenkins.war
webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
Nov 02, 2019 1:33:30 AM org.eclipse.jetty.util.log.Log initialized
INFO: Logging initialized @3749ms to org.eclipse.jetty.util.log.JavaUtilLog

. . . 

INFO: Invalidating Kubernetes client: kubernetes null
Nov 02, 2019 1:35:50 AM hudson.WebAppMain$3 run
INFO: Jenkins is fully up and running
--> setting agent port for jnlp
--> setting agent port for jnlp... done

When you see "INFO: Jenkins is fully up and running", it means that Jenkins has been running. The first startup of Jenkins needs a certain time, and you should wait patiently.

4. login

As mentioned earlier, you can set the network mutual access between the host and the virtual machine in Vagrant. My virtual machine address is "192.168.50.4" and "30080" is the external address of NodePort of Jenkins service, so you can use“ http://192.168.50.4 : 30080 / "access Jenkins.

Before logging in, you need to obtain the initial password. You can obtain the initial password of the administrator user in Jenkins' secretationaladminpassword directory. The host directory where I mount Jenkins is "E:app2kubappjenkins", so the password file is "E:app2kubappjenkins secretationaladminpassword". The password is "072d7157c090479195e0acaa97bc1049". After the first login, the user and password need to be reset.

5. Install recommended plug-ins

After logging in, you need to install the necessary plug-ins before completing the whole installation project. Just select "install recommended plugins".

6 install Kubernetes Plugin

Log in Jenkins homepage with administrator account and find
Manage Jenkins - "Manage Plugins -" Available, check and install "kubernets plugin".

The following figure is after installation:

7. Configure Kubernetes Plugin

Log in the Jenkins Master homepage with the administrator account, and find
Manage Jenkins - "Configure System -" and then configure kubernets plugin. As shown in the figure below:

This is the most important configuration that determines the success or failure of the entire installation. The default "name" is "Kubernetes", which doesn't need to be modified, but will be used later when configuring Pipelines. For "Kubernetes URL"“ https://kubernetes.default ”Just fine. After setting, click "Test Connection" to see "Connection test successful".

"Jenkins URL" is the address to access Jenkins from outside (from the virtual machine rather than the host).
You can find "Jenkins Url" of Kubernetes with the following command:

vagrant@ubuntu-xenial:~$  sudo minikube service k8sdemo-jenkins-service  --url
http://10.0.2.15:30080
http://10.0.2.15:32289

Another parameter is "Jenkins tunnel", which must be configured for communication between Jenkins Master and Jenkins Slave. However, I don't know why many articles on the Internet don't mention this parameter. Maybe the version of Jenkins is different, and some versions may not be required.

View container name

vagrant@ubuntu-xenial:~$ docker ps -a
CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS                      PORTS               NAMES
d15e30169568        f793ea0abe00            "/sbin/tini -- /usr/..."   15 minutes ago      Up 15 minutes                                   k8s_k8sdemo-jenkins-container_k8sdemo-jenkins-deployment-675dd574cb-2thn2_default_fb10e438-0231-4fd2-8dbd-d9e2f0bb9d09_0

To view the container address:

vagrant@ubuntu-xenial:~$ docker inspect d15e |grep _8080
                "K8SDEMO_JENKINS_SERVICE_PORT_8080_TCP_ADDR=10.100.3.79",
                "K8SDEMO_JENKINS_SERVICE_PORT_8080_TCP=tcp://10.100.3.79:8080",
                "K8SDEMO_JENKINS_SERVICE_PORT_8080_TCP_PROTO=tcp",
                "K8SDEMO_JENKINS_SERVICE_PORT_8080_TCP_PORT=8080",

According to the above information, Jenkins's address is "tcp://10.100.3.79:8080". You can replace 8080 with 50000. The final result is "10.100.3.79:50000", take care not to add "http".

8. Test Jenkins:

Now that Jenkins is all installed, let's test it. On the Jenkins main page, click "New Item" to create a New Item, as shown in the figure below, enter the item name, and then select "Pipeline".

Enter the project configuration page, as shown in the following figure. The script file is jenkinsfile test:

This is the simplest test. It directly uses the Jenkins master node (the master node name is the master node), and does not need to start the child nodes, so there is basically no problem.
Select the project "test" on Jenkins main page, then select "Build Now" to run the project, and then view the results in "Console Output" as follows:

Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline[Pipeline] podTemplate[Pipeline] {[Pipeline] nodeRunning on Jenkins in /var/jenkins_home/workspace/test
[Pipeline] {[Pipeline] stage[Pipeline] { (Run shell)[Pipeline] sh+ echo hello world.
hello world.
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: SUCCESS

9. Test child nodes:

This is a more complex test. You need to start the child nodes to really detect the success or failure of the installation. First, create a new project "slave test".


def POD_LABEL = "testpod-${UUID.randomUUID().toString()}"
podTemplate(label: POD_LABEL, cloud: 'kubernetes', containers: [
    containerTemplate(name: 'build', image: 'jfeng45/k8sdemo-backend:1.0', ttyEnabled: true, command: 'cat'),
    containerTemplate(name: 'run', image: 'jfeng45/k8sdemo-backend:1.0', ttyEnabled: true, command: 'cat')
  ]) {

    node(POD_LABEL) {
        stage('build a go project') {
            container('build') {
                stage('Build a go project') {
                    sh 'echo hello'
                }
            }
        }

        stage('Run a Golang project') {
            container('run') {
                stage('Run a Go project') {
                    sh '/root/main.exe'
                }
            }
        }

    }
}

Above is the script (jenkins-salve-test). Among them, "POD_LABEL" can be named by any name (after Kubernetes Plugin version 1.17.0, the system will automatically name it, but before it needs to be named by itself), "cloud: 'kubernetes'" should match the "Kubernetes Plugin" defined earlier. It has two stages, one is "build", the other is "run". In "podTemplate", the image of each stage is defined (so that it can be referenced in the subsequent stage script). Here, for the sake of simplicity, the two images are set to be the same. Because it's a test, the first stage just outputs "echo hello", and the second runs the main.exe program in the image "jfeng45 / k8sdemo backend: 1.0".

On the Jenkins main page, select the project "slave test", then select "Build Now" to run the project, and then view the results in "Console Output" as follows:

Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline[Pipeline] podTemplate[Pipeline] {[Pipeline] node
Still waiting to schedule task
'testpod-f754a7a4-6883-4be0-ba4f-c693906041ae-fjwqs-kbb7l' is offline

Agent testpod-f754a7a4-6883-4be0-ba4f-c693906041ae-fjwqs-kbb7l is provisioned from template Kubernetes Pod Template
Agent specification [Kubernetes Pod Template] (testpod-f754a7a4-6883-4be0-ba4f-c693906041ae): 
* [build] jfeng45/k8sdemo-backend:1.0
* [run] jfeng45/k8sdemo-backend:1.0

Running on testpod-f754a7a4-6883-4be0-ba4f-c693906041ae-fjwqs-kbb7l in /home/jenkins/workspace/slave-test
[Pipeline] {
[Pipeline] stage[Pipeline] { (build a go project)
[Pipeline] container[Pipeline] {
[Pipeline] stage
[Pipeline] { (Build a go project)[Pipeline] sh
+ echo heollo
heollo

[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // container
[Pipeline] }

[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Run a Golang project)[Pipeline] container
[Pipeline] {[Pipeline] stage[Pipeline] { (Run a Go project)
[Pipeline] sh
+ /root/main.exe
time="2019-11-03T01:56:59Z" level=debug msg="connect to database "
time="2019-11-03T01:56:59Z" level=debug msg="dataSourceName::@tcp(:)/?charset=utf8"
time="2019-11-03T01:56:59Z" level=debug msg="FindAll()"
time="2019-11-03T01:56:59Z" level=debug msg="user registere failed:dial tcp :0: connect: connection refused"

[Pipeline] }
[Pipeline] // stage

[Pipeline] }
[Pipeline] // container
[Pipeline] }

[Pipeline] // stage
[Pipeline] }
[Pipeline] // node

[Pipeline] }
[Pipeline] // podTemplate

[Pipeline] End of Pipeline
Finished: SUCCESS

The test phase is complete when the run is successful.

There are two ways to write Pipeline with script: "scripted Pipeline" and "declarative Pipeline". Here is the first way. Please refer to“ Using a Jenkinsfile ”"Declarative pipeline" is a new method, which I will talk about in a future article. Because it's a test, just pass it.

Unnecessary installation steps:

There are also installation steps mentioned in some articles, but they are just icing on the cake and are not necessary. If there's something wrong with your configuration, don't doubt that these steps didn't work.

  1. Configure namespace:

Some installation steps configure a separate namespace for Jenkins, which is better, but you won't have problems even if you don't configure it.

  1. Kubernetes server certificate key:

Some installation steps mentioned to configure "Kubernetes server certificate key", but I did not set it and did not affect the operation.

Part II: common problems

1. Wrong version of Jenkins:

The first one is Jenkins: 2.60.3-alpine (this is the highest version of Jenkins image). This version is too low to install the plug-in successfully, as shown in the following figure

Later, it was changed to jenkins:latest, which should be the latest one. The result version is the same, but Linux is not Apline.

Later, I finally understood that it was the image fault (not the version problem), and it was to use Jenkinsci instead of Jenkins. I used jenkinsci/jenkins:2.150.1-slim, which was ranked first at that time. After installation, all the plug-in errors above disappeared, which was really not easy.

2. Kubernetes Plugin is not supported

However, when installing the Kubernetes Plugin plug-in, the prompt requires 2.150.3 (mine is 2.150.1), which is too much. We had to reinstall it again. This time, we used jenkinsci / Jenkins: 2.154 slim. Fortunately, it was finally successful. But this is still the previous image. The latest one is "jenkins/jenkins".

3. Unable to access Kubernetes

The error message is as follows:

Forbidden!Configured service account doesn't have access. Service account may have been revoked. User "system:serviceaccount:default:default" cannot get services in the namespace "default"

For details, please see Kubernetes log, User "system:serviceaccount:default:default" cannot get services in the namespace.

The reason for the error is that the service account is not established. The solution is to create the "service account. Yaml" file first, and then run the following command:

kubectl create clusterrolebinding service-reader-pod --clusterrole=service-reader  --serviceaccount=default:default

Run again and the error disappears.

4. Jenkins URL is wrong

On the Jenkins main page, enter Manage Jenkins - "System Log -" All Jenkins Logs. The error information is as follows.

SEVERE: http://192.168.50.4:30080/ provided port:50000 is not reachable
java.io.IOException: http://192.168.50.4:30080/ provided port:50000 is not reachable
     at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:303)
     at hudson.remoting.Engine.innerRun(Engine.java:527)
     at hudson.remoting.Engine.run(Engine.java:488)

This error is mainly related to the kubernetes plugin configuration. On the Jenkins main page, go to Manage Jenkins - "Configure System", and click“ http://192.168.50.4 There are two Jenkins URL s in: 30080/configure. Don't confuse them.
The first is "Jenkins URL" under "Jenkins Location", which is the address of the host accessing Jenkins.

The second is "Jenkins URL" under "Cloud", which is the address to access Jenkins from the virtual machine.

In the picture above, I started with“ http://192.168.50.4 : 30080 /, but this is the Url to access Jenkins from the host, not from the internal of the virtual machine.
You can find Kubernetes's "Jenkins Url" with the following command

vagrant@ubuntu-xenial:~$  sudo minikube service k8sdemo-jenkins-service  --url
http://10.0.2.15:30080
http://10.0.2.15:32289

Type the following command to test the URL.

vagrant@ubuntu-xenial:~$ curl http://10.0.2.15:30080
<html><head><meta http-equiv='refresh' content='1;url=/login?from=%2F'/><script>window.location.replace('/login?from=%2F');</script></head><body style='background-color:white; color:white;'>

Authentication required
<!--
You are authenticated as: anonymous
Groups that you are in:

Permission you need to have (but didn't): hudson.model.Hudson.Read
... which is implied by: hudson.security.Permission.GenericRead
... which is implied by: hudson.model.Hudson.Administer
-->

</body></html> 

This means that the URL is good.

5. Unable to connect to slave

After "Jenkins Url" is changed, the address is correct, but it still doesn't work. When running a project, the page displays the following information:

"Console Output" (in Jenkins - > salve test - > build 13) displays the following information:

Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline[Pipeline] podTemplate[Pipeline] {[Pipeline] node
Still waiting to schedule task
'testpod-d56038a0-45a2-41d1-922d-2879e3610900-0hr0m-sfv8s' is offline

Later, it was found that there was another parameter to fill in, "Jenkins tunnel". As shown in the figure below.

See for details. Kubernetes Jenkins plugin - slaves always offline.

After filling in, the original information is gone, and "Agent discovery successful" appears, which is not the original information. But there are new mistakes.
You can view the System Log as follows. On the Jenkins main page, select Manage Jenkins - > System Log - > All Jenkins Logs. The information is as follows:

INFO: Agent discovery successful
  Agent address: http://10.0.2.15
  Agent port:    50000
  Identity:      3e:1b:5f:48:f7:5b:f8:6d:ea:49:1d:b9:44:9a:2f:6c
Oct 30, 2019 12:18:51 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Oct 30, 2019 12:18:51 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to http://10.0.2.15:50000
Oct 30, 2019 12:18:51 AM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: null
java.nio.channels.UnresolvedAddressException
     at sun.nio.ch.Net.checkAddress(Net.java:101)
     at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622)
     at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
     at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:203)
     at hudson.remoting.Engine.connectTcp(Engine.java:678)
     at hudson.remoting.Engine.innerRun(Engine.java:556)
     at hudson.remoting.Engine.run(Engine.java:488)

The reason is that the address of "Jenkins tunnel" is wrong or not. You can find the address of "Jenkins tunnel" as follows:

vagrant@ubuntu-xenial:~$ docker ps -a
CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS                      PORTS               NAMES
d15e30169568        f793ea0abe00            "/sbin/tini -- /usr/..."   15 minutes ago      Up 15 minutes                                   k8s_k8sdemo-jenkins-container_k8sdemo-jenkins-deployment-675dd574cb-2thn2_default_fb10e438-0231-4fd2-8dbd-d9e2f0bb9d09_0

vagrant@ubuntu-xenial:~$ docker inspect d15e |grep _8080
                "K8SDEMO_JENKINS_SERVICE_PORT_8080_TCP_ADDR=10.100.3.79",
                "K8SDEMO_JENKINS_SERVICE_PORT_8080_TCP=tcp://10.100.3.79:8080",
                "K8SDEMO_JENKINS_SERVICE_PORT_8080_TCP_PROTO=tcp",
                "K8SDEMO_JENKINS_SERVICE_PORT_8080_TCP_PORT=8080",

According to the above information, the Jenkins container address is "tcp://10.100.3.79:8080". You can replace 8080 with 50000. The final result is "10.100.3.79:50000", take care not to add "http".
See for details. What setting to use for jenkins tunnel?

6. Image problem

When the image file used is "k8sdemo backend: latest" or "k8sdemo backend: 1.0", the Console Output display error is as follows:

Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline[Pipeline] podTemplate[Pipeline] {[Pipeline] nodeStill waiting to schedule task
All nodes of label 'testpod-2971e0ce-e023-475f-b0ec-6118c5699188' are offline
Aborted by admin[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: ABORTED

Viewing Pod, the error is "ImagePullBackOff":

vagrant@ubuntu-xenial:~$ kubectl get pod
NAME                                                       READY   STATUS             RESTARTS   AGE
envar-demo                                                 1/1     Running            15         28d
k8sdemo-backend-deployment-6b99dc6b8c-tbl4v                1/1     Running            7          12d
k8sdemo-database-deployment-578fc88c88-mm6x8               1/1     Running            9          17d
k8sdemo-jenkins-deployment-675dd574cb-bt7rx                1/1     Running            2          2d
testpod-2971e0ce-e023-475f-b0ec-6118c5699188-xwwqq-vv59p   2/3     ImagePullBackOff   0          38s

To view the image:

vagrant@ubuntu-xenial:~$ docker image ls
REPOSITORY                                TAG                 IMAGE ID            CREATED             SIZE
jfeng45/k8sdemo-backend                   1.0                 f48d362fdebf        11 days ago         14.4MB
k8sdemo-backend                           1.0                 f48d362fdebf        11 days ago         14.4MB
k8sdemo-backend                           latest              f48d362fdebf        11 days ago         14.4MB

There are three "k8sdemo backend" images. Their "Image ID" is the same. The reason why there are three is that I created the tag with the following command

docker tag k8sdemo-backend jfeng45/k8sdemo-backend: 1.0

But after creation, only "jfeng 45 / k8sdemo backend: 1.0" (the latest one) can be used in Jenkins' Pipeline script, and the other two will report errors. After the correct image file is modified, it runs successfully.

7. Slow removal of PV and pvc

When pv is deleted with the following command, the command cannot return later.

kubectl delete pv k8sdemo-jenkins-pv

When you view it, the status display is always "Terminating", but you can't end the exit. pvc is the same.

vagrant@ubuntu-xenial:~$ kubectl get pv
NAME                  CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS        CLAIM                              STORAGECLASS   REASON   AGE
k8sdemo-backend-pv    1Gi        RWO            Retain           Bound         default/k8sdemo-backend-pvclaim    standard                13d
k8sdemo-database-pv   1Gi        RWO            Retain           Bound         default/k8sdemo-database-pvclaim   standard                18d
k8sdemo-jenkins-pv    1Gi        RWO            Retain           Terminating   default/k8sdemo-jenkins-pvclaim    standard                6d8h

The main reason is that the service and deployment using them are still running. After deleting the service and deployment, the deletion of pv and pvc will be finished immediately and return smoothly.

Source code:

github link of complete source code

Note that under 0.1 (tag), the main branch of this program will be modified later.

The following is the directory structure of the program, and the yellow part is the configuration file related to this article.

Indexes:

  1. Kubernetes plugin for Jenkins
  2. Master the important concepts of k8s (Kubernetes) by building MySQL (I): network and persistent volume
  3. Using a Jenkinsfile
  4. Kubernetes log, User "system:serviceaccount:default:default" cannot get services in the namespace
  5. Kubernetes Jenkins plugin - slaves always offline.
  6. What setting to use for jenkins tunnel?
  7. Preliminary test: Jenkins uses Kubernetes Plugin to complete continuous construction and release

This article is based on the platform of blog one article multiple sending OpenWrite Release!

Topics: Go jenkins Kubernetes Ubuntu Java