Knative Practice: Automated Deployment from Source Code to Service

Posted by squiblo on Fri, 09 Aug 2019 06:23:59 +0200

Through the previous articles, I believe you are familiar with Serving, Eventing and Tekton. So in practical use, we often encounter some complex scenarios, which require collaborative processing between components. For example, can we deploy services directly into K8s after we submit the source code? This scenario is attractive to users. Now let's take a look at how to implement from code to service in Knative.

Scenario introduction

Now the scenario is as follows: code building - > event driven - > service deployment. For Knative, Eventing, Tekton and Serving need to work together to achieve this scenario.

Get ready

  • Deploy Knative. Refer to deploying Knative on Aliyun Container Service.
  • Deploy Tekton. Through the Aliyun Container Service Console, the application directory selected ack-tekton-pipelines to install and deploy Tekton.
  • Deploy GitHub event sources. In Aliyun Container Service Console Knative Component Management, you choose to install GitHub Components as shown in the figure:

From Source Code to Service

  • Modify branch code and submit merge request to merge into master branch.
  • Eventing monitors merge events and sends them to GitHub Trigger service.
  • The GitHub Trigger service receives events, executes code building through Tekton, and deploys services through deployer. GitHub Trigger parses the details of GitHub events, converts them into Tekton resources and submits them to Kubernetes to execute Pipeline. Project address: https://github.com/knative-sample/tekton-serving . There are two parts in this project: Trigger and Deployer. Trigger's role is to parse github events and submit Pipeline Run definitions. Deployer's role is to update the Service's mirror information. The key contents of github source pull_request body are as follows:
<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">{
 "action": "closed",
 ... ...
 "merge_commit_sha": "f37cb28b1777a28cd34ea1f8df1b7ebcc6c16397",
 ... ...
 "base": {
 "ref": "master",
 ... ...
 },
 ... ...
}
</pre>
  • action represents the details of the current pull request event. action is opened when pull request is created and closed when pull request is closed.
  • merge_commit_sha obtains the id of merge commit;
  • base.ref can obtain which branch the merge request occurs on.

The code and resource file address involved in this article:

Next, we begin to work step by step.

Deployment of Tekton Service

Let's look at creating code to build Task and deploying service Task.

Code build Task:

<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
 name: source-to-image
spec:
 inputs:
 resources:
 - name: git-source
 type: git
 params:
 - name: pathToContext
 description: The path to the build context, used by Kaniko - within the workspace
 default: .
 - name: pathToDockerFile
 description: The path to the dockerfile to build (relative to the context)
 default: Dockerfile
 - name: imageUrl
 description: Url of image repository
 - name: imageTag
 description: Tag to apply to the built image
 default: "latest"
 steps:
 - name: build-and-push
 image: registry.cn-hangzhou.aliyuncs.com/knative-sample/kaniko-project-executor:v0.10.0
 command:
 - /kaniko/executor
 args:
 - --dockerfile=${inputs.params.pathToDockerFile}
 - --destination=${inputs.params.imageUrl}:${inputs.params.imageTag}
 - --context=/workspace/git-source/${inputs.params.pathToContext}
 env:
 - name: DOCKER_CONFIG
 value: /builder/home/.docker
</pre>

Here, service deployment is performed through deployer-deployer, and service Task is deployed:

<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
 name: image-to-deploy
spec:
 inputs:
 resources:
 - name: git-source
 type: git
 params:
 - name: pathToYamlFile
 description: The path to the yaml file to deploy within the git source
 - name: imageUrl
 description: Url of image repository
 - name: imageTag
 description: Tag of the images to be used.
 default: "latest"
 steps:
 - name: deploy
 image: "registry.cn-hangzhou.aliyuncs.com/knative-sample/deployer-deployer:7620096e"
 args:
 - "--namespace=default"
 - "--serivce-name=hello-sample"
 - "--image=${inputs.params.imageUrl}:${inputs.params.imageTag}"
</pre>

In addition, you need to set up the secret of the mirror warehouse:

<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">apiVersion: v1
kind: Secret
metadata:
 name: ack-cr-push-secret
 annotations:
 tekton.dev/docker-0: https://registry.cn-hangzhou.aliyuncs.com
type: kubernetes.io/basic-auth
stringData:
 username: <cleartext non-encoded>
 password: <cleartext non-encoded>
</pre>

Execute the following commands:

<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"># Create Pipeline
kubectl apply -f tekton/pipeline/build-and-deploy-pipeline.yaml
# Create PipelineResource
kubectl apply -f tekton/resources/picalc-git.yaml
# Create image secret
kubectl apply -f tekton/image-secret.yaml
# Create task: soruce to image
kubectl apply -f tekton/tasks/source-to-image.yaml
# Create task: deploy the image to cluster
kubectl apply -f tekton/tasks/image-to-deployer.yaml
</pre>

Deployment of Knative Serving Service

The deployer-github-trigger service is created to receive GitHub events and trigger the Tekton Pipeline build task. The service.yaml is as follows:

<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">apiVersion: serving.knative.dev/v1alpha1
kind: Service
metadata:
 name: deployer-github-trigger
spec:
 template:
 spec:
 containers:
 - image: registry.cn-hangzhou.aliyuncs.com/knative-sample/deployer-trigger:tekton-v1_74647e3a-20190806093544
 args:
 - --trigger-config=/app/config/deployer-trigger.yaml
 volumeMounts:
 - name: config-volume 
 mountPath: /app/config
 serviceAccountName: tekton
 volumes:
 - name: config-volume 
 configMap:
 name: deployer-trigger-config
 items:
 - key: deployer-trigger.yaml
 path: deployer-trigger.yaml
</pre>

Pipeline Run is set here through ConfigMap deployer-trigger-config. deployer-github-trigger can get the latest information of code warehouse according to github Event information, but can not automatically determine the definition of Pipeline Run, so it is necessary to specify a template for Pipeline Run. Trigger specifies the template for Pipeline Run through the -- trigger-config parameter, which reads as follows:

<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">apiVersion: v1
kind: ConfigMap
metadata:
 name: deployer-trigger-config
 namespace: default
data:
 "deployer-trigger.yaml": |-
 apiVersion: tekton.dev/v1alpha1
 kind: PipelineRun
 metadata:
 name: tekton-kn-sample
 spec:
 pipelineRef:
 name: build-and-deploy-pipeline
 resources:
 - name: git-source
 resourceRef:
 name: eventing-tekton-serving-git
 params:
 - name: pathToContext
 value: "src"
 - name: pathToYamlFile
 value: ""
 - name: imageUrl
 value: "registry.cn-hangzhou.aliyuncs.com/knative-sample/eventing-tekton-serving-helloworld"
 - name: imageTag
 value: "1.0"
 trigger:
 type: manual
 serviceAccount: pipeline-account
</pre>

The executive orders are as follows:

<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"># Create clusterrole
kubectl apply -f serving/clusterrole.yaml
# Create clusterrolebinding
kubectl apply -f serving/clusterrolebinding.yaml
# Create serviceaccount
kubectl apply -f serving/serviceaccount.yaml
# Create configmap
kubectl apply -f serving/configmap.yaml
# Create service
kubectl apply -f serving/service.yaml
</pre>

Configure GitHub Event Source in Eventing

The code merge request triggers the corresponding event and sends the event directly to the deployer-github-trigger service after it is retrieved through Knative Eventing.

Create GitHub Token

Create Personal access tokens to access the GitHub API. In addition, your code will use it to validate the incoming webhook (secret token) from github. The name of token can be set arbitrarily. Source needs to open repo:public_repo and admin:repo_hook to trigger Event events through public warehouses and create webhooks for these public warehouses.

Below is an example of setting up a "GitHubSource Sample" token.

Update the githubsecret.yaml content. If personal_access_token_value token is generated, secretToken needs to be set as follows:

<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">apiVersion: v1
kind: Secret
metadata:
 name: githubsecret
type: Opaque
stringData:
 accessToken: personal_access_token_value
 secretToken: asdfasfdsaf
</pre>

Execute the order to make it effective:

<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">kubectl apply -f eventing/githubsecret.yaml
</pre>

Create GitHub event source

To receive events generated by GitHub, you need to create GitHubSource to receive events.

<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">apiVersion: sources.eventing.knative.dev/v1alpha1
kind: GitHubSource
metadata:
 name: deployer-github-sources
spec:
 eventTypes:
 - pull_request
 ownerAndRepository: knative-sample/eventing-tekton-serving
 accessToken:
 secretKeyRef:
 name: githubsecret
 key: accessToken
 secretToken:
 secretKeyRef:
 name: githubsecret
 key: secretToken
 sink:
 apiVersion: serving.knative.dev/v1alpha1
 kind: Service
 name: deployer-github-trigger
</pre>

Key field interpretation:

  • Specify the github repository: ownerAndRepository: knative-sample/eventing-tekton-service for listening https://github.com/knative-sample/eventing-tekton-serving Warehouse events;
  • Event types: EvetTypes is an array in which you can configure a list of github events.
  • Authentication information: access Token and secretToken refer to the authentication information of github warehouse by secret;
  • The target Service: sink field indicates which service the received event needs to be sent to, which is sent directly to the deployer-github-trigger service defined earlier.

Execute the kubectl command:

<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">kubectl apply -f eventing/github-source.yaml
</pre>

If Istio injection is enabled in the cluster, egress access needs to be enabled:

<pre style="-webkit-tap-highlight-color: transparent; box-sizing: border-box; font-family: Consolas, Menlo, Courier, monospace; font-size: 16px; white-space: pre-wrap; position: relative; line-height: 1.5; color: rgb(153, 153, 153); margin: 1em 0px; padding: 12px 10px; background: rgb(244, 245, 246); border: 1px solid rgb(232, 232, 232); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">kubectl apply -f eventing/egress.yaml
</pre>

After deployer-github-sources are submitted to Kubernetes, the github source controller will http://github.com/knative-sample/eventing-tekton-serving Next, create a webhook, and the callback address is our github_receive_adapter service public network address.

When http://github.com/knative-sample/eventing-tekton-serving When a pull request occurs, the deployer-github-trigger will automatically trigger the execution of the deployer-github-trigger. The deployer-github-trigger compiles the image first, and then updates the hello-sample service image to complete the automatic publishing.

Code - > Mirror - > Service

Let's show you how to automate the build and deployment process from code to service:

Service Access Experience Address: http://hello-sample.default.serverless.kuberun.com

conclusion

From code to service, does Knative give you a different experience through the examples above? Hopefully, Knative will make it easier for you to build code and deploy services so that you can focus more on the business itself. Welcome to share with Knative.

Welcome to Knative Exchange Group

Author: A Green Boat
Read the original text
This article is the original content of Yunqi Community, which can not be reproduced without permission.

Topics: github git Kubernetes Docker