Description of continuous integration process
1) First, developers submit code every day to the Git repository
2) Then, as a continuous integration tool, Jenkins uses Git tool to pull the code from Git warehouse to the integration server, and then cooperates with JDK, Maven and other software to complete code compilation, code testing and review, testing, packaging and other work. If there is an error in each step in the process, the whole process will be executed again.
3) Finally, Jenkins distributes the generated jar or war package to the test server or production server, and testers or users can access the application.
Server list
CentOS7 is adopted for virtual machines.
Gitlab code managed server installation
Introduction to Gitlab
Official website: https://about.gitlab.com/
GitLab is an open source project for warehouse management system. It uses Git as a code management tool and builds a web service on this basis.
GitLab, like GitHub, is a third-party Git based work. It is free and open source (based on MIT protocol). Similar to GitHub, GitLab can register users, submit your code arbitrarily, add SSHKey, etc. the difference is,
GitLab can be deployed to its own server, and all information such as database is in its own hands. It is suitable for collaborative development within the team,
You can't always put the wisdom within the team on someone else's server, can you?
In short, GitLab can be regarded as a personal version of GitHub.
Gitlab installation
- Installation related dependencies – postfix
yum -y install policycoreutils openssh-server openssh-clients postfix
2. Start ssh Service & set to startup
systemctl enable sshd && sudo systemctl start sshd
- Set postfix to start automatically and start. Postfix supports gitlab sending function
systemctl enable postfix
- Open ssh and http services, and then reload the firewall list (if the firewall is closed, the following configuration is not required)
firewall-cmd --add-service=ssh --permanent firewall-cmd --add-service=http --permanent firewall-cmd --reload
- Download the gitlab package and install it
Download the installation package online:
wget https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el6/gitlab-ce-12.4.2-ce.0.el6.x86_64.rpm
Link: https://pan.baidu.com/s/1OaP_x1DaovJLuc518zurqQ
Extraction code: yyds
Upload the downloaded installation package to the server for installation (it takes about 1-2 minutes):
rpm -ivh gitlab-ce-12.4.2-ce.0.el6.x86_64.rpm
6. Modify gitlab configuration
You can find from the console, / etc / gitlab / gitlab RB file, we need to modify the url of gitlab external access
vi /etc/gitlab/gitlab.rb
Modify the gitlab access address (the IP address of the currently installed gitlab server) and port. The default is 80. We change it to 82
external_url 'http://192.168.10.11:82' nginx['listen_port'] = 82
7. Overload configuration and start gitlab (about 3-4 minutes)
gitlab-ctl reconfigure gitlab-ctl restart
8. Add the port to the firewall (if the firewall is closed, the following commands do not need to be executed by default)
firewall-cmd --zone=public --add-port=82/tcp --permanent firewall-cmd --reload
After successful startup, access: 192.168 10.11:82
The above interface appears. Let's wait a minute
See the following page to modify the administrator root password. After modifying the password (the password is determined by yourself), then log in. I modify it here as root123456
Log in
Gitlab add groups, create users, create projects
We use the super management account to add groups, create users and create projects
1) Create group
Use administrator root to create a group. There can be multiple project branches in a group. You can add development to the group to set permissions. Different groups are different development projects or service modules of the company. Adding different development to different groups can realize the management of development setting permissions
We can be in zhgd_group create an item
In that case, we'll be in zhgd_ Create a web group under group_ Demo project
Next, we can create a user zhngsan and assign this user to this zhgd_ Under group
2) Create user
When creating a user, you can select Regular or Admin type.
After creating the user, change the password immediately
3) Add user to group
Select a user group to manage the Members of the group
Gitlab users have five different permissions in the group:
Guest: Can create issue,Comment, can't read or write version Library Reporter: Code can be cloned, but not submitted, QA,PM You can give this permission Developer: You can clone code, develop, submit push，Ordinary developers can grant this permission Maintainer: You can create projects, add tag,Protect branches, add project members, and edit projects. Core development can grant this permission Owner: You can set project access permissions - Visibility Level,The development team leader can grant this permission to delete projects, migrate projects and manage team members
4) Create an item in a user group
Log in to Gitlab as the new user you just created, and then create a new project in the user group
Note that after creating a user and logging in for the first time, Gitlab will still pop up an interface for the user to change the password. We can enter the original set password without changing it
Upload the source code to Gitlab warehouse
Now come to the IDEA development tool. We have prepared a simple Web application for integrated deployment. We will upload the source code to Gitlab's project warehouse.
1) Project structure description
We built a very simple web application with only one index JSP page, if deployed, you can access the page, and it will be successful!
2) Turn on version control
2) Submit code to local warehouse
Add to cache first
Then Commit to the local warehouse
3) Push to Gitlab project warehouse
Enter the user name and password of gitlab, and then you can push the code to the remote warehouse
Refresh gitlab project
For the above operations, we will install gitLab in 192.168.0 On the virtual machine of 10.11, and successfully submit the local project to gitLab
Next we're at 192.168 Install continuous integration environment jenkins on the virtual machine of 10.129
1) Install JDK
Jenkins needs to rely on JDK, so install jdk1 first eight
For the installation process, please refer to the blog:
https://blog.csdn.net/qq_44316726/article/details/116034329
2) Get jenkins installation package
Download page: https://jenkins.io/zh/download/
Installation file: jenkins-2.190 3-1.1. noarch. rpm
Due to the slow access to jenkins official website, baidu online disk resources are provided
Link: https://pan.baidu.com/s/1Ab-tst8oDoXFU_XkHQkbxQ
Extraction code: yyds
3) Upload the installation package to 192.168 10.129 the soft directory of the server for installation
rpm -ivh jenkins-2.190.3-1.1.noarch.rpm
4) Modify Jenkins configuration
vi /etc/syscofig/jenkins
Modify content account and default port:
JENKINS_USER="root" JENKINS_PORT="8888"
5) Start Jenkins
systemctl start jenkins
If the following contents are prompted during startup, it indicates that startup fails
Job for jenkins.service failed because the control process exited with error code. See "systemctl status jenkins.service" and "journalctl -xe" for details.
You can view the blog:
https://blog.csdn.net/weixin_40816738/article/details/91391925
resolvent:
We enter the command according to the prompt to view the failure reason:
systemctl status jenkins.service
We can see that jenkins didn't find the java file. He needs to copy the java file address himself
vi /etc/init.d/jenkins
Manually enter the java path in the corresponding server (this path is the path to install the jdk, and the suffix is / bin/java):
/usr/local/java/jdk1.8.0_171/bin/java
systemctl daemon-reload
Then restart jenkis
systemctl start jenkins
Next, we start jenkins again and we can succeed.
6) Open browser access
http://192.168.10.129:8888
Note: this server has turned off the firewall. If you turn on the firewall, you need to add a port to the firewall
Prompt: we need to find the account and password of admin after jenkins is installed in the specified directory
The password is:
e0701266541149ce88b64d79b5750e25
7) Obtain and enter the admin account password
cat /var/lib/jenkins/secrets/initialAdminPassword
8) Skip plug-in installation
Because the Jenkins plug-in needs to be downloaded from the default official website, the speed is very slow and will fail, so we skip the plug-in installation for the time being
9) Add an administrator account and enter Jenkins background
User name and password: wusong/123456
Start using Jenkins
Continuous integration environment (2)-Jenkins plug-in management
Jenkins itself does not provide many functions. We can meet our needs by using plug-ins. For example, functions such as pulling code from Gitlab and building projects using Maven need to be completed by plug-ins. Next, show me how to download the plug-in.
Modify the Jenkins plug-in download address
The download speed of Jenkins's foreign official plug-in address is very slow, so it can be modified to the domestic plug-in address:
Jenkins - > Manage Jenkins - > Manage plugins, click Available
This is to download the Jenkins official plug-in list locally, and then modify the address file to replace it with the domestic plug-in address
cd /var/lib/jenkins/updates
default. The JSON file records the addresses of all plug-ins downloaded from the official website, but these addresses are foreign and the download speed is very slow. We modify these foreign addresses to domestic download addresses
sed -i 's/http:\/\/updates.jenkins-ci.org\/download/https:\/\/mirrors.tuna.tsinghua.edu.cn\/jenkins/g' default.json && sed -i 's/http:\/\/www.google.com/https:\/\/www.baidu.com/g' default.json
Finally, click Manage Plugins Advanced and change the Update Site to the download address of domestic plug-ins
https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json
After Sumbit, enter in the browser: http://192.168.10.129:8888/restart , restart Jenkins.
Download Chinese plugin
Jenkins - > Manage Jenkins - > Manage plugins, click Available and search "Chinese"
After completion, see the figure below:
When checked, refresh the page
After restarting Jenkins, you will see the Sinicization of Jenkins! (PS: but the Chinese translation of some menus may fail)
Continuous integration environment (3)-Jenkins user rights management
We can use the Role-based Authorization Strategy plug-in to manage Jenkins user permissions
Install the Role-based Authorization Strategy plug-in. Before installing this plug-in, it is recommended to update Jenkins to the latest version, otherwise the installation of Role-based Authorization Strategy plug-in may fail
Enable permission global security configuration
Switch the authorization policy to "role based strategy" and save it
Create role
On the system management page, enter Manage and Assign Roles
Click "Manage Roles"
Global roles: advanced users such as administrators can create global based roles
Item roles: roles for one or more items
Node roles: node related permissions
We add the following three roles:
baseRole: This role is global. This role needs binding Overall Below Read Permission is to bind the most basic permissions to all users Jenkins Access rights. Note: if this role is not bound to subsequent users, an error will be reported: user name is missing the Overall/Read permission role1: This role is a project role. Binding using regular expressions"itcast.*"，It means role role1 Can access operations itcast Items at the beginning. role2: This role is also a project role. binding"itheima.*"，It means role role2 Can access operations itheima Items at the beginning.
Create baseRole global role
Create two project roles, role1 and role2
preservation.
Create user
On the system management page, enter Manage Users
Create two users: jack and eric. Their passwords are 123456
Assign roles to users
The two users created above do not have any roles, that is, they do not have any permissions. After we log in to jenkins with an account, there are no resources to access the page.
On the system management page, enter Manage and Assign Roles and click Assign Roles
The binding rules are as follows:
eric Users bind separately baseRole and role1 role jack Users bind separately baseRole and role2 role
preservation.
Create project test permissions
Create two projects with the wusong administrator account, itcast01 and itheima01
The result is: eric When users log in, they can only see itcast01 project jack When users log in, they can only see itheima01 project
Continuous integration environment (4)-Jenkins credential management
Credentials can be used to store database passwords, Gitlab password information and Docker private warehouse passwords that need ciphertext protection, so that Jenkins can interact with these third-party applications.
Install the Credentials Binding plug-in
To use the credential management feature in Jenkins, you need to install the Credentials Binding plug-in
After installing the plug-in, the "voucher" menu is added to manage all vouchers here
There are five kinds of vouchers that can be added:
Username with password: User name and password SSH Username with private key: use SSH User and key Secret file: Text files that need to be kept confidential when used Jenkins The file will be copied to a temporary directory, and then the file path will be set to a variable. After the construction is completed, the copied Secret file Will be deleted. Secret text: An encrypted text string that needs to be saved, such as a nailing robot or Github of api token Certificate: By uploading the certificate file
Common credential types are Username with password and SSH Username with private key
Next, take the Git tool to pull the project source code from Gitlab as an example to demonstrate how Jenkins manages Gitlab credentials.
Install Git plug-ins and Git tools
In order for Jenkins to support pulling source code from Gitlab, you need to install Git plug-ins and Git tools on CentOS7.
Git plug-in installation. It is recommended to restart Jenkins after installation:
To install Git tools on CentOS7:
install yum install git -y View version after installation git --version
User password type
1) Create voucher
Jenkins - > voucher - > System - > Global voucher - > Add voucher
Select "Username with password", enter Gitlab's user name and password, and click "OK".
2) Test whether the voucher is available
Select the itcast01 project you created earlier
Find "source management" - > "Git", and copy the project URL in Gitlab in the Repository URL
At this time, an error will be reported that the warehouse cannot be connected! If you select the newly added Credentials in Credentials, no error will be reported
After saving the configuration, click "Build Now" to start building the project
Check the / var/lib/jenkins/workspace / directory and find that the code has been successfully pulled from Gitlab to Jenkins.
SSH key type
SSH password free login diagram
1) Generate public and private keys using root
ssh-keygen -t rsa
At / root / The SSH / directory holds the public key and usage
id_rsa: private key file
id_rsa.pub: public key file
2) Put the generated public key in Gitlab
Log in as root - > Click Avatar - > Settings - > SSH keys
Copy ID_ rsa. Here is the content of the pub file. Click "Add Key"
3) Add credentials in Jenkins and configure the private key
Add a new certificate in Jenkins with the type of "SSH Username with private key" and copy the contents of the private file just generated
4) Test whether the voucher is available
Create a new "test02" project - > source management - > GIT. This time, use Gitlab's SSH connection and select SSH credentials
Similarly, try to build the project. If the code can be pulled normally, it means that the voucher configuration is successful!
Continuous integration environment (5)-Maven installation and configuration
On the Jenkins integration server, we need to install Maven to compile and package the project.
Install Maven
First upload Maven software to 192.168 10.129 under the root/soft directory on the server
Link: https://pan.baidu.com/s/1zW2BISo1_PAb4NUmlxkIqg
Extraction code: yyds
Unzip to / usr/local/
tar -xzf apache-maven-3.6.2-bin.tar.gz -C /usr/local/
Configure environment variables
vi /etc/profile
//jdk environment variable export JAVA_HOME=/usr/local/java/jdk1.8.0_171 export JRE_HOME=$JAVA_HOME/jre export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH //$PATH refers to the previously configured command export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/lib:$MAVEN_HOME/bin //maven environment variable export MAVEN_HOME=/usr/local/apache-maven-3.6.2
Configuration effective
source /etc/profile
Find Maven version
mvn -v
Global tool configuration is associated with JDK and Maven
Jenkins - > global tool configuration - > JDK - > JDK is added. The configuration is as follows:
Jenkins - > global tool configuration - > Maven - > Add Maven. The configuration is as follows:
Add Jenkins global variable
Manage Jenkins - > configure system - > global properties, add three global variables JAVA_HOME,M2_HOME,PATH+EXTRA
To make Jenkins aware of jdk and maven commands
Modify Maven's settings xml
Create local warehouse directory
mkdir /root/repo
Modify Maven's settings xml
vi /usr/local/apache-maven-3.6.2/conf/settings.xml
Change the local warehouse to: / root/repo/
Add alicloud private server address:
<mirror>
<id>alimaven</id>
<mirrorOf>central</mirrorOf>
<name>aliyun maven</name>
<url>http://maven.aliyun.com/nexus/content/groups/public/</url>
</mirror>
Test whether Maven is successfully configured
Previously, we only pulled the code from gitLab on jenkins, and saw from the console log that the code was successfully pulled to the server. Next, we configured maven to compile and package the pulled code
Use the previous test02 test project to modify the configuration
Build - > add build steps - > execute shell
input
mvn clean package
Build again. If you can turn the project into a war package, it means that the maven environment configuration is successful!
You can see that the dependencies downloaded from the central warehouse are saved in the local warehouse
Continuous integration environment (6)-Tomcat installation and configuration
Install tomcat8 five
Upload the Tomcat compressed package to 192.168 10.102 server
Extract it to / / usr/local directory
tar -xzvf apache-tomcat-8.5.47.tar.gz -C /usr/local
Start Tomcat (must depend on jdk environment)
/usr/local/apache-tomcat-8.5.47/bin/startup.sh
Note: the server has turned off the firewall, so you can directly access Tomcat
The address is:
http://192.168.10.102:8080
Because Jenkins and Tomcat are on different servers, Jenkins should have permission to connect
Tomcat server for remote transmission, so Jenkins is required to use Tomcat account to realize remote transmission and deployment
Configure Tomcat user role permissions
By default, Tomcat does not have the permission to configure user roles
However, the subsequent Jenkins deployment project to the Tomcat server requires Tomcat users, so modify the following configuration of Tomcat and add users and permissions
vim /usr/local/apache-tomcat-8.5.47/conf/tomcat-users.xml
The contents are as follows:
<role rolename="tomcat"/> <role rolename="role1"/> <role rolename="manager-script"/> <role rolename="manager-gui"/> <role rolename="manager-status"/> <role rolename="admin-gui"/> <role rolename="admin-script"/> <user username="tomcat" password="tomcat" roles="manager-gui,manager-script,tomcat,admin-gui,admin-script"/>
The user and password are: tomcat
Note: in order to log in to Tomcat for the user just configured, the following configuration needs to be modified
vim /usr/local/apache-tomcat-8.5.47/webapps/manager/META-INF/context.xml
Restart Tomcat and access the test
stop it
/usr/local/apache-tomcat-8.5.47/bin/shutdown.sh
start-up
/usr/local/apache-tomcat-8.5.47/bin/startup.sh
visit: http://192.168.10.102:8080/manager/html ，
Enter Tomcat and tomcat, and the following page represents success
As can be seen from the above, for the projects under the webapps directory of tomcat, subsequent jenkins can use the Tomcat account to manage the projects under the webapps directory to realize the remote publishing function of the project
3. Jenkins builds Maven project
Jenkins project construction type (1) - Introduction to project types built by Jenkins
There are many types of automatic construction projects in Jenkins, including the following three types:
Free style software project( FreeStyle Project) Maven Project( Maven Project) Pipeline project( Pipeline Project)
In fact, each type of construction can complete the same construction process and results, but there are differences in operation mode and flexibility, which can be selected according to their own needs and habits in actual development. (PS: pipeline type is recommended because of its high flexibility)
Jenkins project construction type (2) - free style project construction
The following shows how to create a free style project to complete the integration process of the project:
Pull code->compile->pack->deploy
Pull code
1) Create project
2) Configure source code management and pull the code from gitlab
Compile package
Configure - > build - > add build steps - > executor shell
echo "Start compiling and packaging" mvn clean package echo "End of compilation and packaging"
Apply - > save
structure
Deployment (remotely deploy the war package on 129 server to 120 server)
Deploy the project to remote Tomcat
1) Install the Deploy to container plug-in
Jenkins itself cannot realize the function of remote deployment to Tomcat, so it needs to install the Deploy to container plug-in
2) Add post build actions
3) Add Tomcat user credentials
Click "Build Now" to start the build deployment process
4) After successful deployment, access the project
Click item
Demonstrate continuous integration after code changes
1) Modify the source code in IDEA and submit it to gitlab
Push to remote gitLab server
2) Project rebuild in Jenkins
3) Access Tomcat
Jenkins project construction type (3)-Maven project construction
1) Install Maven Integration plug-in (if the download fails, query the download)
2) Create Maven project
3) Configuration item
The process of pulling code and remote deployment is the same as that of free style projects, except that the "build" part is different
Then it's time to rebuild the project
Jenkins project construction type (4)-Pipeline pipeline project construction (*)
Pipeline introduction
1)concept Pipeline，In short, it is a set of running in Jenkins The workflow framework connects the tasks that originally run independently on a single or multiple nodes to realize the complex process arrangement and visualization that is difficult to complete by a single task. 2)use Pipeline There are the following benefits (from translation from official documents): code: Pipeline Implemented in code, it is usually checked into source code control, enabling the team to edit, review and iterate its delivery process. Persistence: whether planned or unplanned server restart, Pipeline Are recoverable. Stoppable: Pipeline Interactive input can be received to determine whether to continue execution Pipeline. multi-function: Pipeline Support complex continuous delivery requirements in the real world. It supports fork/join,Circular execution, the function of executing tasks in parallel. Scalable: Pipeline The plug-in supports its DSL And multiple options for integration with other plug-ins. 3)How to create Jenkins Pipeline And? Pipeline Scripts are created by Groovy Language implementation, but we don't need to learn it alone Groovy Pipeline Two syntax are supported: Declarative(Declarative)and Scripted Pipeline(Scripted)grammar Pipeline There are also two creation methods: you can create them directly in the Jenkins of Web UI Input script in the interface; You can also create a Jenkinsfile Put the script file into the project source code library (generally we recommend Jenkins Directly from source control(SCM)Load directly in Jenkinsfile Pipeline This method).
Installing the Pipeline plug-in
Manage Jenkins - > Manage plugins - > optional plug-in pipeline (reinstall if installation fails)
After installing the plug-in, the "pipeline" type is added when creating the project
Pipeline syntax quick start
1) Declarative declarative Pipeline
Create project
Pipeline - > select HelloWorld template
The generated content is as follows:
pipeline {
agent any
stages {
stage('Hello') {
steps {
echo 'Hello World'
}
}
}
}
stages: Represents all execution stages of the entire pipeline. usually stages There is only one, including multiple stage stage: Represents a stage in the pipeline, which may occur n One. It is generally divided into pulling code, compiling, building, deployment and other stages. steps: Represents the logic to be executed in a phase. steps Inside shell script, git Pull code, ssh Any content such as remote publishing.
Write a simple declarative Pipeline:
pipeline {
agent any
stages {
stage('Pull code') {
steps {
echo 'Pull code'
}
}
stage('Compile build') {
steps {
echo 'Compile build'
}
}
stage('Project deployment') {
steps {
echo 'Project deployment'
}
}
}
}
Click build to see the whole construction process
2) Scripted Pipeline
Create project
Select "Scripted Pipeline" this time
Node: Node, one Node Just one Jenkins Node, Master perhaps Agent，It's execution Step The specific operating environment will be described later Jenkins of Master-Slave Architecture. Stage: Stage, one Pipeline It can be divided into several Stage，each Stage Represents a set of operations, such as: Build,Test,Deploy，Stage Is a concept of logical grouping. Step: Step, Step It is the most basic operation unit. It can print a sentence or build one Docker Mirror image, by various Jenkins The plug-in provides commands such as: sh 'make'，It's equivalent to our usual shell Execute in terminal make Same command.
Write a simple scripted Pipeline
node {
def mvnHome
stage('Pull code') {
echo 'Pull code'
}
stage('Compile build') {
echo 'Compile build'
}
stage('Project deployment') {
echo 'Project deployment'
}
}
The build result is the same as the declarative one!
Pull code (with pipeline syntax)
Compile package
mvn clean package
Rebuild
deploy
Rebuild access project
Pipeline Script from SCM
Just now, we all wrote Pipeline code directly on the UI interface of Jenkins, which is inconvenient for script maintenance and version control. It is recommended to put Pipeline scripts in the project (version control together)
1) Create a Jenkinsfile file in the root directory of the project and copy the contents into the file
And push the script file submission to gitlab
2) Reference the file in the project
Rebuild project test successful