Article background: due to network reasons, accessing Github is often unstable. We are going to deploy a set of Gogs service based on Docker in the company to regularly synchronize projects on Github.
The company's network environment cannot directly use the original construction process in the gogs project. At the same time, in order to save the time spent in downloading the dependent package during the construction process, consider dismantling the construction steps of the compilation environment and operation in the Dockerfile, making two images and putting them in the private warehouse, and then building the gogs image based on the created image. This paper records this process in detail.
The goal of Gogs is to build a self-service Git service in the simplest, fastest and easiest way. Using Go language development makes Gogs
It can be distributed through independent binary and supports all platforms supported by Go language, including Linux, Mac OS X, Windows and ARM platforms.
----Gogs official website: https://gogs.io/docs/
Official documents say that it is very convenient to use binary installation. You only need to download the compiled binary file and start the service.
According to the principle of step-by-step, first install with binary in windows and preview to see what functions gogs has and whether the installation is really simple.
1, Install gogs under window
- Download compressed package
Find the download address of the installation package corresponding to the Windows system on the Internet
- Unzip the compressed package to get the following file contents:
3. Press win+R to enter cmd to open the command line, and cd enters the gogs directory.
For example, my gogs directory path is D:\Program Files\gogs. First execute D: enter to disk D, and then enter cd D:\Program Files\gogs to enter the gogs directory
4. Start the gogs service and execute the command gogs web:
Seeing the above prompt indicates that gogs is started successfully. Open the browser and visit locahost:3000 to see the page of gogs.
Follow up the installation steps according to the prompts on the page, mainly some configuration contents. Click Install immediately after configuration.
If the administrator account is configured, you will automatically log in to the administrator account after installation. Otherwise, you will jump to the login page and the first registered user will be the administrator account.
For the convenience of installation, SQLite3 can be directly used in the database, so there is no need to install another database application.
That's all for the installation steps on windows, mainly to see what application gogs is and what functions it has.
2, Install from the source code under windos
1. Prepare the environment first
This machine is win10 and needs to install go language.
a. Install GO language reference documents: http://c.biancheng.net/view/3992.html , I won't introduce it in detail here.
Go language download address: official https://golang.google.cn/dl/, China Golang community https://studygolang.com/dl
b. Install git: Download address: https://git-scm.com/
2. gogs cloning project
Execute instructions in bash: git clone -- depth 1 https://github.com/gogs/gogs.git gogs
3. Compile the main program
Execute go build -o gogs to compile the main program
This step will download all dependencies. Under normal conditions, you can see that the output content of the dependent package is being downloaded:
There may also be reaction after instruction execution. There are two reasons for this problem:
a. Downloading go module is slow;
b. GCC is not installed correctly.
- Download speed is slow. Set go agent to solve the problem:
Execute go env to view the go configuration and modify it to Seven cattle cloud https://goproxy.cn/ Agent provided:go env -w GO111MODULE=on go env -w GOPROXY=https://goproxy.cn,direct
-
gcc is not installed correctly.
gcc is not installed and an error is reported:cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in %PATH%
Incorrect gcc version and error report:
cc1.exe: sorry, unimplemented: 64-bit mode not compiled in
Installation GCC reference documents: https://blog.csdn.net/xia_2017/article/details/105545789
Download address: https://sourceforge.net/projects/mingw-w64/files/mingw-w64/mingw-w64-release/
Download 64 bit x86_ 64 POSIX sEH compressed package. After decompression, configure the bin directory into the path of the environment variable. Execute gcc -v to see the gcc information:
Try to recompile after the above configuration.
Using the go env command, you can see the GOMODCACHE=C:\Users\Administrator\go\pkg\mod variable. The modules downloaded during build are saved in this directory.
After compilation, gogs.com will be generated in the gogs directory Exe, execute the command gogs web in bash to start the service. The effect is the same as that of the binary file downloaded before. You can see the web page by visiting it.
3, Make docker image
I already know how to compile under windows. Next, I'll change to docker and run this process again, because the ultimate goal is to compile and release a self image according to the source code for running in docker.
Using Docker has many advantages and can avoid various environmental problems.
1. Install Docker:
win10 system, download and install the application package on the official website, which is similar to the installation of ordinary applications. There are many detailed documents on the Internet.
After installing Docker, you can see the version information by executing docker -v, which indicates that the installation is successful.
Open the Dockerfile file in the Gogs directory to view its contents, and manually operate it in the container once according to the steps of Dockerfile.
FROM golang:alpine3.14 AS binarybuilder # Based on golang: alpine3 Follow up step 14
RUN apk --no-cache --no-progress add --virtual \ # Use apk package manager to install some tools needed for compilation
build-deps \
build-base \
git \
linux-pam-dev
WORKDIR /gogs.io/gogs # Enter directory
COPY . . # Copy the contents of the current directory in the host to the container
RUN make build TAGS="cert pam" # Start building.
FROM alpine:3.14
RUN if [ `uname -m` == "aarch64" ] ; then \
export arch="arm64" ; \
elif [ `uname -m` == "armv7l" ] ; then \
export arch="armhf"; \
else \
export arch="amd64" ; \
fi \
&& wget https://github.com/tianon/gosu/releases/download/1.11/gosu-$arch -O /usr/sbin/gosu \
&& chmod +x /usr/sbin/gosu \
&& echo http://dl-2.alpinelinux.org/alpine/edge/community/ >> /etc/apk/repositories \
&& apk --no-cache --no-progress add \
bash \
ca-certificates \
curl \
git \
linux-pam \
openssh \
s6 \
shadow \
socat \
tzdata \
rsync
ENV GOGS_CUSTOM /data/gogs
# Configure LibC Name Service
COPY docker/nsswitch.conf /etc/nsswitch.conf
WORKDIR /app/gogs
COPY docker ./docker
COPY --from=binarybuilder /gogs.io/gogs/gogs .
RUN ./docker/finalize.sh
# Configure Docker Container
VOLUME ["/data", "/backup"]
EXPOSE 22 3000
HEALTHCHECK CMD (curl -o /dev/null -sS http://localhost:3000/healthcheck) || exit 1
ENTRYPOINT ["/app/gogs/docker/start.sh"]
CMD ["/bin/s6-svscan", "/app/gogs/docker/s6/"]
2. Prepare the compilation environment golang: alpine3 fourteen
The process of 1 ~ 9 lines is actually preparing the compilation environment, and then compiling the source code.
It's the same thing as what you do on your local computer.
2.1 running a basic image
Execute the first line in Dockerfile from golang: alpine3 14 as binary builder, the content of this line is based on golang: alpine3 14. This image executes the following instructions. For manual execution, you also need to run a container using this basic image, and then you can carry out the following operations based on this container. Execute the instructions manually:
docker run -it --name golang golang:alpine3.14
Note: after the docker image is started, if there is no process in the container, it will exit automatically. Subsequently, instructions will be executed to install the basic application software, so the container cannot exit. With the - it parameter, it can automatically enter the container after the container is started, so as to prevent the container from exiting. The current directory displayed after entering the container is / go.
You can also use the - itd parameter to start the container in the background. At this time, to enter a container, you need to use the instruction docker attach ContainerID.
Instructions for viewing ContainerID: docker ps view running containers, docker ps -a view all containers.
2.2 install the software in the container of the basic image
2.2.1 configure apk source
Set the pak software source and execute the instruction to quickly transfer the data in the configuration file / etc/apk/repositories https://dl-cdn.alpinelinux.org Replace with http://mirrors.ustc.edu.cn/alpine
sed -i 's#https://dl-cdn.alpinelinux.org#http://mirrors.ustc.edu.cn/alpine#g' /etc/apk/repositories
You can also manually set the source in the configuration file to another available source https://mirrors.alpinelinux.org/ Other available sources can be found in.
2.2.2 installation foundation application
Execute lines 2 to 6 in Dockerfile, which is actually an instruction.
In the Dockerfile, use \ to wrap the line, and write APK -- no cache -- no progress add -- Virtual build DEPs build base git Linux PAM dev.
After installation, you can use apk info to see the list of installed applications. The instructions should be able to work normally. If they cannot work normally, you must reinstall them. Otherwise, subsequent operations based on the problematic image will report errors and it is difficult to troubleshoot.
2.2.4 packaging basic image
After the software in the container is installed, the basic environment will be ready. You can package it into an image. In the future, you need to compile the gogs program and directly use the image with the basic environment.
Instructions needed
docker ps # View the ID of the container docker commit -m "" CONTAINERID TAG # Packaging containers into images
All instructions are performed in the host.
At this point, the compilation environment image of gogs is completed
2.2.5 apk software installation failure problem solving
apk installation software may encounter errors (if no errors are encountered, they can be ignored directly)
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.14/main: temporary error (try again later)
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.14/community: temporary error (try again later)
ERROR: unable to select packages:
build-base (no such package):
required by: build-deps-20220305.081134[build-base]
git (no such package):
required by: build-deps-20220305.081134[git]
linux-pam-dev (no such package):
required by: build-deps-20220305.081134[linux-pam-dev]
It can be seen that the execution reported an error but failed because apk did not find the corresponding software package in the warehouse. You need to modify the source of apk to install successfully.
The source configuration of apk is in the file / etc/apk/repositories. The current configuration is:
https://dl-cdn.alpinelinux.org/alpine/v3.14/main https://dl-cdn.alpinelinux.org/alpine/v3.14/community
here https://mirrors.alpinelinux.org/ Find other available sources, modify the configuration file and re execute the installation instruction.
Execute command sed - I's# https://dl-cdn.alpinelinux.org#http://mirrors.ustc.edu.cn/alpine#g / etc / APK / repositories https://dl-cdn.alpinelinux.org Replace with http://mirrors.ustc.edu.cn/alpine , which can also be modified manually.
https doesn't work. You can try http instead.
At this point, the compilation environment image of gogs is completed. In the future, if the code is modified, you can directly use the produced image. In it, you can use the make build instruction to compile the program directly without downloading the required software first. At this time, you can realize the convenience of docker.
3. Compile gogs to test the availability of basic image
If there is no problem with the image just made, it has the environment to compile gogs. Start this image and try to compile the source code of gogs to see whether it can be compiled successfully.
Lines 8, 9 and 10 of Dockerfile are compilation instructions
WORKDIR /gogs.io/gogs # Enter directory COPY . . # Copy the contents of the current directory in the host to the container RUN make build TAGS="cert pam" # Start building
The WORKDIR in Dockerfie is similar to executing the cd instruction in the container to switch the directory. If the directory does not exist, it will be created automatically.
COPY copies the files in the host to the container. When the Dockerfile file builds the container, the context directory is the directory where the Dockerfile file is located. Therefore, COPY This line is to COPY all the contents of the current directory in the host to the container.
In the original Dockerfile, the construction of the two stages is written in the same file. The files in the container of the previous stage can be copied to the container of the later stage through the COPY --from instruction, - FROM followed by the alias of the container of the previous stage, that is, the name after the FROM instruction line AS of the previous stage.
3.1 running a container with a compilation environment
During manual execution, we directly map the file directory from the host instead of the copy file. When starting the container, we use the - v parameter to map the host directory to the WORKDIR directory of the container. Therefore, we use the image just made to start a container and execute as follows:
docker run -it --name test-build-gogs -it -v "E:\git\gogs:/gogs.io/gogs" golang:buildEnvironment-test
After startup, enter the container, and you can see the files in the directory of the host, and the files modified in the host can also see the changes in the container:
Directory mapping succeeded.
3.2 setting go module agent
ps: the reason why the agent is not configured when creating the basic environment is that the agents required to use the image in different environments may be different. Therefore, configure the agent when using the image. This step can be omitted when the network is normal.
Execute instruction
go env -w GO111MODULE=on go env -w GOPROXY=https://goproxy.cn,direct
3.3 compilation
Run the 10th line instruction of Dockerfie
make build TAGS="cert pam"
If build doesn't respond, set the go agent and try again.
Finally, if the compilation is successful, the file named gogs will be output, which can also be seen in the host:
Here, the image test of the compilation environment of gogs has passed. You can push the basic image to the warehouse and publish it for everyone to use.
4. Make the operation environment of gogs
Why make so many images? Because the compilation environment of the software is very complex. After the software is compiled, the compilation environment is useless in the process of software operation. Therefore, it is OK to make a minimum environment required for software operation separately, so that the efficiency can be improved and the waste of resources can be avoided when the images are distributed.
Dockerfile lines 12-14:
FROM alpine:3.14
RUN if [ `uname -m` == "aarch64" ] ; then \
export arch="arm64" ; \
elif [ `uname -m` == "armv7l" ] ; then \
export arch="armhf"; \
else \
export arch="amd64" ; \
fi \
&& wget https://github.com/tianon/gosu/releases/download/1.11/gosu-$arch -O /usr/sbin/gosu \
&& chmod +x /usr/sbin/gosu \
&& echo http://dl-2.alpinelinux.org/alpine/edge/community/ >> /etc/apk/repositories \
&& apk --no-cache --no-progress add \
bash \
ca-certificates \
curl \
git \
linux-pam \
openssh \
s6 \
shadow \
socat \
tzdata \
rsync
Run a mirror:
docker run -it --name gogsRyantime-test alpine:3.14
Lines 13 ~ 19 are the number of system bits. You need to download a dependent file gosu-amd64 according to the number of system bits,
If the download from github fails when the network is not good, you can find other downloadable addresses on the Internet and replace the download addresses
Or use the - v parameter to mount the path containing the file into the container, and then copy the file to the / usr/sbin / path.
And modify the permissions of the file: chmod +x /usr/sbin/gosu. Remember to modify the permissions, otherwise the following container cannot be started
Install software:
After the software is installed, package it into an image:
docker commit -m "pack gogs Operating environment"c80a108 alpine_3.14:gogsRuntime-test
5. Create gogs docker image
After the running environment image is made, you can use this image to make the image containing gogs. The later steps are no longer used to enter the container for operation, but directly use Dockerfie to operate the subsequent steps
Back up the original Dockerfile and create another Dockerfile.
The contents are as follows:
FROM alpine_3.14:gogsRuntime-test ENV GOGS_CUSTOM /data/gogs # Configure LibC Name Service COPY docker/nsswitch.conf /etc/nsswitch.conf WORKDIR /app/gogs COPY docker ./docker COPY gogs gogs #COPY --from=binarybuilder /gogs.io/gogs/gogs . RUN ./docker/finalize.sh # Configure Docker Container VOLUME ["/data", "/backup"] EXPOSE 22 3000 HEALTHCHECK CMD (curl -o /dev/null -sS http://localhost:3000/healthcheck) || exit 1 ENTRYPOINT ["/app/gogs/docker/start.sh"] CMD ["/bin/s6-svscan", "/app/gogs/docker/s6/"]
Execute docker build - t mygogs test, Generate image with gogs program:
During execution, you may see the error message failed to compute cache key: "xxx" not found: not found
This is because the two phases of the original Dockerfile are merged together, and for some reason, they are in gogs is ignored in the docker ignore file.
Rename gogs to gogsAPP and copy it with the new file name in Dockerfile
In addition, it is specially explained that under the windows system, it is called in the container. sh may fail
Prompt error executor failed running [xxx.sh]:exit code: 127:
This is due to line breaks in the startup script.
The newline character in windows system cannot be correctly resolved when the Dockerfile build image is called. Please use notepad + + to open docker build sh file, check whether the newline symbol is \ n, if not, modify the newline symbol and try again.
After troubleshooting this problem for a long time, I kept deleting and streamlining scripts in window s and virtual machines, and kept trying to build docker. Finally, I found that it was not called when building in windos environment If the sh script can be built, the call will not work. The problem lies in the sh script. Then delete and simplify the sh script line by line without changing the instructions in the sh script. When you are about to give up, do you think it will be a line feed problem? The modification of line breaks was finally solved.
A cup of tea, a cigarette, a question for a long time
Replace all newline characters, and then build successfully:
6. Run the image with gogs container
docker run -itd --name mggogs-test -v "/data:/data" -p 3000:3000 mygogs-test
If the visit goes well, you will see the gogs web page again
So far, we have made three images, one image of compiling gogs program and one image of gogs running environment. On the basis of the running environment, we have added the image containing gogs program.
The final product is a gogs image that can be started and run directly in docker.
A pit is encountered here:
The chmod +x /usr/sbin/gosu instruction was not executed when preparing to run the environment image earlier, resulting in an error when the container was started: standard_init_linux.go:228: exec user process caused: no such file or directory
The problem is even more complicated when the files caused by the system architecture can not be found on the Internet. In addition, the above processes are all done on windows.
When I was about to give up, I looked back at the previous operation process and found that a step had been missed. After modifying the permission again, I continued to do the following work, and finally succeeded.
When writing this document, I went through this process again and had a deeper understanding of docker. I hope it will be helpful to you.