catalogue
3. Significance of warehouse parameters
4 local warehouse creation tool
1 Introduction
Imagine that 10000 Linux developers try to use the Internet to obtain source updates at the same time, but the bandwidth of the source warehouse is only 100M. You can imagine how slow and crowded the whole update process will be. As the population grows, this problem will become more and more obvious.
This article will find a way to solve this problem. In fact, all developers are making the same update, which is repetitive work. Therefore, if the official source of ubuntu can be synchronized locally, the update speed will be greatly improved, and only the first synchronization time and hard disk space will be lost.
2 ubuntu official warehouse
Architecture of xuntu 86: http://archive.ubuntu.com/ubuntu/
ubuntu source of ARM Architecture: http://ports.ubuntu.com/ubuntu-ports/
3. Significance of warehouse parameters
Source. In ubuntu One of the contents of the list:
http://ports.ubuntu.com/ubuntu-ports/ bionic main restricted universe multiverse
The format will be parsed below:
source. Each line in the list can be parsed according to the following figure
The third part explains that "software security classification" is more appropriate.
Interpretation of Part IV:
- Main - the "main" component includes free software, software that can be freely released, and software that is fully supported by the Ubuntu team. This includes most popular and stable open source software, which is installed by default when installing Ubuntu.
- Restricted - the "restricted" component is software that is designed for general use and has no free software copyright, but is still supported by the Ubuntu team.
- Universe - the "universe" component is the epitome of the whole free and open source Linux world. In the "universe" component, you can find most open source software and software under open source copyright, all of which are based on public sources. These software are written using components in "main". They can run together with "main" components safely, but they have no guarantee of security upgrade. The "universe" component contains thousands of software. Although they are public, users must understand the differences and instability between them and the stable Ubuntu core software.
- Multiverse - the "multiverse" component contains "not free" software, which means that these software do not meet ubuntu's various copyright policies relative to the "main" component. When you use these software, how to adjust various rights and abide by the copyright owner is entirely up to you. These software are not supported by the ubuntu team, and usually cannot be modified and updated. If they are used, they will bear any risk.
4 local warehouse creation tool
4.1 rsync
The tool will synchronize all the contents of the specified warehouse. The architecture cannot be specified, so it requires at least 1T of hard disk space.
Create a sync SH script, as follows:
#!/bin/bash STORAGE_PATH="/mirror-directory" #Storage path rsync \ --recursive \ --links \ --perms \ --times \ --compress \ --progress \ --delete \ rsync://archive.ubuntu.com/ubuntu \ # source address $STORAGE_PATH
Parameter interpretation:
- R, - recursive handles subdirectories in recursive mode
- L, - links keep soft links
- P, - perms keep file permissions
- t, - times hold file time information
- Z, - compress compresses the backed up files during transmission
-- progress displays the backup process
-- delete delete files that are not in SRC in DST
-- rsync path = path specifies the path information where the rsync command is located on the remote server
4.2 apt-mirror
"Apt mirror" is a tool developed by Perl, which is used to download and mirror all contents from an open repository. Of course, the tool is the source that supports synchronization of specified architectures.
Configure apt mirror tool, mainly focusing on the following:
- Storage location (base_path)
- Number of threads downloaded (nthreads)
- Version and schema to download
The above configuration is in / etc / apt / mirror List file. Of course, you can also specify the mirror in other locations list.
In the process of installing the tool, a default version will be generated. We need to adjust the above parameters on this basis.
A complete configuration file is listed below, which will mirror the 32-bit and 64 bit versions of Ubuntu 8.04 LTS and synchronize the arm64 and armhf architectures. This requires nearly 500G of storage space,
The settings specify that these contents be synchronized to / media/STORAGE /.
############# config ################## # set base_path /media/STORAGE # # set mirror_path $base_path/mirror # set skel_path $base_path/skel # set var_path $base_path/var # set cleanscript $var_path/clean.sh # set defaultarch arm64 armhf # set postmirror_script $var_path/postmirror.sh # set run_postmirror 0 set nthreads 20 set _tilde 0 # ############# end config ############## # for bionic deb-arm64 https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic main restricted universe multiverse deb-armhf https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic main restricted universe multiverse deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic main restricted universe multiverse deb-arm64 https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-updates main restricted universe multiverse deb-armhf https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-updates main restricted universe multiverse deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-updates main restricted universe multiverse deb-arm64 https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-backports main restricted universe multiverse deb-armhf https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-backports main restricted universe multiverse deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-backports main restricted universe multiverse deb-arm64 https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-security main restricted universe multiverse deb-armhf https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-security main restricted universe multiverse deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-security main restricted universe multiverse #for focal deb-arm64 https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal main restricted universe multiverse deb-armhf https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal main restricted universe multiverse deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal main restricted universe multiverse deb-arm64 https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-updates main restricted universe multiverse deb-armhf https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-updates main restricted universe multiverse deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-updates main restricted universe multiverse deb-arm64 https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-backports main restricted universe multiverse deb-armhf https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-backports main restricted universe multiverse deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-backports main restricted universe multiverse deb-arm64 https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-security main restricted universe multiverse deb-armhf https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-security main restricted universe multiverse deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ focal-security main restricted universe multiverse clean https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/
After the configuration is completed, you can start synchronizing the image with the following command:
Apt mirror (you can specify configuration files in other locations)
5 set timing synchronization
crontab is a command used to execute programs periodically
Parameter Description:
- e: run the text editor to set the schedule. The internal text editor is VI. if you want to use another text editor, first set the VISUAL environment variable to specify which text editor to use (for example, setenv VISUAL joe)
- r: delete the current schedule
- l: list the current schedule
Enter the command {crontab -e
Filling contents:
# For example, you can run a backup of all your user accounts # at 5 a.m every week with: # 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/ # # For more information see the manual pages of crontab(5) and cron(8) # # m h dom mon dow command */1 * * * * /where/directory/test.sh
For example, if it is synchronized once every 6 hours, it is: 0 * / 6 * * / where / directory / test sh
test. The content of the SH script is the synchronization command.
5.1 rules
The time format is as follows:
f1 f2 f3 f4 f5 program
Where f1 is the minute, f2 is the hour, f3 is the day of the month, f4 is the month, and f5 is the day of the week. Program represents the program to be executed.
When f1 is * it means that the program should be executed every minute, when f2 is * it means that the program should be executed every hour, and so on
When f1 is a-b, it means to execute from minute a to minute b, when f2 is a-b, it means to execute from hour a to hour b, and so on
When f1 is * / N, it means to execute every n minutes interval, f2 is * / N, it means to execute every n hours interval, and so on
When f1 is a, b, c Tense means a, b, c Minutes to execute, f2 is a, b, c Time means a, b, c Hours to execute, and so on
* * * * *
- - - - -
| | | | |
||| + ----- day of the week (0 - 6) (Sunday is 0)
|| + ---------------- month (1 - 12)
|| + ----------------- the day of the month (1 - 31)
|+ ---------------- hours (0 - 23)
+-------------------------Minutes (0 - 59)
The user can also save all settings in a file first and set the execution time in the form of crontab file. The execution command is: crontab [- U user] file
reference resources
https://www.jianshu.com/p/84d07a78cd0f
https://blog.programster.org/deploy-a-local-ubuntu-mirror-using-rsync