python multithreaded programming (Part 2)
Link to previous article: python multithreading (Part 1)
preface
I saw it at station b Multithreaded video , please share your notes as follows.
1, Introduction to threads
1.1 another form of multitasking
In python, multithreading can also be used to implement multitasking
1.2 why multithreading
Process is the smallest unit to allocate resources. Once a process is created, certain resources will be allocated, just as it is necessary to open two QQ software to chat with two people about QQ. It is a waste of resources.
Thread is the smallest unit of program execution. In fact, the process is only responsible for allocating resources, and threads use these resources to execute programs. In other words, a process is a container of threads. At least one thread in a process is responsible for executing the program. At the same time, the thread itself does not have system resources and only needs a little essential resources in operation, but it can share all the resources owned by the process with other threads belonging to the same process. This is like opening two windows (two threads) to chat with two people through a QQ software (one process). It realizes multitasking and saves resources at the same time
1.3 role of multithreading
Run task A and Task B at the same time
1.4 role of multithreading
1.5 summary
Another form of multitasking
Thread is the smallest unit of program execution
Multiple threads belonging to the same process share all the resources owned by the process
2, Multithreading completes multitasking
2.1 steps to create a thread
1. Import thread module
import threading
2 create thread objects through thread classes
Thread object = threading.Thread()
3 start the thread to execute the task
Thread object.start()
2.2 creating thread objects through thread classes
Thread object = threading Thread (target = task name)
Parameter name Description
The name of the target task executed by target, which refers to the function name (method name)
Name thread name, which is generally not set
Group thread group. Currently, only None can be used
2.3 code for thread creation and startup
# Create child thread sing_thread = threading.Thread(target=sing) # Create child thread dance_thread = threading.Thread(target=dance) # Start thread sing_thread.start() dance_thread.start()
example
# 1 import process package
import threading
import time
# sing
def sing():
for i in range(3):
print("sing...")
time.sleep(1)
# dance
def dance():
for i in range(3):
print("dance...")
time.sleep(1)
if __name__ == '__main__':
start_time = time.time()
# Create child thread
sing_thread = threading.Thread(target=sing)
# Create child thread
dance_thread = threading.Thread(target=dance)
# Start thread
sing_thread.start()
dance_thread.start()
print(time.time() - start_time)
3, A thread executes a task with parameters
3.1 threads execute tasks with parameters
Parameter name Description
args passes parameters to the execution task as tuples
kwargs passes parameters to the execution task in the form of dictionary
3.2 use of args parameter
# target: the name of the function executed by the thread # args: indicates that parameters are passed to the function as tuples sing_thread = threading.Thread(target=sing,args=(3,)) # Note that if you add, it is a tuple, indicating the parameters that will be passed to the function from 3 sing_thread.start()
3.3 use of kwargs parameters
# target: the name of the function executed by the thread
# kwargs: means to pass parameters to a function in the form of a dictionary
dance_thread = threading.Thread(target=dance,kwargs={"count":3} )
# Start thread
dance_pthread.start()
example
# 1 import process package
import threading
import time
# sing
def sing(num,name):
for i in range(num):
print(name,end="sing...\n")
time.sleep(1)
# dance
def dance(count,name):
for i in range(count):
print(name,end="dance...\n")
time.sleep(1)
if __name__ == '__main__':
start_time = time.time()
# args passes parameters to the execution task as tuples
sing_thread = threading.Thread(target=sing,args=(3,"Xiao Ming"))
# kwargs passes parameters to the execution task in the form of dictionary
dance_thread = threading.Thread(target=dance,kwargs={"name":"Xiao Hong","count":2})
# Start thread
sing_thread.start()
dance_thread.start()
print(time.time() - start_time)
The considerations are the same as for multi process implementation
4, End order of main thread and sub thread
Comparison process
The main thread will wait for all child threads to finish before the main thread ends
import time
import threading
def work():
for i in range(10):
print("work...")
time.sleep(0.2)
if __name__=="__main__":
sub_thread=threading.Thread(target=work)
sub_thread.start()
# The main thread ends after waiting for 1s
time.sleep(1)
print("The main thread is over")
#Conclusion the main thread will wait for all sub threads to finish before ending
4.1 setting the main thread of guard
If you want the main thread not to wait for the execution of the child thread to complete, you can set the guard main thread
# Set the mode of guarding the main thread 1 daemon=True guarding the main thread
work_thread = threading.Thread(target=work,daemon=True)
# Set main thread guard mode 2
# work_thread.setDaemon(True)
work_thread.start()
# Main thread delay 1s
time.sleep(1)
print("over")
example
import time
import threading
def work():
for i in range(10):
print("work...")
time.sleep(0.2)
if __name__=="__main__":
# sub_thread=threading.Thread(target=work)
# sub_thread.start()
# When the main thread ends, you don't want to wait for the sub thread to end. You can set the guard sub thread
# 1. threading.Thread(target=work,daemon=True)
# sub_thread=threading.Thread(target=work,daemon=True)
# sub_thread.start()
# 2.
sub_thread=threading.Thread(target=work)
sub_thread.setDaemon(True)
sub_thread.start()
# The main thread ends after waiting for 1s
time.sleep(1)
print("The main thread is over")
#Conclusion the main thread will wait for all sub threads to finish before ending
5, Execution order between threads
5.1 execution between threads is out of order
for i in range(5): sub_thread=threading.Thread(target=task) sub_thread.start()
5.2 get current thread information
# Through current_ The thread () method gets the thread object current_thread = threading.current_thread() # Through current_ The thread object can know the relevant information of the thread, such as the order in which it is created print(current_thread)
example
import threading
import time
def task():
time.sleep(1)
# Gets the thread object of the current thread
current_thread=threading.current_thread()
print(current_thread)
if __name__=="__main__":
for i in range(5):
sub_thread=threading.Thread(target=task)
sub_thread.start()
# The execution of multithreading is out of order and is scheduled by cpu
6, Process and thread comparison
6.1 relationship comparison
1 thread is attached to the process. Without process, there will be no thread
2 a process provides one thread by default, and a process can create multiple threads
6.2 difference and comparison
1 the resource cost of creating a process is greater than that of creating a thread
2 process is the basic unit of operating system resource allocation, and thread is the basic unit of CPU scheduling
3 threads cannot be executed independently and must exist in the process
6.3 comparison of advantages and disadvantages
1. Process advantages and disadvantages:
Advantages: multiple cores can be used
Disadvantages: high resource overhead
2. Advantages and disadvantages of threading:
Advantages: low resource overhead
Disadvantages: not available multi-core
7, Case multi process implementation of high concurrency copy device for video folder
requirement analysis
1. Whether the target folder exists. If it does not exist, it will be created. If it does not exist, it will not be created
2. Traverse all files in the source folder and copy them to the destination folder
3. Use process to realize multitasking and complete high concurrent copy
Implementation steps
1. Define the path where the source folder is located and the path where the target folder is located
# 1 define the path where the source folder is located and the path where the destination folder is located source_dir = "python Teaching video" dest_dir="/home/python/desktop/test"
2. Create target folder
try:
# 2 create target folder
os.mkdir(dest_dir)
except:
print("The destination folder already exists and was not created~")
3. Through OS Listdir gets the list of files in the source directory
# 3 via OS Listdir gets the list of files in the source directory file_list=os.listdir(source_dir) print(file_list)
4. Traverse each file and define a function to copy the file
# 4 traverse each file and define a function to copy the file for file_name in file_list: copy_work(file_name,source_dir, dest_dir)
5. Use threads to realize multitasking and complete high concurrent copy
# 4 for loop copies each file in turn for file_name in file_list: # copy_work(file_name,source_dir, dest_dir) sub_thread= threading.Thread(target=copy_work,args=(file_name,source_dir, dest_dir)) sub_thread.start()
realization
import os
import threading
def copy_file(file_name, source_dir, dest_dir):
# 1 splice source file path and target file path
source_path = source_dir + '\\' + file_name
dest_path = dest_dir + '\\' + file_name
# 2 open source and target files
with open(source_path, 'rb') as source_file:
with open(dest_path, 'wb') as dest_file:
# 3 loop reading source file to target path
while True:
data = source_file.read(1024)
if data:
dest_file.write(data)
else:
break
if __name__ == '__main__':
# 1 define source and destination folders
source_dir = "Source folder"
dest_dir = "Destination folder 2"
# 2. Create target folder
try:
os.mkdir(dest_dir)
except:
print("The destination folder already exists")
# 3. Read the file list of the source folder
file_list = os.listdir(source_dir)
# 4. Traverse the file list to copy
for file_name in file_list:
# copy_file(file_name, source_dir, dest_dir)
# 5. Use multithreading to realize multitask copy
sub_thread = threading.Thread(target=copy_file, args=(file_name, source_dir, dest_dir))
sub_thread.start()