Many people do not know the knowledge of Python multi process and multi thread details!

Posted by godsent on Mon, 06 Apr 2020 08:41:44 +0200

Single process and single thread: a person eats food on a table.
2. Single process and multithreading: many people eat together on the same table.
3. Multi process single thread: many people, each of them, eat at their own table.

The problem of multithreading is that when more than one person eats a dish at the same time, it is easy to compete. For example, when two people take a dish at the same time, one person just stretches out his chopsticks, and the result reaches the time when he has been taken away... At this point, we must wait for one person to take a bite, and then give another person a dish, that is to say, resource sharing will conflict and scramble.

Two, thread

threading is a programming module used to implement multithreading in a process

We learned multiprocess programming earlier

To complete multiple tasks, you can also use multiple threads in a process. A process includes at least one thread, which we call the main thread. Other threads opened in the main thread are called sub threads

All threads in a process can share resources directly, so the communication between threads is much more convenient than that between processes

The thread module of python is a lower level module. The threading module of python makes some packaging for the thread, which can be used more conveniently

Single thread example:

import time


def say_sorry():
    print("Honey, I'm wrong. Can I eat now?")
    time.sleep(1)


if __name__ == "__main__":
    for i in range(5):
        say_sorry()

Multithreading example:

import threading
import time


def say_sorry():
    print("Honey, I'm wrong. Can I eat now?")
    time.sleep(1)


if __name__ == "__main__":
    for i in range(5):
        t = threading.Thread(target=say_sorry)
        t.start() #Start the thread, that is, let the thread start execution

Explain:

  1. It can be seen that multithreaded concurrent operations are used, which takes much less time
  2. To create a good thread, you need to call the start() method to start it

2.1 threading

Two modules commonly used in Python 3 threads are:

  • _thread

    The thread module has been discarded. You can use the threading module instead. Therefore, the "thread" module can no longer be used in Python 3. For compatibility, python 3 renames the thread to "U thread"

  • Threading (recommended)

    threading is used to provide thread related operations. A thread is the smallest unit of work in an application.

2.2 Thread class

The Thread class represents activities running in a separate control Thread. There are two ways to specify this activity:

2.2.1 pass callback object to constructor

mthread=threading.Thread(target=xxxx,args=(xxxx))
mthread.start()

2.2.2 rewrite the run() method in the subclass. Here is a small example:

import threading
import time
class MyThread(threading.Thread):
    def __init__(self,arg):
        super(MyThread, self).__init__()#Note: be sure to explicitly call the initialization function of the parent class.
        self.arg=arg
    def run(self):#Define the function to run for each thread
        time.sleep(1)
        print('the arg is:%s\r' % self.arg)

for i in range(4):
    t =MyThread(i)
    t.start()

print('main thread end!')

2.2.3 sharing global variables among multiple threads

The biggest difference between multithreading and multiprocessing is that in multiprocessing, there is a copy of the same variable in each process, which does not affect each other. In multithreading, all variables are shared by all threads. Let's see that the same data is shared by two threads.

from threading import Thread
import time 

g_num = 100		# Define global variable G

def work1():
    num = 1		# Define local variable num
    global g_num		# Keyword global mark global variable G
    for i in range(3):
        g_num += 1		# Change the value of a global variable
        print("---Sub thread 1---work1 function---g_num:%d" % g_num)

def work2():
    num = 2
    global g_num		# Keyword global mark global variable G
    print("\t---Sub thread 2---work2 function---individual g_num:%d" % g_num)


if __name__ == "__main__":
    print("Before starting a thread: g_num:%d" % g_num)

    t1 = Thread(target=work1)    # Create t1 sub thread and assign task work1
    t2 = Thread(target=work2)    # Create t2 sub thread and assign task work2
    t1.start()  	# Start t1 thread

    time.sleep(1)	# Wait for t1 thread to finish executing, and observe whether the global variables printed in t2 thread have changed
    t2.start() 		# Start t2 thread

2.2.4 global variables as parameters

Pass in the list as a parameter and append the element at the end of the parameter

from threading import Thread
import time

g_list = [10, 20, 30]

def work1(list):
    for i in range(3):
        list.append(i)    # Change the value of the parameter, append the element at the end of the list
        print("--Sub thread 1--work1----num:", list)

def work2(list):
    print("\t--Sub thread 2---work2---num: ", list)

if __name__ == "__main__":
    print("Main thread access g_list:" , g_list)

    t1 = Thread(target=work1, args=(g_list,))    # Create thread t1 and pass g ﹣ list as parameter to execute function work1
    t2 = Thread(target=work2, args=(g_list,))    # Create thread t2 and pass g'u list as parameter to execute function work2
    t1.start()

    time.sleep(1)
    t2.start()

Pass in the list as a parameter and reset the parameter

from threading import Thread
import time

g_list = [10, 20 ,30]

def work1(list):
    for i in range(3):
        # list.append(i)
        list = [1, 2, 3]	# reset parameters
        print("--Sub thread 1--work1----num:", list)

def work2(list):
    print("\t--Sub thread 2---work2---num: ", list)

if __name__ == "__main__":
    print("Main thread access g_list:" , g_list)

    t1 = Thread(target=work1, args=(g_list,))	# Create thread t1 and pass g ﹣ list as parameter to execute function work1
    t2 = Thread(target=work2, args=(g_list,))	# Create thread t2 and pass g'u list as parameter to execute function work2
    t1.start()  

    time.sleep(1)
    t2.start()  

2.2.5 thread lock

​ The biggest difference between multithreading and multiprocessing is that in multiprocessing, one copy of the same variable exists in each process, which does not affect each other. In multithreading, all variables are shared by all threads, so any variable can be modified by any thread. Therefore, the biggest danger of sharing data between threads is that multiple threads change one change at the same time Quantity, change the content.

Let's see how multiple threads operate on a variable at the same time to change the content:

import time, threading

# Suppose this is your bank account:
balance = 0

def change_it(n):
    # Save before fetching, the result should be 0:
    global balance
    balance = balance + n
    balance = balance - n

def run_thread(n):
    for i in range(100000):
        change_it(n)

if __name__ == "__main__":
	t1 = threading.Thread(target=run_thread, args=(5,))
	t2 = threading.Thread(target=run_thread, args=(8,))
	t1.start()
	t2.start()
    t1.join()
    t2.join()
	
	print(balance)

The result of each execution is not necessarily the same. The reason is that modifying balance requires multiple statements. When executing these statements, the thread may be interrupted, resulting in multiple threads changing the content of the same object.

If two threads deposit and withdraw at the same time, the balance may be wrong. You certainly don't want your bank deposit to become a negative number somehow, so we must make sure that when one thread modifies the balance, other threads cannot.

If we want to ensure that the balance calculation is correct, we need to put a lock on change it(). When a thread starts to execute change it(), we say that because the thread obtains the lock, other threads cannot execute change it() at the same time. We can only wait until the lock is released and the lock is acquired. Since there is only one lock, no matter how many threads, at most one thread holds the lock at the same time, there will be no modification conflict. Creating a lock is achieved by threading.Lock():

#The use of locks
mutex = threading.Lock()  #Create lock
mutex.acquire([timeout])  #locking
mutex.release()  #release
import time, threading

balance = 0
lock = threading.Lock()

def change_it(n):
    # Save before fetching, the result should be 0:
    global balance
    balance = balance + n
    balance = balance - n

def run_thread(n):
    for i in range(100000):
        # To acquire a lock first:
        lock.acquire()
        try:
            # Change with ease:
            change_it(n)
        finally:
            # The lock must be released after modification:
            lock.release() 
            
if __name__ == "__main__":
	t1 = threading.Thread(target=run_thread, args=(5,))
	t2 = threading.Thread(target=run_thread, args=(8,))
	t1.start()
	t2.start()
    t1.join()
    t2.join()
	
	print(balance)

Topics: Python Programming less