python minimalist tutorial 04: processes and threads

Posted by SargeZT on Tue, 11 Jan 2022 22:23:40 +0100

Python wechat ordering applet course video

https://edu.csdn.net/course/detail/36074

Python practical quantitative transaction financial management system

https://edu.csdn.net/course/detail/35475
Test Qitan, the BUG is missing.

Hello, I'm uncle tan.

This one focuses on python processes and threads.

Objective: to master the necessary process and thread knowledge for beginners.

The difference and relation between process and thread

Finally began to deepen the difficulty and came to the knowledge point of process and thread~

These two concepts alone have baffled many beginners - learn the concepts today and forget them tomorrow; Tomorrow I learned examples and forgot concepts.

To understand the relationship and difference between processes and threads, let me give a very simple example:

Your computer has two browsers, a Google browser and a qq browser.

A browser is a process.

Then, you open Google browser, baidu searches Qitan, and opens a new tab to open uncle Tan's article, as shown in the figure below:

You can understand this - * * two web pages opened in the same browser are two threads** If I close the browser, these two threads are gone.

Well, when you have the concept, I can talk about the difference and connection between the two.

process

  • advantage

    • High stability. When the program crashes, it will not affect the use of other processes. That is, after Google browser crashes, qq browser can still be used normally.
  • shortcoming

    • Creating a process is expensive. That is, when you open a variety of browsers, your computer will be particularly stuck, because there is an upper limit on computer memory and CPU.

thread

  • advantage

    • Multithreading is usually a little faster than multiprocessing, but it has little advantage
  • shortcoming

    • Any thread hanging will cause the whole process to crash, because threads share the memory of the process (the browser example is no longer applicable, which can be understood as a grasshopper tied to a ship)

Multi process

Because most small partners use the Windows operating system, the fork() call for Unix/Linux is left aside.

In python, multiprocessing is generally used to implement multiprocessing.

import os
from multiprocessing import Process

def run\_proc(name):
    print('Start sub process execution %s (%s)...' % (name, os.getpid()))

# Code to be executed by child process
if __name__ == '\_\_main\_\_':
    print('Parent process %s.' % os.getpid())
    p = Process(target=run_proc, args=('test',)) # Create a Process instance
    print('The subprocess is about to start.')
    p.start()  # Start the instance with the start() method
    p.join()   # The join() method can wait for the child process to finish before continuing. It is usually used for synchronization between processes
    print('Child process end.')

Do you want to elaborate?

In fact, my notes are very complete. You only need to copy the code and execute it once or debug it once to understand.

Thread pool

How to understand?

That is, the number of processes the code can run. There is a place to control it.

from multiprocessing import Pool
import os
import time
import random

def long\_time\_task(name):
    print('start task %s (%s)...' % (name, os.getpid()))
    start = time.time()
    time.sleep(random.random() * 3)
    end = time.time()
    print('Task %s Execution cost %0.2f second.' % (name, (end - start)))
if __name__ == '\_\_main\_\_':
    print('Father process %s.' % os.getpid())
    p = Pool(3)
    # Because the default size of the Pool is 4, tasks 0, 1, 2 and 3 are executed immediately, while task 4 is executed only after a previous task is completed, but at most 4 processes are executed at the same time
    # Since the default size of the Pool is the number of cores of the CPU, if you have an 8-core CPU, you have to submit at least 9 child processes to see the waiting effect
    for i in range(5):
        p.apply_async(long_time_task, args=(i,))
    print('Wait for the child process to end...')
    p.close()
    # Calling the join() method on the Pool object will wait for all child processes to complete execution
    # Before calling join(), you must call close(). After calling close(), you cannot add a new Process
    p.join()
    print('All child processes have been executed.')

The above is what we must know and know.

Of course, the best time to learn is: when you want to use it, you can learn again, and the effect is the best.

If you have learned and have no use scenarios, I suggest you slow down or use it as a knowledge reserve.

Multithreading

Multithreading has three points that must be made clear in advance:

  • Multitasking requirements can be completed by multiple processes or by multiple threads in a process
  • A process consists of several threads
  • A process has at least one thread

Python's standard library provides two modules:_ Thread and threading_ Thread is a low-level module. We generally use the high-level module threading (encapsulated _thread).

To start a thread is to pass in a function and create an instance of Thread, and then call start() to start executing. Let's look at a simple example:

import time
import threading

# Code executed by the new thread
def loop():
    # Since any process will start a thread by default, we call this thread the main thread, and the main thread can start a new thread,
    # Python's threading module has a current\_thread() function, which always returns an instance of the current thread
    print('thread  ss %s Running' % threading.current_thread().name)
    n = 0
    # The name of the main thread instance is MainThread, and the name of the sub thread is specified during creation. We use LoopThread to name the sub thread and display the name during printout
    while n < 5:
    n = n + 1
    print('thread  ss %s >>> %s' % (threading.current_thread().name, n))
    time.sleep(1)

print('thread  ss %s Has ended.' % threading.current_thread().name)
print('thread  %s is Running' % threading.current_thread().name)

t = threading.Thread(target=loop, name='LoopThread')
t.start()
t.join()

'''
about join()Another important aspect of method is that it doesn't need to be called at all.
Once threads are started, they will continue to execute until the given function is completed and exits.
If the main thread has other things to do instead of waiting for these threads to complete(For example, other processes or waits for new client requests),
You can not call join(). join()Method is only useful when you need to wait for the thread to complete
'''

print('thread  %s Has ended.' % threading.current_thread().name)

Similarly, the above contents must be known and learned.

In addition, in the working scenario, we generally use multithreading to deal with problems rather than multi processes. (Note: General)

As for the knowledge of interprocess communication, thread lock, GIL lock, multithreaded variables, inter thread communication, asynchronous co process and so on, it is more complex, and it is not the core of the minimalist tutorial. You can learn by yourself first, or look at it when you really want to use it.

As always, make a summary

01 multithreading and multi process are necessary to know and know;

02 ponder the learning progress by yourself, neither stubbornly nor simply skip;

03 little friends are more concerned about whether they will be asked during the interview. A: general companies don't, so?

Topics: Python Back-end computer