A story expression: concurrency, parallelism, synchronization, asynchrony, threading, multithreading

Posted by doni49 on Wed, 02 Mar 2022 00:56:27 +0100

A small event describes concurrency, parallelism, synchronization, asynchrony, threading and multithreading

An exhibition invitation was launched to eight companies during a Canton Fair,
There are 8 participating companies, and the venue has an exhibition area of 800000 square meters,
Each exhibitor has 100000 square meters for sale,
Each exhibition company is only equipped with one employee, and there is a distance between each shelf,
When selling, employees need to go to the front of the shelf, and each shelf can only put one commodity.
8 Stalls of Companies:Multi process
8 All companies are on sale:parallel
 The first company has a shelf, and the shelf has a commodity: 1 thread
 The second company has 20 shelves, and each shelf has one commodity: 20 threads
 The third company has a shelf,Shelf configuration: 100 items: single thread concurrent (asynchronous)
Does the fourth company have shelves,Massage service.
The fifth company and the sixth company are the same parent company, named A Company, which means A The company has two stalls: parallel

Corresponding understanding

[[concurrent]: the Canton Fair will launch an exhibition invitation to 8 companies. In order, the invitation letter will be sent to me first and then to you......A total of eight invitations were sent

[[multi process]: all 8 companies have their own stalls

[Parallel]: eight companies are carrying out display tasks at the same time. Each company has its own booth and does not need to share one booth with others in turn

[Thread]: the number of shelves

[Calculation intensive]: the first exhibition company has one shelf equipped with one commodity. It sells large commodities [large nuclear warheads], which need to explain the use method in detail.

[handle I/O [intensive]: the third company has one shelf,It is equipped with 100 commodities and sells small commodities [purified water] for direct payment and delivery,There was little explanation.

[Synch]: the fourth company sells [massage service]: it needs to press from head to foot. [massage sequence cannot be changed]

[Asynchronous] [single thread concurrent]: the third company has one shelf,Shelves are equipped with 100 commodities: employees only need to sell 100 commodities on one shelf, one for each

[Concurrency is a special type of parallelism]
[Concurrency does not mean parallelism]: one shelf displaying 100 kinds of goods does not mean two stalls [the third company has one shelf,Shelf configuration: 100 items: single thread concurrent (asynchronous)-----The fifth company and the sixth company are the same parent company, named A Company, which means A The company has 2 stalls]----[Mononuclear CPU Is it not possible to implement parallelism
[Concurrency does not mean parallelism]: the first company participated in the exhibition first, and the second company is still in the registration form

In a multi-core processor, one process corresponds to one core

[Single core concurrency] multi process in single core CPU In operation, it is understood as:
An exhibition venue has 100000 square meters, and the exhibition of each company needs 100000 square meters,
But there are eight companies here. They can't exhibit at the same time,
Then they will take turns at the venue (and are required to change companies every few seconds).
Therefore, it will take longer to complete the exhibition.
(cpu Last is every 0.00N Millisecond fast switching,So you can't feel it and switch again)
[Mononuclear CPU It is impossible to realize parallelism]

Asynchronous IO, asyncio library is a concurrent programming design, which was introduced from Python version 3.4

Where is asynchronous IO applicable?

Asynchronous IO: a language independent paradigm (model) implemented by multiple programming languages

async/ await:  Keywords used to define collaboration
asyncio/asynchronous io Library: it provides the basis and foundation for running and managing collaborative processes API 

Python Medium asynchronous IO Core of:Coroutine (special generator function)

async IO:asynchronous io
asyncio:python Asynchronous Library in

Python version 3.7 or later is required to perform the following operations

pip install --upgrade pip aiohttp aiofiles
python3 -m pip install --upgrade pip aiohttp aiofiles
  Parallel: perform multiple operations at the same time
 Multi process: a method to realize parallelism
                Multi process: good at dealing with computing intensive( CPU-bound)Tasks: strong dense loops and mathematical calculations

Concurrency: multiple tasks run in overlapping mode. [concurrency is a special type of parallelism]
Threads: concurrent execution model. Multiple threads execute tasks in turn,A process can contain multiple threads.
            Thread: good at handling I/O Intensive tasks. because I/O Need to wait.

Concurrency is a special type of parallelism
 Concurrency includes multiple processes and threads -- [computation intensive]+[I/O Dense]
Multi process is a form of parallelism.

Python provides long-term support for both [parallelism] and [concurrency]
[through its standard library packages multiprocessing, threading and concurrent.futures]
CPython built an independent design: asyncio library.
Asynchronous IO, add asyncio package to standard library.
Keywords [async] and [await].

async IO is not a newly invented concept
async IO already exists in other languages and runtime environments, such as Go, C# and Scala.

The concurrent code library of Python is called a package of asyno.

async IO[asynchronous IO] Not [thread] and [multi process].
async IO[asynchronous IO] 
Using single thread to run in a single process is a single process and single thread design [only using the cooperative multitasking mechanism].
Synergetic process( async IO Can be scheduled at the same time, but it itself is not concurrent. Although it gives people a feeling of concurrency.

Asynchronous properties: asynchronous routines can "pause" while waiting for their final results, and allow other routines to run at the same time.
concurrency Concurrent[ Threading Multithreading, AsyncIO asynchronous I/O]
Parallelism Parallel[ Multiprocessing [multi process]

Case comparison
Asynchronous version

import asyncio

async def count():
    await asyncio.sleep(1)

async def main():
    await asyncio.gather(count(), count(), count())

if __name__ == "__main__":
    import time
    s = time.perf_counter()
    elapsed = time.perf_counter() - s
    print(f"{__file__} executed in {elapsed:0.2f} seconds.")

The output result is:

{File path+file name} executed in 1.00 seconds.

This output order is the core of asynchronous IO,
A single event loop or coordinator is responsible for communicating with each count() method call.
When each task is executed to await asyncio Sleep (1)),
This function will inform [event cycle] and hand over the control authority,
"I'll sleep for a second and continue to do other things during this period.".

Sync version:

import time

def count():

def main():
    for _ in range(3):

if __name__ == "__main__":
    s = time.perf_counter()
    elapsed = time.perf_counter() - s
    print(f"{__file__} executed in {elapsed:0.2f} seconds.")

There are changes in output sequence and time

countsync.py executed in 3.01 seconds.

The above two cases
Use time Sleep () and asyncio Sleep () looks a little crude. It is generally used here to replace time-consuming operations such as standard input.

Syntax async def in asyncio
A native co process or asynchronous generator is introduced.
Expressions async with and async for

async def g():
    r = await f()
    return r

There is an f() function in the g() function

keyword await Pass the controller to the event loop. (that is, suspend the currently running collaboration)
Python When executing, in  g() If an expression is encountered in a function await f()´╝îawait Will tell the event loop,
"Pause first g() Functions, etc f()Function returns the result, and during this period, other functions can be run. "

Use the combination of await and return to create a covariance function. To call a coroutine function, you must use await to wait for the result to be returned.

Case 1

async def f(x):
    y = await z(x)  
    return y
The above code can be executed`await`and`return`Allow in collaboration

Case 2

def m(x):
    y = await z(x)  
#Syntax error in the above code: there is no 'async def'

Case 3

async def g(x):
    yield x  
#The above code can be executed: This is an asynchronous generator
#However, it is rare to use yield in async def code block

Case 4

async def m(x):
    yield from gen(x) 
#Syntax error: syntax error
#Do not use yield from where async def is defined, which will throw an exception SyntaxError.

Demonstration case: 1. The way of becoming a tool - 2 Primary synergy

import asyncio

#1. Generator based approach
def py34_coro():
    yield from stuff()

#2. Original process
async def py35_coro():
    await stuff()

After the emergence of async/await syntax in Python 3.5, async/await replaced the co process mode of generator.
The co process method of generator is out of date.
It is best to use native processes.
Generator based coroutines will be removed in Python version 3.10.

be careful
Using async/await syntax (await cannot be used for generator based collaborations), these rules are largely incompatible with generator based collaborations.

Topics: Python