Test Qitan, the BUG is missing.
Before explaining, let me talk about the difference between my tutorial and other online tutorials:
1 I share the scenes that I use in my work at high frequencies.
Many people learn Python and don't know where to start. Many people learn to look for python，After mastering the basic grammar, I don't know where to start. Many people who may already know the case do not learn more advanced knowledge. These three categories of people, I provide you with a good learning platform, free access to video tutorials, e-books, and the source code of the course! QQ Group:101677771 Welcome to join us and discuss and study together
What I share is the learning method, or that is to say, what you should learn and what you should focus on;
3 based on 1 and 2, you can learn according to my tutorial, find videos online, or read books... You have to understand that mastering learning methods is much more important than finding learning materials.
In the first five issues, I have shared the basic grammar of python. If you practice step by step according to my article, make sure you understand the grammatical characteristics and writing methods of python code, can write some simple function methods independently, and even try to start the practice of automatic testing (based on python grammar).
Of course, if you want to further master the characteristics of python syntax, you must not miss the next few articles.
This one focuses on python} generators and anonymous functions.
Objective: to master the concept and use of these two knowledge points.
generator
01 what is a generator?
Remember two keys:
- Generator is a special function method. This means that it is inseparable from the function (def).
- Based on the above point, as long as the yield keyword appears in the function, it is the generator function.
Beginner you, or too difficult to understand?
02 popular explanation
You can understand the generator as a box. You can add elements to the box and take them out when you need them.
Take the following example:
# Ordinary function
def func():
return 1
f = func()
print("Function return value:",f)
->Function return value: 1
print("Type of return value of function:",type(f))
->Type of return value of function:<class 'int'>
# generator
def gen_func():
yield 1
yield 2
g = gen_func()
print("Generator object:",g)
->Generator object:<generator object gen_func at 0x00000189B8CFF7C8>
print("Type of generator object:",type(g))
->Type of generator object:<class 'generator'>
# Read the value of the generator object, because the generator is also an iterator, which implements python's iterative protocol (that is, implements the _iter method)
for i in g:
print("The value of the generator object:",i)
->The value of the generator object: 1
->The value of the generator object: 2What's the use of 03 generator?
Function: inert evaluation (the mechanism of calculating while cycling) to save performance
04 common uses of generator?
- Read large files
- Web crawler sweep framework
- Synergetic process
For example: Fibonacci sequence (0,1,1,2,3,5...), Print the first 50 elements of the Fibonacci sequence
# Not using the generator consumes a lot of memory
def fib(idx):
res=[]
n, a, b = 0, 0, 1
while n < idx:
res.append(b)
a, b = b, a+b
n += 1
return res
res = fib(100)
print(res)
# Using the generator can save a lot of memory
def gen_fib(idx):
n, a, b = 0, 0, 1
while n < idx:
yield b
a, b = b, a+b
n += 1
for i in gen_fib(100):
print(i)Anonymous function
01 what is an anonymous function?
When:
- The function implementation is relatively simple
- Functions do not need to be called in multiple places
- Don't bother to name this function
We can use anonymous functions.
Beginner you, or too difficult to understand?
02 popular explanation
You want to implement a function to find the square of x, but this function is too simple to be specially defined by def. At the same time, you forget how to spell the square in English. If you name it "pingfang", it looks too low. Therefore, you can implement this function without naming it. This is the anonymous function lambda expression.
For example: find the square of a number
# No lambda expressions
def square(x):
return x * x
print(square(2))
# Using lambda expressions
# Writing method: lambda return value: calculation expression
s = lambda x: x * x
print(s(2))As always, make a summary
01 if you are a beginner, you can not master the generator and anonymous function first, and then think about it after learning python;
02 in practice, generators and anonymous functions are used frequently, and they are high-frequency questions in interviews.
Test Qitan, the BUG is missing.
Hello, I'm uncle tan.
This one focuses on python} processes and threads.
Objective: to master the necessary process and thread knowledge for beginners.
The difference and relation between process and thread
Finally began to deepen the difficulty and came to the knowledge point of process and thread~
These two concepts alone have baffled many beginners - learn the concepts today and forget them tomorrow; Tomorrow I learned examples and forgot concepts.
To understand the relationship and difference between processes and threads, let me give a very simple example:
Your computer has two browsers, a Google browser and a qq browser.
A browser is a process.
Then, you open Google browser, baidu searches Qitan, and opens a new tab to open uncle Tan's article, as shown in the figure below:
You can understand that two web pages opened in the same browser are two threads. If I close the browser, these two threads are gone.
Well, when you have the concept, I can talk about the difference and connection between the two.
process
- The utility model has the advantages of high stability and does not affect the use of other processes after the program crashes. That is, after Google browser crashes, qq browser can still be used normally.
- Disadvantages: creating a process is expensive. That is, when you open a variety of browsers, your computer will be particularly stuck, because there is an upper limit on computer memory and CPU.
thread
- Advantages multithreading is usually a little faster than multiprocessing, but it has little advantage
- Disadvantages: if any thread hangs, it will cause the whole process to crash, because the thread shares the memory of the process (the browser example is no longer applicable, which can be understood as a grasshopper tied to a ship)
Multi process
Because most small partners use the Windows operating system, the fork() call for Unix/Linux is left aside.
In python, multiprocessing is generally used to implement multiprocessing.
import os
from multiprocessing import Process
def run_proc(name):
print('Start sub process execution %s (%s)...' % (name, os.getpid()))
# Code to be executed by child process
if __name__ == '__main__':
print('Parent process %s.' % os.getpid())
p = Process(target=run_proc, args=('test',)) # Create a Process instance
print('The subprocess is about to start.')
p.start() # Start the instance with the start() method
p.join() # The join() method can wait for the child process to finish before continuing. It is usually used for synchronization between processes
print('Child process end.')Do you want to elaborate?
In fact, my notes are very complete. You only need to copy the code and execute it once or debug it once to understand.
Thread pool
How to understand?
That is, the number of processes the code can run. There is a place to control it.
from multiprocessing import Pool
import os
import time
import random
def long_time_task(name):
print('start task %s (%s)...' % (name, os.getpid()))
start = time.time()
time.sleep(random.random() * 3)
end = time.time()
print('Task %s Execution cost %0.2f second.' % (name, (end - start)))
if __name__ == '__main__':
print('Father process %s.' % os.getpid())
p = Pool(3)
# Because the default size of the Pool is 4, tasks 0, 1, 2 and 3 are executed immediately, while task 4 is executed only after a previous task is completed, but at most 4 processes are executed at the same time
# Since the default size of the Pool is the number of cores of the CPU, if you have an 8-core CPU, you have to submit at least 9 child processes to see the waiting effect
for i in range(5):
p.apply_async(long_time_task, args=(i,))
print('Wait for the child process to end...')
p.close()
# Calling the join() method on the Pool object will wait for all child processes to complete execution
# Before calling join(), you must call close(). After calling close(), you cannot add a new Process
p.join()
print('All child processes have been executed.')The above is what we must know and know.
Of course, the best time to learn is: when you want to use it, you can learn again, and the effect is the best.
If you have learned and have no use scenarios, I suggest you slow down or use it as a knowledge reserve.
Multithreading
Multithreading has three points that must be made clear in advance:
- Multitasking requirements can be completed by multiple processes or by multiple threads in a process
- A process consists of several threads
- A process has at least one thread
Python's standard library provides two modules:_ Thread and threading_ Thread is a low-level module. We generally use the high-level module threading (encapsulated _thread).
To start a thread is to pass in a function and create an instance of Thread, and then call start() to start executing. Let's look at a simple example:
import time
import threading
# Code executed by the new thread
def loop():
# Since any process will start a thread by default, we call this thread the main thread, and the main thread can start a new thread,
# Python's threading module has a current_thread() function, which always returns an instance of the current thread
print('thread ss %s Running' % threading.current_thread().name)
n = 0
# The name of the main thread instance is MainThread, and the name of the sub thread is specified during creation. We use LoopThread to name the sub thread and display the name during printout
while n < 5:
n = n + 1
print('thread ss %s >>> %s' % (threading.current_thread().name, n))
time.sleep(1)
print('thread ss %s Has ended.' % threading.current_thread().name)
print('thread %s is Running' % threading.current_thread().name)
t = threading.Thread(target=loop, name='LoopThread')
t.start()
t.join()
'''
about join()Another important aspect of method is that it doesn't need to be called at all.
Once threads are started, they will continue to execute until the given function is completed and exits.
If the main thread has other things to do instead of waiting for these threads to complete(For example, other processes or waits for new client requests)，
You can not call join(). join()Method is only useful when you need to wait for the thread to complete
'''
print('thread %s Has ended.' % threading.current_thread().name)Similarly, the above contents must be known and learned.
In addition, in the working scenario, we generally use multithreading to deal with problems rather than multi processes. (Note: General)
As for the knowledge of interprocess communication, thread lock, GIL lock, multithreaded variables, inter thread communication, asynchronous co process and so on, it is more complex, and it is not the core of the minimalist tutorial. You can learn by yourself first, or look at it when you really want to use it.