iterator
Generally speaking, the process of extracting data from an object in turn is called traversal, and this means is called iteration (repeatedly execute a certain code block, and take the result of each iteration as the initial value of the next iteration).
Iteratable object: refers to the object that can be used for... in... Loop, such as: set, list, ancestor, dictionary, string, iterator, etc.
-
In python, if an object implements__ iter__ Method, we call it iteratable object. You can view the internal implementation of set\list\tuple... And other source codes__ iter__ method
-
If an object is not implemented__ iter__ Method, but using for... in will throw TypeError: 'xxx' object is not iterable
-
You can use isinstance (obj, iteratable) to determine whether an object is an iteratable object. For example:
from collections.abc import Iterable # int a a = 1 print(isinstance(a, Iterable)) # False # str b b = "lalalalala" print(isinstance(b, Iterable)) # True # set c c = set([1, 2]) print(isinstance(c, Iterable)) # True # list d d = [1,2,3,"a"] print(isinstance(d, Iterable)) # True # dict e e = {"a":1,"b":2,"c":333} print(isinstance(e, Iterable)) # True # tuple f f = (1,3,4,"b","d",) print(isinstance(f, Iterable)) # We can also implement it ourselves__ iter__ To turn a class instance object into an iteratable object: -
We can do it ourselves__ iter__ To turn a class instance object into an iteratable object:
Implement the requirements of iterative objects by yourself
1. In python, if an object is implemented at the same time__ iter__ And__ next__ (get the next value) method, then it is an iterator object.2. You can use the built-in function next(iterator) or instance object__ next__ () method to get the value of the current iteration
3. Iterators must be iteratable objects, and iteratable objects are not necessarily iterators.
4. If you continue to call next() after traversing the iteratable object, you will throw a StopIteration exception.
from collections.abc import Iterator, Iterable class MyIterator: def __init__(self, array_list): self.array_list = array_list self.index = 0 def __iter__(self): return self def __next__(self): if self.index < len(self.array_list): val = self.array_list[self.index] self.index += 1 return val else: raise StopIteration # If the parent class is an iterator, the child class will also be an iterator class MySubIterator(MyIterator): def __init__(self): pass myIterator = MyIterator([1, 2, 3, 4]) # Judge whether it is an iterative object print(isinstance(myIterator, Iterable)) # True # Determine whether it is an iterator print(isinstance(myIterator, Iterator)) # True # Subclass instantiation mySubIterator = MySubIterator() print(isinstance(mySubIterator, Iterator)) # True # Iterate print(next(myIterator)) # 1 print(myIterator.__next__()) # 2 print(next(myIterator)) # 3 print(next(myIterator)) # 4 print(next(myIterator)) # raise StopIteration -
Advantages and disadvantages of iterators:
- advantage: The iterator object represents a data flow that can be called when needed next To get a value; Therefore, it always keeps only one value in memory, For small memory consumption, unlimited data streams can be stored. Better than other containers, all elements need to be stored in memory at one time, such as lists, sets and dictionaries...Wait. - Disadvantages: 1.Cannot get the length of stored elements unless the count is complete. 2.The value is not flexible and can only be taken backwards, next()Always return the next value; Unable to fetch the specified value(Can't be like a dictionary key,Subscript of or list)，Moreover, the life cycle of the iterator object is one-time, and the life cycle ends when the element is iterated.
generator
Definition: in Python, the mechanism of calculating while looping is called generator; At the same time, the generator object is also an iterator object, so it has the characteristics of iterator;
For example, it supports for loop, next() method, etc
Function: the elements in the object are calculated according to some algorithm, and the subsequent elements are continuously calculated during the cycle, so there is no need to create a complete list, so as to save a lot of space. Simple generator: you can get a generator object by changing the list generator [] to ().
# List generation
_list = [i for i in range(10)]
print(type(_list)) # <class 'list'>
print(_list) # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# generator
_generator = (i for i in range(10))
print(type(_generator)) # <class 'generator'>
print(_generator) # <generator object <genexpr> at 0x7fbcd92c9ba0>
# Generator object value
print(_generator.__next__()) # 0
print(next(_generator)) # 1
# Start with the third element!
for x in _generator:
print(x) # 2，3，4，5，6，7，8，9
Because the generator object also has the characteristics of iterators, calling the next() method after the element iteration will cause StopIteration.
Function object generator: the return value of a function object with a yield statement is a generator object.
def gen_generator():
yield 1
def func():
return 1
print(gen_generator(), type(gen_generator()))
# <generator object gen_generator at 0x7fe68b2c8b30> <class 'generator'>
print(func(), type(func()))
# 1 <class 'int'>
def gen_generator():
yield "start"
for i in range(2):
yield i
yield "finish"
gen = gen_generator()
print("from gen Object",next(gen))
print("from gen Object",next(gen))
print("from gen Object",next(gen))
print("from gen Object",next(gen))
#
#Take the first value start from the gen object
#Take the second value 0 from the gen object
#Take the third value 1 from the gen object
#Take the fourth value finish from the gen object
#
# StopIteration
#print("take five values from the gen object", next(gen))
#It's equivalent to
#gen2 = (i for i in ["start",1,2,"finish"])
Note: yield returns only one element at a time. Even if the returned element is an iteratable object, it is returned at one time
def gen_generator2():
yield [1, 2, 3]
s = gen_generator2()
print(next(s)) # [1, 2, 3]
Advanced application of yield generator: send() method, pass the value to yield and return it (it will be returned immediately!);
If None is passed, it is equivalent to next(generator).
def consumer():
r = ''
while True:
n = yield r
if not n:
return
print(f'[CONSUMER] Consuming get params.. ({n})')
if n == 3:
r = '500 Error'
else:
r = '200 OK'
def produce(c):
c.send(None) # Start generator
n = 0
while n < 5:
n = n + 1
print(f'[PRODUCER] Producing with params.. ({n})')
r = c.send(n) # Once n has a value, switch to consumer for execution
print(f'[PRODUCER] Consumer return : [{r}]')
if not r.startswith('200'):
print("If the service returned by the consumer is abnormal, the production will be ended and the consumer will be shut down")
c.close() # Close generator
break
consume = consumer()
produce(consume)
[PRODUCER] Producing with params.. (1)
[CONSUMER] Consuming get params.. (1)
[PRODUCER] Consumer return : [200 OK]
[PRODUCER] Producing with params.. (2)
[CONSUMER] Consuming get params.. (2)
[PRODUCER] Consumer return : [200 OK]
[PRODUCER] Producing with params.. (3)
[CONSUMER] Consuming get params.. (3)
[PRODUCER] Consumer return : [500 Error]
If the service returned by the consumer is abnormal, the production will be ended and the consumer will be shut down
The basic function of yield from iterable syntax is to return a generator object and provide a "data transmission pipeline",
Yield from item is the abbreviation of for item in Item: yield item;
And the internal help us achieve a lot of exception handling, simplifying the coding complexity. yield cannot get the return value of generator return:
def my_generator2(n, end_case):
for i in range(n):
if i == end_case:
return f'When i==`{i}`When, interrupt the program.'
else:
yield i
g = my_generator2(5, 2) # call
try:
print(next(g)) # 0
print(next(g)) # 1
print(next(g)) # End to be triggered here_ Case
except StopIteration as exc:
print(exc.value) # When i==`2 ', interrupt the program.
Using yield from can be simplified to:
def my_generator3(n, end_case):
for i in range(n):
if i == end_case:
return f'When i==`{i}`When the program is interrupted.'
else:
yield i
def wrap_my_generator(generator): # Will my_ The return value of the generator is wrapped into a generator
result = yield from generator
yield result
g = my_generator3(5, 2) # call
for _ in wrap_my_generator(g):
print(_)
# Output:
# 0
# 1
# When i==`2 ', interrupt the program.
"""
yield from There are the following conceptual nouns:
1,Caller: the client (caller) code that calls the delegate generator (in the previous section) wrap_my_generator(g))
2,Delegate generators: including yield from Generator function of expression(packing)，The function is to provide a pipeline for data transmission (above) wrap_my_generator)
3,Sub generator: yield from Generator function object added later (above my_generator3 Instance object for g)
The caller interacts with the generator through this "wrapper function", namely "caller"——>Delegate generator——>Generator function
Here is an example to help you understand
"""
# Sub generator
def average_gen():
total = 0
count = 0
average = 0
while True:
new_num = yield average
if new_num is None:
break
count += 1
total += new_num
average = total / count
# Each return means the end of the current collaboration.
return total, count, average
# Delegate generator
def proxy_gen():
while True:
# Only when the sub generator is about to return, the variable on the left of yield from will be assigned and the subsequent code will be executed.
total, count, average = yield from average_gen()
print("Total incoming {} Number of values, sum:{}，average:{}".format(count, total, average))
# Caller
def main():
calc_average = proxy_gen()
next(calc_average) # Activation process
calc_average.send(10) # Incoming: 10
calc_average.send(None) # send(None) is equal to next(calc_acerage), that is, it will go to average_ return statement in gen
print("================== Reopen collaboration ===================")
calc_average.send(20) # Incoming: 20
calc_average.send(30) # Incoming: 30
calc_average.send(None) # End the process
if __name__ == '__main__':
main()
# Output:
# A total of 1 value is passed in, total: 10, average: 10.0
# ==================Reopen collaboration===================
# A total of 2 values are passed in, total: 50, average: 25.0
Decorator
In one sentence, decorators are nested calls to functions
It is mainly applied in three aspects:
- Print program execution time
- Collect program execution logs
- Used for interface access authentication
Let's start with a simple example
def decorator_get_function_name(func):
"""
Get the name of the running function
:return:
"""
def wrapper(*arg):
"""
wrapper
:param arg:
:return:
"""
print(f"Current running method name:{func.__name__} with params: {arg}")
return func(*arg)
return wrapper
# @func_name is the syntax sugar of python
@decorator_get_function_name
def test_func_add(x, y):
print(x + y)
def test_func_sub(x, y):
print(x - y)
test_func_add(1, 2)
# Output:
# Current running method name: test_func_add with params: (1, 2)
# 3
# If you don't use grammar sugar, you can also use the following methods, and the effect is the same
decorator_get_function_name(test_func_sub)(3, 5)
# Remember the quotation mentioned earlier? We can also use another way to achieve the same 👆 Same effect
dec_obj = decorator_get_function_name(test_func_sub) # This is equivalent to the wrapper object
dec_obj(3,5) # This is equivalent to wrapper(3,5)
# Output:
# Current running method name: test_func_sub with params: (3, 5)
# -2
It is often used for authentication verification. For example, the author will use it for login verification:
def login_check(func):
def wrapper(request, *args, **kwargs):
if not request.session.get('login_status'):
return HttpResponseRedirect('/api/login/')
return func(request, *args, **kwargs)
return wrapper
@login_check
def edit_config():
pass
Multiple decorator instances
def w1(func):
@wraps(func)
def wrapper(*args, **kwargs):
print("Here is the first verification")
return func(*args, **kwargs)
return wrapper
def w2(func):
@wraps(func)
def wrapper(*args, **kwargs):
print("Here is the second check")
return func(*args, **kwargs)
return wrapper
def w3(func):
def wrapper(*args, **kwargs):
print("Here is the third check")
return func(*args, **kwargs)
return wrapper
@w2 # This is actually w2(w1(f1))
@w1 # This is w1(f1)
def f1():
print(f"i`m f1, at {f1}")
f1()
Here is the second check
Here is the first verification
i`m f1, at <function f1 at 0x113fe83a0>
Note: when writing a decorator, you'd better add the wrap of functools before the implementation, which can retain the name and properties of the original function
#No wraps
def my_decorator(func):
def wrapper(*args, **kwargs):
'''decoratord'''
print('Calling decorated function...')
return func(*args, **kwargs)
return wrapper
@my_decorator
def example():
"""Docstring"""
print('Called example function')
print(example.__name__, example.__doc__)
# Add wraps
import functools
def my_decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
'''decorator'''
print('Calling decorated function...')
return func(*args, **kwargs)
return wrapper
@my_decorator
def example():
"""Docstring"""
print('Called example function')
print(example.__name__, example.__doc__)
##########################
wrapper decoratord
example Docstring
Log printing case
from functools import wraps
# This is the decoration function
def logger(func):
@wraps(func)
def wrapper(*args, **kw):
print('I'm going to start the calculation:{} Function:'.format(func.__name__))
# The real implementation is this line.
func(*args, **kw)
print('Aha, I've finished the calculation. Add yourself a chicken leg!!')
return wrapper
@logger
def add(x, y):
print('{} + {} = {}'.format(x, y, x+y))
add(200, 50)
##########################
I'm going to start the calculation: add Function:
200 + 50 = 250
Aha, I've finished the calculation. Add yourself a chicken leg!!
Implementation of decorator with parameters
from functools import wraps
def say_hello(contry):
@wraps(contry)
def wrapper(func):
def deco(*args, **kwargs):
if contry == "china":
print("Hello!")
elif contry == "america":
print('hello.')
else:
return
# # Where the function is actually executed
func(*args, **kwargs)
return deco
return wrapper
@say_hello("china")
def chinese():
print("I come from China.")
@say_hello("america")
def american():
print("I am from America.")
chinese()
american()
#######################
Hello!
I come from China.
hello.
I am from America.
Decorator class
The above are decorators based on function implementation. When reading other people's code, you can often find decorators based on class implementation.
Based on the implementation of class decorator, call and__ init__ Two built-in functions.
init: receive decorated function
call: implement decoration logic.
1. Without parameters
class logger(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print("[INFO]: the function {func}() is running..."\
.format(func=self.func.__name__))
return self.func(*args, **kwargs)
@logger
def say(something):
print("say {}!".format(something))
say("hello")
################
[INFO]: the function say() is running...
say hello!
2. Class with parameters
#Decorator with parameters
class logger(object):
def __init__(self,level='INFO'):
self.level = level
def __call__(self,func):
def wrapper(*args, **kwargs):
print("[{level}]: the function {func}() is running..." \
.format(level=self.level, func=func.__name__))
func(*args,**kwargs)
return wrapper
@logger(level='WARNING')
def say(something):
print("say {}!".format(something))
say("hello")
#########################
[WARNING]: the function say() is running...
say hello!