Any language that regards function as a first class object has to face a problem: a function as a first class object is defined in a scope, but it may be invoked in other scopes. How to deal with free variables?

free variable, a variable that is not bound in the local scope.

In order to solve this problem, Guido Van Rossum, the father of python, designed closures, which are like a magic pen and show the aesthetics of the code. Before discussing closures, it is necessary to understand the variable scope in Python.

Variable scope

Coupon website: www.cps3.com cn

Let's take a look at an example of global variables and free variables:

>>> b = 6
>>> def f1(a):
...     print(a)
...     print(b)
...     
>>> f1(3)
3
6

b outside the function is a global variable and b inside the function is a free variable. Because the free variable b is bound to the global variable, print can be correctly in function f1().

If you change it slightly, b in the function body will change from a free variable to a local variable:

>>> b = 6
def f1(a):
...     print(a)
...     print(b)
...     b = 9
...     
>>> f1(3)
3
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "<input>", line 3, in f1
UnboundLocalError: local variable 'b' referenced before assignment

Add b = 9 after function f1(): error: local variable b is referenced before assignment.

This is not a defect, but Python design: Python does not require variables to be declared, but assumes that the variables assigned in the function definition body are local variables.

If you want the interpreter to treat b as a global variable, you need to use the global declaration:

>>> b = 6
>>> def f1(a):
...     global b
...     print(a)
...     print(b)
...     b = 9
...     
>>> f1(3)
3
6

closure

Back to the free variable problem at the beginning of the article, if there is a function called avg, its function is to calculate the mean value of a series of values, which is realized by class:

class Averager():
    
    def __init__(self):
        self.series = []
        
    def __call__(self, new_value):
        self.series.append(new_value)
        total = sum(self.series)
        return totle / len(self.series)

avg = Averager()
avg(10)  # 10.0
avg(11)  # 10.5
avg(12)  # 11.0

Class implementation does not have the problem of free variables because self Series is a class attribute. However, in function implementation, problems arise when functions are nested:

def make_averager():
    series = []
    
    def averager(new_value):
        # series is a free variable
        series.append(new_value)
        total = sum(series)
        return totle / len(series)
    
    return averager

avg = make_averager()
avg(10)  # 10.0
avg(11)  # 10.5
avg(12)  # 11.0

Function make_averager() defines the series variable in the local scope, and the free variable series of its internal function averager() binds this value. However, when calling avg(10), make_ The averager() function has return ed, and its local scope has disappeared. If there is no closure, the free variable series will report an error and cannot find the definition.

So how do closures work? A closure is a function that retains the bindings of free variables that exist at the time of definition, so that when a function is called, those bindings can still be used although the definition scope is unavailable.

As shown in the figure below:

The closure will retain the binding of the free variable series and continue to use this binding when calling avg(10), even if make_ The local scope of the averager() function has disappeared.

nonlocal

Slightly optimize the requirements of the above example, and only store the current total value and the number of elements:

def make_averager():
    count = 0
    total = 0
    
    def averager(new_value):
        count += 1
        total += new_value
        return total / count
        
    return averager

An error will be reported after running: the local variable count is referenced before assignment. Because count +=1 is equivalent to count = count + 1, there is an assignment, and count becomes a local variable. So is total.

It is obviously inappropriate to declare count and total as global variables through the global keyword. Their scope is only extended to make at most_ Within the averager() function. To solve this problem, python 3 introduces the nonlocal keyword declaration:

def make_averager():
    count = 0
    total = 0
    
    def averager(new_value):
        nonlocal count, total
        count += 1
        total += new_value
        return total / count
        
    return averager

The function of nonlocal is to mark the variable as a free variable. Even if the variable is assigned a value in the function, it is still a free variable.

Note that for variable types such as lists and dictionaries, adding elements is not an assignment, and local variables will not be created implicitly. For immutable types such as numbers, strings, tuples, and None, assignment implicitly creates local variables. Example:

def make_averager():
    # Variable type
    count = {}

    def averager(new_value):
        print(count)  # success
        count[new_value] = new_value
        return count

    return averager

Adding elements to variable objects is not assignment, and local variables are not implicitly created.

def make_averager():
    # Immutable type
    count = 1

    def averager(new_value):
        print(count)  # report errors
        count = new_value
        return count

    return averager

Count is an immutable type. The assignment will implicitly create a local variable. An error is reported: the local variable count is referenced before the assignment.

def make_averager():
    # None
    count = None

    def averager(new_value):
        print(count)  # report errors
        count = new_value
        return count

    return averager

Count is None, and the assignment will implicitly create a local variable. An error is reported: the local variable count is referenced before the assignment.

Summary

This paper first introduces the concepts of global variable, free variable and local variable, which is the premise of understanding closures. When a nested variable is bound, the problem of how to handle it will be solved even if the nested variable has disappeared. For immutable types and None, assignment will implicitly create local variables and convert free variables into local variables, which may lead to program error: local variables are referenced before assignment. In addition to using global declaration as global variables, you can also use nonlocal declaration to force local variables into free variables to realize closure.

reference material:

Fluent Python