Python learning note 22: functional programming

Functional programming here does not refer to process oriented programming. More is a description of the flexibility of functions in programming in a programming language that takes functions as first-class objects.

First class object

As for what is a first-class object, Fluent Python explains that if an object is a first-class object, it will have the following characteristics:

• Runtime creation

• Can be assigned to a variable or an element in a container

• It can be passed as a function parameter

• Can be used as the return value result of a function

In my impression, functions in JavaScript and Python have such features and have considerable flexibility. Other languages lack one feature or another.

Function object

As in Python learning notes 0: Variables As explained in, Python is an object-based language, and functions are also objects.

def hello():
    '''Hello'''
    print("Hello world!")

print(hello)
print(type(hello))
print(hello.__doc__)
# <function hello at 0x000001D402AF39D0>
# <class 'function'>
# Hello

Through type, we can clearly see that the custom function hello is an instance of the function class, and its properties can be accessed, such as__ doc__.

In addition, function objects can also be used as parameters and return values, which is clearly reflected when building function modifiers. You can read what you need to review and understand Python learning note 11: function modifiers , I won't go into too much detail here.

Higher order function

We have explained that the function itself can be passed as a parameter, and the function receiving the function through the parameter is called a high-order function.

I know it's funny to say that, but the meaning is not difficult to understand.

There are several high-order functions in common Python functions:

sorted

sorted is often used for sequence sorting. Previously, in Python learning notes 19: List III We introduced it in.

We know that sorted can specify a parameter key to change the default sorting principle.

l = ["aa", "b", "ccc"]
print(sorted(l, key=len))
# ['b', 'aa', 'ccc']

As shown above, key changes the default sorting principle of sorted by receiving single parameter function objects.

So sort is a higher-order function.

map

The map function works as its name reveals. You can apply a function to an iteratable object.

persons = [('Jack chen', 16, 'actor'),
           ('Brus lee', 20, 'engineer')]


def formatPerson(person: tuple):
    return "name:%s,age:%s,actor:%s" % (person[0], str(person[1]), person[2])


formatPersons = list(map(formatPerson, persons))
print(formatPersons)
# ['name:Jack chen,age:16,actor:actor', 'name:Brus lee,age:20,actor:engineer']

The return value of map is also an iteratable object, so we can use list() to undertake and further process it.

The formatPerson function in this example is relatively simple, so we can also rewrite it with an anonymous function:

persons = [('Jack chen', 16, 'actor'),
           ('Brus lee', 20, 'engineer')]

formatPersons = list(map(lambda person: "name:%s,age:%s,actor:%s" % (
    person[0], str(person[1]), person[2]), persons))
print(formatPersons)
# ['name:Jack chen,age:16,actor:actor', 'name:Brus lee,age:20,actor:engineer']

But the readability of such code is not high. Fortunately, python 3 provides two new features: derivation and generator. We can complete similar work with derivation, which is more readable.

persons = [('Jack chen', 16, 'actor'),
           ('Brus lee', 20, 'engineer')]

formatPersons = ["name:%s,age:%s,actor%s" % (name,age,career) for name,age,career in persons]
print(formatPersons)
# ['name:Jack chen,age:16,actor:actor', 'name:Brus lee,age:20,actor:engineer']

Therefore, due to the existence of derivation and generator, map is not used frequently in Python 3.

reduce

The reduce function accepts a two parameter function that will be used to process an iteratable object.

Different from map, the processing logic of reduce function is "cumulative processing". That is, every time the element of an iteratable object is processed, the result will be taken as a parameter in the next processing.

The most common concept of this treatment is in mathematics Factorial.

We use reduce to complete a factorial function:

from functools import reduce


def factorial(n):
    return reduce(lambda a, b: a*b, range(1, n+1))


print(factorial(1))
print(factorial(3))
print(factorial(5))
# 1
# 6
# 120

Anonymous function

As shown earlier, in some scenarios where higher-order functions are used, we may need to use anonymous functions, which will avoid having to create "temporary functions" that are not used frequently.

The syntax of anonymous functions is lambda args:expression.

Where args is the parameter list received by the anonymous function, and expression is the expression as the return value of the anonymous function.

Compared with Java or other mainstream languages, Python's use of anonymous functions is quite limited because of its peculiar syntax characteristics (no {} function body).

The function body of its anonymous function can only contain simple expressions, and can not write complex multi line code.

Callable object

overview

Callable objects are objects that can be executed in obj().

The Python manual summarizes it:

• Custom function
• Built in function
• Built in method
• Custom method
• class
• Class instance
• generator function

The more special ones are class instances and generator functions, which are described in previous articles Python learning notes 16: generator As described in, class instances as callable objects will be introduced later.

callable

In Python, to judge whether an object is a callable object, you can judge it through the callable function.

def hello():
    print("Hello world!")
class Test():
    pass
test = Test()

print(callable(hello))
print(callable(Test))
print(callable(test))
print(callable(len))
# True
# True
# False
# True

The reason why it does not implement (print) is that it does not implement (print)__ call__， We can see examples of callable instances later.

Callable class instance

Python can implement magic methods__ call__ Turn an instance of a class into a callable instance.

This may confuse programmers who have changed careers from other languages.

class Person():
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def __call__(self):
        print("Name:%s Age:%s" % (self.name, self.age))


JackChen = Person('Jack Chen', 16)
BrusLee = Person('Brus Lee', 20)
JackChen()
BrusLee()
# Name:Jack Chen Age:16
# Name:Brus Lee Age:20

Because the current Python actual development experience is not rich, it is difficult for me to judge the necessity of this feature. It may have advantages in building IDE or some special underlying tools.

Function properties

As an object of function class, function naturally has many properties. We can view them through dir function:

def hello():
    print('Hello world!')


print(dir(hello))
# ['__annotations__', '__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__get__', '__getattribute__', '__globals__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__kwdefaults__', '__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']

dir() returns a list of all the attributes.

Interestingly, we can use set operation to further filter out the attributes unique to the function but not in the class:

def hello():
    print('Hello world!')


class Test():
    pass


print(set(dir(hello))-set(dir(Test)))
# {'__annotations__', '__kwdefaults__', '__qualname__', '__call__', '__globals__', '__code__', '__name__', '__get__', '__defaults__', '__closure__'}

parameter

The parameter definition of function signature in Python is quite flexible, which we Python learning note 11: function modifiers I've seen it in.

However, in addition to variable length positioning parameters and variable length keyword parameters, there is also a keyword only parameter.

• The parameters passed here refer to the location parameters.
• ”Variable length positioning parameter "refers to the existence of * args in the parameter signature.
• ”Variable length keyword parameter refers to the existence of * * kwArgs in parameter signature.

Keyword parameters only

This is a rather puzzling noun, but I can't think of an appropriate title. Maybe the keyword limiting parameters are also appropriate, but they are not more helpful to understand.

To put it bluntly, keyword only parameters refer to those parameters that cannot be passed through positioning parameters, but can only be passed through explicit keywords.

Let's illustrate with an example:

def readFile(fileName, *, encoding='UTF-8'):
    content = ''
    with open(file=fileName, encoding=encoding, mode='r') as fopen:
        content = fopen.read()
    return content

print(readFile('test.txt'))
print(readFile('test.txt',encoding='UTF-8'))
print(readFile('test.txt','UTF-8'))
# Hello world!

# Hello world!

# Traceback (most recent call last):
#   File "d:\workspace\python\test\test.py", line 9, in <module>
#     print(readFile('test.txt','UTF-8'))
# TypeError: readFile() takes 1 positional argument but 2 were given

It can be seen that the parameter encoding in the example can only be passed by explicitly indicating the keyword or by default, not by locating the parameter. This is the so-called keyword only parameter.

Function signatures such as fileName, *, encoding='UTF-8 'may be confusing. In fact, * is a variable length positioning parameter that omits the variable name. Because the declaration of keyword only parameters must be after the variable length positioning parameters, you can write this if you need to declare a keyword only parameter but do not need to use the variable length positioning parameters at the same time. This is actually the same as when unpacking_ It's similar.

It should be emphasized that except that it must be declared that after the variable length positioning parameter, only the keyword parameter has no other restrictions, and the default value can also be defined.

Get parameter information

Programming languages often provide some underlying technologies that are not used in normal application development, such as class mapping in Java.

Such mechanisms provide language level support for the development of IDE or other underlying tools.

In Python, we can also detect the parameter composition information of functions through some ways.

Function properties

Before Function properties In, we filter out some unique properties of functions. Some of these attributes record the parameter information of the function, which can be used for parameter analysis.

• __ defaults__， The default values of common parameters are saved.
• __ kwdefaults__， The default values for keyword only parameters are saved.
• __ code__.co_varnames, which saves the variable names in the function.

def person(name, age=15, *args, career='actor', **kwArgs):
    test = 'a function to test'


print(person.__defaults__)
print(person.__kwdefaults__)
print(person.__code__)
print(person.__code__.co_varnames)
# (15,)
# {'career': 'actor'}
# <code object person at 0x000002459B5BB5B0, file "d:\workspace\python\test\test.py", line 1>
# ('name', 'age', 'career', 'args', 'kwArgs', 'test')

You can see that the code information of the function is saved in__ code__ In, this is a code object, and the parameter name exists__ code__.co_varnames, but it also contains the variable name in the function body.

This is undoubtedly inconvenient. Fortunately, Python provides two convenient ways to analyze parameters.

inspect

from inspect import signature


def person(name, age=15, *args, career='actor', **kwArgs):
    test = 'a function to test'


personSig = signature(person)
print(personSig)
for name, param in personSig.parameters.items():
    print(param.kind, ':', name, '=', param.default)

# (name, age=15, *args, career='actor', **kwArgs)
# POSITIONAL_OR_KEYWORD : name = <class 'inspect._empty'>
# POSITIONAL_OR_KEYWORD : age = 15
# VAR_POSITIONAL : args = <class 'inspect._empty'>
# KEYWORD_ONLY : career = actor
# VAR_KEYWORD : kwArgs = <class 'inspect._empty'>

It can be seen that the parameters of a given function can be easily and intuitively analyzed through module inspect.

It is worth noting that through param Kind, you can easily distinguish the parameter types:

• POSITIONAL_OR_KEYWORD: common parameter, which can also be regarded as positioning parameter.
• VAR_ Position: variable length positioning parameter.
• KEYWORD_ONLY: qualified keyword parameter.
• VAR_KEYWORD: variable length keyword parameter.

In addition to analyzing the composition of function parameters, inspect also provides a "binding" parameter list to detect whether the parameters are passed correctly and analyze the function of parameter allocation.

from inspect import signature


def person(name, age=15, *args, career='actor', **kwArgs):
    test = 'a function to test'


personSig = signature(person)
jackChen = {'name': 'Jack Chen', 'age': 16,
            'career': 'actor', 'other': 'no message'}
bindArgs = personSig.bind(**jackChen)
for name, value in bindArgs.arguments.items():
    print("%s=%s" % (name, value))
# name=Jack Chen
# age=16
# career=actor
# kwArgs={'other': 'no message'}

This is very useful for us to analyze the problems encountered in parameter passing.

Function Annotations

The so-called function annotation refers to the Parameter annotation of a function. For example:

def person(name:str, age:int=15, *args:tuple, career:str='actor', **kwArgs:dict)->None:
    test = 'a function to test'

It should be noted that we can add type descriptions to the parameters through annotations, but the Python interpreter does not do type detection, which will only help improve the readability of the code.

Compared with other mainstream languages, Python's function annotation function is quite "weak".

For the time being, as a weakly typed language, PHP is much better in this regard:

<?php

/**
 * Create person
 *@param string $name full name
 *@param int $age Age
 *@param string $career occupation  
 *@return void
 */
function person($name, $age, $career)
{;
}

Intuitive, efficient and humanized. This annotation method is in the same vein as Java.

But anyway, we are also a humble learner, not the boss of the programming community, so it's useless to say more, just learn.

Python's function annotations are stored in properties__ annotations__ Medium:

from inspect import signature


def person(name:str, age:int=15, *args:tuple, career:str='actor', **kwArgs:dict)->None:
    test = 'a function to test'


print(person.__annotations__)
# {'name': <class 'str'>, 'age': <class 'int'>, 'args': <class 'tuple'>, 'career': <class 'str'>, 'kwArgs': <class 'dict'>, 'return': None}

It should be noted that if the parameter has a default value, the default value should be written after the annotation. I often make mistakes...

Similarly, we can use the inspect module to extract comments:

from inspect import signature


def person(name: str, age: int = 15, *args: tuple, career: str = 'actor', **kwArgs: dict) -> None:
    test = 'a function to test'


personSig = signature(person)
for name, param in personSig.parameters.items():
    print(param.annotation, name)
# <class 'str'> name
# <class 'int'> age
# <class 'tuple'> args
# <class 'str'> career
# <class 'dict'> kwArgs

Packages that support functional programming

operator

The operator package provides some basic operations, largely to avoid building anonymous functions frequently.

mul

We are introducing higher-order functions reduce A factorial function was created when:

from functools import reduce


def factorial(n):
    return reduce(lambda a, b: a*b, range(1, n+1))


print(factorial(1))
print(factorial(3))
print(factorial(5))
# 1
# 6
# 120

We can rewrite this code with the mathematical function mul to avoid using anonymous functions:

from functools import reduce
from operator import mul


def factorial(n):
    return reduce(mul, range(1, n+1))


print(factorial(1))
print(factorial(3))
print(factorial(5))
# 1
# 6
# 120

mul is completely equivalent to anonymous function.

itemgetter

Itemsetter is used to get elements from an iteratable object.

from functools import reduce
from operator import itemgetter
persons = [('Jack chen', 16, 'actor'), ('Brus lee', 20, 'engineer')]
getName = itemgetter(0)
getCareer = itemgetter(2)
for person in persons:
    print(getName(person),'->', getCareer(person))
# Jack chen -> actor
# Brus lee -> engineer

It can be seen that itemsetter receives a parameter to describe the obtained index and returns a callable object. Through this callable object, we can obtain the fixed position elements from the iteratable object.

Of course, there is no particularity in this example, because unpacking can be used. Here is a more appropriate example:

from functools import reduce
from operator import itemgetter
from pprint import pprint
persons = [('Jack chen', 16, 'engineer'), ('Brus lee', 20, 'actor')]
getCareer = itemgetter(2)
persons.sort(key=getCareer)
pprint(persons)
# [('Brus lee', 20, 'actor'), ('Jack chen', 16, 'engineer')]

This example sorts the people list by occupation.

Including persons Sort (key = getcareer) and persons Sort (key = lambda person: person [2]) is equivalent.

We build a callable object to get the third element from the iteratable object through getcareer = itemsetter (2), and then pass this object to the sort parameter key.

attrgetter

attrgetter is similar to itemsetter, but the element is obtained not by subscript, but by attribute name.

from collections import namedtuple
from operator import attrgetter
person = namedtuple('person', 'name age career')
jackChen = person('Jack Chen', 16, 'actor')
brusLee = person('Brus Lee', 20, 'engineer')
getName = attrgetter('name')
getCareer = attrgetter('career')
print(getName(jackChen), '->', getCareer(jackChen))
print(getName(brusLee), '->', getCareer(brusLee))
# Jack Chen -> actor
# Brus Lee -> engineer

In particular, attrgetter also supports nested calls. Let's take a more complex example:

from collections import namedtuple
from operator import attrgetter
person = namedtuple('person', 'name age career favorites')
favorites = namedtuple('favorites', 'music dog cat')
jackChen = person('Jack Chen', 16, 'actor', favorites(False, True, True))
brusLee = person('Brus Lee', 20, 'engineer', favorites(False, False, True))
getName = attrgetter('name')
getCareer = attrgetter('career')
isLikeCat = attrgetter('favorites.cat')
isLikeDog = attrgetter('favorites.dog')
print(getName(jackChen), 'like' if isLikeDog(jackChen) else 'not like', 'dog')
print(getName(brusLee), 'like' if isLikeDog(brusLee) else 'not like', 'dog')
# Jack Chen like dog
# Brus Lee not like dog

As you can see, through Operators, we can make nested access.

methodcaller

Through methodcaller, we can call the specified method of the instance.

from operator import methodcaller
toUpper = methodcaller('upper')
s = 'abcdefg'
print(s.upper())
print(toUpper(s))
# ABCDEFG
# ABCDEFG

In the example, the effect of s.upper() and toUpper(s) is exactly the same.

Of course, this example has no practical use, just for demonstration.

functools

Similarly, the functools package also provides some useful higher-order functions.

For example, as mentioned before reduce , and the part to be introduced next.

partial

partial provides an interesting feature that can "solidify" some parameters of functions.

Suppose we have a calculator function:

def calculator(mode, x, y, opertor):
    if mode == 'simple':
        pass
    elif mode == 'math':
        pass
    else:
        pass

Through the parameter mode, we can decide whether the current calculator is a simple mode or a complex mode such as scientific calculation.

If we need to provide this function to some users who only use simple operations and do not use complex modes at all, we can "solidify" a simple calculator through partial:

from functools import partial


def calculator(mode, x, y, opertor):
    if mode == 'simple':
        print('this is a simple calculator')
        pass
    elif mode == 'math':
        pass
    else:
        pass


simpleCal = partial(calculator, 'simple')
simpleCal(1, 2, 'add')

It can be seen that users use the calculator function normally through simpleCal, and will not find that this is a simple version quickly generated by "curing".

Well, after discussing the relevant contents of functional programming, I actually spent a whole afternoon writing this blog. I hope someone will like it.

Thank you for reading.

By the way, I forgot to attach the mind map: