Introduce

• Property dynamic property
• getattr,__ getattribute__ Magic function
• Attribute descriptor and attribute lookup process
• The difference between new and init
• Custom metaclass
• Implement ORM through metaclass

1, Property dynamic property

from datetime import date

class User:
    def __init__(self, name, birthday):
        self.name = name
        self.birthday = birthday
        self._age = 0   # _  A programming specification

    @property
    def age(self):
        return date.today().year - self.birthday.year

    @age.setter  # Set the age property
    def age(self, value):
        self._age = value

    def get_age(self):
        return self._age


if __name__ == '__main__':
    user = User('linda', date(1987, 1, 1))
    print(user.age)     # @property encapsulates logic in the form of variables, and there is no need to put parentheses after age
    user.age = 30      # @age.setter receive parameters
    print(user.get_age())   # self._ There are stored variables inside the age instance_ age
    

33
30

@The user. Property descriptor can be changed into a property descriptor age (attribute), instead of using user age () (how the function is called).

@property is just a get. It's OK to set, which is @ age setter.

• External display user age; Internal storage self_ age

• There is more logical operation space inside the dynamic property

• user.age = 100 carefully experience the internal processing process

2, getattr__ getattribute__ Magic function

2.1,getattr

• getattr, called when the attribute cannot be found
• Similar to else mechanism

When the age attribute cannot be found in the user instance, the__ getattr__ Inside, then you can__ getattr__ Add your own logic to the.

class User:
    def __init__(self, name):
        self.name = name

    def __getattr__(self, item):
        return 'Not found attribute %s' % item

if __name__ == '__main__':
    user = User('linda')
    print(user.age) # Not found attribute age

class User:
    def __init__(self, info=None):
        if not info:
            info = {}
        self.info = info

    def __getattr__(self, item):
        return self.info[item]


if __name__ == '__main__':
    user = User({'name': 'linda', 'age': 18})
    print(user.name)
    print(user.age)

• Magical proxy operation

2.2,getattribute

getattribute__ The priority of is higher. When searching, it will enter first__ Getattribute, and then find the attribute.

class User:
    def __init__(self, name):
        self.name = name

    def __getattribute__(self, item):
        return 'get_attribute'


if __name__ == '__main__':
    user = User('linda')
    print(user.name)    # get_attribute
    print(user.test)    # get_attribute
    print(user.other)   # get_attribute

• Whenever the attribute is called, getattribute is triggered
• If you control the whole property call entry, try not to rewrite this method__ getattribute__ If you don't write it well, the python program will crash.
• When writing a frame, it involves

3, Attribute descriptor and attribute lookup process

The property implementation adds additional logical processing during data acquisition and setting, and provides a simple interface to the outside world

In batch attribute operations, such as verification, each attribute needs to be written once, and the code is repeated

property descriptor

A class implementation __get__(),__set__(),__delete__() Any one of them is an attribute descriptor

There are two kinds of attribute descriptors:

• Data attribute descriptor: implement get and set methods
• Non data attribute descriptor: implement get method

Data descriptors and non data descriptors do not look up attributes in the same order.

import numbers


class IntField:
    # Data descriptor
    def __init__(self):
        self._data = None

    def __get__(self, instance, owner):
        print(instance)     # <__main__.User object at 0x000002B88B270288>
        print(owner)        # <class '__main__.User'>
        print(type(instance) is owner)          # True
        print(instance.__class__ is owner)      # True
        return self._data

    def __set__(self, instance, value):
        if not isinstance(value, numbers.Integral):
            raise ValueError('Need int value')
        # Here's the point. How to save value, instance or self
        # If instance Attribute will trigger again__ set__  descriptor 
        self._data = value

    def __delete__(self, instance):
        pass


class User:
    age = IntField()
    num = IntField()


if __name__ == '__main__':
    user = User()
    user.age = 18  # This statement is equivalent to executing the in the attribute descriptor__ set__ () method.
    print(user.__dict__)    # {} "age" does not enter__ dict__

    print(user.age)

The code is analyzed as follows:

Change the original simple attribute acquisition order

User a class instance, user Age is equivalent to getattr(user, 'age')

First call __getattribute__
    If defined __getattr__ Method, calling __getattribute__ Throw exception AttributeError trigger__getattr__
    And for descriptors(__get__)The call of occurs in __getattribute__inside

user = User(), call user The sequence is as follows:

(1) If 'age' Yes, it appears in User Or base class __dict__ Medium, and age yes data descriptor，Then call its __get__(instance, owner) Method, otherwise
(2) If 'age' Appear in user of __dict__ , then return directly user.__dict__['age']，otherwise
(3) If 'age' Appear in User Or base class __dict__ in
    (3.1) If age yes non-data descriptor, Then call its __get__ Method, otherwise
    (3.2) return User.__dict__['age']
(4) If User have __getattr__ Method, calling __getattr__ Method, otherwise
(5) Throw exception AttributeError

• Attribute descriptor has the highest priority

class NonDataIntFiled:
    # Non data attribute descriptor
    def __get__(self, instance, owner):
        print(instance)
        print(owner)
        return 100

class User:
    age = NonDataIntFiled()

if __name__ == '__main__':
    user = User()
    # user.__dict__['age'] = 18
    # user.age = 18
    # print(user.__dict__)
    print(user.age)

Share a big man's blog:

Chapter 8 metaclass programming - yueqiudian - blog Park (cnblogs.com)

4, Difference between init and new

• In the custom class, new: is used to control the generation process of the object and return the self object. If there is no return value, init will not be called

  __ new__ Method can customize the generation process of the class

• init in custom class: used to perfect objects, such as initialization

  __ init__ The first parameter of the method is the instance object (initializing the instance object)

• new calls before init

  __ new__ The first parameter of the magic function is class__ new__ Allows us to add logic before generating the User object

class User(object):

    # There are only new classes. Add logic before generating the object user
    def __new__(cls, *args, **kwargs):  # cls is the User class
        # args = ('linda', )
        # kwargs = {'age': 20}
        # And in custom metaclass__ new__  There are differences
        print('from __new__')

    def __init__(self, name, age=18):
        self.name = name
        self.age = age
        print('from __init__')


# new is used to control the generation process of objects before they are generated
# init is used to perfect objects
# If the new method does not return an object, the init function is not called

if __name__ == '__main__':
    user = User('linda', age=20)

Operation results:

from __new__

The process has ended with exit code 0

Since the init function will not be called if the new method does not return an object, let's try to return an object in the new method:

class User(object):

    # There are only new classes. Add logic before generating the object user
    def __new__(cls, *args, **kwargs):
        # args = ('linda', )
        # kwargs = {'age': 20}
        # And in custom metaclass__ new__  There are differences
        print('from __new__')
        self = super().__new__(cls)
        return self

    def __init__(self, name, age=18):
        self.name = name
        self.age = age
        print('from __init__')


if __name__ == '__main__':
    user = User('linda', age=20)

Operation results:

from __new__
from __init__

The process has ended with exit code 0

PS: unified description

• Metaclass - > class object
• Class - > instance

5, Custom metaclass

5.1. Create classes with functions

• Class keyword can literally create a class

def create_class(name):
    if name == 'user':
        class User:
            def __str__(self):
                return 'User'

        return User

    elif name == 'company':
        class Company:
            def __str__(self):
                return 'Company'

        return Company


MyClass = create_class('user')
obj = MyClass()
print(obj)
print(type(obj))  # <class '__main__.create_class.<locals>.User'>

Operation results:

User
<class '__main__.create_class.<locals>.User'>

A class can be dynamically obtained through a function and a string, which is very simple in python. However, this dynamic class creation is still complex. We need to define the class statement in the function ourselves. We just put the statement defining class into the function, which is not flexible. How to create classes dynamically? Instead of writing the syntax of class, you need to use type.

5.2. Dynamically create classes with type

• Type can not only be used to obtain the type of an object
• type can also dynamically create classes and dynamically add attributes and methods

Let's look at the source code of the type method:

 def __init__(cls, what, bases=None, dict=None): # known special case of type.__init__
        """
        type(object_or_name, bases, dict)
        type(object) -> the object's type
        type(name, bases, dict) -> a new type
        # (copied from class doc)
        """
        pass

You can see that there are three parameter construction methods, and the results returned by different construction methods are different.

The first method is used to create a class, passing (name, bases, dict).

Remember this format

type("class_name", (base class), (attribute))

Example code:

# type dynamically creates classes
User = type("User", (), {})
# "User" class name,
# () a tuple (the base class inherited by the passing class), which inherits nothing and must be written
# {} is an attribute, {} if there is no attribute

if __name__ == "__main__":
    User = type("User", (), {})
    my_obj = User()
    print(my_obj)

<__main__.User object at 0x0000016A713E2550>

As you can see, classes are created through type and class.

Let's play with the "second parameter" base class again“

When creating a class, there should also be attributes and internal methods. How to use type to create methods?

Let's play with the "third parameter" attribute“

type("class_name", (base class), (attribute))

Example code 1:

# type dynamically creates classes
User = type("User", (), {})
# "User" class name,
# () a tuple (the base class inherited by the passing class), which inherits nothing and must be written
# {} is an attribute, {} if there is no attribute

if __name__ == "__main__":
    User = type("User", (), {"name":"user"}) 
    # This usage is similar to
    # class:
    # 	name="user"
    my_obj = User()
    print(my_obj.name)

user

Example code 2:

# type dynamically creates classes
User = type("User", (), {})
# "User" class name,
# () a tuple (the base class inherited by the passing class), which inherits nothing and must be written
# {} is an attribute, {} if there is no attribute
def say(self): # Must have self
    """
    and class The methods defined in are the same and need to be passed self
    """
    return "i am user"
    # return self.name

if __name__ == "__main__":
    User = type("User", (), {"name":"user","say":say}) # When passing in a function, it must be a function name, not say()
    my_obj = User()
    print(my_obj.say())

i am user

What if User wants to inherit a base class?

Let's try the "second parameter" base class“

type("class_name", (base class), (attribute))

Example code 1:

# type dynamically creates classes
User = type("User", (), {})
# "User" class name,
# () a tuple (the base class inherited by the passing class), which inherits nothing and must be written
# {} is an attribute, {} if there is no attribute
def say(self): # Must have self
    """
    and class The methods defined in are the same and need to be passed self
    """
    return "i am user"
    # return self.name

class BaseClass:
    def answer(self):
        return "i am baseclass"

if __name__ == "__main__":
    User = type("User", (BaseClass,), {"name":"user","say":say}) 
    # When passing in a function, it must be a function name, not say()
    # In () relay class, you must add,
    my_obj = User()
    print(my_obj.answer())

i am baseclass

Example code 2:

def func(self):
    return 'I am from func.'

class Base:
    def answer(self):
        return 'I am from Base.answer.'

# type dynamically creates classes
User = type('User', (Base, ), {'name': 'user', 'func': func})
user = User()
print(user.name)
print(user.func())
print(user.answer())
print(type(user))

user
I am from func.
I am from Base.answer.
<class '__main__.User'>

The process has ended with exit code 0

5.3 category

What is class? As everyone may know, a class is a "template" used to create objects.

What are metaclasses? In a word, generally speaking, metaclasses are "templates" for creating classes.

Why can type be used to create classes? Because it is a metaclass. It makes sense to use metaclasses to create classes.

Type is a metaclass used by Python to create all classes behind it. The ancestor object of the well-known class is also created by type. What's more, even type itself is created by type itself, which is too much.

1>>> type(type)
2<class 'type'>
3>>> type(object
4<class 'type'>
5>>> type(int)
6<class 'type'>
7>>> type(str)
8<class 'type'>

Metaclass create metaclass (type) - > class - > instance

class MetaClass(type):  # Inherits type, so it is a metaclass
    # It is used to control the creation process of User and the__ new__  There are differences
    def __new__(cls, name, bases, attrs, **kw):
        return super().__new__(cls, name, bases, attrs, **kw)


class User(object, metaclass=MetaClass):  # MetaClass controls the process of User instantiation

    def __init__(self, name):
        self.name = name

    def bar(self):
        print('from bar.')

python instantiates user = User()

1. First look for metaclass to create User, otherwise
2. Look for the metaclass of the base class BaseUser again to create the User. Otherwise
3. Then look for the module metaclass to create User, otherwise
4. Finally, the default type is metaclass to create a User

Under normal circumstances, we will not use metaclasses. But that doesn't mean it doesn't matter. If one day we need to write a framework, we may need to use metaclasses.

But why use it? What happens if you don't want it?

The action process of metaclasses is as follows

1. Creation of interception class
2. Modify after interception
3. After modification, return the modified class

Obviously, using metaclasses is to customize and modify classes. Use metaclasses to dynamically generate instances of metaclasses, and 99% of developers do not need to dynamically modify classes, because this should be considered by the framework.

However, in this way, you will not be convinced. What are metaclasses used for?

In fact, the role of metaclasses is to create API s. One of the most typical applications is Django ORM.

6, Implement ORM through metaclass

ORM

ORM is a technology to complete the operation of relational database through the syntax of instance object. It is the abbreviation of "Object/Relational Mapping". ORM maps the database to objects.

Anyone who has used Django ORM knows that with ORM, it is extremely easy for us to operate the database.

First, clarify the requirements

# demand
class User:
    """
    Class maps to a table in the database, operates on the class, writes the data to the database, and can be separated sql sentence
    """
    name = CharField(db_column="", max_length=10)  # db_column is the name of the column in the database, max_length is the maximum length of the database column
    age = IntField(db_column="", min_value=0, max_value=100)

    class Meta:
        """
        An internal class, which is different from the above column names and defines some other things
        """
        db_table = "user"  # To which table


# ORM
if __name__ == "__main__":
    user = User()
    user.name = "linda"  # Define name column
    user.age = 18  # Define age column
    user.save()  # Save to database

Mini ORM

from collections import OrderedDict


class Field:
    pass


class IntField(Field):
    def __init__(self, db_column, min_value=0, max_value=100):
        self.db_column = db_column
        self.min_value = min_value
        self.max_value = max_value
        self._value = None

    def __get__(self, instance, owner):
        return self._value

    def __set__(self, instance, value):
        if not isinstance(value, int):
            raise TypeError('need int value')
        if value < self.min_value or value > self.max_value:
            raise ValueError('need [%s, %s] value' % (self.min_value, self.max_value))
        self._value = value


class CharField(Field):
    def __init__(self, db_column, max_length=32):
        self.db_column = db_column
        self.max_length = max_length
        self._value = None

    def __get__(self, instance, owner):
        return self._value

    def __set__(self, instance, value):
        if not isinstance(value, str):
            raise TypeError('need str value')
        if len(value) > self.max_length:
            raise ValueError('len need lower than %s' % self.max_length)
        self._value = value


# Metaclass injects a series of attributes
class MetaClass(type):
    def __new__(cls, name, bases, attrs, **kw):
        # BaseModel will also call Metaclass, but there is no definition of name, age and other attributes, which can be judged in a special way
        if name == 'BaseModel':
            return super().__new__(cls, name, bases, attrs, **kw)

        fields = {}
        for key, value in attrs.items():
            if isinstance(value, Field):
                fields[key] = value

        attrs_meta = attrs.get('Meta', None)
        _meta = {}
        db_table = name.lower()
        if attrs_meta is not None:
            table = getattr(attrs_meta, 'db_table', None)
            if not table:
                db_table = table

        _meta['db_table'] = db_table
        attrs['_meta'] = _meta
        attrs['fields'] = fields
        if attrs.get('Meta'):
            del attrs['Meta']
        return super().__new__(cls, name, bases, attrs, **kw)


class BaseModel(metaclass=MetaClass):
    def __init__(self, **kw):
        for key, value in kw.items():
            setattr(self, key, value)
        super().__init__()

    def save(self):
        fields = OrderedDict(self.fields)
        fields_str = ", ".join([value.db_column for value in fields.values()])
        values_str = ', '.join([str(getattr(self, field)) if not isinstance(value, CharField)
                                else "'%s'" % str(getattr(self, field))
                                for field, value in fields.items()])
        sql = "insert into %s (%s) values (%s)" % (self._meta['db_table'], fields_str, values_str)
        print(sql)
        # insert into user (name1, age) values ('linda', 20)


# When customizing a class, write a small number of attributes. Metaclasses help us inject many common attributes
class User(BaseModel):
    name = CharField('name1', max_length=16)
    age = IntField('age', min_value=0, max_value=100)

    class Meta:
        db_table = 'user'


if __name__ == '__main__':
    user = User(name='linda')
    user.age = 20
    user.save()

Operation results:

insert into user (name1, age) values ('linda', 20)

ORM design idea

• The data attribute descriptor (set, get) implements the verification operation
• User defined metaclass (MetaClass(type)) implements parameter injection
• The custom ORM class (BaseModel) obtains the parameters injected by the metaclass for additional operations
• Custom metaclass injection objects
• Special attention should be paid to the calling hierarchy order. new precedes init, so metaclasses can be used to register test parameters in init

Share a few big man Blogs:

Meta class programming for advanced Python development - Wang Yibai - blog Park (cnblogs.com)