Learn from Mr. Chen Ru's Python source code analysis
The source code I used is Python 3.7, so I made some changes
PyObject
In python, everything is an object. Objects have the same content. These are defined in PyObject
[object.h] typedef struct _object { _PyObject_HEAD_EXTRA Py_ssize_t ob_refcnt; struct _typeobject *ob_type; } PyObject;
_ PyObject_ HEAD_ The extra macro actually compiles python in release mode and does not define the symbol PyTRACE_REFS
So #define_ PyObject_ HEAD_ Extra is blank
[object.h] #ifdef Py_TRACE_REFS / / undefined /* Define pointers to support a doubly-linked list of all live heap objects. */ #define _PyObject_HEAD_EXTRA \ struct _object *_ob_next; \ struct _object *_ob_prev; #define _PyObject_EXTRA_INIT 0, 0, #else #define _PyObject_HEAD_EXTRA / / is empty #define _PyObject_EXTRA_INIT #endif /* PyObject_HEAD defines the initial segment of every PyObject. */ #define PyObject_HEAD PyObject ob_base; PyObject ob_base;
Then PyObject becomes
typedef struct _object { Py_ssize_t ob_refcnt; struct _typeobject *ob_type; } PyObject;
Py_ssize_t ob_refcnt is related to python's memory management mechanism. It implements a garbage collection mechanism based on reference count. When the reference count is reduced to 0, it can be deleted from the heap to free memory
struct _typeobject *ob_type is a pointer_ typeObject structure pointer, which specifies the type of an object type. Because the operations performed by the same type are the same, it is placed in ob_ Of the type object pointed to by type
Of course, in addition to this header, each object also has some unique things. The PyObject object is shared by other objects, because if this is the header of the structure, we can easily access the ob in the header information of any structure with a pointer of PyObject *_ Refcnt and ob_type (as long as the head of the structure is fixed with these contents), other different objects have different extras except this head
[longobject.h] typedef struct _longobject PyLongObject; /* Revealed in longintrepr.h */ [longintrepr.h] struct _longobject { PyObject_VAR_HEAD digit ob_digit[1]; };
For an integer, its head is the head of a variable length object (the integer in Python 3 has no size limit, so the source code does not simply use int type to store its value, but uses a dynamic array with variable size. The tail size of this structure will be determined when it is created. Here, the default is 1). This will be determined according to ob_size attribute to determine the size, ob_size is a member that becomes an object (structure) holder
Fixed length and variable length objects
For variable and fixed length objects, the space they need cannot be determined, that is, you can't kill their case. Then the memory size of the data storage block that needs to be allocated by this structure needs to be changed according to the length of the actual object
[object.h] typedef struct { PyObject ob_base; //A fixed length structure PyObject Py_ssize_t ob_size; /* Number of items in variable part */ } PyVarObject;
This is the structure head of a variable length object. You can see that its head is first a fixed length object structure, and then an ob that controls the length_ Size, from the head, that is, variable length objects can also be accessed through PyObject *_ refcntļ¼ob_type, because their offsets are the same, ob_ The size member actually indicates how many elements are contained in the variable length object
Type object
[object.h] typedef struct _typeobject { PyObject_VAR_HEAD const char *tp_name; /* For printing, in format "<module>.<name>" */ Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */ /* Methods to implement standard operations */ destructor tp_dealloc; printfunc tp_print; getattrfunc tp_getattr; setattrfunc tp_setattr; PyAsyncMethods *tp_as_async; /* formerly known as tp_compare (Python 2) or tp_reserved (Python 3) */ reprfunc tp_repr; /* Method suites for standard classes */ PyNumberMethods *tp_as_number; PySequenceMethods *tp_as_sequence; PyMappingMethods *tp_as_mapping; ... ... #ifdef COUNT_ALLOCS /* these must be last and never explicitly initialized */ Py_ssize_t tp_allocs; Py_ssize_t tp_frees; Py_ssize_t tp_maxalloc; struct _typeobject *tp_prev; struct _typeobject *tp_next; #endif } PyTypeObject;
PyObject_VAR_HEAD macro
#define PyObject_VAR_HEAD PyVarObject ob_base;
As you can see_ The object object is the head of a variable length object, that is, it has ob_size member, and like PyVarObject, the header can be accessed ob with PyObject *_ refcnt, *ob_ type
Object behavior
[object.h] typedef struct _typeobject { PyObject_VAR_HEAD const char *tp_name; /* For printing, in format "<module>.<name>" */ Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */ /* Methods to implement standard operations */ destructor tp_dealloc; printfunc tp_print; getattrfunc tp_getattr; setattrfunc tp_setattr; PyAsyncMethods *tp_as_async; /* formerly known as tp_compare (Python 2) or tp_reserved (Python 3) */ reprfunc tp_repr; /* Method suites for standard classes */ PyNumberMethods *tp_as_number; PySequenceMethods *tp_as_sequence; PyMappingMethods *tp_as_mapping; ... ... } PyTypeObject;
PyNumberMethods *tp_as_number; PySequenceMethods *tp_as_sequence; PyMappingMethods *tp_as_mapping;
It can be seen from the name that these three methods are the method of numbers, the method of sequences and the method of dictionaries
This is why in python, as long as we implement specific methods, we can obtain sequence operations, such as implementation__ add__ Method can realize addition__ getitem__ Sequence related operations can be realized
Type of type
Because the head of PyTypeObject is a PyObject_VAR_HEAD(PyVarObject ob_base) indicates that there is also a (PyObject) nested in the head, and there is an ob in the PyObject type_ The type pointer points to a PyTypeObject
[object.h] typedef struct _object { _PyObject_HEAD_EXTRA Py_ssize_t ob_refcnt; struct _typeobject *ob_type; } PyObject;
The type of type is ob_type still points to a PyTypeObject. The following is the structure it points to, which initializes some data
[typeobject.c] PyTypeObject PyType_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "type", /* tp_name */ sizeof(PyHeapTypeObject), /* tp_basicsize */ sizeof(PyMemberDef), /* tp_itemsize */ (destructor)type_dealloc, /* tp_dealloc ... };
Header macro
#define PyObject_HEAD_INIT(type) \ { _PyObject_EXTRA_INIT \ 1, type }, #define PyVarObject_HEAD_INIT(type, size) \ { PyObject_HEAD_INIT(type) size },
It can be seen that the value of PyVarObject header is initialized. The default value is
{ { _PyObject_EXTRA_INIT //As mentioned earlier, it is blank 1 // Reference count is 1 type: PyType_Type Point to yourself } PyObject Structure part size: 0 //Variable length object. The default length is 0 }PyVarObject Structure part
summary
Some are their own understanding, but there are mistakes. You are welcome to correct them~