Finally, we decided to choose pybind11 for the following reasons:

1. It looks much more human than python's native C API
2. My C + + code is not ready-made and requires a certain amount of C + + development workload, so I feel that Python is not very convenient. If the C + + interface has been given, it may be better to simply wrap it.
3. pybind11 claims that it only contains header files and can be installed through pip. It feels better than boost_python is lightweight and finally this extension package is easy to distribute. In addition, I feel that its documents are much more friendly than boost python

Setuptools

reference resources Official Setuptools build documentation

This method is suitable for the one-stop service of building, packaging, distributing and uploading python packages to PyPi. python needs to use C + + Extension in setup Py. Here is a setup Example of Py:

import glob
import os.path
from distutils.core import setup

__version__ = "0.0.1"

# make sure the working directory is BASE_DIR
BASE_DIR = os.path.dirname(__file__)
os.chdir(BASE_DIR)

ext_modules = []

try:
    from pybind11.setup_helpers import Pybind11Extension, ParallelCompile, naive_recompile

    # `N` is to set the bumer of threads
    # `naive_recompile` makes it recompile only if the source file changes. It does not check header files!
    ParallelCompile("NPY_NUM_BUILD_JOBS", needs_recompile=naive_recompile, default=4).install()

    # could only be relative paths, otherwise the `build` command would fail if you use a MANIFEST.in to distribute your package
    # only source files (.cpp, .c, .cc) are needed
    source_files = glob.glob('source/path/*.cpp', recursive=True)

    # If any libraries are used, e.g. libabc.so
    include_dirs = ["INCLUDE_DIR"]
    library_dirs = ["LINK_DIR"]
    # (optional) if the library is not in the dir like `/usr/lib/`
    # either to add its dir to `runtime_library_dirs` or to the env variable "LD_LIBRARY_PATH"
    # MUST be absolute path
    runtime_library_dirs = [os.path.abspath("LINK_DIR")]
    libraries = ["abc"]

    ext_modules = [
        Pybind11Extension(
            "package.this_package", # depends on the structure of your package
            source_files,
            # Example: passing in the version to the compiled code
            define_macros=[('VERSION_INFO', __version__)],
            include_dirs=include_dirs,
            library_dirs=library_dirs,
            runtime_library_dirs=runtime_library_dirs,
            libraries=libraries,
            cxx_std=14,
            language='c++'
        ),
    ]
except ImportError:
    pass

setup(
    name='project_name',  # used by `pip install`
    version='0.0.1',
    description='xxx',
    ext_modules=ext_modules,
    packages=['package'], # the directory would be installed to site-packages
    setup_requires=["pybind11"],
    install_requires=["pybind11"],
    python_requires='>=3.8',
    include_package_data=True,
    zip_safe=False,
)

Some points needing attention (pits):

• If you need to publish the package through sdist (i.e. the source code of. tar.gz), the source of the Extension_ The files field must be a relative path. Otherwise, when build ing, it will be because of the source in egg info Txt has an absolute path and an error is reported. But the problem is that we are not sure to run setup What is the working directory in py? To be safe, you need to set it to setup Directory where py is located.

• Before installing the package, setuptools will run setup. Net first to get some metadata Py. If pybind11 is not installed at this time, an error will be reported. To solve this problem:

  • In order to execute the setup function normally, we need to ensure that there will be no error when executing this file without pybind11. So we need to put all the dependencies on pybind11 setup_ All the helpers are put in try.

    There are other methods, such as copying a setup directly_ Helpers or something. See the documentation for details.

  • according to Documentation for setuptools ，setup_requires does not install packages, so pybind11 also needs to be added to install_ Requirements.
  • Finally, before installing this package, setuptools will install dependencies first, and then run setup Py install, and then you can successfully build and install.

• If your add-on library is not in the specified path for the system to find the dynamic library, specify link_ After dirs, there will be no errors in compilation and linking. However, errors will still be reported when the dynamic library cannot be found. You can add runtime_library_dirs (equivalent to - Wl,-rpath), or LD_ LIBRARY_ Add this path to the path environment variable.

• Compiled The location of so and the name of your C++ module in python depend on the name you write for the Extension. For example, you want the file structure to be like this:

  project_dir
      |-- package
      |   |-- __init__.py
      |   |-- this_package.xxxx.so
      |   |-- other.py
      |-- setup.py
  

  In this way, you will only create a new package called package in the site packages.

  In addition, even if your project only wants to export one The module in so will be better packaged in a folder. Because if you just want to export this_ Package, change the configuration in the setup function to packages=['this_package '] So files will be directly added to site packages, which doesn't feel very elegant.

  At this time, the name of your Extension needs to be package this_ package. So The location of so is right. You run import package this_ Package will be found correctly So and executed.
  However, the implementation should be guaranteed so no error, pass pybind11 in C + +_ Module when exposing this extension to python, the name should also correspond to:

  PYBIND11_MODULE(this_package, m) {}
  

CMake

reference resources Official CMake build documentation.

If you are compiling C + + programs embedded in python, you can use CMake, which is more convenient.

Although it seems that CMake can also be used for python extension, setuptools is more convenient.

I mainly use CMake to compile the test of C + + part. CMakeLists.txt looks like this:

# the CMakeList to test the C++ part from a C entry point
cmake_minimum_required(VERSION 3.21)
project(project_name)

set(CMAKE_CXX_STANDARD 14)

# Find pybind11
find_package(pybind11 REQUIRED)

# If any library (e.g. libabc.so) is needed
include_directories(INCLUDE_DIR)
link_directories(LINK_DIR)

# Add source file
file(GLOB A_NAME_FOR_SOURCE CONFIGURE_DEPENDS "source/path/*.cpp")
file(GLOB A_NAME_FOR_TEST CONFIGURE_DEPENDS "test/path/*.cpp")
add_executable(TARGET_NAME test_cpp_part.cpp ${A_NAME_FOR_SOURCE} ${A_NAME_FOR_TEST})
target_link_libraries(TARGET_NAME abc pybind11::embed)

• project_name writes casually
• TARGET_NAME, just add_executable and target_link_libraries is the name of the last executable file
• A_NAME_FOR_SOURCE and A_NAME_FOR_TEST is an intermediate variable name of CMake. Write it casually. They represent a pile of source files found by GLOB and a pile of files used for testing
• INCLUDE_DIR is the header file of abc library, LINK_DIR must contain library files. Dynamic libraries are similar to libabc So, the static library is similar to libabc a.
• The reason to Link to pybind11:: embedded is to prevent the part of C + + code with python object from compiling failure.