SciTech-Python-编译Python的C/C++扩展的setup.py使用pybind映射C/C++到Python库
- pybind:pybind11 — Seamless operability between C++11 and Python
header-only library exposes C++ types in Python and vice versa
,inferring type information using compile-time introspection
.- mainly to create Python bindings of existing C++ code.
- Its goals and syntax are similar to the excellent Boost.Python library by David Abrahams: to minimize boilerplate code in traditional extension modules
- Think of this library as a tiny self-contained version of Boost.Python with everything stripped away that isn’t relevant for binding generation. Without comments, the core header files only require ~4K lines of code and depend on Python (3.6+, or PyPy) and the C++ standard library. This compact implementation was possible thanks to some of the new C++11 language features (specifically: tuples, lambda functions and variadic templates). Since its creation, this library has grown beyond Boost.Python in many ways, leading to dramatically simpler binding code in many common situations.
- pybind(
pybind11-config
)
-
Install:
pip install pybind11 cmake
: as a standard Python package:pip install "pybind11[global]" cmake
: want available directly in your env. root:
it will add files to:
/usr/local/include/pybind11
/usr/local/share/cmake/pybind11
-
Command-line Usage:
pybind11-config [--version] [--includes] [--cmakedir] [--pkgconfigdir]
pybind11-config --includes
:
-I<PYTHON_INSTALL_DIR>/include/python3.12
-I<PY_LIB_DIR>/site-packages/pybind11/include
pybind11-config --cmakedir
:
<PY_LIB_DIR>/site-packages/pybind11/share/cmake/pybind11
pybind11-config --pkgconfigdir
:
<PY_LIB_DIR>/site-packages/pybind11/share/pkgconfig
-
CMake 直接包含 pybind 的头文件方式:
if(APPLE)
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-everything -w -undefined dynamic_lookup")
set (CMAKE_FIND_FRAMEWORK LAST)
endif()
option(PYBIND_INCS "The output of `pybind11-config --includes` " OFF)
IF(NOT PYBIND_INCS)
EXECUTE_PROCESS(COMMAND sh -c 'pybind11-config --includes' OUTPUT_VARIABLE PYBIND_INCS ERROR_QUIET COMMAND_ERROR_IS_FATAL ANY)
SET(PYBIND_INCS "${PYBIND_INCS}" CACHE STRING "The output of `pybind11-config --includes`")
ENDIF()
message(STATUS "***** Detected PYBIND_INCS:${PYBIND_INCS}")
SET(CMAKE_CXX_FLAGS "${PYBIND_INCS} ${CMAKE_CXX_FLAGS}")
SET(CMAKE_C_FLAGS "${PYBIND_INCS} ${CMAKE_C_FLAGS}")
FIND_PACKAGE(pybind11 REQUIRED)
# Define pybind11 tree module.
find_package(PyBind)
pybind11_add_module(_tree tree.h tree.cc)
SET(ABSL_DEPS absl::int128 absl::raw_hash_set absl::raw_logging_internal absl::memory absl::strings absl::throw_delegate)
FIND_PACKAGE(absl REQUIRED COMPONENTS ${ABSL_DEPS})
if (absl_FOUND)
message("Found 'absl':${absl_CONFIG}")
GET_TARGET_PROPERTY(ABSL_INC_DIR absl::strings INTERFACE_INCLUDE_DIRECTORIES)
GET_TARGET_PROPERTY(ABSL_LIBS absl::strings INTERFACE_LINK_LIBRARIES)
message(" 'absl incs':${ABSL_INC_DIR}, ${ABSL_LIBS}" )
target_link_libraries(_tree PRIVATE ${ABSL_DEPS})
else()
message("Not Found: 'absl'")
endif()
-
Compiling the test cases on Linux/macOS:
- On Linux you’ll need to install the
python-dev
orpython3-dev
packages as well ascmake
. - On macOS, the included python version works out of the box, but
cmake
must still be installed.
After installing the prerequisites, run(The last line will both compile and run the tests.)
- On Linux you’ll need to install the
mkdir build
cd build
cmake ..
make check -j 4
- Header and namespace conventions
For brevity, all code examples assume that the following two lines are present(Some features may require additional headers, but those will be specified as needed.):
#include <pybind11/pybind11.h>
namespace py = pybind11;
- Example:
- Python Example: https://github.com/pybind/python_example
- CMake Example: https://github.com/pybind/cmake_example
For simplicity 1, we’ll put both this function and the binding code into a file named example.cpp with the following contents:- In practice, implementation and binding code will generally be located in separate files.
- The
PYBIND11_MODULE()
macro creates a function that will be called when animport
statement is issued from within Python.
The module name (example
) is given as thefirst macro argument
(it should not be in quotes).
The second argument (m
) defines a variable of typepy::module_
which is the main interface for creating bindings.
The methodmodule_::def()
generates binding code that exposes theadd()
function to Python.
#include <pybind11/pybind11.h>
int add(int i, int j) {
return i + j;
}
PYBIND11_MODULE(example, m) {
m.doc() = "pybind11 example plugin"; // optional module docstring
m.def("add", &add, "A function that adds two numbers");
}
https://github.com/google-deepmind/tree/setup.py
"""Setup for pip package."""
import os, platform, sys, sysconfig, shutil, subprocess, setuptools
from setuptools.command import build_ext
here = os.path.dirname(os.path.abspath(file))
def _get_tree_version():
"""Parse the version string from tree/__init__.py."""
with open(os.path.join(here, 'tree', '__init__.py')) as f:
try:
version_line = next(line for line in f if line.startswith('__version__'))
except StopIteration:
raise ValueError('__version__ not defined in tree/__init__.py')
else:
ns = {}
exec(version_line, ns) # pylint: disable=exec-used
return ns['__version__']
def _parse_requirements(path):
with open(os.path.join(here, path)) as f:
return [
line.rstrip() for line in f
if not (line.isspace() or line.startswith('#'))
]
class CMakeExtension(setuptools.Extension):
"""An extension with no sources.
We do not want distutils to handle any of the compilation (instead we rely
on CMake), so we always pass an empty list to the constructor.
"""
def __init__(self, name, source_dir=''):
super().__init__(name, sources=[])
self.source_dir = os.path.abspath(source_dir)
class BuildCMakeExtension(build_ext.build_ext):
"""Our custom build_ext command.
Uses CMake to build extensions instead of a bare compiler (e.g. gcc, clang).
"""
def run(self):
self._check_build_environment()
for ext in self.extensions:
self.build_extension(ext)
def _check_build_environment(self):
"""Check for required build tools: CMake, C++ compiler, and python dev."""
try:
subprocess.check_call(['cmake', '--version'])
except OSError as e:
ext_names = ', '.join(e.name for e in self.extensions)
raise RuntimeError(
f'CMake must be installed to build the following extensions: {ext_names}'
) from e
print('Found CMake')
def build_extension(self, ext):
extension_dir = os.path.abspath(
os.path.dirname(self.get_ext_fullpath(ext.name)))
build_cfg = 'Debug' if self.debug else 'Release'
cmake_args = [
f'-DPython3_ROOT_DIR={sys.prefix}',
f'-DPython3_EXECUTABLE={sys.executable}',
f'-DCMAKE_LIBRARY_OUTPUT_DIRECTORY={extension_dir}',
f'-DCMAKE_BUILD_TYPE={build_cfg}'
]
if platform.system() != 'Windows':
cmake_args.extend([
f'-DPython3_LIBRARY={sysconfig.get_paths()["stdlib"]}',
f'-DPython3_INCLUDE_DIR={sysconfig.get_paths()["include"]}',
])
if platform.system() == 'Darwin' and os.environ.get('ARCHFLAGS'):
osx_archs = []
if '-arch x86_64' in os.environ['ARCHFLAGS']:
osx_archs.append('x86_64')
if '-arch arm64' in os.environ['ARCHFLAGS']:
osx_archs.append('arm64')
cmake_args.append(f'-DCMAKE_OSX_ARCHITECTURES={";".join(osx_archs)}')
os.makedirs(self.build_temp, exist_ok=True)
subprocess.check_call(
['cmake', ext.source_dir] + cmake_args, cwd=self.build_temp)
subprocess.check_call(
['cmake', '--build', '.', f'-j{os.cpu_count()}', '--config', build_cfg],
cwd=self.build_temp)
# Force output to <extension_dir>/. Amends CMake multigenerator output paths
# on Windows and avoids Debug/ and Release/ subdirs, which is CMake default.
tree_dir = os.path.join(extension_dir, 'tree') # pylint:disable=unreachable
for cfg in ('Release', 'Debug'):
cfg_dir = os.path.join(extension_dir, cfg)
if os.path.isdir(cfg_dir):
for f in os.listdir(cfg_dir):
shutil.move(os.path.join(cfg_dir, f), tree_dir)
setuptools.setup(
name='dm-tree',
version=_get_tree_version(),
url='https://github.com/deepmind/tree',
description='Tree is a library for working with nested data structures.',
author='DeepMind',
author_email='tree-copybara@google.com',
long_description=open(os.path.join(here, 'README.md')).read(),
long_description_content_type='text/markdown',
# Contained modules and scripts.
packages=setuptools.find_packages(),
tests_require=_parse_requirements('requirements-test.txt'),
test_suite='tree',
cmdclass=dict(build_ext=BuildCMakeExtension),
ext_modules=[CMakeExtension('_tree', source_dir='tree')],
zip_safe=False,
# PyPI package information.
classifiers=[
'Development Status :: 4 - Beta',
'Intended Audience :: Developers',
'Intended Audience :: Science/Research',
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
'Topic :: Scientific/Engineering :: Mathematics',
'Topic :: Software Development :: Libraries',
],
license='Apache 2.0',
keywords='tree nest flatten',
)
【推荐】国内首个AI IDE,深度理解中文开发场景,立即下载体验Trae
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· 分享一个免费、快速、无限量使用的满血 DeepSeek R1 模型,支持深度思考和联网搜索!
· 使用C#创建一个MCP客户端
· 基于 Docker 搭建 FRP 内网穿透开源项目(很简单哒)
· ollama系列1:轻松3步本地部署deepseek,普通电脑可用
· 按钮权限的设计及实现