What is Pybind use for?

Pybind is a interface that allows data to communicate between C++ and Python. Normally, it is impossible to have C++ and Python to be used together. Pybind allows us to surpass such limitation, and it often can save us a lot of time to rewrite code on old platform and spend even more time debugging for unforseeable bugs.

Embedding vs Extending

The difference between embedding and extending is fairly straight forward. Given an application which is mainly written in C++ and assumed there are some functionalities provide by some Python code that we want to integrate in our C++ application. Since no programmers want to rewrite any code, we can Extend Python with C++. In contrast, if we want to use some C++ function in the Python code, we can Embed Python with C++.

Environment Setup

The following is the structure of my setup for Pybind to work on Ubuntu 18.04 and Mac OSX. You can find my demo code here.

(base) james@james-System-Product-Name:~/Coding/my_website/draft/code/pybind11$ tree .
├── build                                       # building directory for cmake
│   └── run_script                              # customize script to run the application
├── CMakeLists.txt                              # cmake file
├── include                                     # holds all .h files
│   └── simple.h
├── pybind11                                    # directory of pybind11 
├── scripts                                     # holds all .py files
│   └── simple.py
└── src                                         # holds all .cpp files
    └── simple.cpp

CMakeLists.txt

You will need the following lines in your cmake for pybind11 to work

add_subdirectory(pybind11)
pybind11_add_module(pybind_interface ${SOURCE_FILES})
add_executable(${PROJ_NAME} ${SOURCE_FILES})
target_link_libraries(${PROJ_NAME} PRIVATE pybind11::embed)

Be aware of the line pybind11_add_module. The first argument pybind_interface stands for the name of the embedding module generated by pybind11 after the compilation. Such name has to agree with the name used when creating the embedding module.

PYBIND11_EMBEDDED_MODULE(pybind_interface, m)
{
    m.doc() = "Be careful of the name";
}

Application Entry (aka main.cpp)

  • Include Necessary Headers
#include "pybind11/pybind11.h"          # You need this most of the time
#include "pybind11/embed.h"             # You need this when dealing with embedding
#include "pybind11/stl.h"               # You need this when using pybind stl typings
...
  • Create Python Interpreter
pybind11::scoped_interpreter guard{};

Ensure that there’s always no more one interpreter, failure to do so will lead to synchronization issue.

  • Create Pybind Module Macro
// Embedding Module => use this when dealing with embedding
PYBIND11_EMBEDDED_MODULE(pybind_interface, m){
}

// Normal Module
PYBIND11_MODULE(example, m) {
}

Very Basic Example

Extending

In a Python script, we can create a simple add function that returns the sum of two numbers

# In simple.py
def add(a, b):
	return a + b

In your main C++ application, you can utilize such add function by doing

// In simple.cpp
int num1 = 1;
int num2 = 2; 
pybind11::module simpleModule = pybind11::module::import("simple");
auto python_result = simpleModule.attr("add")(num1, num2); 
int result = python_result.cast<int>();  // result should be 3

Let’s take a closer look at the above lines.

  • In line 3, we obtain the module. A module is a python script. In this case, the variable simpleModule is an object which refers to file simple.py.

  • In line 4, we access the add function, which lives in simple.py. The variables num1 and num2 are injected, and the returning value is stored in the vairable python_result, which is a pybind11::object type instead of an integer type variable.

  • In line 5, due to the reason describe above, the pybind11::object is casted into an integer type variable so that it can be utilize in C++.

Embedding

In C++, we can create a simple multiply function that returns the product of two numbers

// In simple.cpp
int multiply(int num1, int num2) {
    return num1 * num2;
}

Now, welcome to the most important step. You will need to write the binding code.

// In PYBIND11_EMBEDDED_MODULE
m.def("multiply", &multiply);

Isn’t it so simple? The first parameter specifies the name of the binded function, while the second parameter contains a reference to the actual C++ function named multiply. It should then be apperant then by now that .def is the syntax of binding a function.

To use the multiply function, do the following

# In simple.py
import pybind_interface
num1 = 2
num2 = 5
cplusplus_result = pybind_interface.multiply(num1, num2)  # should be 10

Now, you are able to call simple Python function in C++ and vice versa. But pybind can do more than these, which I will be cover it the next time.