Python SIP C++ bindings tutorial

by filed under Linux, Python, QGIS.
Python SIP C++ bindings tutorial

Time for something new: recently I’ve been busy with the implementation of QGIS Server Python Plugins, as a part of this efforts, I decided to resume my past century’s C++ skills (C++ has changed a lot in 20 years and it seems like a brand new language) and try to understand how C++ code can be bound to Python modules.

Since QGIS uses QT libraries, SIP is the natural choice for creating the bindings. Here are some random notes about this journey into SIP and Python bindings, I hope you’ll find them useful! We will create a sample C++ library, a simple C++ program to test it and finally, the SIP configuration file and the python module plus a short program to test it.

Create the example library

FIrst we need a C++ library, following  the tutorial on the official SIP website  I created a simple library named hellosip:   
$ mkdir hellosip
$ cd hellosip
$ touch hellosip.h hellosip.cpp Makefile.lib
This is the content of the header file hellosip.h: 
#include <string>

using namespace std;

class HelloSip {
    const string the_word;
public:
    // ctor
    HelloSip(const string w);
    string reverse() const;
};
This is the implementation in file hellosip.cpp , the library just reverse a string, nothing really useful. 
#include "hellosip.h"
#include <string>

HelloSip::HelloSip(const string w): the_word(w)
{
}

string HelloSip::reverse() const
{
    string tmp;
    for (string::const_reverse_iterator rit=the_word.rbegin(); rit!=the_word.rend(); ++rit)
        tmp += *rit;
    return tmp;
}
 

Compiling and linking the shared library

Now, its time to compile the library, g++ must be invoked with -fPIC option in order to generate Position Independent Code, -g tells the compiler to generate debug symbols and it is not strictly necessary if you don’t need to debug the library: 
g++ -c -g -fPIC hellosip.cpp -o hellosip.o
The linker needs a few options to create a dynamically linked Shared Object (.so) library, first -shared which tells gcc to create a shared library, then the -soname which is the library version name, last -export_dynamic that is also not strictly necessary but can be useful for debugging in case the library is dynamically opened (with dlopen) : 
g++ -shared -Wl,-soname,libhellosip.so.1  -g -export-dynamic -o libhellosip.so.1  hellosip.o
At the end of this process, we should have a brand new libhellosip.so.1 sitting in the current directory. For more informations on shared libraries under linux you can read TLDP chapter on this topic.  

Using the library with C++

Before starting the binding creation with SIP, we want to test the new library with a simple C++ program stored in a new cpp file: hellosiptest.cpp: 
#include "hellosip.h"
#include <string>
using namespace std;
// Prints True if the string is correctly reversed
int main(int argc, char* argv[]) {
  HelloSip hs("ciao");
  cout << ("oaic" == hs.reverse() ? "True" : "False") << endl;
  return 0;
}
To compile the program we use the simple command: 
g++ hellosiptest.cpp -g -L.  -lhellosip -o hellosiptest
which fails with the following error: 
/usr/bin/ld: cannot find -lhellosip
collect2: error: ld returned 1 exit status
For this tutorial, we are skipping the installation part, that would have created proper links from the base soname, we are doing it now with: 
ln -s libhellosip.so.1 libhellosip.so
The compiler should now be happy and produce an hellosiptest executable, that can be tested with: 
$ ./hellosiptest
True
If we launch the program we might see a new error: 
./hellosiptest: error while loading shared libraries: libhellosip.so.1: cannot open shared object file: No such file or directory
This is due to the fact that we have not installed our test library system-wide and the operating system is not able to locate and dynamically load the library, we can fix it in the current shell by adding the current path to the LD_LIBRARY_PATH environment variable which tells the operating system which directories have to be searched for shared libraries. The following commands will do just that: 
export LD_LIBRARY_PATH=`pwd`
Note that this environment variable setting is “temporary” and will be lost when you exit the current shell.    

SIP bindings

Now that we know that the library works we can start with the bindings, SIP needs an interface header file with the instructions to create the bindings, its syntax resembles that of a standard C header file with the addition of a few directives, it contains (among other bits) the name of the module and the classes and methods to export. The SIP header file hellosip.sip contains two blocks of instructions: the class definition that ends around line 15 and an additional %MappedType block that specifies how the std::string type can be translated from/to Python objects, this block is not normally necessary until you stick standard C types. You will notice that the class definition part is quite similar to the C++ header file hellosip.h: 
// Define the SIP wrapper to the hellosip library.

%Module hellosip

class HelloSip {

%TypeHeaderCode
#include <hellosip.h>
%End

public:
    HelloSip(const std::string w);
    std::string reverse() const;
};

// Creates the mapping for std::string
// From: http://www.riverbankcomputing.com/pipermail/pyqt/2009-July/023533.html

%MappedType std::string
{
%TypeHeaderCode
#include <string>
%End

%ConvertFromTypeCode
    // convert an std::string to a Python (unicode) string
    PyObject* newstring;
    newstring = PyUnicode_DecodeUTF8(sipCpp->c_str(), sipCpp->length(), NULL);
    if(newstring == NULL) {
        PyErr_Clear();
        newstring = PyString_FromString(sipCpp->c_str());
    }
    return newstring;
%End

%ConvertToTypeCode
    // Allow a Python string (or a unicode string) whenever a string is
    // expected.
    // If argument is a Unicode string, just decode it to UTF-8
    // If argument is a Python string, assume it's UTF-8
    if (sipIsErr == NULL)
        return (PyString_Check(sipPy) || PyUnicode_Check(sipPy));
    if (sipPy == Py_None) {
        *sipCppPtr = new std::string;
        return 1;
    }
    if (PyUnicode_Check(sipPy)) {
        PyObject* s = PyUnicode_AsEncodedString(sipPy, "UTF-8", "");
        *sipCppPtr = new std::string(PyString_AS_STRING(s));
        Py_DECREF(s);
        return 1;
    }
    if (PyString_Check(sipPy)) {
        *sipCppPtr = new std::string(PyString_AS_STRING(sipPy));
        return 1;
    }
    return 0;
%End
};
At this point we could have run the sip command by hand but the documentation suggests to use the python module sipconfig that, given a few of configuration variables, automatically creates the Makefile for us, the file is by convention named configure.py: 
import os
import sipconfig

basename = "hellosip"

# The name of the SIP build file generated by SIP and used by the build
# system.
build_file = basename + ".sbf"

# Get the SIP configuration information.
config = sipconfig.Configuration()

# Run SIP to generate the code.
os.system(" ".join([config.sip_bin, "-c", ".", "-b", build_file, basename + ".sip"]))

# Create the Makefile.
makefile = sipconfig.SIPModuleMakefile(config, build_file)

# Add the library we are wrapping.  The name doesn't include any platform
# specific prefixes or extensions (e.g. the "lib" prefix on UNIX, or the
# ".dll" extension on Windows).
makefile.extra_libs = [basename]

# Search libraries in current directory
makefile.extra_lflags= ['-L.']

# Generate the Makefile itself.
makefile.generate()
We now have a Makefile ready to build the bindings, just run make to build the library. If everything goes right you will find a new hellosip.so library which is the python module. To test it, we can use the following simple program (always make sure that LD_LIBRARY_PATH contains the directory where libhellosip.so is found). 
import hellosip
print hellosip.HelloSip('ciao').reverse() == 'oaic'

Download

The full source code of this tutorial can be downloaded from this link.