Skip to content
This repository has been archived by the owner on Sep 20, 2024. It is now read-only.

ILWIS Python API Requirements

Johannes Kolbe edited this page Aug 26, 2014 · 1 revision

Python (C/C++) Extension

The Problem

Since the standard Python interpreter (CPython) is written in C/C++, there is a variety of tools, which simplify the creation of so called extensions, to make C/C++ code available for programmers via the Python-C-API. Unfortunately C/C++ can be compiled in an even wider variety of compilers to be platform independent, but the python community decided to use MSVC++ as the compiler for CPython on Windows platforms. This way our decision to use minGW/gcc as the main toolchain/compiler for ILWIS, mainly to stay compatible to UNIX systems, has an major impact on the architecture of the ILWIS NG PythonAPI. Even using tools like SIP, SWIG, boost.Python or cython to ease the adoption of ILWIS functionality basically delivered through the ILWISObject Class, will face the problem of incompatible C++-structures due to differences between MSVC++ (CPython) and gcc (ILWIS). This leads to the general structure of a classic C API, which handles pointers to ILWISObjects (classes), wrapped again by self-made Python classes. The decision about, which framework in the end can be useful in this rather complicated setting, can only be made after successfully compiling a non-ILWIS example C++-class using minGW and calling its methods from a MSVC++ compiled standard Python script.

Pros and Cons of the frameworks:

SIP (PyQt)

        + support for PyQt, so signal/slot-mechanism and QtGUI
        + tight C/C++ integration leads to faster execution than SWIG
        - .sip-files (SIP-specific h/i-file copies) are not generated

SWIG

        + support also for Perl, Tcl, C#, Java, Guile
        + best documentation, easy configuration
        - no official support for mingw and python
        - indirection via PySwigObject-layer
        - no inner classes supported (workaround with .i interface declaration)
        - struggle with template calsses

boost.Python(Py++,pygccxml)

        + complete, bug free (supports STL containers, exceptions, inner classes, properties)
        - fat, long complile time, whole boost source, tricky setup
        - incomplete documentation, tutorials
        - additional dependency on run time

Cython(pyrex)

        - only useful to transfer existing Python code to C
        - unreadable c code

pybindgen(pygccxml)

        - quiet new project, incomplete, no documentation
        + direct use of Python C API calls
        + efficient executable
        + uses gcc’s internal gccxml to translate into C library calls
        + Python-only generator provides interactive control on API creation
        + highly readable c code

Decision

Both SIP and SWIG have done their jobs well, compiling of C++ classes (gcc) and preparing them for the use in Python (MSVC++). Both tested in small C++ example tutorials on MS Windows 7 with the current versions SIP 4.15.3 and SWIG 2.0.11 together with Python 3.3.2. After reading through lots of tutorials, examples, issue lists and forum posts, SWIG and  SIP turn out to be the best documented and tested. So I go further trying some examples for both. The SWIG tutorials are perfectly well documented and even though its not recommended to use mingw there are no problems so far. SIPs tutorial misses some part, for example python deployment, configuration. Basically SIP drops some ready-to-run code, that should not be modified, but if you want to use another compiler problems start to arise. Still it was possible to use both frameworks with non-ILWIS and thus non-Qt C++ code, but since SIP did not even bothered to create Python source code to debug classes inside the module, the decision seemed to be made in favour of SWIG.

Integrating the ILWIS headers into the SWIG code generation toolchain ended up in the unsolvable problem of having python.h files (for MSVC) and QtCore header been referred by the generated code in such a way, that MinGW couldn’t compile against cmath.h which is used both by Python and Qt. In the hope, that SIP in connection with PyQt should be able to circumvent this issue because it was made for exactly that problem, I started building a SIP generated ILWIS-extension module ending up in the same cmath.h issue. SIP/PyQt serves pure Qt classes as its own sip-generated Python extension modules only to be used and extended within Python, but does not solve the Python-Qt-compiling-problem.

The next approach for the API is, to replicate the API related ILWIS classes without the use of any Qt functionality in their header files. This header files can then easily be used for a SWIGed extension. The difficulty lies in the efficient but correct creation of the replicated interface classes.


Python2.7 vs. Python3.3

current versions of all the above mentioned code generators are meant to support Python 2.4 and newer including Python 3.x.

Python2.7

        + some improvements of Python 3.1 were backported
        - no further development except long term support

Python3.3

        + since v. 3.2 extensions are programmed against a stable (along Py. versions) ABI[1]

edit

after some development for Python 3.3 using SWIG, there is now special code, which might not be compatible to Python 2.7.

  1. Python 3.3 merges PyLong into PyInt this way standard SWIG has some problems detecting the right PY<->C conversion, so now the ILWIS API always converts to unsigned long long int or long long int or double. This should not be a problem for Python 2.7

edit 2

Introducing compatibility with Python datetime module we included some basic Python C API calls into our customized wrapper classes, which might not be compatible with non-Python3 versions of the C API.


References

[1]: http://legacy.python.org/dev/peps/pep-0384/

Clone this wiki locally