Modernization¶

Before porting a C extension to Python 3, you’ll need to make sure that you’re not using features deprecated even in Python 2. Also, many of Python 3’s improvements have been backported to Python 2.6, and using them will make the porting process easier.

For all changes you do, be sure add tests to ensure you do not break anything.

Comparisons¶

Python 2.1 introduced rich comparisons for custom objects, allowing separate behavior for the ==, !=, <, >, <=, >= operators, rather than calling one __cmp__ function and interpreting its result according to the requested operation. (See PEP 207 for details.)

In Python 3, the original __cmp__-based object comparison is removed, so all code needs to switch to rich comparisons. Instead of a

static int cmp(PyObject *obj1, PyObject *obj2)

function in the tp_compare slot, there is now a

static PyObject* richcmp(PyObject *obj1, PyObject *obj2, int op)

in the tp_richcompare slot. The op argument specifies the comparison operation: Py_EQ (==), Py_GT (>), Py_LE (<=), etc.

Additionally, Python 3 brings a semantic change. Previously, objects of disparate types were ordered according to type, where the ordering of types was undefined (but consistent across, at least, a single invocation of Python). In Python 3, objects of different types are unorderable. It is usually possible to write a comparison function that works for both versions by returning NotImplemented to explicitly fall back to default behavior.

To help porting from __cmp__ operations, py3c defines a convenience macro, PY3C_RICHCMP, which evaluates to the right PyObject * result based on two values orderable by C’s comparison operators. A typical rich comparison function will look something like this:

static PyObject* mytype_richcmp(PyObject *obj1, PyObject *obj2, int op)
{
    if (mytype_Check(obj2)) {
        return PY3C_RICHCMP(get_data(obj1), get_data(obj2), op);
    }
    Py_RETURN_NOTIMPLEMENTED;
}

where get_data returns an orderable C value (e.g. a pointer or int), and mytype_Check checks if get_data is of the correct type (usually via PyObject_TypeCheck). Note that the first argument, obj1, is guaranteed to be of the type the function is defined for.

If a “cmp”-style function is provided by the C library, use PY3C_RICHCMP(mytype_cmp(obj1, obj2), 0, op).

Also, py3c defines the Py_RETURN_NOTIMPLEMENTED macro if it’s not provided by your Python version (3.3 and lower).

Note that if you use PY3C_RICHCMP, you will need to include the header py3c/comparison.h (or copy the macro to your code) even after your port to Python 3 is complete. The is also needed for Py_RETURN_NOTIMPLEMENTED until you drop support for Python 3.3.

Note

The tp_richcompare slot is inherited in subclasses together with tp_hash and (in Python 2) tp_compare: iff the subclass doesn’t define any of them, all are inherited.

This means that if a class is modernized, its subclasses don’t have to be, unless the subclass manipulates compare/hash slots after class creation (e.g. after the PyType_Ready call).

PyObject Structure Members¶

To conform to C’s strict aliasing rules, PyObject_HEAD, which provides members such as ob_type and ob_refcnt, is a separate struct in Python 3. Access to these members is provided by macros, which have been ported to Python 2.6:

Instead of	use
obj->ob_type	Py_TYPE(obj)
obj->ob_refcnt	Py_REFCNT(obj)
obj->ob_size	Py_SIZE(obj)

And for initialization of type objects, the sequence

PyObject_HEAD_INIT(NULL)
0, /* ob_size */

must be replaced with

PyVarObject_HEAD_INIT(NULL, 0)

Adding module-level constants¶

Often, module initialization uses code like this:

PyModule_AddObject(m, "RDWR", PyInt_FromLong(O_RDWR));
PyModule_AddObject(m, "__version__", PyString_FromString("6.28"));

Python 2.6 introduced convenience functions, which are shorter to write:

PyModule_AddIntConstant(m, "RDWR", O_RDWR)
PyModule_AddStringConstant(m, "__version__", "6.28")

These will use native int and str types in both Python versions.

New-Style Classes¶

The old-style classes (PyClass_* and PyInstance_*) will be removed in Python 3. Instead, use type objects, which have been available since Python 2.2.

PyCObject to PyCapsule¶

The PyCObject API has been removed in Python 3.3. You should instead use its replacement, PyCapsule, which is available in Python 2.7 and 3.1+. For the rationale behind Capsules, see CPython issue 5630.

If you need to support Python 2.6, you can use capsulethunk.h, which implements the PyCapsule API (with some limitations) in terms of PyCObject. For instructions, see the chapter PyCapsule API for Python 2.6.

The port to PyCapsule API should be straightforward:

Instead of PyCObject_FromVoidPtr(obj, destr), use PyCapsule_New(obj, name, destr). If the capsule will be available as a module attribute, use "<modulename>.<attrname>" for name. Otherwise, use your best judgment, but try making the name unique.
Instead of PyCObject_FromVoidPtrAndDesc(obj, desc, destr), use PyCapsule_New() as above; then call PyCapsule_SetContext(obj, desc).
Instead of PyCObject_AsVoidPtr(obj), use PyCapsule_GetPointer(obj, name). You will need to provide a capsule name, which is checked at runtime as a form of type safety.
Instead of PyCObject_GetDesc(), use PyCapsule_GetContext().
Instead of PyCObject_SetVoidPtr(), use PyCapsule_SetPointer().
Change all CObject destructors to PyCapsule destructors, which take the PyCapsule object as their only argument.

Done!¶

When your project is sufficiently modernized, and the tests still pass under Python 2, you’re ready to start the actual Porting.