Modernization¶
Before porting a C extension to Python 3, you’ll need to make sure that you’re not using features deprecated even in Python 2. Also, many of Python 3’s improvements have been backported to Python 2.6, and using them will make the porting process easier.
For all changes you do, be sure add tests to ensure you do not break anything.
Comparisons¶
Python 2.1 introduced rich comparisons for custom objects, allowing separate
behavior for the ==
, !=
, <
, >
, <=
, >=
operators,
rather than calling one __cmp__
function and interpreting its result
according to the requested operation.
(See PEP 207 for details.)
In Python 3, the original __cmp__
-based object comparison is removed,
so all code needs to switch to rich comparisons. Instead of a
static int cmp(PyObject *obj1, PyObject *obj2)
function in the tp_compare
slot, there is now a
static PyObject* richcmp(PyObject *obj1, PyObject *obj2, int op)
in the tp_richcompare
slot. The op
argument specifies the comparison
operation: Py_EQ (==), Py_GT (>), Py_LE (<=), etc.
Additionally, Python 3 brings a semantic change. Previously, objects of disparate types were ordered according to type, where the ordering of types was undefined (but consistent across, at least, a single invocation of Python). In Python 3, objects of different types are unorderable. It is usually possible to write a comparison function that works for both versions by returning NotImplemented to explicitly fall back to default behavior.
To help porting from __cmp__
operations, py3c defines a
convenience macro, PY3C_RICHCMP, which evaluates to the right PyObject *
result based on two values orderable by C’s comparison operators.
A typical rich comparison function will look something like this:
static PyObject* mytype_richcmp(PyObject *obj1, PyObject *obj2, int op)
{
if (mytype_Check(obj2)) {
return PY3C_RICHCMP(get_data(obj1), get_data(obj2), op);
}
Py_RETURN_NOTIMPLEMENTED;
}
where get_data
returns an orderable C value (e.g. a pointer or int), and
mytype_Check checks if get_data
is of the correct type
(usually via PyObject_TypeCheck). Note that the first argument, obj1,
is guaranteed to be of the type the function is defined for.
If a “cmp”-style function is provided by the C library,
use PY3C_RICHCMP(mytype_cmp(obj1, obj2), 0, op)
.
Also, py3c defines the Py_RETURN_NOTIMPLEMENTED macro if it’s not provided by your Python version (3.3 and lower).
Note that if you use PY3C_RICHCMP, you will need to include the header
py3c/comparison.h
(or copy the macro to your code) even after your port
to Python 3 is complete.
The is also needed for Py_RETURN_NOTIMPLEMENTED until you drop support for
Python 3.3.
Note
The tp_richcompare
slot is inherited in subclasses together with
tp_hash
and (in Python 2) tp_compare
: iff
the subclass doesn’t define any of them, all are inherited.
This means that if a class is modernized, its subclasses don’t have to be,
unless the subclass manipulates compare/hash slots after
class creation (e.g. after the PyType_Ready
call).
PyObject Structure Members¶
To conform to C’s strict aliasing rules, PyObject_HEAD, which provides
members such as ob_type
and ob_refcnt
, is a separate struct in
Python 3.
Access to these members is provided by macros, which have been ported to
Python 2.6:
Instead of | use |
---|---|
obj->ob_type | Py_TYPE(obj) |
obj->ob_refcnt | Py_REFCNT(obj) |
obj->ob_size | Py_SIZE(obj) |
And for initialization of type objects, the sequence
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
must be replaced with
PyVarObject_HEAD_INIT(NULL, 0)
Adding module-level constants¶
Often, module initialization uses code like this:
PyModule_AddObject(m, "RDWR", PyInt_FromLong(O_RDWR));
PyModule_AddObject(m, "__version__", PyString_FromString("6.28"));
Python 2.6 introduced convenience functions, which are shorter to write:
PyModule_AddIntConstant(m, "RDWR", O_RDWR)
PyModule_AddStringConstant(m, "__version__", "6.28")
These will use native int and str types in both Python versions.
New-Style Classes¶
The old-style classes
(PyClass_*
and PyInstance_*
) will be removed in Python 3.
Instead, use type objects,
which have been available since Python 2.2.
PyCObject to PyCapsule¶
The PyCObject API has been removed in Python 3.3. You should instead use its replacement, PyCapsule, which is available in Python 2.7 and 3.1+. For the rationale behind Capsules, see CPython issue 5630.
If you need to support Python 2.6, you can use capsulethunk.h
, which
implements the PyCapsule API (with some limitations) in terms of PyCObject.
For instructions, see the chapter PyCapsule API for Python 2.6.
The port to PyCapsule API should be straightforward:
- Instead of
PyCObject_FromVoidPtr(obj, destr)
, usePyCapsule_New(obj, name, destr)
. If the capsule will be available as a module attribute, use"<modulename>.<attrname>"
for name. Otherwise, use your best judgment, but try making the name unique. - Instead of
PyCObject_FromVoidPtrAndDesc(obj, desc, destr)
, usePyCapsule_New()
as above; then callPyCapsule_SetContext(obj, desc)
. - Instead of
PyCObject_AsVoidPtr(obj)
, usePyCapsule_GetPointer(obj, name)
. You will need to provide a capsule name, which is checked at runtime as a form of type safety. - Instead of
PyCObject_GetDesc()
, usePyCapsule_GetContext()
. - Instead of
PyCObject_SetVoidPtr()
, usePyCapsule_SetPointer()
. - Change all CObject destructors to
PyCapsule destructors
, which take the PyCapsule object as their only argument.