Annotation-based sugar for ctypes

Ideas
1

Turn this:

class S(ctypes.Structure):
   _fields_ = [ ('a', ctypes.c_int), ('b', ctypes.c_char_p) ]

Into this:

class S(ctypes.Structure):
    a : ctypes.c_int
    b : ctypes.c_char_p

And this:

libm = ctypes.CDLL('libm.so')
pow = libm.pow
pow.restype = ctypes.c_double
pow.argtypes = (ctypes.c_double, ctypes, c_double)

into this:

libm = ctypes.CDLL('libm.so')

@libm
def pow(a: ctypes.c_double, b:ctypes.c_double) -> c_double:
    pass
13 Likes
2

It looks very interesting. I do not see any obvious objections, except possible conflicts with from __future__ import annotations and PEP 563. But dataclasses have to deal with it somehow.

Could you please open two separate issues for this? Do you volunteer to implement these features?

4 Likes
3

Rubicon does something similar; it might be instructive to see what their experience has been with runtime annotations as a FFI.

1 Like
4

I have an implementation in Python. It works but breaks a test so I will need a little bit of C.

Is it necessary to support some of the rarer features like bit fields and self-referential structs? They are always available through setting __fields__

Raise exception if both __fields__and annotations set.

May be a good idea to raise exception if decorated function body is not empty?

5

Function prototypes can be instantiated with a paramflags tuple, which specifies whether each parameter in argtypes is an input parameter (1), output parameter (2), or in-out parameter (3). Optionally, it can also specify a name and default value for each parameter, which allows the function to be called with keyword arguments. For example:

import ctypes
libm = ctypes.CDLL('libm.so.6')

pow_prototype = ctypes.CFUNCTYPE(
                    ctypes.c_double,
                    ctypes.c_double, ctypes.c_double,
                    use_errno=True)
pow_paramflags = ((1, 'x'), (1, 'y', 2)) # default to returning x**2
pow = pow_prototype(('pow', libm), pow_paramflags)

>>> pow(3)
9.0
>>> pow(3, 3)
27.0
>>> pow(x=3)
9.0
>>> pow(x=3, y=3)
27.0
>>> pow(y=3, x=3)
27.0

The parameter names and default values (if any) can be pulled directly from the annotated prototype. If a parameter flag isn’t specified it can be assumed to be handled as an input parameter. An output parameter or in-out parameter could be declared by annotating with a tuple (type, flag).

A function prototype also has a _flags_ value in addition to _argtypes_ and _restype_. The supported flag values are

FUNCFLAG_STDCALL = 0x00 # Windows only
FUNCFLAG_CDECL = 0x01
FUNCFLAG_HRESULT = 0x02 # Windows only
FUNCFLAG_PYTHONAPI = 0x04
FUNCFLAG_USE_ERRNO = 0x08
FUNCFLAG_USE_LASTERROR = 0x10

The last two enable the use of ctypes.get_errno() and ctypes.get_last_error(). Generally scripts rely on the default function prototype of a CDLL, WinDLL, or PyDLL instance to set these flags, or a prototype factory function such as CFUNCTYPE(), WINFUNCTYPE(), or PYFUNCTYPE(). Maybe they could be supported in an annotated prototype as keyword-only parameters such as _stdcall_, _cdecl_, _hresult_, _pythonapi_, _use_errno_, and _use_last_error_. A function pointer also has an errcheck attribute, which is a function that gets called just before returning. Maybe this could be supported as an _errcheck_ keyword-only parameter. If any flag is defined, it will override the _flags_ of the default prototype of a CDLL instance. If a default _errcheck_ is defined, it will be the initial errcheck value of the function pointer.

6

Thanks for the suggestions!

I think only things that map naturally to annotations should be specified as annotations. The rest should just be added as arguments to the @library() attribute. No overloading of the function’s keyword-only arguments.

1 Like
7

Can people forbid creating the class in this way?

class S(dataclass):
    # some annotations

Will there be a switch?

I am more in favour of the decorator style

@dataclass
class S:
    # some annotations

to

class S(dataclass):
    # some annotations

The point here is:
Supporting “annotation define” and “_field_ define” in similar grammar is weird.

8

The first suggestion about Structure looks somewhat normal to me, though making type hints valuable at runtime seems a little bit non-uniform with all other Python type hints which are not meaningful at runtime. Also there is a need to maintain both syntaxes and make them mutually exclusive, but it’s up to you.

But the second suggestion with decorator is very confusing. Please imagine if you are a newcomer and trying to understand why there is an empty function, maybe I need to remove it. :wink: Static analysis may decide something similar. And what if I add some implementation instead of pass statement? Will it be ignored?

Another argument against the decorator is that a dynamic library is pretty low level object, while a decorator is something at high level: it’s typically a transformation of decorated function, but you suggest something different and strange. I think it’s not worth it.

10

Can I recognize that this discussion has been moved to Annotation-based syntax for ctypes structs · Issue #104533 · python/cpython · GitHub?

I have shared my thoughts there.