Implement __class_getitem__ for ctypes.pointer

Ideas
1

Proposal

Make ctypes.pointer support __class_getitem__. As it is currently a function, we can change it to a class, with __new__ implementing the current features of the function. This should not affect existing behavior significantly.

  1. We could make pointer[...] return a GenericAlias as usual
  2. Or alternatively, it can, at runtime, return a call POINTER(...)
    With this option we can effectively eliminate the need for the other POINTER type.
POINTER(POINTER(c_uint)) == pointer[pointer[c_uint]]

Rationale

Currently, ctypes.pointer is used as a generic for type hinting purposes (mypy, pyright, pycharm, etc.) as well as for runtime-libraries that use type hinting.

from ctypes import pointer, py_object

def fn() -> pointer[py_object[str]]:
    ...

But this is not evaluable at runtime since ctypes.pointer is not a function that supports __class_getitem__. So this does not work as a type alias nor with get_type_hints.

We also have another type, ctypes.POINTER, to create a typed pointer, but this doesn’t accept generic aliases, so it also doesn’t work in typing.

from ctypes import pointer, py_object

def fn() -> POINTER(py_object[str]):
    ...
2

Pointer types are subclasses of ctypes._Pointer with a given _type_, such as ctypes.c_int. The exceptions are function pointer types, which subclass _ctypes.CFuncPtr, and simple pointer types, which subclass ctypes._SimpleCData for a given _type_. The latter includes ctypes.c_void_p (type “P”), ctypes.c_wchar_p (type “Z”), and ctypes.c_char_p (type “z”).

The builtin POINTER and pointer functions exist to facilitate creating and caching _Pointer types and instances.

The POINTER function creates and caches a _Pointer subclass for the given type, if one isn’t already cached. For example:

>>> ctypes.c_int in ctypes._pointer_type_cache
False
>>> c_int_p = ctypes.POINTER(ctypes.c_int)
>>> c_int_p._type_ is ctypes.c_int
True
>>> ctypes.c_int in ctypes._pointer_type_cache
True

The pointer function returns a pointer to its argument. It calls POINTER to create and cache a new pointer type, if one isn’t already cached. For example:

>>> b = ctypes.c_ubyte()
>>> ctypes.c_ubyte in ctypes._pointer_type_cache
False
>>> pb = ctypes.pointer(b)
>>> type(pb)._type_
<class 'ctypes.c_ubyte'>
>>> ctypes.c_ubyte in ctypes._pointer_type_cache
True
1 Like
3

The main issue is for typing here. We have class getitem for ctype types like Array for typing

from ctypes import Array, c_uint

x = Array[c_uint]
print(x, type(x))
>> _ctypes.Array[ctypes.c_uint] <class 'types.GenericAlias'>

The wrapping of pointer as a class makes it finally possible to type hint generic pointers, like Array.

4

The base pointer class is ctypes._Pointer. It works similarly to a ctypes.Array subclass, except the latter includes a _length_ in addition to a _type_. For example:

>>> c_int_Array_3 = ctypes.c_int * 3
>>> c_int_Array_3.__bases__
(<class '_ctypes.Array'>,)
>>> c_int_Array_3._type_ is ctypes.c_int
True
>>> c_int_Array_3._length_ == 3
True

We have no operator for creating and caching a ctypes._Pointer subclass. Thus we have the builtin ctypes.POINTER function instead.

1 Like
5

Are you using the latest version of your type checker? The typeshed stubs were updated a few months ago in `ctypes`: `pointer` is a function, not a class by AlexWaygood · Pull Request #8446 · python/typeshed · GitHub, to reflect the fact that at runtime, pointer is actually a function that returns instances of _Pointer. If you’re using the latest version of mypy or pyright, it should be no longer valid to use pointer as a type annotation, since functions can’t be used as type annotations. You’ll need to start using _Pointer instead of pointer in your type hints. So perhaps the question should be: "Should we add __class_getitem__ to ctypes._Pointer?

Having said that, it’s not ideal that users have to access a private class in order to annotate their code. It might be nice to make ctypes._Pointer public.

1 Like
6

One reason that ctypes._Pointer and ctypes._CFuncPtr are private is to discourage manual subclassing that bypasses _pointer_type_cache, _c_functype_cache, and _win_functype_cache, which are used by ctypes.POINTER(), ctypes.CFUNCTYPE(), and ctypes.WINFUNCTYPE(). Since type checking is still based on the Python class, the caches are important in order to avoid creating multiple pointer types for the same _type_, or multiple function pointer types for the same (_restype_, _argtypes_, _flags_). The ctypes.Array type uses the * sequence repeat operator, which has an internal cache.

Possibly the cache support could be relocated to the corresponding metaclasses: PyCPointerType, PyCArrayType, and PyCFuncPtrType. In this case, using the cache would have to be disabled if additional class attributes are defined other than, respectively, ("_type_",), ("_type_", "_length_"), and ("_restype_", "_argtypes_", "_flags_").

2 Likes