Ctypes: improving Structure/Union & basic types

Hello,
I’m spending some of my time improving ctypes.
I’m aiming for incremental changes, seeing where that will take me, so I don’t think I can write a PEP first. On the other hand, individual issues feel too discunnected. Perhaps a topic here will work, like @barneygale’s one on pathlib.

So far, I’ve

  • Finished Rian Hunter’s PR to add memoryview_at
  • merged a few PRs by Jun Komoda with improvements to COM
  • reviewed Peter Bierma’s free-threading improvemens
  • started switching to fixed-width integers: N-byte integers are easier to reason about than ints, shorts and longs. IMO, ctypes badly need that abstraction (which C itself got in 1999). Of course, CPUs, platform ABIs and current ctypes all work in terms of shorts or half-words of platform-dependent sizes; we can’t lose that abstraction.
  • merged a PR by Matthias Görgens to improve the layout of Structures (and Unions), and rewritten the algorithm in Python.

Hope I’m not forgetting too much :‍)

I think I’ll continue working a bit longer on Structures, Unions and basic types, aiming to make them and their fields more usable, introspectable and configurable from Python.

For the next step, I have a pull request up proposing to improve introspectability of Structure/Union fields, like this:

>>> class Color(Structure):
...     _fields_ = (
...         ('red', c_uint8),
...         ('green', c_uint8),
...         ('blue', c_uint8),
...         ('intense', c_bool, 1),
...         ('blinking', c_bool, 1),
...    )
...
>>> Color.red
<ctypes.CField 'red' type=c_ubyte, ofs=0, size=1>
>>> Color.green.type
<class 'ctypes.c_ubyte'>
>>> Color.blue.byte_offset
2
>>> Color.intense
<ctypes.CField 'intense' type=c_bool, ofs=3, bit_size=1, bit_offset=0>
>>> Color.blinking.bit_offset
1

These are quite useful for writing tests for struct layouts, and I hope they’ll also be useful for further simplifying that logic. Maybe we can eventually even make the struct layout fully customizable from Python (although I don’t think that’s on the table for Python 3.14).

What do you think? Is it a good idea? Or should these be private?

5 Likes

If we’re going for usability, then something I’ve been thinking about is using type annotations for defining fields, similar to how things are done for dataclasses and NamedTuple.

In the past, I’ve found that defining _fields_ has been a bit tedious, because it’s difficult to remember some of the rules (e.g. does the type or name go first in the tuple?)

So, something like this:

class Color(Structure):
    red: c_uint8
    green: c_uint8
    blue: c_uint8
    intense: Annotated[c_bool, 1]
    blinking: Annotated[c_bool, 1]

I guess this would break some classes that define unrelated class variables, so it would have to be opt-in, but to me it seems a lot nicer than _fields_.

There’s a feature request for that; I replied there.

1 Like