Deprecate or offer a way to disable ctypes "simple types autocast" behavior

Currently, ctypes has an undocumented feature where collections and structures of “simple” data types are automatically cast to python objects on access.

from ctypes import c_uint32

x = c_uint32(5)
print(x)
>> c_uint(5)
arr = (c_uint32 * 3)(x)
print(arr[0])
>> 5

This behavior is highly unpredictable for typing purposes, end users of ctypes, and library authors. My proposal is that we should deprecate this feature or offer a way to disable this behavior.

Rationales

1. Many APIs within ctypes.pythonapi is unable to be used safely due to the implicit auto-casting.

For example, PyDict_GetItem, which returns a PyObject pointer or NULL pointer.

from ctypes import *

PyDict_GetItem = pythonapi["PyDict_GetItem"]
PyDict_GetItem.argtypes = (py_object, py_object)
PyDict_GetItem.restype = py_object

d = {"A": 1, "B": 2}

print(PyDict_GetItem(d, "C"))
! [139] SIGSEGV (Address boundary error)

Running this above example crashes the interpreter with a segmentation fault when the auto-casting of the null py_object to object occurs.

Versus the alternative, assuming this casting did not occur implicitly:

from ctypes import *

PyDict_GetItem = pythonapi["PyDict_GetItem"]
PyDict_GetItem.argtypes = (py_object, py_object)
PyDict_GetItem.restype = c_ssize_t

d = {"A": 1, "B": 2}

item = cast(PyDict_GetItem(d, "C"), py_object)
print(item)
>> py_object(<NULL>)
print(item.value)
>> ValueError: PyObject is NULL

2. Many ctypes objects are untypable in typeshed due to this behavior, such as ctypes.Array, which, although being a Generic, is forced to have __getitem__ return Any.

Implementing an option to globally disable this behavior would be a development nuisance. Packages that use ctypes would have to support both cases. Deprecating and removing the behavior isn’t much better because packages that need to support older Python versions would have to support both cases.

Currently, the way to disable this behavior is to use a subclass of the simple type. For example:

import ctypes

py_object = type('py_object', (ctypes.py_object,), {})
PyDict_GetItem = ctypes.pythonapi.PyDict_GetItem
PyDict_GetItem.argtypes = (py_object, py_object)
PyDict_GetItem.restype = py_object

d = {'A': 1, 'B': 2}
>>> isinstance(PyDict_GetItem(d, 'C'), py_object)
True
>>> PyDict_GetItem(d, 'C').value
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: PyObject is NULL

I think a less problematic approach would be to add a new ctypes.types module in 3.10+. This would include the same types as the base ctypes module, except that the simple types would be subclasses of the base simple types.