Deprecate the tuple API (object[index]) of structseq objects like os.stat_result

Core Development
1

Hi,

I propose deprecating os.stat(filename)[8] tuple API to migrate to os.stat(filename).st_mtime named attributes. The tuple API exists for backward compatibility with Python 2.1 and older which was released 25 years ago.

The tuple API requires a good memory to remind members by their indexes. What are these members?

  • os.uname()[4]
  • pwd.getpwnam('root')[3]
  • sys.float_info[9]
  • sys.flags[14]

Answers:

  • os.uname().machine
  • pwd.getpwnam('root').pw_gid
  • sys.float_info.radix
  • sys.flags.utf8_mode

In the early days of Python, it was tedious to create an object with attributes in C. So instead, functions returned tuples, since it’s simple to create tuples in C. For example, st = os.stat() returned a tuple of 10 entries (integers) in Python 1.5. The stat provides indices to stat members: st[stat.ST_INO] returns the inode number.

Python 2.2 introduced a new C type structseq type to add attributes to these tuples. The new types inherit from tuple and so remains fully backward compatible. For example, st[stat.ST_INO] is still accepted, but you can now also write st.st_ino instead (the stat module is no longer needed) which is more readable (no need to remind what is st[1]).

Python uses structseq subclasses in many modules: grp, os, resource, signal, etc. Last years, new structseq types have been added. The new types also provide the tuple API, even if they are new and so don’t need backward compatibility. It’s just because it’s currently not possible to select if the tuple API is supported or not.

The os.stat_result type is even more complex because it has “unnamed members”. For example, st[8] (or st[stat.ST_MTIME]) is the modification time in seconds as an integer and it has no corresponding attribute, whereas st.st_mtime is the modification time in seconds as a float and it has not corresponding tuple index (st[11] raises IndexError).

IMO it’s now time to get rid of the backward compatibility with Python 2.1 (released in 2001, 25 years ago) and older. It’s time to migrate code to named attributes: replace st[8] with st.st_mtime.

The good news is that code modified to use named attributes would be compatible with Python 2.2 and newer!

I expect that the majority of code already use named attributes, so no code changed is needed, and that the majority of developers don’t even know that a tuple API exist.

I propose deprecating the tuple API (object[index] and len(object)) in most structseq objects. Examples: os.stat_result, grp.struct_group, sys.float_info, resource.struct_rusage.

The tuple API still makes sense in 2026 for some structseq objects which would be left unchanged: sys.version_info, curses.ncurses_version and time.struct_time. For example, sys.version_info[:2] is commonly used to get (major, minor) version.

Victor

7 Likes
2

The following question is if objects using collections.namedtuple should also deprecate their tuple API. Example of such objects: difflib.Match, functools.CacheInfo, pkgutil.ModuleInfo, platform.AndroidVer, etc.

I suppose that it would be consistent to also deprecate the tuple API of these namedtuple objects. For example, func.cache_info().maxsize is more readable than func.cache_info()[2] on a function decorated with @functools.cache.

Note: I created an issue and a pull request (draft) to deprecate structseq tuple API.

1 Like
3

What kind of deprecation period are you thinking? Since this is a productivity change and not a buggy change I’m assuming you’re thinking a while?

4

What’s the cost/ downside of keeping the tuple API?

2 Likes
5

I would strongly advise leaving namedtuple alone. It has ‘tuple’ in its name, and is a common building block in user code (unlike structseq, which requires C code).

1 Like
6

I propose to emit a DeprecationWarning when the tuple API is used. After 5 years (PEP 387), we can evaluate how many projects would be impacted if the tuple API is removed. If the number is low, we can remove the feature. Other, we just wait longer until this number is low enough.

The usual issue is that DeprecationWarning is ignored by default. But more and more projects are running their test suite with warnings treated as error which catch such DeprecationWarning.

Keeping the tuple API costs nothing. The code is there basically since forever.

Well, it’s not exactly nothing, since we just got a complicated bug report about structseq and os.stat_result unnamed members: issue gh-154387. This issue triggered my old idea of deprecating the tuple API. Removing the tuple API would avoid such issue.

The downside of keeping the tuple API is that developers will continue the cargo cult of getting members by hardcoded index, rather than using named attributes. Code using indexes (tuple API) is harder to read and to maintain.

I propose deprecating the tuple API for most structseq objects. But if these changes affect too many projects, we can reeavalute the plan and only deprecate the tuple API on a case-by-case basis (ex: only start with os.stat_result).

7

It seems like deprecating and removing is therefore more costly than doing nothing. You mention the recent bug, which is a cost to keeping the code, but unless we get a lot of such bugs, is probably not high.

I personally don’t really care, but I know some people are frustrated by unnecessary “churn” in Python, so the bad feelings this deprecation might cause could be another hidden cost of the removal.

6 Likes
8

I think removing the tuple API from namedtuple() is out of the question, as Guido already mentioned.

For structseq, you have to take the historic reason for it’s existence into account.

Several APIs returned tuples for easy access to combined output (usually structs in C).

The problem was: How can you add new values to the set without breaking backwards compatibility – common usage is to do tuple unpacking to access the values, which is both very efficient and readable. This was the reason for adding named access to additional fields.

I agree that remembering the positions of fields isn’t easy and named access to single fields is more readable as well, but the use case of getting all fields at once (or common subset) becomes a lot harder and slower, if you remove tuple access.

So overall -1 on the idea.

Let’s keep both, since there are use cases for tuple access as well as named access.

3 Likes
9

One thing I love about namedtuple: it provides a surprising but pithy and insightful explanation of how tuples and lists differ:

A tuple is a namedtuple without the names.

2 Likes
10

I think it makes sense to emit warnings to suggest accessing members with names instead of indices. But removing the __index__/__len__ methods does not because structseq types are subclasses of tuple.

def function_accepts_tuple(tup: tuple[Any, ...]) -> ReturnType: ...

function_accepts_tuple(structseq_instance)  # should pass without error

In comparison, we add the new frozendict type parallel to the existing mutable dict rather than subclassing dict with removed __setitem__.

Posted in `structseq_repr` mislabels unnamed `PyStructSequence` fields (e.g. `os.stat_result` slots 7–9) · Issue #154387 · python/cpython · GitHub

deprecate the tuple API of structseq objects.

I wonder if such deprecation would break tuple unpacking behavior of packages in the wild. For example, torch.return_types.* ( `structseq_repr` mislabels unnamed `PyStructSequence` fields (e.g. `os.stat_result` slots 7–9) · Issue #154387 · python/cpython · GitHub ):

structseq is a subtype of tuple. So it can be unpacked like:

seq = some_c_function(...)
x, y, z = seq

Here is a real-world example:

>>> import torch

>>> seq = torch.sort(torch.tensor([3, 2, 1]))
>>> seq
torch.return_types.sort(
values=tensor([1, 2, 3]),
indices=tensor([2, 1, 0]))

>>> values, indices = seq  # tuple unpack
>>> values
tensor([1, 2, 3])
>>> indices
tensor([2, 1, 0])

>>> seq.values             # access by name
tensor([1, 2, 3])
>>> seq.indices
tensor([2, 1, 0])
11

structseq is a structure and a sequence at the same time. Maybe one need transition from structseq to just struct for that cases (not removing tuple API from structseq)?

12

To me, this looks like something a linter could complain about. Not a reason to break working code.