PEP 737 – Unify type name formatting

[vstinner (Victor Stinner)]

@encukou

But why add a shortcut for __name__, in the same PEP that recommends not using __name__ in new code?

Many people asked for it in the discussions so far.

[encukou (Petr Viktorin)]

It’s a question for them, then.

[vstinner (Victor Stinner)]

@vstinner

Many people asked for it in the discussions so far.

I tried to find references.

2018:

• Serhiy Storchaka: “I think we need to handle only two cases: short and fully qualified names. (…)” message
• MRAB: “%T for short form, %#T for fully qualified name”. email
• Barry Warsaw: “%t (short) and %T (long) do seem like the logical choices” email

2023:

• Serhiy: “If add a specialized format, we should provide options for all reasonable variants: (…)” message
• Serhiy: “For different kinds of names we can use the “size” modifier.” message
• Petr Viktorin: “There’s also __module__ + ':' + __qualname__, separated by semicolon rather than a dot.” message

[sirosen (Stephen Rosen)]

I’m chiming in with a small +1 for anything which encourages the :-separated format as “the primary way that a type name is stringified”.

I only learned about that format last year, in spite of years of using Python, when writing some code to resolve a string to a class. It’s a nicer format for that sort of resolution (since you don’t need to walk modules; I think this is well understood?), but it’s a little bit obscure right now. Proliferating that style more widely helps guide people like me who may not have otherwise seen the format.

[vstinner (Victor Stinner)]

I updated the PEP (commit):

• Add N and #N formats to type.__format__() (Python API).
• Add PyType_GetModuleName() function.
• Only recommend to no longer truncate type names in new code (leave existing code unchanged).
• Add “Formats Summary” section.

This version should address all post requests and have the minimum effects on the backward compatibility.

@encukou

It seems to me that these aren’t really worth it:

• {type:n} instead of {type.__name__}
• {type:q} instead of {type.__qualname__}
• {type:m} instead of {type.__module__}

Since no one else expressed their opinion about these formats, I remove these variants to only keep the “fully qualified name” formats. We can reconsider adding them later. Let’s start with the minimum.

[vstinner (Victor Stinner)]

@vstinner

I updated the PEP (commit):

I submitted PEP 737 to the Steering Council: PEP 737 -- Unify type name formatting · Issue #227 · python/steering-council · GitHub

[MegaIng (Cornelius Krupp)]

@vstinner

Since no one else expressed their opinion about these formats, I remove these variants to only keep the “fully qualified name” formats. We can reconsider adding them later. Let’s start with the minimum.

I did voice my support for at least :n, but deleted my reply for unrelated reasons.

[encukou (Petr Viktorin)]

Thank you for the PEP, and for addressing all my concerns!

For the record, I still think the attribute (__fully_qualified_name__) is unnecessary. As far as I can see, it’d be the first attribute on similar types that’s rather easily derived from the other ones (rather than being the source of truth). Typically, operations that convert the raw data in these attributes to more useful forms live as functions in inspect - e.g. signature, getcallargs, unwrap, etc. With the helper API that the PEP now specifies, the argument against a function in inspect loses much of its weight – you don’t need to import it.

But, you don’t need to convince me. All I ask is that you think about my concerns, which you did. I’m simply summarizing my own remaining reservations, since they’re hard to find in the big thread.

[vstinner (Victor Stinner)]

@encukou

it’d be the first attribute on similar types that’s rather easily derived from the other ones (rather than being the source of truth).

I’m always surprised that the PyTypeObject.tp_name member is used by static types to compute:

• type.__name__ and PyType_GetName()
• type.__qualname__ and PyType_GetQualName()
• type.__module__ and (the recently added) _PyType_GetModuleName()

Each access/function call creates a new string object.

Heap types have a more regular design, these attributes are just members of the PyHeapTypeObject.

See also Investigate how the PyTypeObject members can be removed from the public C API.

[barry (Barry Warsaw)]

@vstinner - The 2024 Steering Council discussed PEP 737 today, so with that hat on, I’d like to share our feedback. First of all, thank you for putting this PEP together along with the associated PR.

We can see the usefulness of the C API changes proposed by the PEP and would likely accept those changes as is.

We see less justification for the Python level changes. We especially question the need for __fully_qualified_name__. As a side note, we prefer __fullname__ as previously suggested in an earlier post although that is moot if we do not accept the Python level changes.

Our question to Victor is: would you be willing to update the PEP to propose only the C API changes, deferring the Python changes to some separate, future PEP?

[vstinner (Victor Stinner)]

I wrote that PEP to make some C extensions compatible with the limited C API. Unifying type formatting between Python and C was a secondary goal for me. I can live without it.

Next week I will work on a PEP update to move aside the Python changes.

[vstinner (Victor Stinner)]

@barry

Our question to Victor is: would you be willing to update the PEP to propose only the C API changes, deferring the Python changes to some separate, future PEP?

I prepared PEP 737: Remove Python changes by vstinner · Pull Request #3704 · python/peps · GitHub to update the PEP. Would you mind to review the changes?

It changes the purpose of the PEP. It’s no longer about making type name formatting consistent between Python and C. Now it’s only adding C API to format a type fully qualified name.

I have a concern: in Python, I didn’t find any function which formats a “type fully qualified name” the same way than PyType_GetFullyQualifiedName(): omit the module if it’s equal to "builtins" or "__main__" (or if the module is not a string). In general, it’s bad to add C functions which can do things which are not (easily) doable in Python.

Is the Steering Council aware of this issue and fine with it? If the SC is fine with my updated PEP, I’m also fine with it

[steve.dower]

@vstinner

In general, it’s bad to add C functions which can do things which are not (easily) doable in Python.

Not “in general” - we regularly add C functions that can do things that aren’t doable in Python, and there’s nothing wrong with that (provided it’s an operation that makes sense in C and doesn’t make sense in Python).

However, in this case, it does seem a bit weird to not have the C API map closely to an accessible Python value. Perhaps that means that the C API should be more like the available Python operations rather than creating something new?

[thomas (Thomas Wouters)]

@vstinner

I have a concern: in Python, I didn’t find any function which formats a “type fully qualified name” the same way than PyType_GetFullyQualifiedName(): omit the module if it’s equal to "builtins" or "__main__" (or if the module is not a string). In general, it’s bad to add C functions which can do things which are not (easily) doable in Python.

Is the Steering Council aware of this issue and fine with it? If the SC is fine with my updated PEP, I’m also fine with it

Our main concern was with adding attributes to all types, and the added complexity of the two ways of formatting type names in the proposed type.__format__ (although very much more the former than the latter). It’s not that inconvenient to format type names in Python, since it’s just a string operation. Users can use cls.__module__ and cls.__qualname__ to format their own types. The only real difference is in the special-casing of builtins, but formatting the names of types that might be builtin (and where leaving off the builtin. matters), that’s such a rare need I don’t think it warrants adding a __format__ method. (On the other hand, it is inconvenient to format type names in C, because it’s just string operations.)

If there really is a desire for formatting types the exact same way the C API does it, a utility function would make more sense to me, personally, than type.__format__, but I think the SC could be persuaded given some concrete use-cases.

[vstinner (Victor Stinner)]

@thomas

If there really is a desire for formatting types the exact same way the C API does it, a utility function would make more sense to me, personally, than type.__format__, but I think the SC could be persuaded given some concrete use-cases.

As I wrote, I’m fine with not making Python changes, that’s why I prepared PEP 737: Remove Python changes by vstinner · Pull Request #3704 · python/peps · GitHub waiting for your review/approval

I mostly worked on the C API side. If tomorrow, someone comes with a strong use case, we can obviously consider extending the Python API.

[steve.dower]

@vstinner

If tomorrow, someone comes with a strong use case, we can obviously consider extending the Python API.

It’s still today But every time I’ve wanted that kind of name formatting in Python code (debuggers, typically), I’ve already had inspect imported, so it would’ve been fine as a function in there.

[vstinner (Victor Stinner)]

I updated the PR to remove Python API changes: commit. I’m now waiting for the Steering Council to review the updated PEP.

[thomas (Thomas Wouters)]

The SC has accepted the updated PEP: PEP 737 -- Unify type name formatting · Issue #227 · python/steering-council · GitHub

[vstinner (Victor Stinner)]

Thank you very much to everybody who was involved in the scattered discussion about PEP 737, and thanks to the Steering Council for reviewing and accepting my PEP!

It’s now implemented in the main branch. Let’s see how it goes