PEP 675 – Arbitrary Literal String Type

Typing
1

This PEP is already implemented in Python, however, I found some uncovered cases. This includes other literal values and their conversion to string.

The only example provided about other literal types is the next:

Other literal types, such as literal integers, are not compatible with LiteralString :

some_int: int
expect_literal_string(some_int)  # Error: Expected LiteralString, got int.

literal_one: Literal[1] = 1
expect_literal_string(literal_one)  # Error: Expected LiteralString, got Literal[1].

And this makes total sense: a literal string expected, not an integer, literal or not.

But what about the next case:

some_constant: Literal[1] = 1
some_constant_f = f'{some_constant}'
some_constant_str = str(some_constant)

Are some_constant_f and some_constant_str of type LiteralString or str? If I mark them as LiteralString, should type checker emit an error?

I would like answers to be “LiteralString” and “no”, but maybe there is some other opinion upon this case.

Generally speaking:

  • Do functions format, str, repr, ascii return LiteralString if literal values are passed to them?
    • Literal values are any literal value, not just literal strings.
    • Some specific examples:
      • Are format specifiers allowed in f-strings and str.format? Example: f'{literal_string:literal_format}'.
      • Are conversions allowed in f-strings and str.format? Example: f'{literal_string!r}'.
  • Is self-documenting allowed in f-strings? Example: f'{literal_string=}'.

All of the cases are not covered by the PEP explicitly. It would be good to address them in the PEP itself or any addendum PEP.

UPD: I checked the list of compatible str methods and found some inconsistencies:

  • The second overload for method str.format accepts only strings as arguments, but value of any type can be passed as an argument.
  • Method str.__format__ is not listed.
  • Method str.__mod__ accepts two literal strings, but returns a non-literal string.
  • Method str.__iter__ accepts a literal string, but yields non-literal strings.
1 Like
2

The implementation for PEP 675 requires changes to the runtime, typeshed stubs, and type checkers.

  • The runtime implementation was included in Python 3.11 (and included in the typing_extensions library for backward compatibility).
  • The typeshed stubs have also been updated to include support for LiteralString.
  • Only two of the four major type checkers (pyright and pyre) have implemented support for this feature leaving mypy and pytype without support for LiteralString at this time.

It would be good to address them in the PEP itself or any addendum PEP.

PEPs are historical documents, and they are generally not updated after they are accepted. Clarifications and extensions can be made in the official typing specification.

We’ve also been working to write conformance tests for type checkers. You can find the source code for the LiteralString type checker conformance tests here. Some of the cases you’ve mentioned are not currently covered by the conformance test, but they probably should be.

To answer your specific questions…

  • The specification is very clear that f-strings have the type LiteralString “if and only if its constituent expressions are literal strings”. That doesn’t accommodate other literal types (like literal integers, bytes, enums, etc.).
  • The spec doesn’t specifically mention the intended behavior of the str constructor. I wouldn’t expect that to ever produce a LiteralString. If you want to create a literal string, you should use a literal string expression form, not call the (dynamic) str constructor.
  • The spec doesn’t specifically mention the format, ascii or repr functions, so by default I would assume these never return a value of type LiteralString. You may be able to convince the typeshed maintainers that some subset of these should optionally return LiteralString under certain circumstances. If so, an overload could be added.
  • You asked whether format specifiers are allowed in f-strings. In pyright’s implementation (which I wrote), I interpreted the spec to mean that an f-string that uses only literal strings (even if in a format specifier) should be treated as a LiteralString. In all other cases, it’s treated as a regular str. That means, for example, if you use a non-literal str or a non-string literal, the f-string type becomes str. Here is an example in the pyright playground.
3 Likes