PEP 686: Make UTF-8 mode default

PEPs
25

For example, see this pull request.

ipython added 84 encoding="utf-8" to fix the EncodingWarning. They added 0 encoding="locale" or locale.getpreferredencoding(False).

I did similar commit by adding hundreds of encoding="utf-8". Most of them are just ASCII, although few of them can be a real bug found by the warning. And those few hidden bugs are fixed when UTF-8 become default.

So I suppose many people don’t want to add dozens or hundreds of encoding="utf-8". They may want to wait UTF-8 become default. For example, craft-parts maintainer rejected adding dozen encoding="utf-8":

One example in the pull request tried to add encoding="utf-8":

    with open(call_fifo, "w") as fifo:
        fifo.write(json.dumps(data))

Although json must be UTF-8, json.dumps generates only ASCII unless ensure_ascii=False is specified. So omitting encoding here is not a bug.

I re-read the PEP 387.

  • Adding a warning is required by the policy. But DeprecationWarning is not required. Other warning category or compiler warning can be used when there is a reason.
  • “Wait for the warning to appear in at least two minor Python versions” is required.

Now I am considering postpone the target to Python 3.13.
Although EncodingWarning was added in Python 3.10, there was no official plan for changing the default encoding. So I don’t count the Python 3.10 in the deprecation period.

If this PEP accepted before 3.11 beta:

  • Fix encoding="locale" in UTF-8 mode in Python 3.11
  • Advertise the EncodingWarning and UTF-8 mode in the release note of Python 3.11 and 3.12
  • Make the UTF-8 mode default in Python 3.13

If no objections, I will update the PEP.

1 Like