PEP 597: Enable UTF-8 mode by default on Windows

[methane (Inada Naoki)]

@steve.dower

Changing the default is a massively breaking change. It’s a 4.x change, in my opinion, not a 3.y.

How do you think about enable UTF-8 mode automatically when console code page is 65001 (or based on some better flag if Microsoft introduce) ?

I can not wait Python 4.x. More and more people including kids in kinder school start learning Python. UTF-8 must be the default for them. And setting an environment variable is a complex step for them.

@steve.dower

It’ll break cache and configuration files that apps expect to survive updates, or cross versions (pip.ini, tox.ini, setup.cfg are easy examples).

On the other hand, current default value broke these files many times, because legacy encoding can not represent some paths and tool authors forget the default encoding is not always UTF-8.

@steve.dower

Alternatively, we could make a more intelligent default decoder for Windows that will read UTF-8 until it fails, then assume ACP. Because that’s what we’re going to tell libraries to do anyway, so may as well make it easy on them.

If the default decode become it, do you think we can change the default encode for writing file to UTF-8?

@steve.dower

I regularly make the argument that if you don’t specify an encoding when you write text to a file, you can’t possibly read it back. If we change the default, everyone is going to learn that very quickly and painfully.

After default encoding is changed to UTF-8, everyone can omit encoding and no pain. They are not forced to learn what is “encoding”, until they are required to handle legacy encoding.
I want to provide such experience to people start learning Python in 2020s.