Integrating with app installers (as we discussed in Allow uploading .pyz/zipapp files to PyPI? would be neat.
One of the advantages of keeping the .pyz around is it fits people’s expectations of “download and run”. If running it is able to transparently reuse cached installs, nobody ever has to really know about the install step - it’ll only occur on the first run. It could even be used for automatic updates (ClickOnce is a similar kind of approach for .NET).