I was redirected to this forum by a member of github vscode-cpptools.
Please refer to the github link https://github.com/microsoft/vscode-cpptools/issues/5695 to see details of the idea and its comments. It is about how to reduce the size of C/C++ compiler that is required by Python packages, take dtaidistance for instance, among many others. It would be good to be able to run them with MinGW instead, or to reduce MSVC strongly in size.
There was a workaround until Python 3.4 which might also be an approach now: Use MinGW compiler till Python 3.4 by adding a “distutils.cfg” to the folder “\Lib\distutils” in Python install directory. It would be nice to have that MinGW “distutils.cfg” workaround for the recent Python versions as well.
The people who need to see this question are likely the setuptools and pip folk, since what you need is some kind of option to override the compiler used for build options. (Historically, this would also have involved contributing code to the distutils module in CPython, but now that it’s moving into setuptools you’re best to start there).
If anyone knows the current active MinGW folk, it may be valuable to loop them in. I have no idea what their status is on making it look more like MSVC.
I do know that both the MSVC and Windows teams are very aware of the size of their respective packages and the complexity in acquiring/installing them. I’m not sure what changes will come about yet, but as those are the tools that are properly supported we’ll make sure that Python continues to work with them.
As for the distutils.cfg workaround, I doubt that was ever removed, so not sure why it stopped working there. It may be when setuptools started being injected by pip, in which case you’ll still want to work with someone from that project (@jaraco probably) to figure out what broke it, though it’s also possible that MinGW will say they shouldn’t be used anymore.
Thank you for taking this up and redirecting it to the Python community.
I am myself just a normal end user who caught the idea due to a nearly full disk on the netbook, that is why I cannot contribute anything. I can perhaps guess that the workaround with distutils.cfg seems most promising to revive. It would already be a good step to know why it does not work anymore. By the way, I have not tested the workaround. Unlikely, as it is not documented, but not that it is actually still working?
I’m pretty sure that part of the problem was that mingw didn’t really support linking with any C runtime other than msvcrt. As Python uses a different C runtime, this has the potential to cause issues (in practice, the risk seems to be mostly theoretical, though). I don’t think enough people found using MSVC an issue (particularly once it became free to use) that anyone with sufficient technical knowledge was interested in working on maintaining that option.
I’ve no idea whether any of the above is still true or relevant, though…
Still all true, though I’ll add some clarifications for the record.
Correct, though some work was done to support msvcrt120 (circa 2013) and UCRT (circa 2015). It never came to full fruition.
“Theoretical to subtle” is true enough. People have noticed our internal inconsistencies here (such as the text/binary mode of standard IO changing), and would certainly notice these when there are two+ C runtimes loaded.
But many people would never encounter things like mismatched file descriptors, memory allocators, locales, or errno state. CPython’s API is nearly clean in this respect, but not quite.
I think this is 100% true. Perhaps netbook users are the audience who are impacted enough to motivate a fix?
Or maybe they’ll all just fork the packages they need, add CI to build a wheel on GitHub or equivalent, (cough submit a PR to the original project cough) and then just copy a pre-built binary onto the netbook?
Would it be an option to ask those / the core programmer who have implemented the distutils.cfg in the past to just have a look at it again and estimate if it is possible?
This made me smile, the “just so simple thing” :))))) It sounds so far for a common user:
Or maybe they’ll all just fork the packages they need, add CI to build a wheel on GitHub or equivalent, ( cough submit a PR to the original project cough ) and then just copy a pre-built binary onto the netbook?
The C++ ABIs (possibly headers even) of GCC and MSVC are just incompatible. You might have some luck using Clang, but I don’t guarantee anything. /// If you’re concerned about disk space, you might try installing the requisites and then stripping the parts irrelevant to Python packages. There might even be a way to make something of a portable MSVC toolchain, but don’t distribute it (I believe it’s not allowed in the license).
The question is perhaps also what the license allows at all.
I didn’t implement the mingw support previously, but I did work on it. Personally, I’m not going to go back to it. At least one other person who worked on it is no longer active in core Python development.
Maybe you’ll get someone else who might offer to help. But I wouldn’t be too optimistic…
That is understood, the whole try was unrealistic from the start. What remains is just that msvc might one day compress the installation size. I have switched away from a netbook now because I could not stand the constant struggle with disk space in every aspect, msvc was just one of them. Thanks for the insights!
I am not a lawyer, but the license forbids redistribution (send people to the original download instead), reverse engineering (to copy obfuscated functionality into your own projects), and targeting platforms other than Windows (because they’re not supported, and “support” means something real compared to what most OSS projects are used to).
Once you’ve got a licensed copy, you can do whatever else you want with it on your own machine(s). And of course you can do anything with your own compiled binaries.
The license really isn’t that much more restrictive than any other compiler, and is cheaper than most. It’s just the packaging of the tools that frustrates people.
As an aside, now that setuptools has fully taken on distutils, we’ll be deprecating it in the standard library (soon). So this request in future would have to be made to each project implementing a build tool.
Then there simply remains waiting for a change from MSVC. It is not completely unlikely because of possibly confusing legacy code in the 4.5 GB installer, no matter whether this will be for the cleaning or for the size of it. This can be closed in my opinion, if such option exists here on python.org.
