Planning for an MSVC ABI break

[steve.dower]

@steve.dower

More recently, I’ve been suggesting that extension modules statically link their C++ runtime .

The PR I linked has the flags you need. The challenge is that we want practically all extensions to dynamically link the C runtime (which always has shared state), and to statically link the C++ runtime (which mostly doesn’t). So typically the choice would be between /MT and /MD, but it’s not enough to get the hybrid arrangement. (You need to pass /MT to the compiler to generate statically linked C++ code, then use linker arguments to remove the static CRT libs and replace them with the CRT import libs.)

The easiest way to check given a specific binary is to inspect the import table for msvcp*.dll. And usually a text-like search of the file is going to be sufficient (IIRC it’s stored as UTF-16), but the PE structure isn’t too complicated to check it properly using the struct module if you really want.

[pitrou (Antoine Pitrou)]

@steve.dower

The challenge is that we want practically all extensions to dynamically link the C runtime (which always has shared state), and to statically link the C++ runtime (which mostly doesn’t). So typically the choice would be between /MT and /MD, but it’s not enough to get the hybrid arrangement. (You need to pass /MT to the compiler to generate statically linked C++ code, then use linker arguments to remove the static CRT libs and replace them with the CRT import libs.)

That sounds like a problem, since an average non-expert like me wouldn’t know where to start. It would be nice if there was a cookbook somewhere (including e.g. CMake instructions)…

@steve.dower

The easiest way to check given a specific binary is to inspect the import table for msvcp*.dll. And usually a text-like search of the file is going to be sufficient (IIRC it’s stored as UTF-16), but the PE structure isn’t too complicated to check it properly using the struct module if you really want.

… and it would be extra nice if there was a tool (ala auditwheel) that could be run automatically on CI to check these requirements.

(yes, I know, this is basically asking others to do work. Sorry for that)

[steve.dower]

@pitrou

It would be nice if there was a cookbook somewhere

Yeah, we’re in need of a new doc on how to build extension modules on each platform. Choosing the right sysconfig vars to base your build on isn’t obvious either, so it’s basically all black magic that very few people know how to do.

Choosing a good build backend should be the only important thing for most users. But that does also require a backend that allows a lot of compiler customisation, because it’s always necessary for one reason or another. setuptools wins on the weight of existing docs, and it’s customisable enough, but because the customisation usually depends on the defaults not changing it means we really can’t do a lot there without breaking people.

This is a big part of why I keep recommending people not just assume they should use setuptools (or any other obvious default), and to take an interest in the build backend they choose.