Figure I ought to make some kind of announcement, though I’m not really expecting this to be of broad interest.
I’ve finally bitten the bullet and made a PEP 517 backend that does what I need. Quite literally, it covers my needs, and likely nobody else’s
PyPI page is here: https://pypi.org/project/pymsbuild/
Firstly, it only works on Windows and always requires Visual Studio to be installed (possibly just Build Tools, but I haven’t tried that yet). This is because it generates an MSBuild project to do most of the heavy lifting. 
The downside of this is that it’s somewhat overkill for plain Python packages. That’s fine, the only one of those I’ve done recently is this package
The upside is you get much better incremental builds for native packages, including (as I shake the bugs out) in-tree builds (which are essentially “installs” into the source tree, as all build artifacts naturally live in separate directories and you have to work harder to mix them up with your sources).
Secondly, you have to specify metadata manually and precisely - there’s no validation built in at all. Even the metadata version is something you have to specify! I trust myself to get this right (a.k.a. clean up my own mess when I get it wrong), and it was much simpler to code!
But probably the biggest feature (for me) is that you can just specify files and where they should go in the final package! My biggest annoyance with all the rest is trying to coerce them to put certain files in certain places, so now it’s easy to make thing go where I want (under my top-level package directory).
I don’t really see this as competing with any other build tools, and I’ll likely keep using those too (or at least setuptools, which tends to annoy me less than flit), and it won’t build up a huge userbase I’m sure. But I’m already finding it a very useful escape hatch for complex package builds (my current project has multiple Cython modules in a PEP 420 namespace package with adjacent type stubs).
1: Although, MSBuild now runs anywhere .NET Core runs, so it could become cross-platform. But I’ll do that when I need it