This could be another use case to add to the list in @takluyver’s new “Pip options for controlling use of prebuilt packages” thread.
The simplest API might wind up being one that lets the user opt in or out individual packages or types of packages, in which case this would be a special case.
Also, another thing to consider is whether we could let pip re-use from previous invocations dependencies previously built from source. I’m not sure how easy or hard that would be to implement or if it would violate some other assumption.