Sorry, I’m really not sure what I can add. You seem to be trying to use pip in a way that’s very different from the intended usage, and without putting far more effort than I have time for into understanding how p4a works, I can’t see how your proposed “short/medium term” solutions would work.
I honestly have no idea what you mean by this paragraph, sorry.