Conflicting binary extensions in different packages

[youben11 (Ayoub Benaissa)]

@steve.dower

Ah, this is a slightly different kind of problem than what I imagined. You would have to convince one of the two packages to change to use the other OpenMP runtime, which may not be source compatible.

I figured after reading what you referenced and found BLAS, LAPACK and OpenMP - pypackaging-native

@steve.dower

Best way to do this (today and likely forever) is with separate processes. Depending on how the libraries are being used, multiprocessing is likely your best bet, but subprocess may be sufficient if it looks more like sequential tasks (i.e. you run a big job with Torch and then serialise the result to return back for later use).

Indeed, but why I said imaginary is because I meant this is handled automatically, and the user doesn’t have to write the multiprocessing logic but more seriously, I like the “Potential solutions or mitigations” part, and I think those are more probable solutions to land than what I just described