What's the deal with add_dll_directory?

Python Help
1

Can someone please address my points on this thread (OP and answer):

2

Please copy the question here so that the answer will be with it.

3

Copying the question:

The python [doc][1] explains the breaking change:

New in version 3.8: Previous versions of CPython would resolve DLLs
using the default behavior for the current process. This led to
inconsistencies, such as only sometimes searching PATH or the current
working directory, and OS functions such as AddDllDirectory having no
effect.

I’ve never had such issues, and I’d be happy for a specific example.
I’ve never heard about AddDllDirectory before, and I was happy. I used to have a reliable process: dll (shared lib) dirs coincide with the environment path–like in any other OS. This allowed me to easily test if necessary dlls are in the path, e.g., via [dlldiag][2]. I was able to easily read and modify the env path. This applies to any executable on Windows, and I had robust tools to handle any dll issues.

Now, I have only this os.add_dll_directory(). I don’t see any way to read it, and there’s no convenient var to modify. I can’t compare what’s there in order to verify locations of my dlls. Rumor has it, that this function uses in the end AddDllDirectory, and there are no additional convenient functions in its [family][3].

I tried the following simple solution:

import os
for p in os.environ[ 'path' ].split( ';' ):
    if os.path.isdir( p ):
        os.add_dll_directory( p )

This resolved the useless error message

ImportError: DLL load failed while importing foo: The specified module could not be found

which meant couldn’t find a dependent dll (god forbids it will tell me clearly that a dll is missing and which one).

Now, however, I have a completely new useless message:

ImportError: DLL load failed while importing foo: The operating system cannot run %1

(ah, of course, it can’t run %1, how silly of me) which apparently means that there are two candidate dlls in the path, and it doesn’t know what to do (or the wrong one was loaded or something).

This is a new can of worms. I usually have in the path both release and debug versions of the dlls, and there was never such an issue. Now, I have to be careful to separate them and specify each dir with the proper release or debug that I’m running. If in my pipeline I have a script that executes another one as a different process, well, bad luck, these new dll paths aren’t inherited anymore, and you need somehow to pass them down the line.

How am I suppose to conveniently handle that without burdening the user with these dir specification?
How do I keep this cross platform? Now, I need special code for Windows.

Python suddenly decides to break from its nice cross-platform behavior (os.add_dll_directory() is available only on Windows) and starts meddling with obscure Windows functions.
Moreover, the function of win32api.LoadLibrary() is now inconsistent with how ctypes loads a dll.

How can I revert back the behavior of loading dlls to the way it was before Python 3.8?

Copying the answer:

Looks like my conclusion on the second error was wrong. It wasn’t an issue related to debug vs release (thank god–just an expression, I’m an atheist). It was probably the wrong dll version since I set the dirs in a reverse order. The corrected code is:

import os

def add():
    # Windows only
    if not hasattr( os, 'add_dll_directory'):
        return

    print( 'Adding dll dirs from env path' )

    paths = os.environ[ 'path' ].split( ';' )
    paths.reverse()
    for p in paths:
        if not os.path.isdir( p ):
            continue

        print( "Adding a dll dir:", p )
        os.add_dll_directory( p )

I think this solves my main problem, although I’m still unhappy about the whole thing.

To rehash, the problem was that the dir order didn’t match what is set on the system, and I had to reverse it. Otherwise, it chose the wrong dll.

Now, I don’t want to alarm you, but the [doc][1] says:

If AddDllDirectory is used to add more than one directory to the process DLL search path, the order in which those directories are searched is unspecified.

It worked in practice, but it shouldn’t have. I have no clue what they expect you to do. Maybe copy all the dlls to a single directory before running (yeah, right…).

Maybe [they][2] can shade some light on this whole mess.