How to Unload Native Extensions ( .pyd ) on Windows

Python Help
1

Hey there!

I’m using a native Python extension written in Rust in order to power a Blender plugin, but on Windows I have a problem when I try to uninstall the plugin. The issue is that, when a Blender plugin is installed, it runs a register function in a Python script that, in my case, imports my Rust Python module, in this case called core.

My Rust library is compile to a core.pyd file that is loaded by the Blender plugin when I from . import core in the Blender plugin. This works all fine and dandy until Blender wants to uninstall the plugin. When Blender uninstalls the plugin it runs an unregister function to allow the plugin to clean up, and then it tries to remove the directory containing the plugin. This is where things go wrong. Because the Python interpreter embedded in Blender is still running, it still has the core.pyd file open, and Windows ( :roll_eyes: ) doesn’t let you remove the file, causing an exception in the Blender Python interpreter and a failure to uninstall the Plugin. The recourse is to close Blender and try to remove it again.

My question is whether or not there is a way to tell the Python interpreter to close that core.pyd file so that Windows will let you remove it. Unix has the concept of unlinking files instead of deleting them outright so this problem thankfully doesn’t exist there.

2

Because the Python interpreter embedded in Blender is still running, it still has the core.pyd file open, and Windows ( :roll_eyes: ) doesn’t let you remove the file

A file with executable image sections mapped cannot be unlinked, even with POSIX semantics. (WINAPI DeleteFileW uses POSIX delete semantics by default now in Windows 10, with supported filesystems such as NTFS.) The request is denied as STATUS_CANNOT_DELETE, which maps to WINAPI ERROR_ACCESS_DENIED, C EACCES, and Python PermissionError.

That said, you are allowed to rename a loaded DLL to another directory in the same filesystem, which may allow the unregister function to succeed.

My question is whether or not there is a way to tell the Python interpreter to close that core.pyd file so that Windows will let you remove it.

AFAIK, the interpreter doesn’t implement detailed reference tracking to know whether it’s safe to unload an extension module. They’re loaded for the lifetime of the process.

Unix has the concept of unlinking files instead of deleting them outright so this problem thankfully doesn’t exist there.

Windows filesystems also support hard links. But Windows uses the term “delete” (e.g. DeleteFileW) instead of “unlink” when deleting a link. Like Unix, a file is deleted from disk when its link count is 0. There are two fundamental differences compared to Unix.

  • Windows filesystems implement a cooperative data access model, which requires all opens to agree to share read/execute, write/append, and delete/rename access. A link cannot be renamed or unlinked if existing opens do not share delete/rename access.
  • Classically, Windows guarantees that a link will remain named in the filesystem as long as it’s referenced by an open File object. It can be renamed to another directory, but not removed entirely. This is implemented by making a ‘delete’ request just set a delete disposition on the underlying filesystem File Control Block (FCB). This disposition is executed once the last File object reference is cleaned up. Until then, a ‘deleted’ link remains linked in the directory and cannot be opened again for any access.

Windows 10 supports unlinking with POSIX semantics. This still requires shared delete/rename access, but the target file in this case gets immediately unlinked from the directory. Interestingly, NTFS still keeps the file linked in the filesystem, but renames it according to its file ID in an inaccessible system directory. For example:

>>> f = tempfile.NamedTemporaryFile()
>>> os.remove(f.name)
>>> h = msvcrt.get_osfhandle(f.fileno())
>>> print(GetFinalPathNameByHandle(h, 0))
\\?\C:\$Extend\$Deleted\000F0000000011946CE51530

This behavior isn’t specified, so other filesystems that implement POSIX delete semantics may have undefined behavior (e.g. fail the request) if GetFinalPathNameByHandleW or GetFileInformationByHandleEx: FileNameInfo is called on an existing handle for a file that was deleted POSIX style.

1 Like
3

Thanks for the detailed reply!

I think I misunderstood some of the nuances of the filesystem interactions, I just new I only ran into the problem on Windows. :wink:

Anyway, if I can rename the file, that could work. So I would just move the DLL to a temp directory in the unregister() function and then when Blender tries to delete the plugin dir after it runs unregister() the dll will no longer be in that dir, it will be in the temp directory and the delete should succeed.

I’ll try that out. Thanks for the help!

4

So I would just move the DLL to a temp directory

That should work as long as the target temp directory is in the same filesystem, i.e. you can’t rename a file across drives. Also, you need permission to rename the file. If the DLL is installed in %ProgramFiles%, a standard user won’t be allowed to rename it.

1 Like
5

OK, good to know.

That won’t be a problem in my case because Blender plugins are always installed to a user-specific dir, %APPDATA%/blender/something if I remember right.