TerminateProcess via os.kill on Windows?

Ideas
1

On Windows there isn’t really a SIGKILL signal, hence it not existing in signal on Windows.

Could we define SIGKILL on Windows, then internally take:

os.kill(<pid>, signal.SIGKILL)

and have that call TerminateProcess?

I know it isn’t a 100% mapping of SIGKILL → TerminateProcess but its pretty darn close. From the winapi docs:

The TerminateProcess function is used to unconditionally cause a process to exit.

vs the GNU docs:

The SIGKILL signal is used to cause immediate program termination

TLDR: I propose defining signal.SIGKILL on Windows and if it is used by os.kill, internally call TerminateProcess; after all they seem to be functionally similar.

2

I think there is strong precedent that functionality in os is supposed to reflect the actual functionality offered by the operating system, including mirroring its API as closely as makes sense in Python; and then platform-agnostic wrappers are put at a higher level.

3

Actually upon looking at the source, it seems it actually already sort of does what I was asking about: https://github.com/python/cpython/blob/main/Modules/posixmodule.c#L8915

So I guess sending a kill with any regular signal is more/less the same on Windows… Weird.

4

Unfortunately, Python’s implementation of os.kill() on Windows is kind of a mess.

First and foremost, the use of GenerateConsoleCtrlEvent() is completely wrong. This API function requires a console process group ID (pgid). The current implementation of GenerateConsoleCtrlEvent() in the console host (i.e. conhost.exe or openconsole.exe) is ridiculously buggy in many cases if it gets passed a pid that’s not a pgid. The only way to get a pid value that’s known to also be a pgid value is to spawn a process with the creation flag CREATE_NEW_PROCESS_GROUP. Thus os.kill() should only call GenerateConsoleCtrlEvent() when passed a negative process ID, which, as specified by POSIX, indicates that it’s a process group ID. The pid value -1 should be special cased as Windows console process group 0, which includes every process in the current console session. The pid value 0 (i.e. every process in the current process group, according to POSIX) should fail with a ValueError. It can’t be implemented on Windows since there’s no documented way to query the process group ID of the current process.

Also, when the pid value is negative, the implementation of os.kill() should map signal.SIGINT and signal.SIGBREAK to the console control events CTRL_C_EVENT and CTRL_BREAK_EVENT. Those are the only two values that should be supported in this code path. Any value other than SIGINT or SIGBREAK should fail as a ValueError. In particular, the special signal value 0 that POSIX specifies can’t be reasonably supported to test for the existence of a process group.

If the pid value is positive, then the implementation of os.kill() should take the code path that calls OpenProcess() and TerminateProcess(). This code path can allow any value for the signal number, which will be used as the exit status value of the process. To better support cross-platform code, signal.SIGKILL could be defined as the integer enum value 1. It’s common on Windows to use 1 as the exit status when forcibly terminating a process. If the signal value is 0, only OpenProcess() should be called, to test for existence and PROCESS_TERMINATE access, as specified by POSIX.

3 Likes
5

Hi Eryk, I’m not sure if there has been any changes to the implementation of os.kill in the last year, but I wanted to add to the discussion something I noticed when working with FastAPI and uvicorn.

Since Process.terminate() is using TerminateProcess(), using it for hot-reloading in uvicorn caused on_shutdown events to be skipped, so it was changed to os.kill(self.process.pid, signal.CTRL_C_EVENT) which correctly handles termination, but also kills any process that has been opened when using the multiprocessing module or Popen without the CREATE_NEW_PROCESS_GROUP flag. This kills an Python console app when using uvicorn with reload in a subprocess.

A discussion was opened about a half year ago at the uvicorn repo, but hasn’t gained much traction, I guess because there is really no solution that covers all cases, short of CPython doing something about it or going back to Popen.

Even if technically they are using os.kill wrong according to the docs because they’re not using the proper flag, I think it’s somewhat frustrating that the higher-level multiprocessing module does not offer a way to gracefully terminating a process in Windows, having to resort to os.kill which is incompatible with multiprocessing due to the impossibility of using creationflags.

Also the os.kill documentation is confusing as it suggests that you’re killing as specific process when in reality you’re calling GenerateConsoleCtrlEvent on the whole group. Moreover, it does not refer to the need of using subprocess.CREATE_NEW_PROCESS_GROUP, which might lead to incorrect use (such is the case with uvicorn)

I think the culprit is Win32 at the end of the day for not providing a API function to gracefully terminate a single process, but perhaps CPython could work around it by providing multiprocessing.Process with a new parameter like create_new_group, or maybe just take creationflags like Popen does.

6

You should be able to call TerminateProcess() using the _winapi private extension and OpenProcess(). Pseudo-code:

import _winapi

def kill(pid, exitcode):
    handle = _winapi.OpenProcess(_winapi.PROCESS_ALL_ACCESS, False, pid)
    _winapi.TerminateProcess(handle, exitcode)
    _winapi.CloseHandle(handle)

It’s basically the same as os.kill(), except that it doesn’t call GenerateConsoleCtrlEvent() for CTRL_C_EVENT and CTRL_BREAK_EVENT, and it gives you a better control on the exit code of the terminated process.

On Windows, subprocess.Popen.kill() calls TerminateProcess(handle, 1).

We don’t add constants which don’t exist on a platform. If you’re unhappy about os.kill() API, we should design a new API around TerminateProcess() in the os module. It was discussed previously, but no one proposed any concrete API so far.

7

I’d really like to try this, but I am not sure how to apply it to a long-running process created with subprocess.Popen; could you clarify how this would be done?

Like @toni-neurosc, I am trying to run Uvicorn in reload mode inside a subprocess.