Os.kill Signals not being received correctly, alternative is kill -SIGUSR1 command

Python Help
1

I’m trying to use the os.kill, Popen.send_signal, or otherwise some kind of pure-python implementation of sending a signal to a process. Unfortunately, none of them work, and instead, I spawn a kill process to do it for me with Popen.

You can read my original issue on StackOverflow here, but someone just downvoted it and I have my doubts it will be answered by the bag of dicks at SO, and I’ve rewritten my latest progress in this topic here.

Here’s my code, you should be able to run this (I am on Linux Pop!_OS) without install. Developed on Python 3.7.

import os
import subprocess
import signal
import time
import sys

# Define the command to execute
command = ["tar", "-xpf", sys.argv[2], "-C", sys.argv[1], "--totals=SIGUSR1"]

# Start the subprocess
print(' '.join(command))
process = subprocess.Popen(command, preexec_fn=os.setsid, stderr=subprocess.PIPE)

try:
    while True:
        # Ping the subprocess with SIGUSR1 signal
        # NOTWORK: process.send_signal(signal.SIGUSR1)
        # NOTWORK: os.killpg(os.getpgid(process.pid), signal.SIGUSR1)
        subprocess.Popen(["kill", "-SIGUSR1", str(process.pid)])

        print(process.stderr.readline().decode("utf-8").strip())
        # print(process.stdout.readline().decode("utf-8").strip())

        # Wait for a specified interval
        time.sleep(1.9)  # Adjust the interval as needed

except KeyboardInterrupt:
    # Handle Ctrl+C to gracefully terminate the script
    process.terminate()

# Wait for the subprocess to complete
process.wait()

For a bit of context, I found out that tar allows you to query progress reports on it by sending SIGUSR1 signals to it. You can confirm this yourself by running pkill -USR1 tar in the CLI after starting a tar process with --totals=USR1.

You can see my three different styles of sending signals above:

  1. Popen.send_signal - One would believe this should work just fine, but after inspecting the source code, I am inclined to believe it might never work.
  2. os.kill and os.killpg. I tried both, and I think os.kill is the intended method, but I can’t be sure.
  3. Spawning a kill process with -SIGUSR1 to select the signal, and the process PID given from the subprocess instance.

Only the third option works.

Additionally, the other two options kill the process, the return code being based on the signal: SIGUSR1 gives -10, SIGUSR2 is -12, and SIGHUP is -2. This is documented based on a program’s default behavior for signal handling.

So, what gives?

2

You are right os.kill() is what you need to use.
That is the same the kill command that you spawn,
Is the progress write to stdout or stderr?
I see that you capture stderr and are not reading it.
If that is where the progress us written that will explain why you see nothing.

3

It’s written to stderr. Can’t get the link on mobile email, but it’s in the stackoverflow post - documentation on how the signal works.

Additionally, the kill Popen method works fine and reads properly, and any python based signal methods terminate the process, so the issue lies within how python sends signals.

4

python sends then using the kill system call. python is not special in that regard.

You can use strace to check exactly what python is doing.

And use strace on the /usr/bin/kill and you will see its the same system call.

Oh and signal numbers are not negative. SIGUSR1 is 10 for example not -10.
On my Fedora 38 I see these defs:

/usr/include/asm-generic/signal.h:21:   #define SIGUSR1		10
/usr/include/bits/signum-arch.h:49:     #define SIGUSR1		10
5

That’s true, but the corresponding return codes are negated. If a process terminates due to SIGUSR1, its exit code will be -10. So the OP’s analysis is correct when noting that “the other two options kill the process” successfully.

I’m confused by why os.kill() and subprocess.send_signal() don’t work, since they would be the two most obvious ways to do it. I tried both using a simple sleep command, and it worked fine:

>>> proc = subprocess.Popen(["sleep", "60"], preexec_fn=os.setsid, stderr=subprocess.PIPE)
>>> proc.send_signal(signal.SIGUSR1)
>>> proc.wait()
-10
>>> proc = subprocess.Popen(["sleep", "60"], preexec_fn=os.setsid, stderr=subprocess.PIPE)
>>> os.kill(proc.pid, signal.SIGUSR1)
>>> proc.wait()
-10

Maybe there’s something about the way the tar command works?

6

The script is sending the signal before the child process has a chance to actually set a handler for SIGUSR1. Try the following instead:

import sys
import signal
import subprocess
import time

if sys.platform == 'linux':
    def wait_until_ready(pid, sig, timeout=1):
        deadline = time.monotonic() + timeout
        while time.monotonic() < deadline:
            mask = 0
            with open('/proc/{}/status'.format(pid)) as f:
                for line in f.readlines():
                    if line[:7].lower() == 'sigcgt:':
                        mask = int(line[7:], 16)
                        break
            if mask & (1 << (sig - 1)):
                break
else:
    def wait_until_ready(pid, sig, timeout=1):
        time.sleep(timeout)

cmd = ['tar', '-xpf', sys.argv[2], '-C', sys.argv[1], '--totals=SIGUSR1']
with subprocess.Popen(cmd, stderr=subprocess.PIPE, text=True) as p:
    try:
        wait_until_ready(p.pid, signal.SIGUSR1)
        while p.poll() is None:
            p.send_signal(signal.SIGUSR1)
            time.sleep(1)
            print(p.stderr.readline().strip())
    except KeyboardInterrupt:
        p.terminate()

On Linux, this waits until the child process sets a handler for SIGUSR1. On other POSIX platforms (macOS, BSD, etc), it simply waits for a second because I couldn’t be bothered to look up how to implement it properly.

2 Likes
7

Fantastic work! Works perfectly. I do wonder about a Windows implementation, but I’m working on Linux for both my development and production environments, so it doesn’t concern me personally.

Thank you.

8

Windows 11 includes “tar.exe” from libarchive 3.5.2. While this implementation of tar does support reporting read/write status to stderr, it’s only implemented on platforms that support Unix signals and either SIGINFO or SIGUSR1. The Windows kernel doesn’t implement Unix signals[1], so there’s no implemented way to signal “tar.exe” to report its read/write status to stderr. A named event would do the job, but the source would have to be modified to support passing the event name on the command line, and to create a thread to wait on the event and set the siginfo_occurred global variable.

  1. The C runtime emulates SIGSEGV, SIGILL, and SIGFPE based on a structured exception handler. It emulates SIGINT and non-standard SIGBREAK based on a console control handler for CTRL_C_EVENT, CTRL_BREAK_EVENT, CTRL_CLOSE_EVENT, CTRL_LOGOFF_EVENT, and CTRL_SHUTDOWN_EVENT. The latter 4 are all mapped to SIGBREAK. It emulates SIGABRT and SIGTERM only for use with C raise() and abort(). ↩︎