@barry-scott and @onePythonUser – When I first coded the program I used shell=True because I was in total control of the input string and did not have any of the security concerns that can exist in other applications. When I encountered the shutdown problem, I searched for solutions and found a few examples where the shell had been the problem. I switched to what you see now and the problem persisted.
Incidentally, Paul that is why the string.format() was present in the code. I had used it to put the filename into the shell command. When I converted to a list, I broke the command into pieces and mindlessly kept the format() command. Good catch on your part.
Barry, when you asked about strace, were you referring to using it on the thread-launched subprocesses or on the main Python program that creates these threads? I have not yet tried strace on the main Python program. That is next on my list of things to try.
As for the threads, I am not sure if what you mean by “exited” is the same thing as no longer being “alive.” As you may see in the sample program, it checks all of the threads that were created and does a join() on any thread where thread.is_alive() is True.
Perhaps you mean something else? If so, could you explain more fully? How can I determine what you asked about?
@barry-scott I ran the main Python program with strace, using -ff to follow the forks and print each process in a separate file. I set the program to start 3 threads, each launching a subprocess.Popen instance.
Running the program with only three threads resulted in 93 separate process files. I am an strace novice. This is beyond my ability to analyze. I am looking for the proverbial needle in a very large haystack and I do not even know what the needle looks like.
I tried running your code with a small number of changes.
My system is Fedora 41 with the KDE plasma desktop and ffmpeg 7.1.1 and I used python 3.13.2.
I used PyQt6 not 5 (You should use PyQt6 unless you are using an old OS).
To test I used a .mov file and some .mp4 files.
The program ran and when I click on close it prints FINI and exits.
The only odd thing is that something turned off echo in the terminal.
I fix that with stty sane after the test.
For reference here is the code I used.
#!/usr/bin/env python3
#
import sys, subprocess, queue, threading, os
from PyQt6 import QtWidgets, QtCore
class AudioPlayerDialog(QtWidgets.QDialog):
def __init__(self, mediaList):
super().__init__()
vbox = QtWidgets.QVBoxLayout()
pb = QtWidgets.QPushButton('CLOSE')
pb.clicked.connect(self.close)
vbox.addWidget(pb)
self.setLayout(vbox)
self.calcGainTimer = None
self.gdCalcThreads = None
self.finished.connect(self.closeEvent)
self.mediaList = mediaList
self.mediaListLen = len(mediaList)
self.normGainList = [-1. for x in mediaList]
self.buildNormGainList()
def closeEvent(self, event):
print('**** close event')
if self.calcGainTimer is not None:
print('**** stopping calcGainTimer')
self.calcGainTimer.stop()
if self.gdCalcThreads is not None:
print('**** checking gain calc threads')
for idx, t in self.gdCalcThreads.copy().items():
if t.is_alive():
print('**** waiting for thread {} to finish'.format(idx))
t.join()
print('**** closing ...')
def buildNormGainList(self):
self.calcGainQueue = queue.LifoQueue(maxsize=self.mediaListLen + 10)
self.calcResultsQueue = queue.Queue(maxsize=self.mediaListLen)
for i in range(self.mediaListLen-1, -1, -1):
self.calcGainQueue.put_nowait(i)
# setup gain/dur calculation thread dictionary
self.gdCalcThreads = dict()
# For each expiration of the following timer, a thread will be
# started to calculate the replay gain for a particular file
# and a check will determine if all files have been processed.
print('@@@@ starting updt timer')
self.calcGainTimer = QtCore.QTimer()
self.calcGainTimer.timeout.connect(self.stopGainDurCalcWhenDone)
self.calcGainTimer.setInterval(500) # interval in msec.
self.calcGainTimer.start()
def stopGainDurCalcWhenDone(self):
# this is the target of the updt timer
try:
# start a thread if the queue is not empty
idx = self.calcGainQueue.get_nowait()
if idx not in self.gdCalcThreads:
print('@@@@ creating thread for', idx)
thread = threading.Thread(None, self.requestGain, args=(idx,))
self.gdCalcThreads[idx] = thread
thread.start()
except queue.Empty:
pass
except Exception as err:
print('**** ERROR: get request from calcGainQueue failed.\n\t'\
'{}'.format(err))
# check if any thread finished but updates not yet finished
try:
idx, gain = self.calcResultsQueue.get_nowait()
self.normGainList[idx] = gain
print('@@@@ update finished for', idx)
except queue.Empty:
pass
except Exception as err:
print('**** ERROR: get request from calcResultsQueue failed.\n\t'\
'{}'.format(err))
# check if all files have been processed; stop updt timer if so
done = True # assume all calc done
for i in range(self.mediaListLen):
if self.normGainList[i] < 0.:
done = False
break
if done:
print('@@@@ all calc are done -----------------------')
# all files have been processed
print('@@@@ stopping updt timer')
self.calcGainTimer.stop()
self.calcGainTimer = None
self.gdCalcThreads = None
TEST = False # set True to pass dummy filename to ffmpeg
def requestGain(self, idx):
print('@@@@ start subprocess for', idx)
if self.TEST:
fileName = 'XXXX'
else:
fileName = self.mediaList[idx]
cmd = ['ffmpeg', '-i', fileName, '-hide_banner',
'-af', 'replaygain', '-f', 'null', '-']
proc = subprocess.Popen(cmd, stderr=subprocess.PIPE)
try:
_,err = proc.communicate(timeout=60)
text = err.decode('utf8')
except subprocess.TimeoutExpired:
print('**** ERROR: Wait for report from subprocess timed out')
proc.kill()
text = ''
except Exception as err:
print('**** ERROR: Unable to get report from subprocess: {}'.format(err))
proc.kill()
text = ''
# actual program parses gain from text
gain = 1.
self.calcResultsQueue.put_nowait((idx,gain))
if __name__ == '__main__':
app = QtWidgets.QApplication(sys.argv)
d = '/shared/Media/iTunes/Music/Vangelis/Direct'
# put names of media files in list
mediaList = [os.path.join(d, f) for f in os.listdir(d)]
apd = AudioPlayerDialog(mediaList)
apd.exec()
print('\nFINI')
sys.exit()
@barry-scott I will probably transition to PyQt6 with my next OS change. I am not eager to run all of my Qt programs and discover one-by-one what changes may be needed to get everything running again.
I tried running my code without any changes and had the same experience you reported. Echoing in the terminal window was turned off as you indicated. By entering stty sane, character echoing was restored.
But consider the first odd behavior I observed that started this investigation. I use Geany as my code development tool. When I press F5 to run the program, it starts a gnome-terminal window and runs the code in that window. When the program completes, the following messages appear:
(program exited with code: 0)
Press return to continue
When no threads overlapped (no concurrency) in my testing, pressing Return/Enter closed the window. When overlap did occur, pressing Return/Enter did not close the window. As far as I could tell, all key presses were ignored except for ctrl+C, which closed the window. This suggests to me that more is going on than just echoing being turned off.
In any case, I tried launching my code in two different situations: (1) from a context menu in my file manager, and (2) by an IR remote using irexec in the lirc package. Happily, both use cases seem to work OK. I can launch repeatedly and cannot see any unwanted processes hanging around afterwards.
I think it is fair to say there is a bug somewhere but apparently harmless and surely of such low priority that valuable time would not be spent trying to find it and fix it.
