Subprocess stderr outputs even though the process works just fine

Python Help
1

Configs

Using macOS, python v3.12.4

Problem

So I am doing some bioinformatics stuff and I wanted to automate some commands using subprocess. I was using subprocess.Popen().
Here is the code I was testing my usages.

import subprocess

cmd = "pdb2pqr --ff=AMBER --apbs-input=7y6i.in --keep-chain --whitespace --drop-water --titration-state-method=propka --with-ph=7 7y6i.pdb 7y6i.pqr"

process = subprocess.Popen(
    cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True
)
output, error = process.communicate()
results = {"output": output, "error": error, "pid": process.pid}
output = results["output"]
error = results["error"]
print("*********ERROR IS **************") # to just get the error
print(error)

And it gave me the output, which is actually showing some warnings and minor errors but the overall process is a success. But it contains many more things apart from what should not be considered as just warnings and errors.
Output(some part only from starting)-

INFO:PDB2PQR v3.6.2: biomolecular structure conversion software.
INFO:Please cite:  Jurrus E, et al.  Improvements to the APBS biomolecular solvation software suite.  Protein Sci 27 112-128 (2018).
INFO:Please cite:  Dolinsky TJ, et al.  PDB2PQR: expanding and upgrading automated preparation of biomolecular structures for molecular simulations. Nucleic Acids Res 35 W522-W525 (2007).
INFO:Checking and transforming input arguments.
INFO:Loading topology files.
INFO:Loading molecule: 7y6i.pdb
ERROR:Error parsing line: invalid literal for int() with base 10: ''
ERROR:<CRYST1    1.000    1.000    1.000  90.00  90.00  90.00 P 1>
ERROR:Truncating remaining errors for record type:CRYST1
WARNING:Warning: 7y6i.pdb is a non-standard PDB file.

ERROR:['CRYST1']
INFO:Dropping water from structure.
INFO:Setting up molecule.
WARNING:Unable to find amino or nucleic acid definition for NAG.  Parsing as new residue.
WARNING:Unable to find amino or nucleic acid definition for NAG.  Parsing as new residue.
WARNING:Unable to find amino or nucleic acid definition for NAG.  Parsing as new residue.
WARNING:Unable to find amino or nucleic acid definition for NAG.  Parsing as new residue.
WARNING:Unable to find amino or nucleic acid definition for CL.  Parsing as new residue.
WARNING:Unable to find amino or nucleic acid definition for NA.  Parsing as new residue.
WARNING:Unable to find amino or nucleic acid definition for CL.  Parsing as new residue.
WARNING:Unable to find amino or nucleic acid definition for NA.  Parsing as new residue.
WARNING:Unable to find amino or nucleic acid definition for Y01.  Parsing as new residue.
WARNING:Suppressing further "Unable to find amino or nucleic acid definition for" messages
INFO:Created biomolecule object with 975 residues and 8170 atoms.
INFO:Setting termini states for biomolecule chains.


---------  -----   ------   ---------------------    --------------    --------------    --------------
                            DESOLVATION  EFFECTS       SIDECHAIN          BACKBONE        COULOMBIC
 RESIDUE    pKa    BURIED     REGULAR      RE        HYDROGEN BOND     HYDROGEN BOND      INTERACTION
---------  -----   ------   ---------   ---------    --------------    --------------    --------------

ASP 244 A   7.29*   83 %    2.57  514   0.94    0   -0.01 GLN 445 A    0.00 XXX   0 X   -0.20 ARG 158 A
ASP 244 A                                           -0.04 ARG 243 A    0.00 XXX   0 X    0.02 GLU 248 A
ASP 244 A                                            0.00 XXX   0 X    0.00 XXX   0 X    1.10 GLU 240 A
ASP 244 A                                            0.00 XXX   0 X    0.00 XXX   0 X   -0.88 ARG 243 A

ASP 255 A   4.12     0 %    0.34  186   0.00    0    0.00 XXX   0 X    0.00 XXX   0 X   -0.01 ARG 243 A
ASP 255 A                                            0.00 XXX   0 X    0.00 XXX   0 X   -0.03 ARG 261 A
ASP 255 A                                            0.00 XXX   0 X    0.00 XXX   0 X    0.02 ASP 259 A

ASP 259 A   3.89     2 %    0.87  288   0.01    0    0.00 XXX   0 X   -0.69 ILE 253 A   -0.10 ARG 243 A

ASP 311 A   3.81    22 %    0.75  343   0.02    0   -0.12 SER 314 A    0.00 XXX   0 X   -0.02 LYS 312 A

...

This is all part of “error” I printed. But clearly many things are not actually stderr.

Questions

  1. Is there anything wrong I am doing in python code specifically?
  2. How does subprocess tell if that text should be in stderr?
  3. How can I fix it? Cause I get an email saying it’s an error and I get scared.

(if you wanna know how I coded my mail to say error, if stderr=“” then success else error, if you need this to solve my issue…)

2

Check the return code instead of the stderr. Subprocess doesn’t decide what goes where, whatever process you’re running does. It’s entirely up to the program as to what it writes to stdout, stderr, or even what exit codes it uses.

1 Like
3

Following on from what @Monarch says, you could try running it from the command line, directing stdout and stderr to separate files, something like this:

pdb2pqr --ff=AMBER --apbs-input=7y6i.in --keep-chain --whitespace --drop-water --titration-state-method=propka --with-ph=7 7y6i.pdb 7y6i.pqr >output.txt 2>error.txt

Do output.txt and error.txt contain the same text as the Python script shows from stdout and stderr?

If yes, then it’s not something that Python’s doing, it’s something that pdb2pqr is doing.

4

The return code is 0, which means I can use that nicely!

Yes both output.txt and error.txt are same.