Problem with subprocess.run working and not working

i have the following code

cmd1 =         f"openssl req -nodes -new -out {ca_base_folder}/base.domain.csr -config {ca_base_folder}/req.base.domain.conf"
subprocess.run([f"openssl req -nodes -new -out {ca_base_folder}/base.domain.csr -config {ca_base_folder}/req.base.domain.conf"], shell=True)

the content of cmd1 is
‘openssl req -nodes -new -out /Users/mk/CA_test/base.domain.csr -config /Users/mk/CA_test/req.base.domain.conf’

the result if the subprocess.run is
CompletedProcess(args=[‘openssl req -nodes -new -out /Users/mk/CA_test/base.domain.csr -config /Users/mk/CA_test/req.base.domain.conf’], returncode=0)

entering the cmd1 at the command line the .key file is created, but not with the subprocess.run
while with the next lines in my code

subprocess.run([f"openssl ca -batch -config {ca_base_folder}/sign.cert.conf -extfile {ca_base_folder}/req.base.domain.conf -extensions my_extensions -out {ca_base_folder}/{cert_common_name}.crt  -infiles {ca_base_folder}/base.domain.csr"], shell=True)

the .crt file is created and all following subprocess.run lines

    subprocess.run([f"openssl x509 -in {ca_base_folder}/{cert_common_name}.crt -out {ca_base_folder}/{cert_common_name}.pem -outform PEM"], shell=True)
subprocess.run([f"openssl x509 -in {ca_base_folder}/{cert_common_name}.pem -out {ca_base_folder}/{cert_common_name}.der -outform DER"], shell=True)

are working as expected

Any hint ?

Rather then use a single string as the command line it’s recommended to use a list of args. This then avoid issues with parsing the string into args, as you made the args explicit. For example:

subprocess.run(['/usr/bin/echo', 'arg1', 'arg2'])

If you use the list form does that fix the issue?

I will tie this, but I in total 6 or 7 quit similar openssl commands and only this one does not work, even if the returned result i 0

That is one command openssl with a lot of args if I split them openssl does not accept that. On the other side the next openssl command is much longer and works

Please share the code showing you call to run() please.
When you say one command works and another fails please share code of both.

These are working

 subprocess.run([f"openssl x509 -in {ca_base_folder}/{cert_common_name}.crt -out {ca_base_folder}/{cert_common_name}.pem -outform PEM"], shell=True)
subprocess.run([f"openssl ca -batch -config {ca_base_folder}/sign.cert.conf -extfile {ca_base_folder}/req.base.domain.conf -extensions my_extensions -out {ca_base_folder}/{cert_common_name}.crt  -infiles {ca_base_folder}/base.domain.csr"], shell=True)
 subprocess.run([f"openssl req -nodes -new -x509 -days 3650 -config {ca_base_folder}/req.ca.conf -out {ca_base_folder}/{CA}.crt"], shell=True)

This one does not

subprocess.run([f"openssl req -nodes -new -out {ca_base_folder}/base.domain.csr -config {ca_base_folder}/req.base.domain.conf"], shell=True)

perhaps this helps, created a test

import sys
import os
import subprocess

ca_base_folder = "/Users/mk/CA_test"
cmd1 =          f"openssl req -nodes -new -out {ca_base_folder}/base.domain.csr -config {ca_base_folder}/req.base.domain.conf"
print(cmd1)
subprocess.run([f"openssl req -nodes -new -out {ca_base_folder}/base.domain.csr -config {ca_base_folder}/req.base.domain.conf"], shell=True)

the base.domain,csr

 cat base.domain.csr
-----BEGIN CERTIFICATE REQUEST-----
MIIDVjCCAj4CAQAwgZIxCzAJBgNVBAYTAkRFMQ8wDQYDVQQIDAZCQVlFUk4xFzAV
BgNVBAcMDlJFRE5JVFpIRU1CQUNIMSIwIAYDVQQKDBlJbmcuIEJ1ZXJvIE11ZWxs
ZXItS25vY2hlMSIwIAYDVQQLDBlJbmcuIEJ1ZXJvIE11ZWxsZXItS25vY2hlMREw
DwYDVQQDDAh6ei5hYS5kZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
ALsUg6u+MjkIY39pN866gUMjO+tTglAMzlN0tUzAMfMYAgDzn9ipIKSuyIahj+p1
XZMxJIkS56JvgNhwXLJMttQtjQcb+47rrSSfPuiWqmWOIi91s3cbFbhZCEqA5WX2
iZfZf+5CWHYAod97UdJawEuksu+xLyKcsP/w1yDNeXLIidVy5HuaJXHoWNWJh2yE
QyLD35fS19YkklKvPQpNlNnZm+/cDGfA3CbEiDzopYF+7b7+yS2jM3ESNPYWQoGJ
tpwKphU7L0t9Mz/HKes014Rx/nVzntFvwM1pPp/J5mzBbfuMggiVUFR/QMvlHyHC
jwvLDRWz5KP+pZy3W1Tos68CAwEAAaB+MHwGCSqGSIb3DQEJDjFvMG0wDgYDVR0P
AQH/BAQDAgKkMA8GA1UdEwEB/wQFMAMBAf8wFgYDVR0lAQH/BAwwCgYIKwYBBQUH
AwEwEwYDVR0RBAwwCoIIenouYWEuZGUwHQYDVR0OBBYEFM/7QVu64e7MTSqAB9mm
Wiq/951tMA0GCSqGSIb3DQEBCwUAA4IBAQCkoUXVzIJZTF8P5oKaLBusA3Gi1HNI
6ngA20XMeRRke4EDX8h6sh23w4P5bHR+Lc7PCtLL8QpVY2Iwisx2SFLiiW4mI3YM
TW3WvC3/C2wfhdbXKOuroQm0lo3dYu2jZwPOMBIo+090C60HLAnbemH7/dBCAj0W
I19TWjWCPoP/U7Ms6Ypk822/59ssMfaoJcaDQjyH+kNJgSYzSgwEopy5w7UBjnEf
Q1BLWt7anxx+4mu0IGObXDg1+5jAcDAPTRmnEpfNko5zICMIWCnlcn08sQm589H5
DRqm4TOMAOeurw6ilqeLDH0VOTmp/DqQ/nJsHVnGjmlNrXF5BIB5yUWt
-----END CERTIFICATE REQUEST-----

and the req.base.domain.conf

cat req.base.domain.conf
[ req ]
default_bits = 2048
default_keyfile = zz.aa.de.key
encrypt_key = no
default_md = sha256
prompt = no
utf8 = yes
distinguished_name = my_req_distinguished_name
req_extensions = my_extensions

[ my_req_distinguished_name ]
C=DE
ST=BAYERN
L=REDNITZHEMBACH
O=Ing. Buero Mueller-Knoche
OU=Ing. Buero Mueller-Knoche
CN=zz.aa.de

[ my_extensions ]
keyUsage=critical, digitalSignature, keyEncipherment, keyCertSign
basicConstraints=critical,CA:TRUE
extendedKeyUsage=critical,serverAuth
subjectAltName=@my_subject_alt_names
subjectKeyIdentifier = hash

[ my_subject_alt_names ]
DNS.1 = zz.aa.de

The smal test creates a command line that does the same as the subprocess line.
The command line creates the .key file, the subprocess line does not
While the other subprocess lines creates the files.
Appreciate any help

Sorry Guys, it works but not as expected. The .key and .crt Files etc. are created but not where expected in the certificate directory but in the program directory
Have a nice Sunday
Rainer

I am recommending you stop using shell=True and instead pass the command as a list or command args.

Look closely at the echo example I posted.

Sounds like you either need to set the current working directory or find how to use absolute paths for the outputs.

It was a little bit tricky to find. In the commands I had everywhere the full path, but in one of the generated conf files I forgot to enter the path variable at one entry.
Now everything works.
What it does generated a self signed CA and then to create certificates signed by this CA as much as you want for internal use.
create a CA and a first certificate in less than one minute in GUI
All entries are re checked if valid
Is some one is interested let me know, it will be availabe at GIT soon

Glad you found it.

FYI this is what I mean by using a list and not a string for the commands:

subprocess.run(
    ['openssl', 'req', '-nodes', '-new',
     '-out', f'{ca_base_folder}/base.domain.csr',
     '-config', f'{ca_base_folder}/req.base.domain.conf'])

thank you
Rainer

by the way I use another solution now. I write the full command to disk, make this file executable and execute it then. Positive is that I can whats really done for debug purpose by displaying the command file.

# create certificate if not exists
def create_certificate ():
    # check cert exists
    global ca_base_folder
    if os.path.exists(f"{ca_base_folder}/{cert_common_name}.crt"):  # Cert exists
        show_error(data_frame,"!!! CERTIFICATE EXIST WE STOP HERE !!!")
        sys.exit()
    # conf file for new cert
    create_req_base_domain_conf_file()
    create_sign_cert_conf_file()
    subprocess.run([f"rm {ca_base_folder}/cmd*"], shell=True)
    try:
        with open(f"{ca_base_folder}/cmd1", 'w') as f1:  # try open
            f1.write( f"openssl req -nodes -new -out {ca_base_folder}/base.domain.csr -config {ca_base_folder}/req.base.domain.conf\n")  # read
            f1.close()  # close
    except IOError as e:
        print("An error occurred:", e)
    subprocess.run([f"chmod +x {ca_base_folder}/cmd1"], shell=True)
    subprocess.run([f"{ca_base_folder}/cmd1"], shell=True)

#
    try:
        with open(f"{ca_base_folder}/cmd2", 'w') as f2:  # try open
            f2.write(f"openssl ca -batch -config {ca_base_folder}/sign.cert.conf -extfile {ca_base_folder}/req.base.domain.conf -extensions my_extensions -out {ca_base_folder}/{cert_common_name}.crt  -infiles {ca_base_folder}/base.domain.csr\n")  # read
            f2.close()  # close
    except IOError as e:
        print("An error occurred:", e)
    subprocess.run([f"chmod +x {ca_base_folder}/cmd2"], shell=True)
    subprocess.run([f"{ca_base_folder}/cmd2"], shell=True)

Regards Rainer

I usually have logging that logs the commands my scripts are going to run so that I can try them in a shell if I need to.

I have moved away from shell scripts to using python and its subprocess.run() for anything that is not trivial to write.

And I learnt, the hard way, to avoid using the shell=True so that my scripts are robust.

1 Like