Why would a relative path in xyz.py work but not the compiled xyz.exe not?

Python Help
1

First, I am a Python noob… (using 3.11.4) I am creating a simple test program which takes a file and converts to another type or file using a conversion program that I call with a relative path. The sticking point is that I need this complied program to be portable and work on various systems and pull everything it needs from the root file.
xyz.exe file path: “Portable1/dist/xyz.exe”
xyz.py path Portable1/xyz.py
Conversion file path: “Portable1/bkg/bin/convert.exe”

The code that I have works very well when I run from xyz.py. I have used a number of different methods to call the conversion file, all of which work as long as I use a valid form of relative path. This is the current:

Get the root folder of the project

    project_root = os.path.dirname(base_dir)

    converter_path = os.path.join('bkg','bin','convert.exe')
    converter_command = f'"{converter_path}"'

    for output_type in selected_output_types:
        output_file = os.path.join(project_root,'Portable1', f'{base_file}.{output_type}')

        if selected_input_type.get() == 'xyz' and output_type == 'zyx':
            command = f'{converter_command} -i {input_file} -c:f libx {output_file}'
        else:
            command = f'{converter_command} -i {input_file} -c copy {output_file}'

Every iteration of code that works calling the conversion program, I compile. Each time I do, I get a message stating that it cant find the convert.exe. Note: if I change the path to call the convert program which is installed on the computer, it does work, but since I need it to be mobile… Relative path needs to work.

Message:
Error occurred during conversion:
The system cannot find the path specified.

Any help would be appreciated.
Thanks,
Noobie

2

Not sure if this is going to help you: I’ve coded a file backup app, from which I can offer you this snippet:

from pathlib import Path, PurePath
from os import walk, makedirs

for path, dirs, files in walk(SRC):
    for file in files:
        dst_path = Path(path.replace(ROOT, DST))
        if dst_path.exists():
            pass
        else:
            makedirs(dst_path)
        src_path = PurePath(Path(path).joinpath(Path(file)))
        dst_path = PurePath(dst_path.joinpath(Path(file)))
        print(f"Copying {file}")
        RW = read_write(src_path, dst_path)
        COUNT += 1

Maybe you can study what I have done and apply some of the techniques to your own project.

EXAMPLE_CONFIG = """
[root]
root = /home/user_name
[source]
src = /home/user_name/python
[destination]
dst = /home/user_name/backups
"""
3

I am not new to coding, I am new to coding in Python. I have a working knowledge of dirs/subdirs structure. My fundamental question is “why”… does python work in a “.py” and not work in a complied “.exe”. Is my code fundamentally flawed?

4

A couple of questions that might clarify things:

  1. How are you building the .exe?
  2. What is base_dir in this case? Is this supposed to be relative to the current working directory or relative to the location of the .exe?
5

Root “Portable1”

xyz.exe file path: “Portable1/dist/xyz.exe”
xyz.py path Portable1/xyz.py
Conversion file path: “Portable1/bkg/bin/convert.exe”

In this instance, the actual path is: C:\Users\Owner\Desktop\Portable1

I am using: “pyinstaller Portable1.spec” to build.

And yes, I want everything to be relative to Portable1, wherever it might be copied to…

FYI… When using xyz.py, at file conversion all converted files are deposited in the root “Protable1” dir.

6

If it’s not finding convert.exe you may want to look at what os.path.realpath(converter_path) is in each case.

Potentially this is just due to the difference in working directory?

The .py file can resolve to “Portable1/bkg/bin/convert.exe”
While the .exe may resolve to “Portable1/dist/bkg/bin/convert.exe”

There are some complications if you’ve used __file__ to locate the .py file as that doesn’t work as expected once bundled into a pyinstaller .exe but I don’t think that’s the case here?

Perhaps you can try:

import sys

if getattr(sys, "frozen", False):
    # Running as .exe
    converter_path = os.path.join("..", "bkg", "bin", "convert.exe")
else:
    # Running as .py
    converter_path = os.path.join("bkg", "bin", "convert.exe")
7

Since Python doesn’t truly compile to .exe, what you’re really doing there is taking some sort of stub .exe file, slapping your Python code onto the end of it, and then running that. So there definitely will be changes to the behaviour, usually subtle ones, but importantly, the exact changes depend on how you built that exe file. In general, I advise against trying to convert .py to .exe, since it seldom gives any advantages, and makes it impossible for end users to upgrade the Python version (or to select 32-bit or 64-bit, or any other change).

8

This really depends on who the end users are. I’ve found it much easier to get someone to use something if all they have to do is run an .exe and don’t need to install python/setup an environment.

I generally use pyinstaller (or cx_freeze) for windows programs that are intended for people who don’t have a python install and wouldn’t otherwise have one. There are complications but once you know what those are you can usually find a way to make things work.

Also on the flipside to the argument about them updating python it also means you don’t have to worry if the end user does have python but it’s an old version that doesn’t have some of the features you’ve used.

9

True… This is why I am making sure that everything is contained in one folder, and everything required is contained within. David: I am currently integrating your code, into mine… It sems to hang at “copying Portable.py”

10

Yes, and importantly, it depends on how consistent the end users are. Is this a corporate environment where the IT department specifies exactly what OS you’re running, what hardware it’s running on, etc? Then it’s probably fine - a single Python deployment will work on them all. People on the internet who’ve just purchased your device and want the corresponding software? That’s more of a pain if you get something wrong, and more limiting if you aim for the lowest common denominator.

How do you choose which Python to use?

That’s true, but ultimately, what it means is that the distributor of the app is now ALSO responsible for the distribution of Python. If you ship a version of Python that is old and has known flaws, you and you alone are responsible for putting this on people’s systems, since (short of ripping your code out of your executable and running it independently) end users have no power to update their Python, even bugfix/security releases. If you stop making new releases, they can’t just start using your program using ActiveState or another commercial Python provider.

11

The intended user group is a fairly large group of peers who need this as a shortcut to performing a tedious and repetitive task to complete a desired outcome. This will save a huge amount of time and frustration. Versions and updates will be easily managed as its not a commercial product.

12

I am using variable which you select the file to convert, using radio buttons, confirm the file type. then using check boxes, select the output types (one or many)… It all works well in .py, and in compiled format, but only if I code the file being called from the computer (convert.exe program installed on the computer, as a registered “program”).

13

I feel like this has gone off topic but the alternative is not that the user has the power to update the python used to run your program, the alternative is they don’t use the program.

That’s true of any compiled release though, I don’t see how it’s any different using python and pyinstaller than if you’d made the software with any other language with a library that turns out to have a security flaw. This isn’t about making an .exe to hide the source code, it’s about making an .exe so the end user doesn’t have to learn how to setup a python environment and install dependencies if all they want to do is run your program. (In my case the source is also available, so they could do whatever they want - but the bundle is far more convenient.)

14

I would log all of the commands somewhere to check that they’re the same in both instances.

Possibly worth calling os.path.realpath on all of the paths to make sure they are also correct when you do.

If they are identical there could be some security feature causing it to hang?

15

All I really need to do is figure out how to call the file from this (which works in .py):
ffmpeg_path = os.path.join(‘bkg’, ‘bin’, ‘convert.exe’)
ffmpeg_command = f’“{convert_path}”’

to "something that works in a compiled format…

16

it dosen’t “hang”… The message is:
“Error occurred during conversion:
The system cannot find the path specified.” <— (the “convert.exe”)

I can call the convert file in a number of different ways, which I have done, all of which work in .py… If I intentionally change the path, I get the same message in .py as the compiled version:
“Error occurred during conversion:
The system cannot find the path specified.”

Therefore I know its a way that the compiled exe is looking for the “Convert.exe”

17

LOL… I understand in “simply using the .py file”… but I use this code and convert.exe so I don’t want to give away any of my “trade secrets”… I am the person that people come to the do these conversions which eats into my own personal productivity. If I can furnish a program without divulging what I’m doing and how I’m doing it, it is a win for me. I no that its selfish, but I use the tools that I have to make $$ for the general public and don’t want my peeps to become competitors.

18

The entire point of dynamic linking is that this is NOT the case for the vast majority of libraries. For example, I have a huge number of compiled binaries on my computer, and a good few of them will be linked against openssl. What happens when there’s an openssl update? Do I have to recompile them all? No - they will use the updated shared object the next time they run. (Linux calls it a .so or “Shared Object” file, Windows calls it a .dll “Dynamically Linked Library”, they have the same effect.)

And yes, it’s true that you COULD statically link with your libraries, but this is only done in a very small set of circumstances (eg busybox) where you really truly do need it to be 100% self-contained.

I’m aware of this and never said anything about hiding the source code, although to be fair, that is the usual (and most flawed) reason for wanting to convert .py to .exe.

That’s a really nice theory, and works great if you know EVERYTHING that a person will be running on. Hence my questions about whether you build a 32-bit version as well as a 64-bit version. (And ARM too? I think Python supports ARM on Windows.) Depending on how broad your userbase is, you might need to support newer and older Python versions; or maybe you can get away with just using an older Python version so it runs on all the Windowses you support, but forego the significant performance improvements in recent versions.

Effectively, your program now needs as many options as Python itself has. Is this really easier? Is it easier enough to justify extra debugging work from “it works in .py but fails in .exe”?

And we STILL don’t know from the OP what method is actually being used here - no information about how the exe is being built.

19

Your issue seems to be relative paths, the base path has changed when you built the .exe but you have not changed your code to accommodate for this. You can either move the converter.exe so it’s in the same location (relatively) to the .exe as it was to the .py file or you can change the code.

20

Oh. Well, then… I take back what I said about hiding the source code.

Fun fact: You aren’t hiding much when you convert to .exe. All your code is still there, just hidden a little. So, save yourself a ton of trouble, stop trying to make exe files when it won’t help you anyway.

Then there’s an easier way: don’t give them the program at all. Put it on a web site, have them upload their files to it. You do the conversion on your own server, and give back the file. This is the ONLY way to truly stop them from accessing the code.

It also happens to be a well-known technique for which there are lots of tutorials around. Searching the web for “build a web app with Python” will give you a ton of good hits.