I have cobbled something together in my main script and it is running the main program now, which it wasn’t before, but the program is not seeing the files even in the same directory as well as other directories I want it to source from.
I tried to make a simple test to forget about the main script for now like so:
The echo shows the $PATH but the ls returns directory not found where mo1_dir1 is a subdirectory of the folder $PATH. I would like to have several folders that the main .bat file can look from but I have to figure how it works first! I am finding it strange that it doesn’t even see files within its own directory, even though I explicitly changed the directory with os.chdir(). It complains of file not found for even files in that same directory while if run outside of python it would see them.
So how to resolve these issues? It will really be a level up if I can learn how to add multiple sources for executables to digest as pathing issues are a big source of problems and being able to source multiple would be very liberating.
There is not a feature that searches for data files in a set of paths.
You will need to write your own code to do that.
In particular there is no linux feature that does this for external commands.
If you’re starting out with the subprocess module then it’s recommend to use the highest-level subprocess call available which is subprocess.run. This call automatically waits on the command to finish.
Modifying $PATH is used for very specific effects recognized only by a select few programs. If what you actually wanted was the effects of os.chdir() then you’d instead use the cwd= (change working directory) keyword.
# list "mod1_dir1" within "/path/to/other/folder/mod1"
result = subprocess.run(
["ls", "mod1_dir1"],
cwd="/path/to/other/folder/mod1",
capture_output=True,
text=True,
check=True,
)
print(result.stdout)
If you need to work with multiple working directories then you would iterate over them from Python code and call subprocess on each. You would do this using os.listdir, os.walk, or Path.walk depending on the folder layout.
If the layout is complex then you should only work with resolved/absolute paths to mod folders rather than trying to change $PATH or the working directory. Check pathlib for a friendly way to work with these folders.
Using pathlib.Path() you could also do something like Path(Path.home()).rglob("*/*") to access sub directories.
My understanding of what you are asking is that I’m not sure why you need subprocess.run at all!
Just to note, bash has a separate variable CDPATH that is uses to resolve directory names for the built-in cd. Any individual command like ls could, in theory, define an environment variable to be used in a similar fashion, but that’s not something that you can “impose” on a command that doesn’t support it. In short, the shell only provides access to an environment variable that a command knows to look for; it doesn’t provide the lookup functionality itself.
Thanks for the suggestions and others who replied.
Running the process many times on different paths is no good because I have to have them all available on the single call because the program needs to see all the files, from multiple different directories, or it won’t run. It is a patcher for a game (Skyrim) but is very finecky. Took me loads of testing to figure out how to pretty much ‘hack’ it to work in this way. It is because I am using through wine whereas it was designed for windows and an expected setup, which of course is different with wine.
It will shut down if it doesn’t see what it expects to see but I figured out I can trick this by copying all files it asks for in to one folder before running it. This works but it uses a lot of disk space to copy loads of mods to one folder. It is a dirty way to make it work but making it see multiple paths would be much cleaner which is why I ask if it can be done. So it must see all files in all working directories as one in that single process or else it will refuse to run complaining of missing files.
Also I would like it to be OS agnostic because even some Windows users, maybe with just different setups to the norm experience such issues and so this could help them too. I thought I might release it if I get it working smoothly for myself so making it work on any OS would be required for that. This isn’t a dealbreaker but just something to keep in mind.
Does the patcher support absolute paths? If it does then pass those all at once. Otherwise you’re stuck with whatever the program you’re calling allows you to do. In that case you can ask the patcher dev upstream for more support.
If you’re stuck having move folders around then you’ll just have to deal with it. I wouldn’t worry too much about performance if your script is only meant to run once or rarely.
If you’re okay with users running your script as admin on Windows then os.symlink would be the simplest and fastest method of organizing these folders.
shutil has more options if you’re willing to compromise on safety for performance. Will be slower than symlinks.
Best case scenario is that your initial solution was overthinking it and your patcher can simply take the absolute paths to all mods at once.
What do you mean does it support them? There is no option to put in paths anywhere in the program. You just run it and it either complains that it has missing files and refuses to run or it works.
I have already contacted the dev of it and he has no interest in supporting none windows users despite numerous none linux users on the mod having issues. He just said “skyrim doesn’t support none windows OSes so neither do we” so we are on our own, hence why I am making a script for it.
In the mod manager though there is an arguments path so not sure if that wouild worK? the mod manager is different from the patcher but you can run the patcher from the mod manager and put in arguments there for running the .bat patcher. I looked into that before starting writing this script but couldnt find any windows args that worked - that will be windows because the mod manager runs through wine.
As above it can’t take any paths, as there is nowhere to put them in and also as above symlinks to files didn’t work when I tried, the patcher didn’t see them. So what would the shutil option be. I don’t see that security is a problem since this is a script I am making myself so know I am secure. :)?
Ye I am not bothered about performance, it works and fast, I just thought there might be a cleaner way to do things than the moving files to and froe.
You never actually told us what the tool is, so this whole discussion has been an XY problem.
If other tools can already call this tool with “multiple paths” then your script can as well, but you’ll need to learn or ask how this is communicated between those programs.
Don’t know what you mean by ‘learn to ask how this is communicated’? ask who to communicate what?
I didn’t mention specifically the mod because I don’t see the point? the question is about python tools not the mod itself. The patcher is part of a mod and unless you play the game and have used the mod itself it is not useful to add that because you would have to download and run it but you can’t run it unless you have the game installed, which is the crux of the problem since it needs all the files to run. If you happen to program in mutagen language then it is open source and you could inspect the code I guess but thinking that is a long shot? hence why I simply didn’t mention it.
If you still want to know it is Skyrim Requiem on nexusmods and you can see in the files once downloaded the recitifcator, which you must make an account to download it, or it is on github which would be easier to inspect: GitHub - ProbablyManuel/requiem · GitHub
Because you are using subprocess in your example, the Command Line Interface (CLI) of the program you intend to call becomes important. This may be documented on the program itself with a --help flag, or you can inspect the source code or external documentation of the program, or ask the program’s author as a last resort.
shutil.move operates fast, but only when the folders are on the same disk, at the same time it risks folders being misplaced if your script is interrupted. Whatever you’re doing now should work better.