I have never run python from a terminal before so I don’t know what is involved, I have always run my project from PyCharm. My project runs fine from PyCharm. I am running on Linux. My file structure is as follows:
BackTesting - This is my project
BackTestPkg - One of my packages
__init__py
Parameters.py
...
MAX1 - My main development
MAX1_Controller.py
MAX1_Main.py
...
TradingPkg - My other package
__init__.py
PlatformLib.py
...
MAX1_Controller.py calls MAX1_Main.py, and both call modules in both BackTestPkg and TradingPkg packages, which in turn call other modules in these 2 packages.
When I perform the following at the linux terminal:
Traceback (most recent call last):
File "/data/stuart/Projects/Python/BackTesting/MAX1/MAX1_Controller.py", line 10, in <module>
import MAX1_Main as main
File "/data/stuart/Projects/Python/BackTesting/MAX1/MAX1_Main.py", line 8, in <module>
import TradingPkg.PlatformLib as pl
ModuleNotFoundError: No module named 'TradingPkg'
I have never worked with python outside PyCharm so I don’t know what I can try. Other posts I looked at looked complicated and overkill for what I am sure is a simple solution. Can anybody point me in the right direction please.
cd /data/stuart/Projects/Python/BackTesting
/data/stuart/Projects/Python/Env/bin/python MAX1/MAX1_Controller.py 'EpicConfigs.csv' 'MAX1.param'
If so, the packages are not properly installed. The reason it works in pycharm is that your working directory is set to /data/stuart/Projects/Python/BackTesting, where Python’s import machinery can find the packages despite them not being installed.
Thanks for the suggestion, but it didn’t seem to make any difference. I tried running my CLI prompts from the same folder where the 2 packages are located but that made no difference either.
Do I need to put anything in the PYTHONPATH environment variable? At the moment it is empty. Any other suggestions?
what you can do is move the modules BackTestPkg and TradingPkg sub-packages under the MAX1 main development directory (where the program/script begins - the root folder).
I believe that this should work.
BackTesting - This is my project
...
MAX1 - My main development
MAX1_Controller.py
MAX1_Main.py
BackTestPkg - One of my packages
__init__py
Parameters.py
...
TradingPkg - My other package
__init__.py
PlatformLib.py
if you add the following script to the MAX1_Controller.py module:
import sys
for index, path in enumerate(sys.path):
print(f'{index}. {path}')
you will notice that the very first path that is listed will be the MAX1 directory. Thus, from wherever you run the start module from, that path will will added to the PYTHONPATH at run-time (in this case, MAX1). The way that you had it before or currently, both the BackTesting and TradingPkg packages would not fall under this path (where your program looks when importing modules).
So can I/should I add the paths to my 2 packages to sys.path? Is this normal? What is the best way to do that? Is there something else I should be doing? I am a newbie when it comes to python from a CLI.
you don’t need to add a path. I was merely pointing out that Python implicitly adds the root directory to the PYTHONPATH at run time. What you need to do is move the two folders as I described in my prior post.
Move TradingPkg and BackTestPkg packages under the MAX directory. As you currently have them, they are adjacent to MAX1.
Yes, this does work. However… the previous structure was logically correct because the 2 packages are (or are likely to be) used by other ‘sub-projects’ such as MAX2 or MAX3 (for example). The other possible alternative is to put the contents of MAX1 in the root of my project, but then things start to get messy (possible really messy as time goes by). At this stage it is nice to know my code does run from a CLI, but I really would like to keep the original structure as much as possible.
Is there some way to make these 2 packages known to python, by editing PYTHONPATH, or modifying sys.path, or some other method (see my previous response to you)?
Thanks for the response. Unfortunately no it doesn’t work. Logically I would have to put ‘MAX1’ before the 2 file names ‘EpicConfigs.csv’ and ‘MAX1.param’ because that is where they sit. As per the following:
As it is I get exactly the same response either way. I suppose the validation of the 2 supplied file names has not happened before python complains that the modules cannot be found.
Is there any way to make python aware of my 2 packages even though I have not ‘installed’ them? Do I need to install them (I hope not)? Surely there is a simple solution - or am I being naive?
I don’t think that you need to modify the contents of folder MAX1. You can add the path of these folders to MAX2, …, MAXn via the following commands at runtime:
# Assume module "some_module" exists in the "TradingPkg" sub-folder and the function "func_1" exists in the module "some_module".
import sys
sys.path.append(r'C:\folder1\MAX1\TradingPkg') # add system path
from some_module import func_1 # assuming func_1 is a function that exists
Your other question:
Or, if these are libraries (sub-folders) that you think will be used by multiple applications, it is possible to add them to the PYTHONPATH. Create a directory and add these two folders. You can then add them to the system PATH via the following instructions.
just keep in mind that adding the modules at runtime requires adding them for EVERY single application. If you add these modules to the system PATH, as described above, you only have to do it once. Something to ponder.
Paul,
So are you suggesting that I have a folder specifically for my packages, e.g. ‘my_packages’, that contains BackTestPkg and TradingPkg and have PYTHONPATH point to the my_packages folder?
Regards, Stuart
Yes, OK. It didn’t work anyway. That bit seems to have got lost in the ether when I posted that response. Also, what seems to have got lost is that I tried updating PYTHONPATH as follows:
If I did have a folder specifically for my packages (‘my_packages’) would that be within my current project, or in a new project? I ask this beacuse the example you gave:
You can place the new path anywhere you want, but preferably outside of any active project/application so that it is independent, just like all other paths in PYTHONPATH. I just added this arbitrary path because it was in the list of the paths that you provided is all.