I’m new to Python. Config: Python 3.12, Pycharm 2023.3.2, on Windows 10 Pro.
I know very little about modules as I just learned about them from a tutorial. I made a small custom module. I put it in c:\users<username>\Pycharm Projects. The module is just named “util.py”. But when I open a new blank .py file and type “import” a list of modules pop up, and mine is not in there, so it’s not possible to enter “import util”.
What do I need to set up to import my own util.py file?
Thank you!
EDIT: I created util.py manually in that directory using a text editor. Perhaps I created it all wrong and should go through Pycharm to do that. But I don’t know how to create a module through Pycharm.
Edit 2: This util.py file will be used by many projects later.
You should specify where you’re trying to import util.py from. Is it from the same project, or another one ? From a file at the same hierarchical level ?
Also, note that not because PyCharm doesn’t propose a module as a suggestion doesn’t mean it cannot be imported when running the code. A good example would be if you modify sys.path at runtime.
This util.py file will be used by many projects later
If you want to use this file in many projects, your best option is to make a package out of it. This way, you’ll be able to install it in the Python environment associated with each of your projects, and PyCharm (or any other IDE) will be able to provide support for it.
For guidelines on how to package a Python project, you can refer to this very nice resource:
Note that packaging your util.py file will require you to write at least a second file (pyproject.toml), I don’t think you can currently distribute a single Python file as a module. If I understood correctly, this will change once PEP 723 (PEP 723 – Inline script metadata | peps.python.org) gets implemented,
Do I specify the file using a whole path like this? import "c:\users\<username>\PyCharm Projects\util"? I still may be thinking in terms of Perl which I’m used to using. Double quotes would have to be used since there’s a space in the path.
Util.py is not in a project, it’s a plain text library file by itself which is how I did things in Perl.
I haven’t learned about packages yet in my tutorial. Maybe I should wait until the tutorial covers those.
Maybe I’m not understanding how Python projects work. Do I have to copy the util.py file to every project I use? Or install it to every project I make? That would really negate the use of a library file at all. The purpose of a library file is to include code that is reused a lot and to have only one single copy of the library. Thus when I fix an error in a library function it is fixed in all projects that use it.
Oh sorry, I assumed you were more familiar with the import and package system and asking more specifically about PyCharm.
Do I specify the file using a whole path like this? import "c:\users\<username>\PyCharm Projects\util"?
One option for you would indeed be to tell Python where to look for the util.py file. This can be done by adding the path to your file in the PYTHONPATH, accessible in sys.path. In practice, your script would contain something like:
Do I have to copy the util.py file to every project I use?
That’s an other temporary option while you’re still learning. It has the advantage that your IDE will be able to provide support for this file (likely not the case if you modify the PYTHONPATH). Lots of drawbacks though, as you mentioned.
Or install it to every project I make?
It’s actually a good practice to isolate your different projects in separate Python environments (virtual environments for example), to avoid dependency conflicts. But you can also install libraries system- or user-wide, they are then accessible to all the projects that do not rely on a specific environment. I shamelessly use my system environment for very small projects and tests, but always switch to a virtual environment when getting serious
Thank you for the more detailed answer. I will not have any dependencies that I create with my custom library, I will only have the util.py library. There may be other dependencies which I use with standard and added libraries, as in my real programs that I will write later, I will be reading and writing Excel, CSV, and tab-delimited files.
My tutorial uses the Pycharm IDE, but my actual programs will use the Microsoft Azure environment, which I will have to learn later. In Azure I will only specify using one Python version to simplify things. I am unlikely to use other python environments in our Azure account.
When I get to the point of designing a program set in Azure I will design all the programs to use one version of Python. If MS forces me to upgrade Python I will go through an upgrade and testing process and test all the programs I wrote, which takes time, but it will, in the long term, simplify things.
No; Python’s import statement uses symbolic names, not file paths. (Also, Python does not have anything like Perl’s barewords; if something is a string, it needs to be in quotes, even if there is no whitespace.) The complete system is very complex, and it does offer many tools that can be used to import a module “dynamically” (i.e., given a string representing the path to the file). But none of the ways to use these tools are especially straightforward - plain import is certainly simpler.
Generally, Python will use a special value sys.path to look up file paths and translate the symbolic name into a path on disk. (As it does so, it needs to consider many other special cases: built-in modules that are implemented by the interpreter itself rather than by a source file; the bytecode-caching system that uses .pyc files generated from the .py source; etc. etc.) These are the rules for how sys.path gets initialized when you start Python.
I can’t comment on what Pycharm (or any other IDE) does to look up modules and pop up suggestions, as I don’t use such tools. That said, you can always just type whatever you want into the file, even if it doesn’t pop up as a suggestion. The Python interpreter is the ultimate arbiter of what’s valid and how it works - not your IDE. Some IDEs might make it more or less difficult to bypass the list of suggestions, but a .py file is ultimately just a file. (Which is exactly why you were able to create util.py in the first place without using Pycharm.)
Python doesn’t formalize the “project” concept - as far as I’m aware, no programming language does. A “project” is either an abstraction created by your IDE (for example, by leaving behind IDE-specific configuration files in a top-level folder) or that you create yourself (by arranging your files on disk within a containing folder.
When you “install” something in Python, you install it to a Python environment. This can mean the root Python installation folder itself (something like C:\Program Files\PythonXY\site-packages), or a user-specific folder (probably something like C:\Users\<username>\AppData\PythonXY\site-packages). Do not attempt to do this manually. Installation is performed by creating a distributable package from your code (ideally, as a “wheel”), and then using the provided Pip tool to install it.
After the code is installed, it is available to all Python code that is executed in the same environment.
By following these steps, you get the advantage of being able to share your library automatically: putting it on PyPI (covered by the same tutorial above) means that anyone can install it locally using the same Pip tool. Congratulations; you’re now an open-source developer
If you want to fix a problem in the “library”, the idea is to re-package and re-install. Generally, Python will only support installing one version of the same package at a time; re-installing will update the code.
However, if you just want to hack something together, then there’s nothing that prevents you from just putting all your source code for all the programs you want to write into the same “project”. It’s just that eventually you will suffer from the lack of organization, and it makes sharing the code harder as well.
The key idea behind “installing a package” is that the code will, ultimately, get copied to a library location that’s always put in sys.path, so import can find it. But as seen in the link above, typicallysys.path will include (and give priority to) a location that depends on how you start up Python. So if you want import util to work, you just need to make sure that util.py is in that location. If you have several “projects”, then you would have to copy the file around, true. But if foo.py and bar.py both need to use util.py, then nothing prevents just putting them all in the same folder - even if foo.py and bar.py don’t have anything to do with each other.
(It’s also important to keep in mind that it’s easy to hide other modules accidentally this way. For example, if you instead wanted to call your “utility” library, say, math.py, that would cause problems.)
I’m speculating, of course, but this seems very unlikely. Windows doesn’t come with anything that depends on Python (which is why you had to install it first), so Microsoft has no motivation to force an upgrade. However, the Python development team does regularly drop support for older versions, which is a good reason to update voluntarily. For reference:
I’m afraid I can’t comment meaningfully on Azure, as I don’t even really understand what it is.
Sorry I wasn’t clear. I wasn’t talking about Windows the OS, I was talking about the online web app environment MS Azure. MS Azure is for cloud computing and web apps, it supports many tools like Django and Python, and the MS Azure ecosystem would likely upgrade it’s own Python that it supports, which would require me to test all programs that use Python in the Azure ecosystem. Azure is constantly upgrading so fast, that a tutorial will often become obsolete in just 6-12 months, simply because the screen change monthly. For a newbie to Azure this presents more challenges which complicate the learning process. Screens are not the only thing that change in Azure though, thus making a tutorial obsolete.
Thank you for the great answer! I know programming, but I’m new to Python. I’m part way through a Python tutorial on Udemy.
Oh. Well, if the goal is to learn Python, it’s easy enough to set it up on your own computer first. You should never feel like you have to rely on someone else’s service in order to pursue a craft or hobby.
