Python for IA beginners : local, VENV & Embeded : Frustrations with Python and Dependencies - Need Help Understanding Embedded Environments

Dear Community,

I’m new to working with AI, and as expected, I’m diving into Python. Honestly, I’m finding Python incredibly challenging to understand, especially when it comes to making everything work together.

One of the projects I’m working on, ComfyUI, requires a bunch of different libraries and components. But here’s the catch: many of these libraries need different versions of Python! So, when I try to get everything working, I end up facing what feels like an insurmountable mess, and it’s honestly exhausting.

I’m working on Windows, which isn’t terrible compared to Linux or Python’s command line, but the experience is still frustrating.

Today, I came across a promising project on GitHub called Docling, which is designed to process documents for LLM (Large Language Models). I thought, “Great, this could be exactly what I need!” I even found that LM Studio makes it easy to run LLMs on Windows. But of course, when I tried to install it, things went sideways.

I decided to go with an embedded Python installation to keep things local. After hours of struggling and Googling, I just couldn’t make it work. Here’s a rough outline of my journey:

  1. Downloaded the zip file: I extracted it, but why was there no README file in the zip? I wasn’t sure what to do with the python3.x.x.zip file. After unzipping, should I unzip it again? And where should I place everything?

  2. Pip issues: Once I unzipped everything, I tried running pip install docling—but it failed because pip wasn’t recognized! I then followed instructions to manually install pip via get-pip.py. This seemed to work, but when I tried to run pip install docling, it still didn’t recognize the command.

  3. Path and environment issues: I spent hours trying to figure out why pip wasn’t working, including editing the pythonxx._pth file and creating a sitecustomize.py file (as suggested by various online guides). Finally, I got pip to work by navigating directly to the Scripts folder, and I was able to install Docling—sort of.

  4. Installation problems: After installing, the library was only in the Scripts folder and not in the expected location. When I tried uninstalling, the cleanup didn’t remove all the files, so I had to manually delete them.

  5. Running Docling: I followed the instructions to run Docling but ended up with an error: ModuleNotFoundError: No module named 'docling'. The installation didn’t work as expected.

This process was confusing and frustrating, and I feel like I’m constantly battling with Python’s dependency management, especially with embedded installations. I often face these kinds of issues when installing or updating libraries for ComfyUI as well. The constant struggle with ensuring the right Python versions are installed for each library, handling virtual environments, and managing dependencies has become a nightmare.

I’m not an IT expert, and I don’t have a deep understanding of embedded Python environments, venvs, or all the tools needed to manage them. Every time I try to install or update a package, I end up with errors and conflicts that leave me feeling stuck.

My request: Is there a simpler way to deal with these issues? I’m not looking to learn Python itself (I just want it to work), but I’d love some guidance on understanding embedded environments, virtual environments (venvs), and dependency management in Python. If anyone has beginner-friendly tutorials, tips, or resources that could help, I would really appreciate it.

Also, I can’t help but wonder—why isn’t there a simple installer for these types of projects? Something that handles dependencies and paths automatically would save a lot of us from this kind of frustration. Not everyone is an IT expert—some of us just want to use the software, not get bogged down with technical details.

Thanks in advance for any advice or resources you can share!

There is a simpler way to deal with these issues, particularly for machine learning.

Install anaconda. https://www.anaconda.com/

Use conda install for everything you can.

You’ll need to read the anaconda docs, but I found them pretty straight forward.

There are things you can’t conda install, but those are generally beyond what you should deal with as a beginner.
If you’re not familiar with python, you definitely shouldn’t try to combine libraries that require different versions of python. But that only happens when you need a deprecated python library and a very new one at the same time, which can usually be avoided.

Conda is a dependency management tool (among other things), usually you can feed it your list of requirements and it will find a way to install all of them with compatible versions.

2 Likes

The embedded Python is meant for embedding Python into an application. For using Python to write your own code in it, just use the regular installer.

None of the problems you mentioned happen with the normal Python from the normal installer.

2 Likes