Installing Packages in Applications with Python

Looking for advice on how to install package in python embedded in LibreOffice without affecting main python on my win10.

How is a python package for the Libre Office Version installed? pip installed it to my system version of python (V3.9.0) which LO doesn’t access. (Common problem without a real answer.)

APSO python console [LibreOffice]
3.7.10 (default, May  6 2021, 16:23:28) [MSC v.1928 64 bit (AMD64)]
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os
<module 'os' from 'C:\\Program Files\\LibreOffice\\program\\python-core-3.7.10\\lib\\os.py'>
C:\Users\User>pip install --prefix="C:\Program Files\LibreOffice\program\python-core-3.7.10\lib\site-packages" scipy.optimize
ERROR: Could not find a version that satisfies the requirement scipy.optimize (from versions: none)
ERROR: No matching distribution found for scipy.optimize

I understand this is a mismatch of python versions (System is 3.9 and LO is 3.7). LO has no pip installed. What is the way forward?

C:\PROGRAM FILES\LIBREOFFICE\PROGRAM\PYTHON-CORE-3.7.10
├───bin
└───lib
    ├───asyncio
    ├───collections
    ├───concurrent
    │   └───futures
    ├───ctypes
    │   └───macholib
    ├───distutils
    │   └───command
    ├───email
    │   └───mime
    ├───encodings
    ├───html
    ├───http
    ├───importlib
    ├───json
    ├───lib2to3
    │   └───fixes
    ├───logging
    ├───multiprocessing
    │   └───dummy
    ├───pgen2
    ├───pydoc_data
    ├───site-packages
    ├───unittest
    │   └───test
    │       └───testmock
    ├───urllib
    ├───wsgiref
    ├───xml
    │   ├───dom
    │   ├───etree
    │   ├───parsers
    │   └───sax
    └───xmlrpc

There are so many problems with doing this; it’s really just not a great idea.

Since the distributed Python in the LO suite doesn’t have pip, you’d have to do a manual installation of pip into its environment, or just manually install the package you want.

You could download a wheel of pip and then use it like this (no strict need to install pip itself):

path\to\python.exe path\to\pip.whl/pip install Something

Off the top of my head, without testing, some tweaking might be needed.

In a sane Python environment this should work without issue. But no guarantee this would work in your case. It could fail for many reasons. One of them I can think of is that I suspect this Python installation is not full and might lack some of the modules and packages of its standard library.

That this works is borderline accidental. It is not a detail you should rely on unless you know what you’re doing.

4 Likes

The LibreOffice team has fully catered for pip with the following in C:\Program Files\LibreOffice\program\python-core-3.8.10\lib\site-packages\README.txt"

This directory exists so that 3rd party packages can be installed
here. Read the source for site.py for more details.

site.py

"""Append module search paths for third-party packages to sys.path.

****************************************************************
* This module is automatically imported during initialization. *
****************************************************************

This will append site-specific paths to the module search path.  On
Unix (including Mac OSX), it starts with sys.prefix and
sys.exec_prefix (if different) and appends
lib/python<version>/site-packages.
On other platforms (such as Windows), it tries each of the
prefixes directly, as well as with lib/site-packages appended.  The
resulting directories, if they exist, are appended to sys.path, and
also inspected for path configuration files.

If a file named "pyvenv.cfg" exists one directory above sys.executable,
sys.prefix and sys.exec_prefix are set to that directory and
it is also checked for site-packages (sys.base_prefix and
sys.base_exec_prefix will always be the "real" prefixes of the Python
installation). If "pyvenv.cfg" (a bootstrap configuration file) contains
the key "include-system-site-packages" set to anything other than "false"
(case-insensitive), the system-level prefixes will still also be
searched for site-packages; otherwise they won't.

All of the resulting site-specific directories, if they exist, are
appended to sys.path, and also inspected for path configuration
files.

A path configuration file is a file whose name has the form
<package>.pth; its contents are additional directories (one per line)
to be added to sys.path.  Non-existing directories (or
non-directories) are never added to sys.path; no directory is added to
sys.path more than once.  Blank lines and lines beginning with
'#' are skipped. Lines starting with 'import' are executed.

For example, suppose sys.prefix and sys.exec_prefix are set to
/usr/local and there is a directory /usr/local/lib/python2.5/site-packages
with three subdirectories, foo, bar and spam, and two path
configuration files, foo.pth and bar.pth.  Assume foo.pth contains the
following:

  # foo package configuration
  foo
  bar
  bletch

and bar.pth contains:

  # bar package configuration
  bar

Then the following directories are added to sys.path, in this order:

  /usr/local/lib/python2.5/site-packages/bar
  /usr/local/lib/python2.5/site-packages/foo

Note that bletch is omitted because it doesn't exist; bar precedes foo
because bar.pth comes alphabetically before foo.pth; and spam is
omitted because it is not mentioned in either path configuration file.

The readline module is also automatically configured to enable
completion for systems that support it.  This can be overridden in
sitecustomize, usercustomize or PYTHONSTARTUP.  Starting Python in
isolated mode (-I) disables automatic readline configuration.

After these operations, an attempt is made to import a module
named sitecustomize, which can perform arbitrary additional
site-specific customizations.  If this import fails with an
ImportError exception, it is silently ignored.
"""

It’s so good the default LibreOffice python package installer is a pip shell written by one of the community [in Spanish] - elmau/zaz-pip: Extension for install and admin Python Pip in LibreOffice. - zaz-pip - Git para Cuates

So if you used that facility and it installed the package into the wrong location, then it sounds like you need to report that to the LibreOffice community.

Also, one of your messages says LO includes Python 3.7, but the most recent message says 3.8. Does that mean you’ve changed LO versions? If so, have you tried installing the package again with the new version?

Also the package installer screenshot says Python 3.6.9… And it’s on Linux, but your initial command line was on Windows.

Focusing purely on the original command you quoted:

C:\Users\User>pip install --prefix="C:\Program Files\LibreOffice\program\python-core-3.7.10\lib\site-packages" scipy.optimize

You’re trying to install a package called scipy.optimize. But I don’t see that on PyPI. Is it an actual package, or just a component of scipy? (In which case you should be installing scipy).

It sounds like managing the Python stuff embedded in LibreOffice is pretty complex and non-standard, so you may have other problems getting things to work (the --prefix flag to pip isn’t something intended for non-expert users to have to use). You might need to get a better understanding of Python packaging if you’re trying to do this by yourself, or get help from the LibreOffice community if possible (as they will have people familiar with the quirks of their environment).

It works, but you have to install a very recently created LibreOffice extension.

No, that’s not my screenshot, I’m on Win10 and Yes I initially had wrong package name but that wasn’t the issue. Not really any expertise from the LO community on this, not surprising after 20 years python is hardly used but the interest is growing.


Don’t assume LibreOffice developers know anything about python, it’s the user domain and since the 1980’s the native language is basic. The biggest issue is python is not in the path. It would be really handy for someone that knows their way around python took a look. Maybe it should be in a venv??

Way forward:

  1. That extension is annoying but with a bit of a cleanup it works. Any thoughts on missing dll and cleanup. What is f2py.exe?
  2. Advice on something better for the README.txt would be useful.

I’m not, but equally don’t assume that the people in the Python packaging community have any knowledge of LibreOffice. Generally, as soon as a Python installation stops looking like a “standard” installation (in this case, you don’t have pip installed in the LibreOffice installation of Python, and you therefore need to use --prefix to do the install) you’re likely to be on your own, sorry.

Picking up on some specific points you asked:

No idea, that’s entirely down to the LibreOffice folks. They could do any one of a number of things, and there’s no way of knowing without looking at a LibreOffice installation.

If you’re not using a version of pip installed into LibreOffice, that’s expected.

Basically you’re in unsupported territory at this point. The supported answer is “install pip into the LO Python environment and use that”. You can also use pip install --prefix or pip install --target, but those are highly dependent on the target Python’s expectations, and you’re back to “it depends what the LO install expects”.

That’s very far from “fully catered for pip”. To fully cater, they would need to include pip in their Python site-packages and provide a way for you to invoke it, like a standard Python install does. Can you invoke the LO python from the command line, even? If so, then python -m pip should give you a running pip, unless the LO python doesn’t include pip.

Can you not use that to install Python packages then? If it’s a wrapper round pip, it seems possible that it would be usable that way.

Sorry, that’s about all I can offer. You’ll need to do some more investigation and digging on your own, I think. Maybe if you can come up with some more specific questions that can be explained without relying on people knowing about LO, you’ll be able to get additional answers.

No, that’s the default README from Python itself, that may or may not apply to LibreOffice’s installation: cpython/Lib/site-packages at main · python/cpython · GitHub

The site module docstring you pasted also seems like the upstream one.

I’m confident there is no pip in LibreOffice but it’s certainly there after running that extension and installing python V3.9. Anyway, this covers everything except downloading pip.

C:\Program Files\LibreOffice\program>dir python.exe /s

 Directory of C:\Program Files\LibreOffice\program
11/08/2021  11:04 AM            31,280 python.exe

 Directory of C:\Program Files\LibreOffice\program\python-core-3.8.10\bin
11/08/2021  11:05 AM           103,472 python.exe


C:\Program Files\LibreOffice\program>python
Python 3.8.10 (default, Aug 10 2021, 19:39:20) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> ^Z


C:\Program Files\LibreOffice\program>python -m pip

Usage:
  C:\Program Files\LibreOffice\program\python-core-3.8.10\bin\python.exe -m pip <command> [options]
...

C:\Program Files\LibreOffice\program>python -m pip install fire
Defaulting to user installation because normal site-packages is not writeable
Collecting fire
  Downloading fire-0.4.0.tar.gz (87 kB)
     |████████████████████████████████| 87 kB 633 kB/s
Collecting six
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting termcolor
  Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Building wheels for collected packages: fire, termcolor
  Building wheel for fire (setup.py) ... done
  Created wheel for fire: filename=fire-0.4.0-py2.py3-none-any.whl size=115943 sha256=7ba8877ce397e185e7d1c9a74744d4371418ac62c0d44d77497372d454b153f9
  Stored in directory: c:\users\user\appdata\local\pip\cache\wheels\1f\10\06\2a990ee4d73a8479fe2922445e8a876d38cfbfed052284c6a1
  Building wheel for termcolor (setup.py) ... done
  Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4847 sha256=9a1fa612ae2fd7d2d96df6f9e2462491ddac6d0497930c5731df49a5c5736af5
  Stored in directory: c:\users\user\appdata\local\pip\cache\wheels\a0\16\9c\5473df82468f958445479c59e784896fa24f4a5fc024b0f501
Successfully built fire termcolor
Installing collected packages: termcolor, six, fire
Successfully installed fire-0.4.0 six-1.16.0 termcolor-1.1.0

C:\Users\User\AppData\Roaming\Python\Python38\site-packages>dir f*
 Volume in drive C is OS

 Directory of C:\Users\User\AppData\Roaming\Python\Python38\site-packages

16/08/2021  11:29 PM    <DIR>          fire
16/08/2021  11:29 PM    <DIR>          fire-0.4.0.dist-info

Firstly, I use How update Libre Office Python on windows? - Stack Overflow to install pip.

The LO people are of the view they just take the package from Python and include it. It seems to me 1. Embedding Python in Another Application — Python 3.12.1 documentation should include https://bootstrap.pypa.io/get-pip.py by default with all the caveats. I’d like to pass that message to the python dev team.


Now when pip isn’t enough in LibreOffice, there’s trickery.

I’d argue that it’s on the app developer to make that decision. Virtually every time I’ve embedded Python, and most of the times I’ve worked with colleagues who were doing it, we haven’t wanted to allow people to modify the app. That’s not been the point.

If you want to embed Python and allow users to mod your app, by all means allow them to use pip. But I don’t see any reason to recommend that by default (which is going to be interpreted by some readers as a “requirement,” no matter how many caveats we apply).

“How to embed Python and let users use and modify it” might be a good new page in the docs, but it’s also a perfectly good opportunity for someone to publish outside of the core docs, which lets them be more opinionated and also receive more of the credit for their work. If a lot of people are writing this up and giving contradictory advice, or are making the Python ecosystem worse as a result, I’d be more inclined to write a canonical guide in the docs. But most likely, they’ll make things better for those who want to do something like this.