How to install opencve in Debian 12

Python Help
1

I am running a freshly installed Debian 12 VM with Python version 3.11.2, and pip3 version 23.0.1. When I try to install opencve I get the following:

# pip3 install opencve
error: externally-managed-environment

   This environment is externally managed
   To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

So I tried with --break-system-packages - after all, this is just a VM:

# pip3 install opencve --break-system-packages
Collecting opencve
  Using cached opencve-1.4.1-py3-none-any.whl (1.4 MB)
Collecting Flask==1.0.2
  Using cached Flask-1.0.2-py2.py3-none-any.whl (91 kB)
Collecting Flask-Admin==1.5.5
  Using cached Flask_Admin-1.5.5-py3-none-any.whl (1.9 MB)
Collecting Flask-Assets==0.12
  Using cached Flask-Assets-0.12.tar.gz (22 kB)
  Preparing metadata (setup.py) ... done
Collecting Flask-DebugToolbar==0.10.1
  Using cached Flask_DebugToolbar-0.10.1-py2.py3-none-any.whl (326 kB)
Collecting Flask-SQLAlchemy==2.3.2
  Using cached Flask_SQLAlchemy-2.3.2-py2.py3-none-any.whl (16 kB)
Collecting Flask-Migrate==3.0.1
  Using cached Flask_Migrate-3.0.1-py2.py3-none-any.whl (12 kB)
Collecting Flask-Login==0.4.1
  Using cached Flask-Login-0.4.1.tar.gz (14 kB)
  Preparing metadata (setup.py) ... done
Collecting Flask-WTF==0.14.3
  Using cached Flask_WTF-0.14.3-py2.py3-none-any.whl (13 kB)
Collecting Flask-User==1.0.2.2
  Using cached Flask-User-1.0.2.2.tar.gz (74 kB)
  Preparing metadata (setup.py) ... done
Collecting Flask-Gravatar==0.5.0
  Using cached Flask_Gravatar-0.5.0-py2.py3-none-any.whl (7.5 kB)
Collecting Flask-Paginate==0.5.2
  Using cached flask-paginate-0.5.2.tar.gz (5.5 kB)
  Preparing metadata (setup.py) ... done
Collecting Flask-RESTful==0.3.8
  Using cached Flask_RESTful-0.3.8-py2.py3-none-any.whl (25 kB)
Collecting Flask-Limiter==1.4
  Using cached Flask_Limiter-1.4-py3-none-any.whl (15 kB)
Collecting SQLAlchemy==1.3.1
  Using cached SQLAlchemy-1.3.1.tar.gz (5.9 MB)
  Preparing metadata (setup.py) ... done
Collecting SQLAlchemy-Utils==0.33.11
  Using cached SQLAlchemy-Utils-0.33.11.tar.gz (128 kB)
  Preparing metadata (setup.py) ... done
Collecting psycopg2-binary==2.8.4
  Using cached psycopg2-binary-2.8.4.tar.gz (378 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  python setup.py egg_info did not run successfully.
  exit code: 1
   [25 lines of output]
      /usr/lib/python3/dist-packages/setuptools/config/setupcfg.py:515: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
        warnings.warn(msg, warning_class)
      running egg_info
      creating /tmp/pip-pip-egg-info-r7rjid_r/psycopg2_binary.egg-info
      writing /tmp/pip-pip-egg-info-r7rjid_r/psycopg2_binary.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-pip-egg-info-r7rjid_r/psycopg2_binary.egg-info/dependency_links.txt
      writing top-level names to /tmp/pip-pip-egg-info-r7rjid_r/psycopg2_binary.egg-info/top_level.txt
      writing manifest file '/tmp/pip-pip-egg-info-r7rjid_r/psycopg2_binary.egg-info/SOURCES.txt'
      
      Error: pg_config executable not found.
      
      pg_config is required to build psycopg2 from source.  Please add the directory
      containing pg_config to the $PATH or specify the full executable path with the
      option:
      
          python setup.py build_ext --pg-config /path/to/pg_config build ...
      
      or with the pg_config option in 'setup.cfg'.
      
      If you prefer to avoid building psycopg2 from source, please install the PyPI
      'psycopg2-binary' package instead.
      
      For further information please check the 'doc/src/install.rst' file (also at
      <http://initd.org/psycopg/docs/install.html>).
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

Encountered error while generating package metadata.
See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

So I made sure that the Debian 12 python3-psycopg2 package got installed, tried again, and ended up with the same error as above. Plus pg_config is nowhere to be found in the system. Somebody online suggested that I should install it with pip3 install psycopg2-binary. I tried that, and had to add the --break-system-packages to proceed. That seem to work fine (with a warning against doing things as root, however) and then tried the opencve installation command again.

Same problem. And pg_config still nowhere to be seen. At this point I clearly do not know what the heck I am doing, and my VM probably is utterly messed up by now, when it comes to the Python framework.

Anybody out there with Python savvy can provide idiot-proof steps to install opencve under Debian 12?

2

You are being forced to use a venv.
Create a venv and the install opencve into that venv.

For example try this (untested):

python3 -m venv create myvenv
myvenv/bin/python3 -m pip install opencve
1 Like
3

Thanks. Unfortunately, the psycopg2 problem remains, exactly as before.

4

Try installing psycopg2 from debian using sudo apt rather then from PyPI.
At a guess (not near a debian system to try) it will be sudo apt install python3-psycopg2 should install it.

5

I did. No change :frowning: Still no pg_config executable to be found anywhere.

6

On Debian, you can use the apt-file utility to search for packages
that provide a specific filename and path by substring:

fungi@dhole:~$ apt-file search bin/pg_config
libpq-dev: /usr/bin/pg_config
postgresql-client-15: /usr/lib/postgresql/15/bin/pg_config
postgresql-client-16: /usr/lib/postgresql/16/bin/pg_config
postgresql-common: /usr/bin/pg_config

Note that the results will vary depending on the version you’re
running. For example I’m using Debian’s “sid” (unstable) version, so
if you’re on “bookworm” (stable) or “bullseye” (oldstable) then what
apt-file reports for your available packages may differ.

7

That did help a lot, but ultimately the whole thing failed when trying to compile something to do with psycopg2 during the build process:

     x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC "-DPSYCOPG_VERSION=2.8.4 (dt dec pq3 ext lo64)" -DPG_VERSION_NUM=150003 -DHAVE_LO64=1 -I/home/jca/OpenCVE_Env/include -I/usr/include/python3.11 -I. -I/usr/include/postgresql -I/usr/include/postgresql/15/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-cpython-311/psycopg/psycopgmodule.o -Wdeclaration-after-statement
      psycopg/psycopgmodule.c: In function ďż˝:
      psycopg/psycopgmodule.c:920:36: error: lvalue required as left operand of assignment
        920 |         Py_TYPE(typetable[i].type) = &PyType_Type;
            |                                    ^
      psycopg/psycopgmodule.c: In function ďż˝:
      psycopg/psycopgmodule.c:952:30: error: lvalue required as left operand of assignment
        952 |     Py_TYPE(&pydatetimeType) = &PyType_Type;
            |                              ^
      psycopg/psycopgmodule.c: In function ďż˝:
      psycopg/psycopgmodule.c:1084:28: error: lvalue required as left operand of assignment
       1084 |     Py_TYPE(&typecastType) = &PyType_Type;
            |                            ^
      psycopg/psycopgmodule.c:1087:25: error: lvalue required as left operand of assignment
       1087 |     Py_TYPE(&chunkType) = &PyType_Type;
            |                         ^
      psycopg/psycopgmodule.c:1090:25: error: lvalue required as left operand of assignment
       1090 |     Py_TYPE(&errorType) = &PyType_Type;
            |                         ^
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  Rolling back uninstall of psycopg2-binary
  Moving to /home/jca/OpenCVE_Env/lib/python3.11/site-packages/psycopg2/
   from /home/jca/OpenCVE_Env/lib/python3.11/site-packages/~sycopg2
  Moving to /home/jca/OpenCVE_Env/lib/python3.11/site-packages/psycopg2_binary-2.9.7.dist-info/
   from /home/jca/OpenCVE_Env/lib/python3.11/site-packages/~sycopg2_binary-2.9.7.dist-info
  Moving to /home/jca/OpenCVE_Env/lib/python3.11/site-packages/psycopg2_binary.libs/
   from /home/jca/OpenCVE_Env/lib/python3.11/site-packages/~sycopg2_binary.libs
error: legacy-install-failure

ďż˝ Encountered error while trying to install package.
��> psycopg2-binary

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

I give up - I’ll just use the web interface that is provided under opencve.io - I am sure there is a way to pull this off, but the OpenCVE people do not seem to be particularly interested to share it: according to them, just doing pip install opencve should work. Heck, it is not even clear what it is one that one should do about the blasted psycopg2 package: whether or not to install it before hand, and if the former, how.

8

pip is compiling because you have not installed the debian package that is already compiled i assume?

Did apt install python-psycopg2 fail to install?

9

I was able to do apt install python3-psycopg2 without any problems. It makes no difference.

10

But you did not install pg config using the package found by searching as @fungi show you how to do.

There are package system skills that when you are comfortable make solving this type of problem yourself easier. like the use if apt-file to find packages that contain stuff you want.
I don’t remember if apt-file is setup by default, i use debian rarely myself.

11

I did install postgresql and postgresql-contrib, and that eliminated the pg_config problem. The psycopg2 problem is totally unrelated, as far as I can tell.

12

Hi, I followed everything I found (the venv, install python3-psycopg2…), but after also installing postgresql and postgresql-contrib, the problem with the build process of psycopg2 still arises when I try to install opencve in the venv.
I have uninstalled and reinstalled psycopg2 also.
Could you please write down the steps you followed or that you would follow from stage 0? Thanks!