Creating egg-info failing in Docker Container

Hello everyone,

I’ve been tearing my hair out for a few hours over this and I don’t seem to be getting anywhere. I’ve got a Python package with a file which imports this carries out some useful work as per the design of the package. I don’t need the setup.py file right now as I’m not packaging this but I would like to get this working for when the codebase hopefully expands in the future, all I need from setup.py at the moment is for the dependencies to be installed!

My setup.py file is as below…

import setuptools

with open("README.md", "r", encoding="utf-8") as file:
    long_description = file.read()

setuptools.setup(
    name="appName",
    version="0.0.1",
    author="myName",
    author_email="myEmail",
    description="appDescription",
    long_description="test",
    #long_description=long_description,
    long_description_content_type="text/markdown",
    url="myUrl",
    packages=setuptools.find_packages(),
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    python_requires='>=3.8',
    install_requires=[
        'BAC0 @ git+https://github.com/ChristianTremblay/BAC0.git@develop',
        'bacpypes>=0.18.3',     
        'requests>=2.25.1',
        'pytest>=6.2.4',
        'pytz>=2021.1',
        'PyYAML>=5.3.1'       
    ]
)

One of the commands in my dockerfile is…

RUN python setup.py install

…but I always get an error when it’s running through the setup.py/setuptools process, which is as below…

Step 5/6 : RUN python setup.py install
 ---> Running in 27e35f2300ab
running install
running bdist_egg
running egg_info
creating bmsapiconn.egg-info
error: [Errno 1] Operation not permitted
The command '/bin/sh -c python setup.py install' returned a non-zero code: 1

I’ve tried various different versions of Docker, Python and even implementing a venv in the container but I just cannot get this to proceed through the setuptools process. The python setup.py install works fine on my machine but it just will not work in the container.
I had convinced myself that this was working a few weeks ago but now I don’t even know which way is up with getting this working!

Can anyone point anything obvious out or ask any relevant questions which might help?

Thanks!

Is the “python setup.py install” command running as a non-root user? (That shouldn’t make a difference if you were using a virtualenv, but…) Can you paste your entire Dockerfile?

Also, note that “setup.py install” is deprecated to some extent; the modern equivalent is “pip install .”. Does using that command instead work?

John,

Thanks for getting back to me so quickly!

I’m not setting any users in the dockerfile, am I correct in thinking that means everything will run as root?

This is my dockerfile…

FROM python:3.9-slim
COPY ./ /bmsapiconn/

WORKDIR /bmsapiconn

RUN echo 'CONFIG_DIR="bmsapiconn/config"\n\
INITIALISED="${CONFIG_DIR}/.initialised"\n\
\n\
if [ ! -f ${INITIALISED} ]; then\n\
    cp -r $CONFIG_DIR/default/* $CONFIG_DIR/\n\
    touch ${INITIALISED}\n\
fi\n\
python ./bmsapiconn/BMSAPIConnector.py\n\
'\
>> start_connector.sh && chmod +x start_connector.sh
RUN python setup.py install
CMD "./start_connector.sh"
#CMD [ "bin/sh", "ls -la" ]

If I change the RUN python setup.py install line to RUN pip install . then I get the below…

Step 5/6 : RUN pip install .
 ---> Running in be1c513ed972
Processing /bmsapiconn
  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
ERROR: Could not install packages due to an OSError: [('/bmsapiconn/README.md', '/tmp/pip-req-build-pd5pclfm/README.md', '[Errno 1] Operation not permitted'), ('/bmsapiconn/setup.py', '/tmp/pip-req-build-pd5pclfm/setup.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/.dockerignore', '/tmp/pip-req-build-pd5pclfm/.dockerignore', '[Errno 1] Operation not permitted'), ('/bmsapiconn/docker/dockerfile', '/tmp/pip-req-build-pd5pclfm/docker/dockerfile', '[Errno 1] Operation not permitted'), ('/bmsapiconn/docker', '/tmp/pip-req-build-pd5pclfm/docker', '[Errno 1] Operation not permitted'), ('/bmsapiconn/LICENSE', '/tmp/pip-req-build-pd5pclfm/LICENSE', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/BMSAPIConnector.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/BMSAPIConnector.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/config/default/config.yaml', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/config/default/config.yaml', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/config/default', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/config/default', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/config', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/config', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/connector/connector_service.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/connector/connector_service.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/connector/__init__.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/connector/__init__.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/connector', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/connector', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/bms/protocols/bacnet.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/bms/protocols/bacnet.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/bms/protocols/__init__.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/bms/protocols/__init__.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/bms/protocols', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/bms/protocols', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/bms', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/bms', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/api/endpoints/open_weather_map.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/api/endpoints/open_weather_map.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/api/endpoints/carbon_intensity.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/api/endpoints/carbon_intensity.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/api/endpoints/__init__.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/api/endpoints/__init__.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/api/endpoints/octopus_energy.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/api/endpoints/octopus_energy.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/api/endpoints', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/api/endpoints', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/api/common/common.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/api/common/common.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/api/common/__init__.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/api/common/__init__.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/api/common', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/api/common', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/api/__init__.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/api/__init__.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/api', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/api', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn/__init__.py', '/tmp/pip-req-build-pd5pclfm/bmsapiconn/__init__.py', '[Errno 1] Operation not permitted'), ('/bmsapiconn/bmsapiconn', '/tmp/pip-req-build-pd5pclfm/bmsapiconn', '[Errno 1] Operation not permitted'), ('/bmsapiconn/start_connector.sh', '/tmp/pip-req-build-pd5pclfm/start_connector.sh', '[Errno 1] Operation not permitted'), ('/bmsapiconn', '/tmp/pip-req-build-pd5pclfm', '[Errno 1] Operation not permitted')]

The command '/bin/sh -c pip install .' returned a non-zero code: 1

Yes, most Docker images (including the python images) use the root user by default.

If I use that Dockerfile to build an image for one of my own Python projects, everything works out fine, even if I copy the install_requires from your setup.py. The only thing I see that might be the source of the problem is if root somehow doesn’t have permission to read your README.md, but I’m not sure how that could happen. Can you paste the permissions on your project directory as shown by ls -l? Do you have a repository on GitHub or similar where all the code for this project can be found?

That’s really promising and frustrating at the same time! :rofl:

The error when using RUN pip install . looks to be failing for all file operations but I take on board what you say about the permissions to read README.md.

Thanks for your suggestions and efforts so far, I’ll check out the folder permissions in the morning and get something posted up.

Thanks again.

I’ve had a good look around the file structure and absolutely every directory/file is owned by root and in the root group. The ls -la result for the project structure, in the container is as below…

root@680ef6d6889a:/bmsapiconn/bmsapiconn# ls -la
total 32
drwxr-xr-x 6 root root 4096 Aug  4 14:38 .
drwxr-xr-x 1 root root 4096 Aug 18 08:52 ..
-rw-r--r-- 1 root root  714 Jul 30 11:33 BMSAPIConnector.py
-rw-r--r-- 1 root root   57 Jul 30 09:55 __init__.py
drwxr-xr-x 4 root root 4096 Jun 25 14:50 api
drwxr-xr-x 3 root root 4096 Jul 23 10:32 bms
drwxr-xr-x 3 root root 4096 Aug 17 17:11 config
drwxr-xr-x 2 root root 4096 Jul 22 16:03 connector

I’ve since tried adding --use-feature=in-tree-build to the pip install . command with no luck.
Also, I’ve tried commenting out the two "long description lines from the dockerfile.

The error when running RUN pip install --use-feature=in-tree-build . is…

    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/bmsapiconn/setup.py'"'"'; __file__='"'"'/bmsapiconn/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-7b1c1i0f
         cwd: /bmsapiconn/
    Complete output (3 lines):
    running egg_info
    creating /tmp/pip-pip-egg-info-7b1c1i0f/bmsapiconn.egg-info
    error: [Errno 1] Operation not permitted
    ----------------------------------------
WARNING: Discarding file:///bmsapiconn. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I’m weary of publishing this publicly as it’s related to a university assignment and I don’t want to court any risks of being accused of plagiarism, with my own code! Rest assured that this problem isn’t directly associated with the core assignment so this isn’t a request for help with an assignment per se.

I’m just about to try this out on a different (more powerful) machine, to see if it’s related to the Raspberry Pi host!

Thanks.

The same dockerfile runs fine on a different (Ubuntu) host, as previously verified by @jwodder, so I can only assume that the problem was something with the host.

I tried apt remove --purge docker-ce docker-ce-cli dockerd.io follwed by a re-install on the Raspberry Pi but I still didn’t have any luck!