Creating a package that depends on a private third-party package

Packaging
1

I am working with a third-party Python package that provides various classes and functions for interacting with a manufacturer’s lab equipment. This third-party package is meant to be used as a software development kit (SDK) for your own Python code. However, due to licensing, the third-party package is not open-source and not available on PyPI and cannot be shared with others. To make working with this third-party package a little more convenient, I have created my own Python package that is basically a wrapper for the third-party package. My code looks like this where MyDeviceController is a class in my package:

import sys
sys.path.append("/Users/bart/projects/sdk/")  # <-- path to third-party package

from sdk import DeviceController  # <-- DeviceController class from third-party package

class MyDeviceController:

    def __init__(self):
        self.device = DeviceController

    def get_serial_number(self) -> str:
        x, y, z = self.device.get_serial()
        serial = f"{x}.{y}.{z}"
        return serial

This approach works fine for me and I would like to give my package to some colleagues who also have the third-party package on their computers. But if I give this code to my colleagues, the path to the third-party package may be different on their computer. To solve this issue, I can initialize the MyDeviceController with the path to the third-party package:

import sys

class MyDeviceController:

    def __init__(self, path: str):
        sys.path.append(path)  # <-- path to third-party package
        from sdk import DeviceController  # <-- DeviceController class from third-party package
        self.device = DeviceController

    def get_serial_number(self) -> str:
        x, y, z = self.device.get_serial()
        serial = f"{x}.{y}.{z}"
        return serial

But importing the third-party package within the class init method feels like a hack. Is there a better way to work with third-party packages that need to be imported via a path parameter?

2

If the latter is a requirement, no, not really.

But even private, local third party packages can be normal python packages using normal install systems. This would require changes to that package. Your company could even setup a local package server, but I don’t have any experience with that.

1 Like
3

The license of the third-party package prohibits us from distributing it within our company. If someone else wants the third-party package then they must purchase it from the third-party. Otherwise, this could be handled like any other package on PyPI.

4

… That is a stupid license.

Then no, I fear there is nothing that could significantly improve this. But I would seriously consider if there isn’t some alternative to that third party package that doesn’t actively slow down your workflow.

5

I think you’re basically on the right track for this use case. The thing I’d suggest is making the addition to sys.path into a function, rather than doing it in class init.

But that’s because I think modifying global state should be kept separate from class definitions, and I think it will serve you better if you add other classes, etc. Not because what you’re doing is “wrong”.

Is this vendor providing their sdk in an installable format, like an sdist or wheel? Needing to manipulate the path at runtime comes from not having their code installed as a package. That seems like something they could make easier for you, regardless of the licensing.

2 Likes
6

Is this what you mean by put the sys.path into a function?

import sys


def get_device_controller(path: str):
    sys.path.append(path)
    from sdk import DeviceController
    return DeviceController


class MyDeviceController:

    def __init__(self, path: str):
        self.device = get_device_controller(path)

    def get_serial_number(self) -> str:
        x, y, z = self.device.get_serial()
        serial = f"{x}.{y}.{z}"
        return serial
7

In whichever ways the users must input the path to the third-party package.

In this case I think the cleanest way is to use env var PYTHONPATH=<path_to_package> and just import the third-party module in your package. Treat it as an external dependency, and configure it using an external way, instead of hacking into your own code. So that users don’t have to hardcode the path in the code, but make it configurable in their bashrc files

9 Likes
8

I meant that if you absolutely must make the changes to sys.path at runtime, I think a better paradigm is to separate sys.path manipulation from anything else, e.g.:

def make_sdk_importable(path: str):
    if path in sys.path:
        return

    if not (pathobj := pathlib.Path(path)).exists():
        raise RuntimeError(f"{path} did not exist")
    if not (pathobj / "sdk").exists():
        raise RuntimeError(f"sdk in {path} did not exist")

    sys.path.append(path)


class MyDeviceController:
    def __init__(self):
        import sdk
        self.device = sdk.DeviceController

and if these are isolated into separate modules, you don’t need the deferred import.

The simplest way is probably, as noted above, to just use PYTHONPATH. If you choose to do this at runtime, do it in a way which is easy for people to swap out for alternatives.

9

Where does the make_sdk_importable() function get called?

10

Is setting the PYTHONPATH environment variable the same on macOS, Linux, and Windows? I am familiar with this approach on macOS but have no idea if it is the same on Linux and Windows. I agree this approach would make the underlying source code of the package cleaner. But from a user perspective it requires extra steps outside of the Python environment which the user may not be comfortable with. So that is why I like doing this in the Python code because the user does everything in Python.

11

Setting it in bash and zsh are the same as export PYTHONPATH=...
And in Windows cmd its set PYTHONPATH=..., in powershell: $env:PYTHONPATH=...