Can we do something about the difficulty of installing compatible packages/versions?

I love installing new repos and experimenting with new code. However, my enthusiasm is sometimes dampened by the 1 to 2 week installation process.

For instance, on a recent repo, I had to make my windows PC a dual-boot system because the software required “bare metal” Linux. I purchased a second internal hard drive, then installed the exact required version of Linux, then the exact required version of Tensorflow GPU with the exact version of CUDA and the exact version of Python and Unity.

If I install another repo of similar complexity, I will likely have to install another version of Linux, Tensorflow, and Python, as well as the package dependencies. Having virtual environments to segregate the dependencies isn’t enough; I’d like to see us get to the point where we can have a “one-click” installation of even complicated programs with many dependencies, so even newbies can install intricate projects.

Hello Tyler,

You say that you had to make your Windows PC dual-boot because “the
software” required “bare metal Linux”.

What software are you referring to? Surely not Python, it runs happily
on Windows. If not Python, but some other software, what do you expect
us to do about third-party software we don’t control?

What do you mean by “bare metal”? To me, that term applies to running
software without an operating system. You might mean something else,
since you were running Linux.

Tensorflow runs on Windows; CUDA runs on Windows; Python runs on
Windows; the Unity game engine runs on Windows. So it isn’t clear to me
why you needed Linux or why it specifically had to be a dual boot system
rather than running in a VM, and it certainly isn’t clear to me why it
is Python’s responsibility to install the full stack of software
including the operating system.

I’m not unsympathetic to your problem, it is just that you seem to be
venting your frustrations rather than giving sufficient detail to allow
constructive discussion to take place.

I think what that Tyler is emphasizing here is the general problem of package installation due to version incompatibilities.

For this general discussion he doesn’t need to describe the process for CUDA and CUDnn exactly to get his point across.

If you insist on some “measure” of difficulty and frustration from the community at large:

YES, Python, Tensorflow and CUDnn all run on windows, but there is no straightforward set of steps to install them for every windows user: For example, after doing the steps from StackOverflow suggestions, I still could not access my gpu. Someone pointed out that specific CUDA versions require specific TensorFlow versions → I started again → still nothing happened → then someone else said “use conda” → still nothing.

Tyler is more or less wondering what we can do so that such issues are minimized or eliminated. That is how I found this post, I am also frustrated because such things take so much time.

Also, like myself, Tyler might not be a programmer with extensive experience to the point that he understands the inner workings of the setup process in Python. But, again like me, he likes to program and get into the essence of things but instead hits a wall of unnecessary work. He then opened a general discussion as to what is the problem, which I think is a good question, because:

  • One could argue that it is the issue from the side of the developers: They did not take into account multiple version cases and only made corrections after people like Tyler wrote their comments on their repo/blog (after all, they might not be getting paid for their coding work)

  • One could argue that expert developers don’t sympathize with people who know how to code but have limited “IT knowledge”. Especially nowadays, since interdisciplinary research is more common than ever, even people in the medical field try to get their hands on python and deep-learning. I think this is great, but they are faced with tasks that are easy only for IT people and IT hobbyists. If one derives pleasure from learning all those inner workings of Linux, Windows, software package setup tricks etc, good for them, but all the others should not have to suffer

  • One could argue that maybe it is the price we pay for opting for open-source software and not paid subscriptions to software such as MATLAB, wherein we can ask for customer service

2 Likes

Ah, yes, because when you ask for customer service from a company you’ve paid money to, they always and perfectly solve your problem.

But I suspect that, in the OP’s case, the problem isn’t Python, nor Tensorflow, nor Linux, but that the tool itself didn’t adequately specify its dependencies. And unfortunately, there’s little that anyone else can do.

1 Like

Exactly! That is one of the cases mentioned above.

Well, regarding paid packages, I doubt such dependency and versioning issues would be an obstacle. I suspect that those happen in OpenSource because the devs either don’t have time to rigorously document how to solve such issues or automatically revert to the correct versions, or they do not want to spend that time because they don’t have a significant incentive to do so

Ah yes, because paid packages ALWAYS get tested on every possible platform, and someone makes absolutely sure that it works perfectly. That’s definitely been my experience in the past.

I’d say it’s more likely to be the opposite; open source tools can have end users like yourself submit pull requests, and as long as they don’t break anything on the tool’s primary platform(s), they can be merged. So you end up with far better platform support than if some individual or organization has to do all the testing themselves.