Turn shutil into a runnable module

I don’t see any of those problems as big as trying to guess is 42 a number or a string in an environment where you cannot reliably use quotes. More precisely,

  • type hints can be added and they have also other benefits,
  • unions can be handled by trying conversion with all types in the specified order (e.g. with int | float try integer conversion first and if it fails try float), and
  • ABCs can be handled by converting to concrete classes (e.g. with Mapping convert to dict).

That existis. It is called python -c

I have a system here that consumes command-line arguments (good design, btw) in a YAML list of strings and puts them in a CMake variable.

Let’s imagine we come up with a practical, generalized approach that interprets functions and makes them usable as command-line. Then, sometime later, someone proposes a better heuristics. Now, do we know whether our shutil 1) works exactly the same as before, 2) breaks slightly and requires changes such as hints, or 3) breaks silently and end-users’ system changes behavior? If we convinced ourselves by saying “we know,” sometime later, another person proposes a better way to annotate shutil. Will we know whether the changes break its command-line interface interpreted by our heuristics?

We don’t truly know unless building a regression test for shutil. Which means we are effectively mandating the outcome, not the underlying implementation.

To me, a heuristics-based solution is okay to play with when I sit in front of a desktop. But I will not use it in an automated system. My proposal here in this thread implies a stable contract between the command-line interface and the end-users, if that is not obvious.

1 Like

Considering I removed the pending label on the gh issue, here’s my 2c:

  • Progression bar should be considered separately honestly.

  • runpy is something I only discovered recently and I think I’m probably not the only one even though it’s been years I’ve been programming in Python.

  • Personally, I would implement python -m shutil [...] to be consistent with other modules as well (e.g., dis, symtable, etc). I don’t know why we shouldn’t do this for shutil since it appears that people would like a CLI (see below for additional discussion). Since we only expose some functions, we can use subparsers for that and that would be sufficient (a bit like git). It would be python -m shutil copy F1 F2. Typing wouldn’t be a problem.

Concerning cross-platform compatibility, we could either 1) disable flags that would not be supported everywhere or, 2) make the silently ignored, or 3) something else (?). Note that using the platform-dependent commands may not necessarily be an alternative because Linux commands such as cp or mv do not exist on pure Windows shells (on PowerShell, those commands exist IIRC).

Now, for the runpy alternative, I think it should be something we discuss separtely as it can affect any kind of module. I find passing arguments as JSON through the CLI really painful (for instance, jq is great but it’s hard to type inline jq commands because it’s barely readable). It wouldn’t be an issue if you write a script but invoking it in the terminal would be less convenient.

So there is a question: should shutil be considered a complete platform-agnostic replacement of copy and co. functions or not? if yes, it’s probably preferrable to strive towards a runpy solution while providing a minimal shutil CLI that can be invoked outside a script.

  • The minimal CLI would be used for inline commands. I don’t see why someone would use shutil in the terminal if they can actually use the platform command directly. Well I can see some reasons such as remembering only one syntax but apart from that I think that the caller would be more comfortable with the platform commands. Nevertheless, since I cannot know whether this is the case or not, I would be willing to simply expose the compatible commands (with or without the platform-dependent flags, that can be discussed later as well).

  • The runpy solution would allow complex copy/move scripts to be platform-agnostic. This also means that passing arguments through JSON would be easier. So it’s no more a blocker.

Concerning the issue itself, having a minimal implementation of shutil as a CLI, the same way as zipfile, could be a worthwhile addition. It would be possible to have a platform-agnostic solution without blocking future improvements of runpy. I am not saying that a PR can be opened and that the feature will be accepted as is, but I’d like to share my ideas and try to advance the discussion towards a “planned/not planned” for shutil, or “improve/do not improve runpy” conclusion (so that we don’t have yet-another-forgotten-idea-with-an-opened-issue).

1 Like