There’s a misunderstanding here. This thread started with a proposal to un-soft-deprecate getopt and optparse, but there was never any suggestion to remove them in the first place, even before this proposal.
Yeah, this is essentially the current policy. The docs say:
Deprecated since version 3.2: The optparse module is soft deprecated and will not be developed further; development will continue with the argparse module.
Deprecated since version 3.13: The getopt module is soft deprecated and will not be developed further; development will continue with the argparse module.
A soft deprecated API should not be used in new code, but it is safe for already existing code to use it. The API remains documented and tested, but will not be enhanced further.
Soft deprecation, unlike normal deprecation, does not plan on removing the API and will not emit warnings.
Perhaps “discouraged”? I.e. “use of the getopt module is discouraged, and it will not be developed further…”. We should ideally use a term/phrase that doesn’t require reading a glossary entry…
I’m bike-shedding, but I would prefer “not recommended” or “not recommended for new code”. I.e. don’t write new code that uses getopt. However, you shouldn’t feel you need to change existing code that uses getopt assuming it already works fine.
Even that is overly negative. There are genuine trade-offs, so the way I’d describe the three modules is:
getopt: very specifically focused on producing CLIs that can be readily replicated in C code. Not recommended as a general purpose tool. Considered feature complete - any enhancements should be developed as third party modules on PyPI.
optparse: lower level API that provides the essentials of breaking up a command line API into named options and positional arguments, but requires the application developer to be explicit in how more complex processing should be implemented. Considered feature complete - any enhancements should be developed as third party modules on PyPI.
argparse: higher level opinionated API that attempts to allow application developers to support common CLI use cases with less application specific code. By virtue of being opinionated, its behaviour may sometimes clash with the way an application developer would like their CLI to work, which may mean having to switch to optparse or a third party library to obtain the desired behaviour. Remains open to the addition of new features in new Python versions.
(I know @storchaka has a more negative view on argparse than I do, but I also think we need to remember that the vast majority of argparse users don’t encounter the situations where it misbehaves, or if they do, the problem is readily handed by saying “don’t call this CLI like that, it doesn’t work that way”)
For the case of getopt Is the current note at the top of the documentation not sufficient as recommendation?
The getopt module is a parser for command line options whose API is designed to be familiar to users of the C getopt() function. Users who are unfamiliar with the C getopt() function or who would like to write less code and get better help and error messages should consider using the argparse module instead.
It could perhaps be a little shorter but I think it gets the general message across.
It’s worth being explicit that it won’t be developed further–i.e. people shouldn’t request new functionality to be added to getopt. But the deprecation note (above the one you quoted) makes it sound like it shouldn’t be used.
Yes, you could state it’s considered complete and won’t get new features, but in terms of what’s recommended and what it’s for (and what you might want to use instead) I think the present note covers that reasonably well.
Actually, this is exactly the reason why we should remove getopt from
the standard library. Even for the getopt’s afficionados as you it is
obviously not enough to use, so you have your own library.
No. I have my own library to address my use habits. I’ve got a lot of
these, for many things.
I’d have a library for argparse if that were the only option (but I
wouldn’t, because I’d be using getopt from wherever it went).
This is not an argument for getting rid of getopt.
To me, argparse is cumbersome bloatware; I do not want it. Many people
feel otherwise, and more power to them!
How different would be your life if you just included those 215 lines
of Python code to your argument parsing library? Why do you need it in
the standard library at all?
I’ll second Paul Moore’s sentiment here: what if I vendored everything
into my code. I could drop the entire stdlib!
I think a better question would be: why is the standard library a better place for that module than a third party package? I understand no one is asking for its removal but I would like an answer to the question from the advocates of the standard library necessity point of view.
In the first instance: backwards compatibility. The stdlib was supposed to be something that you could depend on in batteries-included Python so people wrote code that depended on it. Any removal from stdlib breaks stuff that was written a long time ago and that should still work fine and has no reason to break.
Your question here is backwards like “please justify why we should not break your code” when it should be the other way round like “please justify why you want to break my code”.
Of course these modules would not be added to the stdlib now but now is not then. They are here now and are depended on and if you want to change that then you break a lot of code.
On my Win10 machine, idlelib is 5.75MB. Of that, 3.91MB is for __pycach__ and idle_test.__pycache__. If one does not fill those either on install or by use, idlelib proper is 1.84MB.
I would seriously argue that no module can be proposed for removal from the stdlib unless it’s possible to demonstrate that more time has been spent maintaining the module over the last (say) 3 years than would be spent discussing the removal. Otherwise the proposal is clearly a net loss in terms of developer productivity.
I don’t think so: there are also user expectations. When a normal user sees a module in the standard library, they may conclude that it is a good idea for them to use it. Having three modules for more or less the same thing is confusing (even more so for optparse), and misleading (I don’t think 90% of Python developers should ever touch getopt). Yes, getopt is so stable and mature that it is mostly maintenance-free, but I really believe it belongs more to PyPI for specialized use it is best for (and former C-programmers who haven’t shaken the habit yet).
You are underestimating the value of having such a simple to use, no maintenance arg parser in the stdlib.
It’s most likely going to be used for simple scripts and for those you don’t want to have to install a venv just to get the script running. It’s a perfect example of the classic batteries included advantage of Python.
My stance is that it was a mistake to recomend the argparse module as the only true CLI parsing module and drop-in replacement of the optparse module, while it was not. It’s not only a different API, but different behavior. We cannot undo this. But we can stop doing more harm, stop misleading users. Most of current argparse users are victims of our advertising. Users should be able to make informed choices between simpler API that implements traditional CLI and more feature-rich experimental API that implements non-orthodox CLI.
Ideally, we should have a feature-rich API with configurable behavior, which provides user-friendly but non-ambiguous CLI. But argparse is still in experimental stage, after 15 years.
‘Experimental’ seems like an unrealistic characterisation here. I’ve used argparse for any number of CLIs, and while the code I write with it is not especially beautiful, it’s been very capable & reliable, and I have little fear that it will break underneath me. If its behaviour differs from time-honoured tradition in specific cases, it doesn’t bother me. Traditional utilities have some pretty mixed CLI styles anyway - tar, ps and dd come to mind.
I really don’t want to make an informed choice about my argument parsing library. I’m quite happy to have one recommended stdlib option that I use without thinking much about. I’d almost always much rather be spending the mental energy on what the code does after successfully parsing arguments.
Users should be able to make informed choices between simpler API that implements traditional CLI and more feature-rich experimental API that implements non-orthodox CLI.
Do you want to say that Linux is all about choice?
(and no I don’t think the link is off-topic, when you click on the answer, it is exactly about the confusing choice between three more or less identical solutions, moreover “There should be one-- and preferably only one --obvious way to do it.”; what he says about the need to fix bugs in the end, is the same as what I am saying about the need to fix design)
In my usage experience, argparse and optparse/getopt are pretty much different tools for different jobs, and – from the outside – it seems like there is no harm in keeping them all in the stdlib.
argparse is by far the most used tool, as it covers most needs most of the time. This is the one module that cannot be removed, and it’s also the one module that requires active maintenance.
optparse and getopt are the tools for when argparse doesn’t cut it, e.g., when one has to create something with unix-like or git-like behaviour. It’s great to have these tools available out of the box for distributing a script or similar with minimal fuss. These modules could probably be removed, but they also seem to require very little maintenance.
While I appreciate this view, it’s important to realise how much heavy lifting “traditional” and “non-orthodox” are doing here. Traditional… amongst who? Non-orthodox… from what perspective?
When argparse rejects “-o -v” or “-o --” as ambiguous, or treats “-o=foo” as equivalent to “-o foo”, a lot of users will be happy with that behaviour.
It’s true that argparse won’t let you completely mimic a traditional getopt-based Unix CLI with full fidelity, but optparse won’t let you mimic a traditional DOS CLI at all (since you can’t use “/” as the option prefix).
So for a lot of developers, if you give them a list of specific behavioural differences between argparse and optparse, many of them will say that it is optparse which is doing the wrong thing, regardless of any Unix traditions that consider it to be expected behaviour.
argparse not supporting full fidelity equivalency with optparse is a valid argument for reversing the formal deprecation of the latter (if you want or need that behaviour, argparse can’t and won’t give it to you, so we can’t legitimately expect all optparse users to switch to using argparse instead). But for most users that don’t have a specific need for the features that only optparse offers, the additional conveniences that the argparse API offers is a substantially more valuable benefit than adhering more strictly to Unix CLI conventions.
I’m quite happy to have one recommended stdlib option that I use without thinking much about. I’d almost always much rather be spending the mental energy on what the code does after successfully parsing arguments.