FRE - Frequently Requested Enhancements

In my short time contributing to cpython, I’ve noticed that a few changes are frequently requested, and there are repetitive explanations for why the change won’t be made. The first and foremost one that springs to mind is the CSV module. People often ask for various tweaks or improvements, but ultimately it never gets changed because :

  • too many people depend on it
  • any change to it’s behavior would cause regressions
  • there is no strict CSV file format. Therefore, the implementation defines the specification, and changing the implementation will always be painful
  • any new features will face the maintainability pain that the current features face

This is just one example, but anyone who’s spent a while on the bug tracker can probably think of a handful of examples like this. I wonder if we can maintain a section of the dev guide that documents these, adds new ones as they come up, and references a handful of bpo’s as examples.

This could help core devs and triagers close issues more quickly, because they can have a knowledge base to point issue reporters towards whenever they see bug reports that fall into these common categories, rather than having to repeatedly explain the same idea in different ways.

What do you think? Would this be a useful resource? What common non-bug or unfixable bug report would you want to see documented here?

1 Like

This is a promising suggestion, but I see many complications.

It’s not clear to me why we only care about frequently made
enhancements as opposed to other forms of bug reports.

Firstly, how do we decide what counts as “frequently requested”? Is this
just based on a subjective “feel” for what’s common?

For example, you mention the CSV module. I don’t recall seeing any
requests for enhancements, but then if I had, I would have probably just
deleted the email and ignored it. Whereas my entirely subjective feeling
is that barely a week goes by without yet another spurious “Python can’t
do maths” bug report.

In truth, my subjective feeling is probably not true: I don’t think that
there are 40 or 50 such bug reports each year. Perhaps ten? I don’t
know. That’s frequent enough for there to be an entry in the FAQs.

Another issue is that today’s frequently rejected enhancement can be
tomorrow’s accepted feature. Just a few examples that I remember:

  • ternary if

  • list.clear()

  • “adding” dicts (we use | rather than +)

  • strict zip that raises if the input iterables are unequal length

  • sorted() and reversed()

Sometimes the right response to a frequently rejected feature is not to
reject it again but to reconsider why we’ve rejected it in the past and
whether we should just add it.

A “Frequently Rejected” section of the docs could become an excuse to
mindlessly reject features that shouldn’t be rejected. How do we take
care to prevent that?

Your explanation for why the CSV enhancements are rejected doesn’t sound
right to me. If they are enhancements, then they would be new
functionality, not changes to existing functionality. So:

  • no behaviour that people are relying on will change;

  • new behaviour shouldn’t cause regressions;

  • the non-existence of a strict CSV file format is a red herring;

  • naturally every new feature increases the maintenance load, but
    that should be weighed against the usefulness of the feature.

There may be other reasons to reject an enhancement, but they will be
specific to the enhancement request.

Even if there are changes to behaviour, they may be accepted. See for
example:

https://bugs.python.org/issue43625

which pushed through a change in behaviour to the header detection
heuristic.

On the other hand:

https://bugs.python.org/issue44677

is still open, and the final comment (as of the time of writing)
suggests that it probably won’t be accepted.

So I don’t think we can do better than judge each request on its merits.
There’s certainly no rule about no new enhancements to the csv module.

Response to Steven

Thank you for the detailed reply Steven! I agree that there are plenty of nuances to work out. I’ll respond to the points you’ve raised.

It’s not clear to me why we only care about frequently made
enhancements as opposed to other forms of bug reports.

You are right, we should document anything that ends up in the bug tracker repetitively here. I invite anyone to brainstorm a better name! Here are a few suggestions:

  • Frequently Reported Enhancements and Bugs
  • Frequently Reported Things
  • BPO Frequent Flyers
  • Repetitive BPOs

etc…

For example, you mention the CSV module. I don’t recall seeing any
requests for enhancements, but then if I had, I would have probably just
deleted the email and ignored it. Whereas my entirely subjective feeling
is that barely a week goes by without yet another spurious “Python can’t
do maths” bug report.

I think this gets at the fact that everyone has their own pet peeve issues on the bpo. A goal of this document would be to make it easier for core devs and triagers to link to a definitive source about the history of the discussion of an issue without needing to provide an explicit explanation or search up some past example bpo’s as I normally see.

Another issue is that today’s frequently rejected enhancement can be
tomorrow’s accepted feature

I definitely understand this, and of course it’s important to discuss these issues over time. I think that the document should start with a paragraph to contextualize what follows, and to ultimately communicate these points:

  • These topics are not “banned” or even discouraged – this is just a knowledge base for frequent (or sometimes repetitive) discussions; HOWEVER
    • You should read the former discussions, and see if there are perspectives you haven’t considered
    • You should message python-ideas (email or discourse), which is a friendlier and more contemplative discussion forum than the python bug tracker. If you can achieve some consensus about a fix or forward-step there, a request for concrete changes in the bug tracker will be much better received.

Benefits and Use-Cases of This Doc

I want to talk a little bit more about the benefits and use cases for this document to further clarify what I’m envisioning.

Strengthen New BPOs

When contributors read this document first, new bug reports about repetitive issues will be stronger. They might take the issue to this forum or the python-ideas mailing list, or at the very list they will read and be prepared to respond to the ideas in this document. Discussions about these long-running and common problems will be built upon a stronger foundation, which will actually help to push discussions forward faster.

Service Core Dev’s Pet Peeves

I think that a lot of core devs have pet peeve issues that they are tired of dismissing in the same way. Obviously actual core devs will know better than me whether that is actually the case :slight_smile: If so, this document can become a definitive source about the problem. If bug reporters appear to not have read the document, they can just link to it without putting too much mental energy towards the issue, and the reporter can follow-up after they have absorbed the additional context.

Index of Related BPOs

Topics in this doc should end with a list of the BPOs related to the issue. Of course, it would be ideal to include BPOs about the issue that have lead to merged pull requests to show that this isn’t an off-limit topic, it just has nuances that must be considered. This would be doubly good, because it provides a model for which reports led to positive changes, versus ones that hit dead ends.

Citing BPOs also ensures that the topics in this document are grounded in real repetitive bug reports. I think it is important to ensure that it doesn’t become a divisive place where people try to shoehorn in the things that they just don’t want to see changed or discussed.

Maintaining this Document

Avoid Pedantic-ism

The document should establish a culture of discussion and provide the context around these complex issues in a neutral way without implying that the problem is off-limits. We also don’t want it to become a place where people try to shoehorn in things that they don’t want to see discussed.

Automation?

I’d be happy to solicit suggestions and then write this document by hand to start with, but I’m thinking that the burden of maintaining this document might be problematic, especially since its purpose is supposed to be to smooth out discussion and save time!

I’m not really an expert in automated workflows, but I’m hoping others might share some bright ideas!

I have some sympathy to the request, but also most the objections. But note:

If the existing functionality is not buggy, then a change as opposed to addition request is an enhancement issue that if accepted would likely break code and that is rejected for that reason. Some people confuse ‘design choice I don’t like’ with ‘behavior that violates the doc’ (and sometime the doc is a bit vague).

There is currently no active csv expert; it needs one. Searching the titles of open issues for ‘csv’ and ‘sniffer’ return 38 and 8 hits respectively. I suspect this is larger than usual.

Someone (Jd?) could usefully do a review issue (such are allowed) of the Sniffer issues. Are there really 8 different issues? or are some duplicates? Which can really be classified as bugs? The Sniffer docstring says “Return a dialect object”. That is wrong, as calling Sniffer() returns a Sniffer object. So the issues must be about the sniffer method, which does return a dialect object. As long as it does so, when can we semi-objectively say the return is wrong? IE, when or how can ‘the current behavior is the authority’ be rebutted? Any change could potentially break code that depends on the current return.

Separate reviews of reader and writer issues would likely be useful.

I wonder if adding a few strategic sentences to the csv doc about its design philosophy would help triaging. Such would more easily be found than anything in the devguide.