Proposal: canned explanations for issue/PR closure decisions (Language Summit follow up)

Core Development
1

The 2022 Language Summit [1] included a discussion about the issue and PR backlog. The premise was Sam Schillace’s observation that “a large pile of old, low-ish priority bugs […] obscures both newer quality issues as well as the overall drift of the product. It’s far better to resolve something to the right bucket or ‘will not fix’ to keep the tool clean for things that matter.” [2]

There were a number of interesting suggestions, but not enough time to discuss any of them at depth. In the next few weeks we will be creating a thread for each of the ideas, so that it can be discussed and we can reach a collective decision about which changes we want to implement in our work practices. This is the first in the series.

@pablogsal mentioned in the discussion a problem that I will call “core developer fatigue”, where it is sometimes easier to leave an issue open than discuss the closure decision with the person who reported it. In other words, letting someone down is an unpleasant task, and many low-value issues remain open because we would rather avoid it.

In a slightly different context, @tiran suggested that we create standard explanations for the most common closure reasons, which a core dev can apply without having to type repetitive comments.

I think Christian’s suggestion may actually help not only with reducing the typing workload, but also with the emotional fatigue that Pablo described, because canned responses tend to feel less personal to both sides of the exchange.

We could implement Christian’s suggestion by adding a number of close-reason labels, like close-out-of-date, close-cannot-reproduce, close-unsupported-platform, close-abandoned etc. We can then have a bot that responds to this label by closing the issue with the canned response. The core dev can always add information in an additional comment if necessary.

[1] PyCon 2022 - Language Summit
[2] Bonus post - Noisy Monitors - by Sam Schillace

6 Likes
2

It was also mentioned (by @willingc IIRC?) that issues that have actionable items are more likely to be resolved. Maybe it would be worth it to create templates (or guides) for how to proceed to solve various types[1] of issues?

  1. C bugfix, Python bugfix, new minor feature, new major feature, etc. ↩︎

3

I don’t know if there’s a list forming, but IMO one good canned response category for PRs would be “PRs that only adjust minor code style/aesthetics”.

2 Likes
4

I’m not sure that receiving a canned response is a better experience for the recipient. Less personal, for sure, but that’s not a good thing, right? In most cases I’d rather read a brief personal response containing typos and grammar errors than a standard form response written by an English major. And I’d worry that a user receiving a canned response from a bot after a human applied a specific label might miss the fact that a human made a decision there, and wonder why their issue was “auto-closed” by a bot.

Then again from the developer’s POV I very much agree that sending a negative response is emotionally draining (I have to do this frequently to people inviting me to talk at their event).

Maybe the developer could put in a brief personal message and close the issue, and a bot could add a longer explanation of why humans close issues? (Or perhaps better, a link to an explanation?) The bot could be triggered by the human putting in a special code. Just like there are some phrases such as “closes #NNNNN” that auto-close issues from PRs, we could have some other phrases like “closed out-of-date”, “closed cannot-repro” etc. that trigger the bot.

In fact, not using labels would be better because labels are a scarce resource (the labels menu is already too long). I’d prefer to reserve labels for filtering, and I don’t see a reason to filter for closing reasons.

7 Likes
5

Rather than a label, would it be possible to add one section per reason in the devguide with a short text, and then copy/paste this text? Maybe also add link to the devguide (about closing old issues) when closing an issue?

1 Like
6

GitHub has support for canned responses, they call then “saved replies”. Each person has to create them separately, but it can save repeated copy/pasting.

After choosing the saved reply, it’s inserted into your comment box and you get to edit before posting.

3 Likes
7

Oh, that’s very convenient! But still, IMO it would be nice to have a more elaborated explanation in the devguide for the reporter of the bug. Explain why core devs cannot fix all bugs, why it’s better to close a bug than leave it open, etc. Well, what has been said here :wink:

4 Likes
8

I suggested this in Triaging/reviewing/fixing issues and PRs. As I mentioned there, I closed a number of issues by linking to the “Why did changing list ‘y’ also change list ‘x’?” FAQ, usually after a shorter/personalized explanation.

This could be useful. We might also be able to implement it with autolinks (at least initially, if creating a bot is too involved), and have a reason-xxx that autolinks to devguide.python.org/faqs.html#reason-xxx.
Eventually having a bot/action that either edits the dev message or adds a new one with an explanation and a link to the devguide would be better.

1 Like
9

+1 from me.

10

FWIW, I used this feature for the Python extension for VS Code when I worked with @brettcannon. I found it helped me feel less bad about telling people “no”.

2 Likes
11

I was going to bring this up at the language summit, but I assumed most core devs were generally aware of this, since it has been a (very useful) feature for many years now. In fact, the top 10 (at least) responses get automatic keyboard shortcuts, so they can be applied in a second or two. They get posted as you, which is rather more personal than a bot, and can be edited/expanded upon as needed to customize them for the situation. Also, they can contain a link to a more detailed explanation in the devguide, if desired.

This was all certainly quite invaluable when I managed Spyder’s issue tracker, saving me countless hours of time while providing a clear, consistent and reasonably complete explanation for the user, while minimizing tedious mechanical effort on my part.

1 Like
12

I wouldn’t assume people’s knowledge of GitHub. A lot of core devs only have CPython as their only open source project, so if we don’t promote it or use it, they very likely don’t know about it.

3 Likes
13

Sorry about that. I just didn’t want to make a fool of myself at my first language summit suggesting something that most people already knew, but by the time I heard the last comment suggesting a bot just to post canned responses, I’d concluded a fair number of people must not be. Unfortunately, that was the last question before they moved on so I missed my chance, heh.

14

Having canned responses that coredevs can use as themselves (and not bots) with optional customization/personalization sounds great. Using auto-linking and/or bots that respond to certain incantations in the human response could work too.
Expecting coredevs to maintain their own library of “saved replies” adds significant friction to adopting this tactic, and would easily lead to inconsistencies and drift between the saved replies of different individuals.
Ideally this would be solved by a repo or org level shared saved replies. afaik this doesn’t currently exist in GitHub, but maybe GitHub would add this feature if requested by enough people. Another option is to use a third-party product that allows sharing snippets in a group (e.g. TextExpander or something similar).

I’m not sure though whether this would solve the “coredev fatigue” issue. Even with canned responses, the recipient might push back and ask for further explanation or justification. Maybe all canned responses can link to a FAQ that addresses that, and hope that the receiver takes the time to read that (and/or close the issue).

3 Likes
15

To summarise the discussion so far, it seems that everyone agreed that some form of canned responses can be helpful for core devs, in some cases based on previous experience.

The mechanics are yet to be determined, but labels+bot is not the way to go. There were a few votes for autolinks, and the GitHub “saved responses” feature was mentioned, but it’s unclear whether it currently supports shared responses.

5 Likes
16

This is not necessarily a bad thing: we could have a page in the devguide with some instructions and templates that can be copied by the devs to quickly set up canned responses. By doing this they can select the ones that they are more likely to use, and customize them to their liking.

4 Likes
17

The main advantage of canned responses is not so much that they save some typing, but rather that they reflect collective policy decisions of the core developers (this is how they reduce the emotional toll of having to let people down, by making the responses less personal).

It’s fine (even if a bit clumsy) if we say that each core dev creates their own set of saved replies on GitHub, and of course each core dev will make adjustments to the replies they use as is appropriate for each case. But that can only come after we have achieved the agreed baseline policies.

I’d suggest at this stage that we focus on the contents of our responses (in other words, our shared principles) rather than the mechanics of how the canned responses are stored, maintained and dished out. Once the contents of the replies is agreed, we will have a better feel for the appropriate mechanics.

4 Likes
18

Based on experience closing multiple cpython issues on GH (I think at least 10), I am happy with the ‘Close with comment’ as ‘completed’ or ‘not planned’ we now have + plus any additional comment someone cares to add. I prefer ‘completed’ to ‘fixed’ because it can cover action (at least one merge) that does not really ‘fix’ the issue as conceived by the OP, but does all we are going to do. (Documenting a workaround or bug avoidance method is an example.)

2 Likes