The Python Typing Survey is back for its third year! Last year we received 1,241 responses (up 15% YoY), and we’ve been pleased to see the survey become a recognised reference point for the direction of Python’s type system. It’s being referenced in PEPs and presentations to the Python community (including Guido’s typing summit talk). If you’re interested, you can read the analysis of last year’s results.
We’ve pulled together a draft for this year’s survey, featuring mostly the same questions as last year with a few adjustments. We’ve also opted to try out SurveyMonkey this year instead of Google Forms, it’ll give us some nicer tooling to pull together graphs etc. at the end, and should be a nicer experience for survey takers.
Feel free to click through the draft to try it out, responses won’t be saved. There’s one branching point this year (e.g. you’ll see different questions depending on whether you answered “never” to the question of type checker usage), so it’s worth clicking through to see how the flow feels.
If you have any feedback on the questions or structure, please leave suggestions in this thread. Also happy to hear ideas for how we can spread the word more broadly when it comes time to publish.
Q17 [1] could use an “other” option, even if it doesn’t come with a free-form response, if the free-form response is not seen as useful for this question (given the specific options listed, I could see this being used to try and prioritize specific ongoing work)
I didn’t notice any other things like that, if these questions cover what people need from the ecosystem, the way the questions are presented seems fine.
Thanks for running the survey!
As labeled in the overview, it’s question 16 as presented to users after the skip logic for “Never” ↩︎
Q9 is phrased as “Which IDE [singular] do you mainly use? (Select all that apply)”. The two clauses are contradictory; maybe something like “which IDEs do you use regularly” would be better?
Tangentially, specifying “for Python development” might be good; some people use different IDEs for different tasks.
Pedantically, “IDE” might be the wrong term here since “(Neo)vim/VIM (without a language server protocol)” is an option.
Q12 asks you to “rate” 6 options on a scale of 1-5, which is fine. However, the labels for 1 and 5 are confusing; 1 reads “not useful” (which matches a “rating”), while 5 reads “most useful” (which implies a “ranking” instead). Perhaps “very useful” or similar would be a better label?
You might want to ask a question relating to “how” people use type checkers. Some are integrated into IDE’s, some are only available via the command line, some have both. So something like:
Do you use a type checker via (check all that apply)
Command line
My IDE
CI
Another question could relate to how people configure their type checkers, i.e., via pyproject.toml, or type-checker specific configuration files.
Another idea is to have a question about how people are reconciling different behaviors between type checkers, if they use more than one. For pandas-stubs (I’m the lead maintainer), this is something we have to deal with. I don’t know how to word this question, but maybe something along the lines of “If you use multiple type checkers on the same code base, are differences in features and behavior an issue?” (Yes or No). If yes, ask for a free-form text answer to understand the issues.
Thanks everyone for the feedback, really appreciate it! I’ve worked through all the suggestions here (as well as some additional comments received externally) and incorporated what I can.
I’ll keep the draft open for another few days. If there are no further comments, the plan is to finalize this week and launch next week. When we do (I’ll post separately), any help spreading the word would be great!
On Q6, if the respondent hasn’t used typing in that context or hasn’t been aware of whether it was useful in that context, are they meant to answer “1 (not useful at all)”?
Q11 I think “In my IDE / editor (real-time feedback)”, “From the command line (on-demand)” should be split into
Automatically run by my IDE
Manually run in my IDE
From the command line
I manually run dmypy in my IDE with a hand-written tasks.json that runs command dmypy run in the integrated terminal[1]. I also have Pylance/Pyright enabled (they update in real time). So I would select #1 and #2[2], and if I didn’t use Pylance/Pyright I would select #2. It’s clearer.[3]
In addition, #1 covers “update-on-save” in addition to “instant feedback” (if this finegraining isn’t important).
Some remarks:
Shouldn’t Q4 have the option of typing for runtime use (eg pedantic or beartype)?
Shouldn’t Q7 include IDE?
Answering Q18 requires quite some background; perhaps slight explanation?
