Typed Python Developer Survey

Typing
1

Hi all, I’ve chatted with a number of folks who have expressed interest in a survey to get feedback from a wide variety of Python developers on the current state of the Python Typing System. Our goal is to get a better understanding of how Python Developers are using (or not using) the type system, type checkers and IDEs. Luciana from VSCode and Galina from PyCharm have offered to share the survey across their groups and I will, of course, post a link here that can be shared as well. The ask for this group:

  • Leave feedback on the survey proposal if there is something you would like included: Python Typing Survey - Google Docs
  • Are there social media or other distribution channels you would want to share this on?

Thanks all and looking forward to feedback or suggestions!

6 Likes
2

Some thoughts:

  1. What type checking tools does your project use? (Select all that apply)

    •  mypy
    •  pyright
    •  pytype
    •  pyre
    •  typeguard
    •  pydantic
    •  typeshed
    •  Other (please specify)

typeshed is not a “type checking tool” per se. Most people don’t know what it is, yet all who use type checkers/Pylance/PyCharm/Jedi are already using it.

There are other tools that should be explicitly mentioned as well: Beartype, Pylyzer, Pyanalyze, etc.

  1. Which IDE do you mainly use?
  • VS Code
  • PyCharm
  • Emac or (Neo)vim/VIM
  • Other (please specify)

The “PyCharm” option should also mention other JetBrains IDEs, as they all use the same plugin under the hood.

Vim, Neovim and Emacs should each be listed on its own line. Other IDEs/editors like Visual Studio, Sublime Text, Spyder and VS Codium should also be mentioned.

  1. Which IDE plugins/extensions do you use for typing? (Select all that apply)

    •  MS.Python (pylance/pyright)
    •  Mypy
    •  PyCharm defaults
    •  Not sure
    •  Other (please specify)

The “Mypy” option could use some clarification. There are two Mypy plugins for PyCharm (three if you count this non-public one), three extensions for VSCode and many others for various editors/IDEs. Remember that there are people who use neither VS Code nor PyCharm. From what I heard, Neovim users love their experience with Pyright.

Just a FYI, I myself wrote two Pyright plugins for PyCharm. They aren’t exactly popular, however, and thus might not worth mentioning.

  1. What are your main goals for typing your project? (Select all that apply)

    •  Preventing bugs
    •  Documentation
    •  Runtime guarantees
    •  Faster code review
    •  Better IDE support (e.g. autocomplete)
    •  Other (please specify)

Instead of asking for the “main” goals, respondents should be able to give each choice a score (e.g. from 1 to 5) representing how much it contributes towards the overall usefulness of type hints in their code.

  1. Where do you hear about new type system features? (Select all that apply)

    •  Python version release notes
    •  Type checker release notes
    •  PEPs
    •  Reddit
    •  Hacker News
    •  Discord
    •  Coworkers
    •  Fellow Pythonistas
    •  Other (please specify)

This forum, Python YouTubers, Stack Overflow and the @python/typing GitHub repository are four other possible choices.

  1. How did you learn to use Python typing? (Select all that apply)

    •  Online tutorials
    •  Official documentation
    •  Blog posts
    •  Books
    •  Courses
    •  Code reviews
    •  Mentorship from colleagues
    •  Other (please specify)

Again, Stack Overflow is a likely choice; according to SO’s 2023 survey, 83% of respondents use Stack Overflow as a learning resource. The same survey also shows that three out of every five respondents learn from how-to videos.

2 Likes
3

I am not sure if this makes sense to add or where it would go, but I would be interested in seeing if folks add type annotations mainly to prevent bugs in code internal to the library OR for downstream users of their library.

6 Likes
4
  1. What type checking tools does your project use? (Select all that apply)
  • mypy
  • pyright
  • pytype
  • pyre
  • typeguard
  • pydantic
  • typeshed
  • Other (please specify)

Pylance is not listed. I’m guessing that many VS Code users are not savvy enough to know which type checker is used under the hood by default (ex. Pylance is based on Pyright).

Could we add something like “Pylance (VS Code default)”?

I’m not sure what PyCharm uses by default, but maybe a similar item for PyCharm would also make sense?

  1. Which IDE plugins/extensions do you use for typing? (Select all that apply)
  • MS.Python (pylance/pyright)
  • Mypy
  • PyCharm defaults
  • Not sure
  • Other (please specify)

I’m confused about how this question differs from question 3. But assuming both are kept:

  1. I’m guessing that “MS.Python” refers to the Python extension in VS Code? I think that that terminology is going to confuse readers. I’d suggest something like “Python VS Code extension (Pylance)” instead.
  2. Pyright is not run through the Python extension in VS Code, so I’d suggest that Pyright be a separate checkbox.
5

There are other tools that should be explicitly mentioned as well: Beartype , Pylyzer , Pyanalyze , etc.

Added

The “PyCharm” option should also mention other JetBrains IDEs, as they all use the same plugin under the hood.

I’m pulling these from the top IDEs used according to JetBrains survey. Given the low response on everything past vim, I don’t want to add too many options. I am open to expanding, but also want to keep it shorter if possible. (I still use and love Sublime BTW, but I realize there are fewer of us everyday)

Which IDE plugins/extensions do you use for typing? (Select all that apply)

I worry a little about getting into the weeds here. With that said, if there is an important distinction to be made here I am open to rewriting the question. Perhaps “Do you use a type checking extension? If so which one (short answer box)”

What are your main goals for typing your project? (Select all that apply)

I like this! How about as a separate question? I’m quite interested in the distinction here for use cases, but perhaps general usefulness is more important than use cases.

This forum, Python YouTubers, Stack Overflow and the @python/typing GitHub repository are four other possible choices.

Great additions!

Again, Stack Overflow is a likely choice;

Agreed. Will add.

6

I think this is a really interesting point. I will add it to “What are your main goals for typing your project? (Select all that apply)”.

1 Like
7

Random thoughts:

  • In question 2, I’m not quite sure how to interpret “lack of tool support”. Maybe this is more explicitly phrased as “shortcomings of type checking related tooling”?
  • In question 2, it might be worth trying to tease apart “not required for my projects” more. Currently that doesn’t give me enough “why”
  • Like InSync mentions, typeshed isn’t really a user-facing tool, just shared infrastructure. I’d just remove it from the list
  • In question 11, it could be worth putting discuss.python.org as an option (if only in case it helps more people find out about it)
1 Like
8

Updated, good suggestions!

9

There is an extension named “Python” from Microsoft that is not Pylance.

It says this about their relationship:

The Python extension will automatically install the following extensions by default to provide the best Python development experience in VS Code:

  • Pylance - to provide performant Python language support
  • Python Debugger - to provide a seamless debug experience with debugpy

These extensions are optional dependencies, meaning the Python extension will remain fully functional if they fail to be installed. Any or all of these extensions can be disabled or uninstalled at the expense of some features.

Many VSCodium users use this Microsoft “Python” extension without Pylance (since Pylance doesn’t work in VSCodium).

10

There is an extension named “Python” from Microsoft that is not Pylance.

Yup, I understand. I work on the Pylance team btw.

Many VSCodium users use this Microsoft “Python” extension without Pylance (since Pylance doesn’t work in VSCodium).

That’s true. I think that’s another argument that the current option of “MS.Python (pylance/pyright)” doesn’t make sense. Here’s my list:

  1. “MS.Python” doesn’t mean anything to users.
  2. Technically the Python extension doesn’t provide typing support itself. This may be pedantic.
  3. Python extension users on VSCodium don’t get Pylance or Pyright out of the box – they get Jedi.
  4. The Pyright VS Code extension works without the Python extension installed.
  5. Many Pyright users don’t use VS Code at all – they’re using Neovim, etc. What option will they select for this question?

At a higher level, what are the goals of question 5? What are we hoping to learn from the answers to this? It seems very similar to question 3 (type checking tools) or perhaps a cross of 3 and 4 (primary IDE).

4. Which IDE do you mainly use?

  • VS Code
  • PyCharm
  • Emac or (Neo)vim/VIM
  • Other (please specify)

Your point about VSCodium users above makes me think that we should add that to the IDE list.

11

There’s a fork of pyright that has support for vscodium, don’t know the specifics of all of that, but it seems like asking about IDE could very quickly turn into chasing details that may not be the most useful or that users might not even be 100% sure of how they are receiving typing related diagnostic info. (I’m not saying not to ask it, just to be careful about what you try to infer from it)

2 Likes
12

I think it might be interesting to have one or two questions about what developers are using Python for, to see if there are any differences in responses between categories. Taking inspiration from the annual JetBrains survey, my suggestion would be to ask (1) whether developers are mainly using Python for personal or work projects (or both), and/or (2) what type of development (data analysis, web development, etc.) they’re doing.

3 Likes
13

While I use typing heavily, on question 2 my own biggest pain point is not mentioned. I guess type check tooling is closest. Many packages I use are still untyped. For some of the key ones I ended up making my own stubs, but I’d expect most users to give up here.

There are still many packages in broader pypi ecosystem with no types. It’s definitely gotten much better over past couple years (only ~3ish years ago numpy mostly lacked typing), but some questions about what percent of libraries do they use come with types and which libraries they would most want type stubs (or inline types) added to would be useful. This also touches my own answer for 13 is biggest value gain I’d have for my team’s typing experience is increased typing coverage of ml/data ecosystems.

1 Like
14

Thanks for all of the feedback. I’ve put them survey in a form. I tried to keep it as accessible to all swaths of developers as possible and keep it short to increase the response rate. This may not address everyone’s questions, but I hope we can gain some insight from it and iterate on it as we see the results: https://forms.gle/XdSNR42LUuj58ghdA

2 Likes
15

@lolpack how long are you planning to run the survey for?

1 Like
16

Most of this thread has been about providing feedback on the survey design itself. @lolpack it may be worth creating a separate thread asking users to actually fill out the survey, to bring more attention to the survey.

5 Likes
17

As of now, I’m hoping to run it for all of August and release a full analysis in early September. Here are some early results for those interested:

  • 240 responses as of this post
  • Scripting/Automation: 71%, Web Development: 61%, Data Analysis: 47%, DevOps/Build/System Administration: 46%, AI/ML: 33%, Teaching/Learning: 18%
  • 58% say they ‘Always use type hints’ with only 2% saying ‘Never’ and 4% saying ‘Rarely’
  • ‘Not required for my projects’, ‘They’re too complex’, ‘Type check tooling is hard to use/set up’, ‘Lack of familiarity’ cited as top reasons for not using types (in that order)
  • Mypy (69%), Pyright (52%), Pydantic (44%) for the top type checking tools
  • Do you projects have type checking in CI? Yes: 60%, No: 35%, Not Sure: 5%
  • Better IDE support cited as ‘most useful’ reason to use types

I will post a more thorough analysis and make the data available for those interested when I have time next week. Here are some quick word clouds I made for the short answer portions:

LoveTypeSystem
LoveTypeSystem1000×500 210 KB

MissingFeature
MissingFeature1000×500 224 KB

HardestPartTypeSystem
HardestPartTypeSystem1000×500 225 KB

I’m open to it, but I also worry that the respondents skew toward heavy type system users already. Here is a blurb folks can use if they are okay sharing through social media or other channels they have access to distribute through:

📢 Calling all #Python developers! 🐍

We’re gathering insights on types in Python and need your help! :star2: Your input will help us improve the Type System and IDE tooling. Please take a moment to fill out our quick survey and share with other developers: https://forms.gle/XdSNR42LUuj58ghdA. Thank you!

18

What do you love about the Python type hints?

b u g s

19

I think basedpyright should get a mention in the type checkers section.

1 Like