Python Packaging in GSoC 2020

Google Summer of Code 2020 has been announced.

Students work with an open source organization on a 3 month programming project during their break from school.

Do we have projects related to Python Packaging, that are the right kind1 for GSoC 2020? If yes, do we have mentors available for such tasks?

Let’s start compiling a list of projects and figuring out mentors for them. Please leave a comment below if you can think of a project that’s compatible. If we have more than two projects with mentors, I’ll reach out to PSF’s GSoC admins, to become a sub-org of the PSF in mid-Jan 2020.

We’re accepting a limited number of new sub-orgs through March 5th! Email gsoc-admins(at)python.org if interested.


1 Ideally, the project ideas for the students to work on would:

  • be well-defined and well-scoped projects
  • be ~8 weeks of full time work from a junior developer
    • some students will have overlapping college days, during start/end of GSoC’s coding period.
  • allow for stretch goals
    • in case the student does achieve the main task earlier than 12 weeks
5 Likes

There are two migration tasks that seem like they would be useful but I am not sure if they are sized correctly, and I can’t really commit to mentoring for them next summer (but I can try):

  1. Migration of distutils out of the standard library and into setuptools
  2. Merging of wheel into setuptools.

One thing I’ve been thinking about lately is that with the constantly shifting sands of Python packaging, I am constantly finding packages that use outdated packaging advice. It might be nice to have some tools for automated code cleanup (or at least linting) that would do things like:

  1. Convert your setup.py into a setup.cfg in the trivial case.
  2. Add a pyproject.toml (populated with setup_requires if necessary)
  3. Remove setup.py upload from make files if they exist.
  4. (Ambitious) Convert if / else code in install_requires over to using env markers.
  5. (Ambitious) Convert a project over to using a src/ layout

I think at least the first 2 of these would be useful, and we can probably add in a Github bot that makes a PR for you if you call it (e.g. open an issue that says “@pkg_fixer_bot: Fix my build” and it comes around with a PR.

Semi-related to this would be if we can add pyproject.toml support in setuptools, along with a way to convert setup.cfg files to pyproject.toml files, that would be very nice.

It seems like Fundable packaging improvements would be another good place to mine for ideas. Specifically package preview, metadata audit/update and robust interoperability testing seem achievable by the right students.

3 Likes

I feel like there are quite a few GSoC-sized projects just lingering within pip’s codebase.

pip’s test suite can definitely use a lot of TLC. Similarly, we can also bundle our “switch to this other library” tasks to be a GSoC sized project (although, it’d be composed of a lot of smaller tasks):

  1. Refactoring infrastructure around pip’s tests

    • setting up code coverage metric tracking
    • decoupling certain bits and pieces into independent libraries
    • improving our pytest fixtures/configuration
    • better code organization
  2. Improving pip’s tests

    • removing redundant/no-op tests
    • enhancing existing tests
      • making them self-contained
    • writing new tests (maybe?)
    • improving test suite run times
  3. Switch out libraries that pip’s using for newer/better supported variants

    • optparse to click/argparse/something else – #4659
    • pytoml to tomlkit – #6120
    • retrying to tenacity – #7530
    • more?

There are also a whole bunch of improvements we can have around CI tooling, automation and more – all of which would make for good stretch goals.

2 Likes

I’ve contacted the GSoC Admins to apply as a GSoC sub-org under the PSF. I’ll be filling a proper form containing all the details (including an “Ideas page” location) later this week. If no project other than pip is interested, I’ll fill in the details to be specific to pip itself.

If you’re a maintainer of a Python Packaging related project and are willing to participate in GSoC 2020 as a mentor, the next couple of days would be a really good time to let me know that you’re interested.

3 Likes

While technically, tox is not a PyPa project, I would be willing to participate with its rewrite as mentor.

Alternatively, I think we could make editable installs as a GSOC project too, and myself and @pganssle (haven’t asked him yet, but I’m wishful) maybe can co-mentor on that.

6 Likes

I would love to mentor or even participate if i am allowed.

2 Likes

Hello everyone, I am an undergraduate student, I would love to contribute to pip as said by @pradyunsg. please point me to some resources to get started.

2 Likes

I hope to see the editable installs working with pyproject.toml because that aspect drives me crazy. From the perspectice of someone that packages mutiple pure-python libraries, I can say that all these atempts to fix the setup.py issue produced far more pain and wishes me to not ever tried to go the “toml” way.

Those that broke deeply were not those that had a setup.py but those that adventured to have a pyproject.toml one. Installing a package takes considerably more time now and even worse, no longer being able to perform editable installs, something that is a productivity killer. You can no longer test making a patch to a python based tool because you have to reinstall it after each modification.

2 Likes

Python Packaging Authority has applied to be a sub-organization under the Python Software Foundation for GSoC 2020. Our project ideas page has also been published publicly:


Students: Take a look at the ideas page for getting started on Python Packaging related projects, and https://python-gsoc.org/index.html#apply, for a checklist provided by the Python Software Foundation on how to apply for GSoC. Python Packaging Authority will be listed on the PSF’s sub-organizations page in a few days.

4 Likes

@rohitsanj That’s great! Please take a look at https://github.com/pypa/packaging.python.org/wiki/GSoC-2020-Ideas-Page#getting-started.

2 Likes

@pradyunsg I am interested to work upon improving the pip’s test helper. I have experience with Python in general and have written tests using Pytest and Python’s mock library. Apart from it, I have the drive to contribute to open source in general. I would be glad to be a part of pip.

Looking forward to contributing to pip.

Some testing related work in other OSS project

[1] django-phone-verify test refactor
[2] nexus-linkedin-integration-tests

Thanks.

3 Likes

As a part of learning, I write blogs. Some testing related blogs, I have written

[1] Python testing with pytest
[2] Mock Testing in Python

3 Likes

Hi @gutsytechster! Please take a look at https://github.com/pypa/packaging.python.org/wiki/GSoC-2020-Ideas-Page#getting-started for details on how to get started. :slight_smile:

2 Likes