Announcement: Pipenv Beta Release

Packaging
1

Pipenv (pre)release announcement

We have a new (pre)release out

I am happy to announce the (pre)release of pipenv v2020.4.1b1, available on PyPI for testing. To install: pip install --pre --upgrade pipenv.

To leave time for testing, we are targeting Thursday, 7 May for the final release - so please help us test this, and report your bugs soon.

What’s new

This release has a ton of bugfixes, performance improvements and enhancements, including:

  • Improved dependency and dependency conflict resolution for VCS dependencies, specified in Pipfile or as direct URLs in setup.py
  • Extensive improvements to output and error formatting
  • Better hash discovery, caching, & resolution
  • Fixes for encoding issues, input parsing/escaping, and various bugs (which could sometimes cause confusing issues when pipenv would search for python on your path)
  • Support for the environment management utility ASDF + improved support for pyenv
  • A whole lot more

For a draft of the changelog, see below. (After the release, it will go in our docs.)

What we need help testing

You can use this testing guide for some ideas and workflows to get you started.

If you have a bug to report, please file it at https://github.com/pypa/pipenv/issues . If you need to ask a question, please ask on https://stackoverflow.com/ (with the tag “pipenv”), or visit the #pypa channel on Freenode IRC https://webchat.freenode.net/?channels=%23pypa .

Other changes in the project

This is a little lengthy – feel free to skim/skip.

  • Process changes: I’ve been a single point of failure in the release pipeline, which is a reason this release is so delayed. To mitigate this bus factor, I’ve documented release process) and automated it. Even though Pipenv has had ongoing development, it has been impossible for anyone to push new releases to PyPI – addressing this is a top priority. Similarly, I need to be more of a reviewer and less of a PR author, to enable other contributors and reduce review lag.
  • Communications changes: We are looking ahead with an eye to collaboration and community engagement – we have had some past challenges and have been looking to improve how we approach our interactions. I personally joined the project because my coworkers who were new to Python struggled with specific workflows. I wanted to help address this genuine need I saw around me. I recognize that you are all, similarly, contributing your time and effort in good faith to make things better for everyone. We don’t all face the same challenges, but hopefully we can leverage our diverse backgrounds to build even better tools.
    • Looking forward, I am hoping to convey this as I engage with you all about how to make things better for users of Pipenv and of Python in general. Part of this means my role will likely shift somewhat, and I will focus more on engagement, review, triage, and onboarding new maintainers. If you see me writing a bunch of PRs instead of encouraging and reviewing others’, feel free to gently poke me about that. :slight_smile: Habits can be hard to break.
  • Release cadence & financial support: Pipenv hasn’t pushed a new release in over a year – that is super frustrating. Canonical is paying me and enabling me to work on Pipenv as part of my job (20% of the time), but that’s not enough to get us sustainably on a better release cadence, take care of bug reports and PRs, manage development on tricky bugs, and set us up for a future where I might not be around. In fact, this release was only possible because of the generous project management contributions donated by Changeset Consulting LLC. If your company uses Pipenv, ask them to sponsor Python packaging via the PSF so there’s a stronger fund for packaging work.

Thanks

  • All of the other maintainers, especially @frostming and @uranusjr, who keep the project running whether or not I’m around
  • Changeset Consulting LLC, particularly Sumana Harihareswara @brainwane, for donating a ton of project management help to get this release out
  • All of the contributors who have stewarded their changes through the sometimes frustrating pull request process (continuous integration is fixed now!)
  • Those of you who have filed issues or spoken to me directly to share your needs and use cases to help make the software better
  • Our users

Changelog for 2020.4.1b1 (2020-04-30)

Features & Improvements

  • Added support for resolution of direct-url dependencies in
    setup.py files to respect PEP-508 style URL dependencies. #3148
  • Added full support for resolution of all dependency types including
    direct URLs, zip archives, tarballs, etc.
    • Improved error handling and formatting.
    • Introduced improved cross platform stream wrappers for better
      stdout and stderr consistency. #3298
  • Pipenv will now successfully recursively lock VCS sub-dependencies. #3328
  • Added support for --verbose output to pipenv run. #3348
  • Pipenv will now discover and resolve the intrinsic dependencies of
    all VCS dependencies, whether they are editable or not, to
    prevent resolution conflicts. #3368
  • Added a new environment variable, PIPENV_RESOLVE_VCS, to toggle
    dependency resolution off for non-editable VCS, file, and URL based
    dependencies. #3577
  • Added the ability for Windows users to enable emojis by setting
    PIPENV_HIDE_EMOJIS=0. #3595
  • Allow overriding PIPENV_INSTALL_TIMEOUT environment variable (in
    seconds). #3652
  • Allow overriding PIP_EXISTS_ACTION evironment variable (value is
    passed to pip install). Possible values here:
    https://pip.pypa.io/en/stable/reference/pip/#exists-action-option
    Useful when you need to PIP_EXISTS_ACTION=i (ignore
    existing packages) - great for CI environments, where you need
    really fast setup. #3738
  • Pipenv will no longer forcibly override PIP_NO_DEPS on all vcs and
    file dependencies as resolution happens on these in a pre-lock step. #3763
  • Improved verbose logging output during pipenv lock will now stream
    output to the console while maintaining a spinner. #3810
  • Added support for automatic python installs via asdf and
    associated PIPENV_DONT_USE_ASDF environment variable. #4018

Behavior Changes

  • Make conservative checks of known exceptions when subprocess returns
    output, so user won’t see the whole traceback - just the error. #2553
  • Do not touch Pipfile early and rely on it so that one can do
    pipenv sync without a Pipfile. #3386
  • Re-enable --help option for pipenv run command. #3844
  • Make sure pipenv lock -r –pypi-mirror {MIRROR_URL}
    will respect the pypi-mirror in requirements output. #4199

Bug Fixes

  • Raise PipenvUsageError when [[source]] does not
    contain url field. #2373
  • Fixed a bug which caused editable package resolution to sometimes
    fail with an unhelpful setuptools-related error message. #2722
  • Fixed an issue which caused errors due to reliance on the system
    utilities which and where which may not always exist on some
    systems.
    • Fixed a bug which caused periodic failures in python discovery
      when executables named python were not present on the target
      $PATH. #2783
  • Dependency resolution now writes hashes for local and remote files
    to the lockfile. #3053
  • Fixed a bug which prevented pipenv graph from correctly showing
    all dependencies when running from within pipenv shell. #3071
  • Fixed resolution of direct-url dependencies in setup.py files to
    respect PEP-508 style URL dependencies. #3148
  • Fixed a bug which caused failures in warning reporting when running
    pipenv inside a virtualenv under some circumstances.
    • Fixed a bug with package discovery when running pipenv clean. #3298
  • Quote command arguments with carets (^) on Windows to work around
    unintended shell escapes. #3307
  • Handle alternate names for UTF-8 encoding. #3313
  • Abort pipenv before adding the non-exist package to Pipfile. #3318
  • Don’t normalize the package name user passes in. #3324
  • Fix a bug where custom virtualenv can not be activated with pipenv
    shell #3339
  • Fix a bug that --site-packages flag is not recognized. #3351
  • Fix a bug where pipenv --clear is not working #3353
  • Fix unhashable type error during
    $ pipenv install --selective-upgrade #3384
  • Fixed a keyerror which could occur when locking VCS dependencies in
    some cases. #3404
  • Fixed a bug that ValidationError is thrown when some fields are
    missing in source section. #3427
  • Updated the index names in lock file when source name in Pipfile is
    changed. #3449
  • Fixed an issue which caused pipenv install --help to show
    duplicate entries for --pre. #3479
  • Fix bug causing [SSL: CERTIFICATE_VERIFY_FAILED] when Pipfile
    [[source]] has verify_ssl=false and url with custom port. #3502
  • Fix sync --sequential ignoring pip install errors and logs. #3537
  • Fix the issue that lock file can’t be created when PIPENV_PIPFILE
    is not under working directory. #3584
  • Pipenv will no longer inadvertently set editable=True on all vcs
    dependencies. #3647
  • The --keep-outdated argument to pipenv install and pipenv lock
    will now drop specifier constraints when encountering editable
    dependencies.
    • In addition, --keep-outdated will retain specifiers that would
      otherwise be dropped from any entries that have not been
      updated. #3656
  • Fixed a bug which sometimes caused pipenv to fail to respect the
    --site-packages flag when passed with pipenv install. #3718
  • Normalize the package names to lowercase when comparing used and
    in-Pipfile packages. #3745
  • pipenv update --outdated will now correctly handle comparisons
    between pre/post-releases and normal releases. #3766
  • Fixed a KeyError which could occur when pinning outdated VCS
    dependencies via pipenv lock --keep-outdated. #3768
  • Resolved an issue which caused resolution to fail when encountering
    poorly formatted python_version markers in setup.py and
    setup.cfg files. #3786
  • Fix a bug that installation errors are displayed as a list. #3794
  • Update pythonfinder to fix a problem that python.exe will be
    mistakenly chosen for virtualenv creation under WSL. #3807
  • Fixed several bugs which could prevent editable VCS dependencies
    from being installed into target environments, even when reporting
    successful installation. #3809
  • pipenv check --system should find the correct Python interpreter
    when python does not exist on the system. #3819
  • Resolve the symlinks when the path is absolute. #3842
  • Pass --pre and --clear options to pipenv update --outdated. #3879
  • Honor PIPENV_SPINNER environment variable #4045
  • Fixed an issue with pipenv check failing due to an invalid API key
    from pyup.io. #4188

Vendored Libraries

  • Updated pip_shims to support --outdated with new pip versions. #3766

  • Update vendored dependencies and invocations

    • Update vendored and patched dependencies
      • Update patches on piptools, pip, pip-shims, tomlkit
    • Fix invocations of dependencies
      • Fix custom InstallCommand instantiation
      • Update PackageFinder usage
      • Fix Bool stringify attempts from tomlkit

    Updated vendored dependencies:

      - **attrs**: `18.2.0` => `19.1.0`
  - **certifi**: `2018.10.15` => `2019.3.9`
  - **cached_property**: `1.4.3` => `1.5.1`
  - **cerberus**: `1.2.0` => `1.3.1`
  - **click**: `7.0.0` => `7.1.1`
  - **click-completion**: `0.5.0` => `0.5.1`
  - **colorama**: `0.3.9` => `0.4.3`
  - **contextlib2**: `(new)` => `0.6.0.post1`
  - **distlib**: `0.2.8` => `0.2.9`
  - **funcsigs**: `(new)` => `1.0.2`
  - **importlib_metadata** `1.3.0` => `1.5.1`
  - **importlib-resources**: `(new)` => `1.4.0`
  - **idna**: `2.7` => `2.9`
  - **jinja2**: `2.10.0` => `2.11.1`
  - **markupsafe**: `1.0` => `1.1.1`
  - **more-itertools**: `(new)` => `5.0.0`
  - **orderedmultidict**: `(new)` => `1.0`
  - **packaging**: `18.0` => `19.0`
  - **parse**: `1.9.0` => `1.15.0`
  - **pathlib2**: `2.3.2` => `2.3.3`
  - **pep517**: `(new)` => `0.5.0`
  - **pexpect**: `4.6.0` => `4.8.0`
  - **pip-shims**: `0.2.0` => `0.5.1`
  - **pipdeptree**: `0.13.0` => `0.13.2`
  - **pyparsing**: `2.2.2` => `2.4.6`
  - **python-dotenv**: `0.9.1` => `0.10.2`
  - **pythonfinder**: `1.1.10` => `1.2.2`
  - **pytoml**: `(new)` => `0.1.20`
  - **requests**: `2.20.1` => `2.23.0`
  - **requirementslib**: `1.3.3` => `1.5.4`
  - **scandir**: `1.9.0` => `1.10.0`
  - **shellingham**: `1.2.7` => `1.3.2`
  - **six**: `1.11.0` => `1.14.0`
  - **tomlkit**: `0.5.2` => `0.5.11`
  - **urllib3**: `1.24` => `1.25.8`
  - **vistir**: `0.3.0` => `0.5.0`
  - **yaspin**: `0.14.0` => `0.14.3`
  - **zipp**: `0.6.0`
    • Removed vendored dependency cursor. #4169

  • Add and update vendored dependencies to accommodate safety
    vendoring:

    • safety (none) => 1.8.7
    • dparse (none) => 0.5.0
    • pyyaml (none) => 5.3.1
    • urllib3 1.25.8 => 1.25.9
    • certifi 2019.11.28 => 2020.4.5.1
    • pyparsing 2.4.6 => 2.4.7
    • resolvelib 0.2.2 => 0.3.0
    • importlib-metadata 1.5.1 => 1.6.0
    • pip-shims 0.5.1 => 0.5.2
    • requirementslib 1.5.5 => 1.5.6 #4188

  • Updated vendored pip => 20.0.2 and pip-tools => 5.0.0. #4215

Improved Documentation

  • Added documenation about variable expansion in Pipfile entries. #2317
  • Consolidate all contributing docs in the rst file #3120
  • Update the out-dated manual page. #3246
  • Move CLI docs to its own page. #3346
  • Replace (non-existant) video on docs index.rst with equivalent gif. #3499
  • Clarify wording in Basic Usage example on using double quotes to
    escape shell redirection #3522
  • Ensure docs show navigation on small-screen devices #3527
  • Added a link to the TOML Spec under General Recommendations &
    Version Control to clarify how Pipfiles should be written. #3629
  • Updated the documentation with the new pytest entrypoint. #3759
  • Fix link to GIF in README.md demonstrating Pipenv’s usage, and add
    descriptive alt text. #3911
  • Added a line describing potential issues in fancy extension. #3912
  • Documental description of how Pipfile works and association with
    Pipenv. #3913
  • Clarify the proper value of python_version and
    python_full_version. #3914
  • Write description for --deploy extension and few extensions
    differences. #3915
  • More documentation for .env files #4100
  • Updated documentation to point to working links. #4137
  • Replace docs.pipenv.org with pipenv.pypa.io #4167
10 Likes
2

Congrats! And really appreciate your openness about the issues and difficulties managing a significant project.

2 Likes
3

Per Work towards next pipenv release , Pipenv 2020.5.28 is now out!