3.12.3 --> 3.12.4 regression? [deep rabbit hole warning]

Python Help
, ,
1

I’m working on docs for a project, and while building them, I stumbled upon a case where sphinx (and a bunch of extensions) works fine under 3.12.3 and fails under 3.12.4.

Same OS (tested: Linux and macOS), same arch (tested: arm64), same deps (pinned).

I’ve made a reproducer: GitHub - dimaqq/MRE-sphinx-PurePath: Reproducer for Sphinx regression under Python 3.12.4

(all you need to try it out: pyenv and tox)

Sphinx maintainers commented that my Sphinx version is out of support, and that’s fair, as I was able to get docs to build with much newer Sphinx.

I’ve spent a fair amount of time just tracking this down, and the docs are rather complex, with custom config, many deps, etc.

Perhaps this all is obvious for a Sphinx expert?

Or maybe some Python (core?) dev would like to validate that it’s not a Python regression?

  • Py 3.12.3 – OK
  • Py 3.12.4 – 2 errors, which in itself is odd (*)
  • PY 3.130b4 – many errors of similar kind (**)

(*) given from pathlib import PurePath and many uses of PurePath in type hints, Sphinx/autodoc somehow gives warnings for only 2 uses.
(**) similar kind, though not exactly same, can’t find pathlib._local.Path, etc.

Edit: I’ve made the MRE repo much smaller by cutting out most of the code, tests, and pyproject.toml which seemed to confused a few here.

2

For the MRE - you installed the packages globally.

Better to create separate venvs for 3.12.3 and 3.12.4, install the packages directly in the venvs, and then provide/compare the pip freeze for both venvs to ensure you’re comparing apples to apples.

1 Like
3

Neither you post nor the linked stuff say what out-of-support Sphinx version you are using. Hypothesis: that version depends on a bug, perhaps in pathlib, which got a lot of patches in the last year, some backported, that was fixed in 3.2.4. Why don’t you just use the newer sphinx that works?

2 Likes
4

Looks like 6.2.1, but I agree that upgrading seems like the most reasonable solution.

If the question is just “what changed that could cause this” I would start with the changelog between 3.12.3 and 3.12.4

1 Like
5

git has a binary search function that, given a range of merges, will find the one that caused a behavior change. I have never used it, but since this could look for a change in subprocess return code, it should be doable.

2 Likes
6

git bisect is the git command that Terry mentioned for isolating where a change occurred.

It looks like the project dependencies are pinned, but the optional dependencies (Sphinx and the extensions) are not all pinned. I would recommend doing a pip freeze on both versions. My gut instinct is that one of the extensions for docs is differently versioned.

2 Likes
7

Thank you for the suggestions, I shall certainly rely on your experience.

Though if someone else gets an itch to poke at this, that would be amazing!

That’s an interesting idea, because I could automate that. :thinking:

Another way I was considering would be to manually copy Python files between 3.12.3 and 3.12.4 stdlib to narrow down the issue.

They are, although in a non-standard location: MRE-sphinx-PurePath/docs/requirements.txt at main · dimaqq/MRE-sphinx-PurePath · GitHub

I did read it, but I couldn’t find any obvious culprit. Thus this post.

I’m pretty sure that tox whips up a venv on the fly. I can see it and inspect it.

I am installing pinned dependencies, same in both environments and I have validated that what gets installed is in fact the same. Only Python is different.

I have moved to a newer major version that doesn’t have this error.

Note the subject of this thread, it’s a deep rabbit hole that someone may want to go down. I may as well given some ideas in this thread.

8

Actually now that I am looking closer, there are optional dependencies in pyproject.toml and also a frozen requirements in the docs directory. When doing the autodoc are the same dependencies being used?

Another interesting thing that the example does is use a custom conf.py for autodoc that is different than the main conf.py.

9

One more thought on the @dimaqq is that it looks like the failure is due to intersphinx. Can you run this: sphinx.ext.intersphinx – Link to other projects’ documentation — Sphinx documentation in each environment and check if the diff is the same.

2 Likes
10

intersphinx generates identical docs/_build/html/objects.inv under the two Python versions and parses these files identically as well.

https://docs.python.org/3/objects.inv is parsed by interphinx in exactly same way under the two Python versions as well. I guess that’s not surprising.

I’ll bisect on cpython 3.12 branch and see where I get with that.

1 Like
11

Wow, git bisect surprised me once again.
(I guess we mere mortals often fail to understand the power of the exponential / the usefulness of the logarithmical)

first bad commit: [5430f614371530aab1178b3b610add4bf54c383e] [3.12] gh-114053: Fix bad interaction of PEP-695, PEP-563 and get_type_hints (#118009) (#118104)

1 Like
12

Before the backport, Sphinx/autodoc parses the regular methods correctly:

.. py:method:: Container.make_dir(path: str | ~pathlib.PurePath, *, make_parents: bool = False, permissions: ...

And type overloads for methods correctly:

.. py:method:: Container.pull(path: str | ~pathlib.PurePath, *, encoding: None) -> ~typing.BinaryIO
               Container.pull(path: str | ~pathlib.PurePath, *, encoding: str = 'utf-8') -> ~typing.TextIO

After the backport, regular methods are still parsed correctly:

.. py:method:: Container.make_dir(path: str | ~pathlib.PurePath, *, make_parents: bool = False, permissions: ...

But type overloads for methods are not:

.. py:method:: Container.pull(path: Union[str, PurePath], *, encoding: None) -> BinaryIO
               Container.pull(path: Union[str, PurePath], *, encoding: str = 'utf-8') -> TextIO

Here Sphinx seems to keep the signature verbatim as it’s stated in the source code.

Note that the code in question has to work under py3.8~3.13, so we use older-style annotations, like Union[str, PurePath. These seems to be decoded by Sphinx running under Python 3.12.3 into str | ~pathlib.PurePath.

The regression is that, somehow, the type annotation is not decoded specifically in overloads.

It’s also possible that Sphinx tries to decode this, hits an exception, and keeps the literal type hint / signature as a fallback.

2 Likes
13

Thanks for checking. That makes me feel better :smile:

14

The regression seems to be basically that Sphix 6.2.1 tried to run code equivalent to below for overloads, and that works in 3.12.3 and doesn’t in 3.12.4 due to a signature change:

import typing
annotation = 'Union[str, PurePath]'
fr = typing.ForwardRef(annotation, True)
print(fr._evaluate(globals(), locals(), frozenset())) 

# There is a new argument in 3.12.4
# The new signature is: _evaluate(self, globalns, localns, type_params, *, recursive_guard)

This happens in sphinx.util.inspect.evaluate_signature, which is called by sphinx.ext.autodoc.MethodDocumenter.format_signature to handle overloads. That function has an exception guard that results in the signature remaining a string instead of a becoming a type.

I think that’s it. You can test this by editing evaluate_forwardref inside sphinx.util.inspect.evaluate_signature and adding an if branch for 3.12.4 to call ref._evaluate with the correct arguments.

3 Likes
15

That explains it, thank you, Daniel!

16

To complete the story, here is the relevant fix in Sphinx:

A

2 Likes
17

Thank you @AA-Turner that’s totally it.

Wrt. Python, how do we normally go about this?

Given that it’s a change in a _ “private” method that’s undocumented in py 3.12, itshouldn’t count as a breaking change, yet this is the newest mainline Python release. I’d be tempted to contribute something to the docs about the change, but there’s nowhere to plug that in.

Wrt. application, I’ll ask Sphinx is they want to a) do nothing, b) make a patch release for 6 series, or c) state that py 3.12.4 is not supported.

18

The version of Sphinx in question is already out of support, so I’m pretty sure they’ll go for “do nothing”.

1 Like