Rules on bytecode generation

Documentation
1

From experimentation/inspection, it seems the CPython interpreter’s rules for writing bytecode into a __pycache__/ directory are:

  1. Bytecode will never be written for Python code executed as a script filename passed to the interpreter:

    python3.13 ./my_code.py

  2. Bytecode will never be written when a script is executed via a shebang

    ./my_code.py  # A file beginning '#!/usr/bin/env python3.13'
              # with the executable permission bit set
./my_code  # Same as the previous, just renamed

  3. Bytecode will never be written when a script is executed via runpy.run_path()

    >>> import runpy
>>> runpy.run_path('./my_code.py')

  4. Bytecode will be written IFF a .py file is used as a module, provided the user has permission to create or write to the directory located at

    $(dirname $(realpath module_file.py))/__pycache__/

    Any of the following will attempt to create the bytecode file ./__pycache__/my_code.cpython-313.pyc:

    python3.13 -c 'import my_code'
python3.13 -m my_code
echo "import my_code" > my_wrapper.py \
  && python3.13 ./my_wrapper.py

    >>> import runpy
>>> runpy.run_module('my_code')

  5. (Perhaps just as significantly, Python will only ever try to read a bytecode file under the same rules that govern when it would write one. i.e. python3.13 my_code.py will never attempt to read ./__pycache__/my_code.cpython-313.pyc, even if it exists and is up to date.)

…So, that’s all well and good, but are the rules documented explicitly anywhere? And are they even rules of the language itself, or specific to the CPython implementation?

Best I can tell, the docs only address bytecode-file output in the command-line docs for the interpreter, when discussing the PYTHONDONTWRITEBYTECODE environment variable and the -B flag. Those descriptions only sort of obliquely allude to the majority of the observed behaviors, saying (for example) of -B…

If given, Python won’t try to write .pyc files on the import of source modules.

You could make an argument for that one sentence covering, by implication, all of the scenarios presented in my numbered statements above — well, except 5 which doesn’t appear to have any documentation, perhaps as that really is considered just an implementation detail.

Still, I wonder if a bit more explicit-better-than-implicit documentation of the whole bytecode-file “thang” might be warranted?

(Full disclosure: This post was largely inspired by a review comment on a KDE change expressing concern about Python attempting to create a bytecode file in a context where it never would. Because the rules on that are often poorly understood. Possibly because they’re documented poorly, if at all.)

3 Likes
2

Correct: .pyc files are a part of the import system. But you can do it via code if you want (e.g. at installation time, pip will do it for you).

It’s actually considered an implementation detail of CPython, so I don’t know how critical it is to document. While it hasn’t changed in a while, it has changed (e.g. dropping .pyo files, the creation of __pycache__ itself, etc.).

I don’t think we spell them out since they are an implementation detail.

4 Likes