Referring to packages as "modules"

Documentation
1

[Note: I have had to strip most of the citation links from this post, as the interface won’t allow me to include more than two links. :thinking:]

Technically speaking, a Python package is a module. However, throughout the Python documentation and the PEPs we find abundant instances of language which implies that “modules” by itself would not include packages. For example:

Python supports modules and packages …

… or in the unittest documentation:

… all of the test files must be modules or packages …

… or in the official Python tutorial:

A module is a file containing Python definitions and statements.

The result of these discrepancies is that (at least in my experience) the average Python user assumes that “modules” does not include packages. That’s usually not a serious problem, but it does result in some occasional head-scratching, for example as when reading this paragraph in the Django documentation:

Once you have defined your models, you need to tell Django you’re going to use those models. Do this by editing your settings file and changing the INSTALLED_APPS setting to add the name of the module that contains your models.py.

Strictly. speaking, the use of the word “module” in that passage is correct, but if the reader has just finished reading the official Django tutorial, where that location is referred to as a “package,” I can easily imagine this being a “Wait, what?” moment.

I wonder if it might be helpful to acknowledge this second (arguably more common) meaning of the word “module” explicitly somewhere in the Python documentation (perhaps in the glossary) with some encouragement to use “package” instead when the word in question can only refer to a package, and to use “module or package” when referring to a module which might or might not also be a package.

What do others think?

1 Like
2

Modules don’t include packages. A module is a single importable .py file. A package is an importable directory, usually containing an __init__.py. The two never overlap.

3

Modules don’t include packages.

In common parlance, I agree that this is true. However, this is the official Python definition of package:

A Python module which can contain submodules or recursively, subpackages. Technically, a package is a Python module with a __path__ attribute.

This definition is echoed in the language reference:

It’s important to keep in mind that all packages are modules, but not all modules are packages. Or put another way, packages are just a special kind of module. Specifically, any module that contains a __path__ attribute is considered a package.

It’s this language putting forth a different interpretation which makes it a little more difficult to push back (for example) with the maintainers of the Django documentation that the use of the word “module” in the passage I quoted above in my first message is misleading or at least confusing.

1 Like
4

In common usage, there’s a fair amount of overlap. But that’s the result of people speaking imprecisely (because we’re having discussions, not writing reference manuals!) rather than because the definitions need fixing.

This isn’t accurate (although a single importable .py file underlies the vast majority of modules that aren’t packages)[1]. A module is an instance of the code ModuleType type, which is created by the import system. A package[2] is a particular type of module which acts as a container for other modules. As per the section of the documentation that @bkline quoted, what makes a module a package is the presence of a __path__ attribute.

There’s historical baggage involved here as well. Originally, Python only supported modules, and there was no nesting at all. When nested modules were introduced, the term “package” was added to distinguish these “new style” objects. And I can’t remember the details now, but it’s quite possible that at the time, packages weren’t the same (in an implementation sense) as modules. A lot of the documentation has existed for a long time, and quite possibly inherited that distinction from then.

There’s also parallel, but subtly different, terminology for what exists on the file system. A module, in that context, is an importable .py file, and a package is a directory with an __init__.py file[3]. In this context, a package and a file are different.

It’s also worth remembering that sometimes, even though packages are a type of module, it’s worth being redundant to make things clearer - “xxx supports modules and packages” makes it explicit that packages (nested modules) are supported. It costs very little, and saves any chance of confusion for people thinking in terms of the filesystem terminology.

Personally, I think the Python docs are probably fine, although they are less formal than maybe some people would prefer. I think the quoted Django documentation is wrong. In terms of the import system, modules don’t contain .py files, and in terms of the filesystem, packages contain other .py files, not modules. I’d suggest that the simplest fix is to replace the term “module” with “package” and move on - although strictly speaking, it’s still wrong, as you add the name of the import system module, and import system modules don’t contain .py files[4].

The reality is that the words “package” and “module” get used to cover a variety of concepts, simply because there aren’t enough good words to go round. So there’s some level of imprecision, and mostly people can just live with that.

An incidental anecdote - the packaging ecosystem went around this a few times with the word “package”. There’s import system packages, filesystem packages, distributable packages, etc., etc. We tried defining precise terms for them all - “import package”, “distribution package”, … No-one used them. And yet, we still manage to communicate successfully. As I say, not every interaction is a legally precise document :slightly_smiling_face:

  1. But see below, about the filesystem interpretation ↩︎

  2. In the import system sense, not the package distribution sense!!! ↩︎

  3. There’s also implicit namespace packages, which are just directories, but let’s not get even more confusing… ↩︎

  4. What if you implemented a module hook that loaded your models module from somewhere other than a .py file? ↩︎

10 Likes
5

Thanks, Paul. This is helpful background information. I’ll do my best to convince the Django folks that this passage in their documentation needs to be fixed, even though parts of the Python docs might appear to support the existing language in their view.

… although strictly speaking, it’s still wrong, as you add the name of the import system module, and import system modules don’t contain .py files

Would this version address that concern?

… add the name of the package which contains your models module.

1 Like
6

Apart from the fact that “model module” just sounds weird, I don’t think it helps. From what I understand, Django people tend to think in terms of model.py, not “the model module”.

I think you’re trying too hard to emphasise precision. Readability is more important.

7

As a dev using django, this sounds clear to me – but fix this to: models module !

1 Like
8

I think you’re trying too hard to emphasise precision.

Not my intention. Since you had taken the trouble to point out the remaining fIaw in the wording I was just responding to that comment. Basically I’m just trying to eliminate the confusion which that Django documentation is going to present to any reader whose understanding of modules and packages matches Brénainn’s (which is most of them, in my experience).

1 Like
9

Oops, right. Silly typo on my part. Fixed in the comment above. Though I’m inclined to agree with Paul that just changing “module” to “package” is all that really needs to happen.

Thanks for catching that!

10

Sigh. I carried the conversation over to the Django forum, asked for feedback, offered to create a pull request, got one response encouraging me to do so, waited a few days to make sure there were no objections, and proceeded with a ticket and PR. It was closed pretty quickly by one of the maintainers, with the justification that in the Django world they don’t want to use the word “package” for something which isn’t “packaged for distribution” (which seems odd, given that the first page of the official Django tutorial goes to the trouble of pointing out that any directory containing an __init__.py file is “a Python package”). :thinking:

Oh well, thanks anyway for the feedback. :folded_hands:

2 Likes