Creating project files from a package's command line interface

Python Help
,
1

I made a Python package that is a static site generator with a command line interface to build the site and perform other operations. I would like to add a CLI command that generates a new project that is used to create a site. For example, running the command mypkg new would create a directory with the following content:

mysite/
├── docs/
│   ├── images/
│   │   └── apple.png
│   └── styles.css
├── posts/
│   ├── fruit.md
│   └── veggie.md
├── pages/
│   └── about.md
├── templates/
│   ├── post.html
│   ├── page.html
│   └── index.html
└── config.toml

Some ideas I have on how to implement this are:

  • Put the needed files in the package itself then copy them to the user’s computer when the mypkg new command is run.
  • Have the contents of the files as strings in the package then write them out to file when the user runs the mypkg new command.

Which approach should I take or is there a completely different way I could accomplish this?

2
3

That’s what I would do. Then it is easier with source code versioning, syntax highlighter, and so on.

4

I made an example package to test copying the files from within the package. The structure of the package is shown below. The static directory contains the files that I want to copy to the user’s working directory.

mypackage/
├── src/
│   └── mypackage/
│       ├── static/
│       │   ├── __init__.py
│       │   ├── config.toml
│       │   ├── index.html
│       │   └── post.html
│       ├── __init__.py
│       └── cli.py
└── pyproject.toml

The pyproject.toml contains the following:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "mypackage"
version = "24.11"
authors = [{name = "Bart Simpson"}]
description = "A small example package"
requires-python = ">=3.10"

[project.scripts]
mypkg = "mypackage.cli:main"

The cli.py contains the following:

import argparse
import importlib.resources
import pathlib
import shutil

from . import static

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("command", choices=["new"], help="commands")
    args = parser.parse_args()

    if args.command == "new":
        static_dir = importlib.resources.files(static)

        config_toml = static_dir / "config.toml"
        index_html = static_dir / "index.html"
        post_html = static_dir / "post.html"

        shutil.copy(f"{config_toml}", ".")

        pathlib.Path("./web").mkdir(exist_ok=True)
        shutil.copy(f"{index_html}", "./web")
        shutil.copy(f"{post_html}", "./web")

Running cd my-project then mypkg new on the command line creates the following:

my-project/
├── web/
│   ├── index.html
│   └── post.html
└── config.toml

This seems to be doing what I need. But I have a few questions:

  1. The importlib.resources.files creates a Traversable so what is the proper way to handle that as a path? In my example, casting it as a string seems to work but is there a better approach?
  2. Instead of creating a directory then copying a file into the directory, is there a way to copy the entire directory including its contents?
  3. Will the static directory be included with the package when I publish it to PyPI or do I need to explicitly include it using the pyproject.toml file?
5

Building a wheel, and installing that wheel in a fresh new virtual environment is probably your best way to verify that.

I recommend to play around with the importlib.resources API until you find what fits best your use case. I do not recall that it allows copy whole directory but maybe it does.

6

Verified that the static directory is included with the package after installing it with pip install . in a virtual environment. So it looks like I don’t need to explicitly include it in the pyproject file. I guess it works because it’s already in the src directory as a subpackage.

7

What files get automatically included in the wheel (or the sdist) is entirely dependent on the build backend. I see that your build backend is hatch/hatchling, and for this build backend the relevant part of its documentation seems to be “file selection”.

8

Just to back up a second, have you heard of cookiecutter? It seems like it does everything you want with this tool so you don’t have to reinvent the wheel.

If you have heard of it, can you explain what you want your tool to do that cookiecutter can’t do?

9

Yes, I am aware of cookiecutter. But I don’t want to rely on an external package just to copy files. I can use standard Python features to do this which I demonstrated in the example above.

10

Yes, I’m using hatchling for the build backend but I haven’t had to explicitly declare anything in the pyproject file to include data files in the package. So I guess hatchling automatically includes the files as long as they are in the src/package directory.

11

Probably. I do not know what the exact logic is. I also seem to recall that hatch tries to follow what’s in the .gitignore file if there is one (if you tell git to ignore something, then hatchling will also ignore it and this file will not be in the distribution files). But I am not sure, you really should check the documentation I linked.