Best practices for packaging Python + JavaScript projects

Hi all, I’m wondering what are the best practices when packaging Python + JavaScript projects?

I know that Jupyter started using Hatch and created their own extension Packaging for Jupyter in 2022. Packaging has been difficult and… | by Steven Silvester | Jupyter Blog

Basically I’m trying to address some problems in GitHub - kedro-org/kedro-viz: Visualise your Kedro data and machine-learning pipelines and track your experiments., because now our sdists are not self-contained, and installing from GitHub cannot be done in one step because there’s a make build command required to build the JS assets.

Any recommendations or examples are more than welcome.

Historically, the most trotted approach is to simply treat the JavaScript end result the same as “binary” files (or more broadly, data files). Most users are directed to install wheels, and building from source (either sdist or an ad-hoc source tree) is an exercise targeting more/very/extremely experienced people. This would put the package in a similar position to e.g. Numpy and Pandas, except you have JavaScript code instead of CPython extension modules.

Just noting that even in the case of sdists, the built JS components can be bundled pre-built inside the sdist. This is what we do for Bokeh, for instance. We simply don’t want to tolerate any risk that a downstream packager uses a slightly different Typescript compiler or JS toolchain and ends up with a slightly different BokehJS that does not match exactly what we publish to CDN, since those kinds of bug reports would be a nightmare to deal with.

Hi @astrojuanlu! Adding to what Bryan said, in Jupyter we also include JS files in the sdist for similar reasons. If you want a build step to work from a git install, you’ll need a build backend that supports them. Previously we had used custom Setuptools classes and cmdclass, and now as you’ve seen we’re using a hatch plugin. Alternatively, you could subclass a simple build backend like flit and add your own build step on top of it. That is a path we had started to take in Add flit backend by blink1073 · Pull Request #128 · jupyter/jupyter-packaging · GitHub.

That’s very helpful folks, thanks a lot! Didn’t occur to me that the sdist could be different from the GitHub checkout, will definitely look at how we can bundle the JS files within the sdist. I’m still interested in supporting git installs, so I’ll have a look at hatch plugins.