Hi, I am interested to move Bokeh away from
setup.py  in the near-ish future. However, Bokeh is a cross-language project with compiled TypeScript components that need to be identically included in all published packages (wheel, sdist, conda). Our current build automation does this to build the packages before publishing:
def build_sdist_packages(config: Config, system: System) -> ActionReturn: try: system.run("python setup.py sdist --install-js --formats=gztar") return PASSED("sdist package build succeeded") except RuntimeError as e: return FAILED("sdist package build did NOT succeed", details=e.args) def build_wheel_packages(config: Config, system: System) -> ActionReturn: try: system.run("python setup.py bdist_wheel --install-js") return PASSED("wheel package build succeeded") except RuntimeError as e: return FAILED("wheel package build did NOT succeed", details=e.args)
The question comes down to that
--install-js option that we pass in. That option is currently handled by code in our
setup.py and what it does is copy an existing, built BokehJS into the Python source tree for inclusion in the package. Without that option, BokehJS gets built from scratch  every time. This is undesirable from a package automation standpoint:
- It is somewhat time-consuming to build BokehJS, so it’s preferable to do it only once, rather than once for every package type build.
- It is crucial that every package type (wheel, sdist, conda) has the exact same BokehJS files (i.e with identical hashes). While the risk of somehow getting slightly different TS build outputs from subsequent BokehJS builds is very small , any risk at all here is unacceptable. We simply must use a single source of truth for BokehJS across all packages.
So what are our options here? Looking at
build it does not seem sufficient to support a customization like this. Are there other tools that support defining custom steps as part of a build, or have extension APIs that we can leverage? 
TLDR; How can we support custom build steps in two cases:
- A default build should build BokehJS from scratch and move it into the Python package.
- An “install-js” build should move a pre-buit BokehJS into the Python package.
cd bokehjs; node make↩︎
Maybe some datetime-dependent codegen is erroneously introduced somewhere, etc ↩︎
Certainly we could “shell-script” our way out of this but I would much prefer to stick to community standard commands and tools to the extent possible. ↩︎