# Script to get top-level packages from source tree **URL:** \ **Category:** Packaging\ **Created:** [December 3, 2023, 6:00pm UTC](https://discuss.python.org/t/script-to-get-top-level-packages-from-source-tree/40232 "2023-12-03T18:00:12Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![astrojuanlu](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/astrojuanlu/32/703_2.png) [@astrojuanlu](https://discuss.python.org/u/astrojuanlu)\ **Post date:** [December 3, 2023, 6:00pm UTC](https://discuss.python.org/t/script-to-get-top-level-packages-from-source-tree/40232/1 "2023-12-03T18:00:12Z")
Hi all, for some time I’ve wanted a standards-compliant way of getting the top-level packages of a source tree (the ones that will end up in a wheel). It turns out that the file collection mechanisms are not standardised, so I figured that the only way of doing this in a 100 % correct way (modulo bugs) was to go through installing the build backend in an isolated environment, building the wheel, and extracting the resulting `RECORD` file. That’s what I did here: > It works on itself: ```auto >>> from pygetpackages import get_packages >>> get_packages(".") ['pygetpackages'] ``` It’s annoyingly slow and requires an Internet connection, so if there’s a better way of doing this, I’ll be happy to know. ---
**Author:** ![jeanas](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jeanas/32/8115_2.png) [@jeanas](https://discuss.python.org/u/jeanas)\ **Post date:** [December 3, 2023, 6:31pm UTC](https://discuss.python.org/t/script-to-get-top-level-packages-from-source-tree/40232/2 "2023-12-03T18:31:48Z")
I don’t see why you need to parse RECORD. Can’t you just get the file list from zipfile? Also note that by looking for ` __init__.py` files you are excluding namespace packages. Depending on your use case, this might be good or bad. ---
**Author:** ![sinoroc](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sinoroc/32/1607_2.png) [@sinoroc](https://discuss.python.org/u/sinoroc)\ **Post date:** [December 3, 2023, 6:46pm UTC](https://discuss.python.org/t/script-to-get-top-level-packages-from-source-tree/40232/3 "2023-12-03T18:46:38Z")
I would guess that building is the only way, yes. If we consider things like setuptools’ `package_dir` which kind of allows rewriting the project directory structure at build time. If I am not mistaken pymsbuild and probably other build backends allow this kind of things, where the installed tree ends up being completely different than what is in the source tree. ---
**Author:** ![CAM-Gerlach](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cam-gerlach/32/3688_2.png) [@CAM-Gerlach](https://discuss.python.org/u/CAM-Gerlach)\ **Post date:** [December 4, 2023, 2:40am UTC](https://discuss.python.org/t/script-to-get-top-level-packages-from-source-tree/40232/4 "2023-12-04T02:40:27Z")
Note that as a bit of a hacky shortcut for (the still large majority of extant) wheels that are built with `setuptools`, a list of the top-level import packages can be trivially parsed from `top_level.txt` in `dist-info`, but its an unstandardized relic of eggs, and other backends do not appear to emit it (at least checking meson-python and hatchling). ---
**Author:** ![pradyunsg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pradyunsg/32/206_2.png) [@pradyunsg](https://discuss.python.org/u/pradyunsg)\ **Post date:** [December 4, 2023, 5:42am UTC](https://discuss.python.org/t/script-to-get-top-level-packages-from-source-tree/40232/5 "2023-12-04T05:42:20Z")
I should share my Gist from Oct 2022 here, which handles namespace packages as well as `.data` directory magic, when someone uses that. > I had a bit of deja-vu, and realised we’ve discussed this specific topic before and I posted this link theere too. > [@Record the top-level names of a wheel in \`METADATA\`?](https://discuss.python.org/t/record-the-top-level-names-of-a-wheel-in-metadata/29494): > > I had another instance where I wished I could map a project name to the modules and packages they provide. I also realized that I suspect people come to PyPI looking for a project by its import name and is not always successful (look at [sklearn · PyPI](https://pypi.org/project/sklearn/) as a redirect example for this), so having a reverse lookup at [pypi.org](http://pypi.org) might be useful. Would it make sense to have METADATA have a repeatable Import-Name field that lists a top-level name the wheel provides? I’m fine if this this optional, but f… ---
**Author:** ![astrojuanlu](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/astrojuanlu/32/703_2.png) [@astrojuanlu](https://discuss.python.org/u/astrojuanlu)\ **Post date:** [December 4, 2023, 10:22pm UTC](https://discuss.python.org/t/script-to-get-top-level-packages-from-source-tree/40232/6 "2023-12-04T22:22:59Z")
@pradyunsg While we wait, I took your gist and turned it into a package [pygetimportables · PyPI](https://pypi.org/project/pygetimportables/) ---
**Author:** ![willingc](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/willingc/32/20_2.png) [@willingc](https://discuss.python.org/u/willingc)\ **Post date:** [December 5, 2023, 12:10am UTC](https://discuss.python.org/t/script-to-get-top-level-packages-from-source-tree/40232/7 "2023-12-05T00:10:35Z")
@pradyunsg and @astrojuanlu Thanks for sharing these helpful resources. I often wish the lock files separated highest level dependencies from the dependencies brought in by those libraries. ---
**Author:** ![CAM-Gerlach](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cam-gerlach/32/3688_2.png) [@CAM-Gerlach](https://discuss.python.org/u/CAM-Gerlach)\ **Post date:** [December 5, 2023, 8:08am UTC](https://discuss.python.org/t/script-to-get-top-level-packages-from-source-tree/40232/8 "2023-12-05T08:08:38Z")
Sorry, I’m rather confused by what you are describing and how it connects to the topic here. Reading between the lines, it sounds like you might be intending to suggest that lock files distinguish direct from transitive dependencies, but this topic is discussing how to enumerate the top-level import packages in a given project source tree, which is entirely unrelated. Did you mean to post this in another topic, sorry? Or was this all one big misunderstanding brought on by the unfortunate overloading of the word “package” (meaning both import package, as in the case here, and distribution package, as perhaps was mistakenly assumed), coupled with a misinterpretation of “top-level packages” to instead imply “direct dependencies”?