import importlib.metadata as importlib_metadata
importlib_metadata # NO error!
I have seen other similar odd issues with this module. What is going on? I can provide more details if necessary, it would just be a bit of a slog to write up. Mostly I’m hoping that to zeroth order someone can just shed light about any special assumptions this module has, or under what circumstances the metadata submodule would just not exist.
Most of the time I have seen people run into that is when they do the latter and then try to use importlib.metadata. But if you only import importlib you don’t get importlib.metadata, you have to import the submodule explicitly.
@saaketp Yes, 100% sure, which is why I am very confused.
Edit:
bk301-py10 ❯ bokeh serve --show foo.py
2022-11-10 14:23:59,567 Starting Bokeh server version 3.0.1 (running on Tornado 6.2)
2022-11-10 14:23:59,900 User authentication hooks NOT provided (default user enabled)
2022-11-10 14:23:59,903 Bokeh app running at: http://localhost:5006/foo
2022-11-10 14:23:59,903 Starting Bokeh server with process id: 30450
> /Users/bryan/tmp/foo.py(28)<module>()
-> import pyproj
(Pdb) import importlib.metadata
(Pdb) importlib.metadata
*** AttributeError: module 'importlib' has no attribute 'metadata'
(Pdb)
FWIW Bokeh server is in the unusual position of being a Python application that runs other Python applications, which sometimes introduces weirdness. I can believe there is something we might need to change or start doing on our end to make things more “normal” for the app code, but I also don’t know what that might be.
I cannot reproduce this on my Windows Store install of Python 3.10:
C:\>python3.10
Python 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib.metadata
>>> importlib.metadata
<module 'importlib.metadata' from 'C:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.10_3.10.2288.0_x64__qbz5n2kfra8p0\\lib\\importlib\\metadata\\__init__.py'>
Is it possible that Bokeh server is invoking a different interpreter version than the 3.10 you think it is?
What’s the output of import sys; sys.version_info in that same context?
Anyone likely to be doing reloads or messing with sys.modules?
Often the second import of a module can do weird things like this, though there shouldn’t be a second import because it should be cached. But then, people like to try and “clear the cache,” which is much harder than del sys.modules[name]…
@steve.dower Sort of, Bokeh programmatically creates a module for the app code to be executed in for each session, then installs that module in sys.modules. But nothing is deleting modules at the point this error happens. [1] This has all worked fine for many years but it seems like some recent developments mean we need to update some aspect of how this is handed.
bk301-py10 ❯ bokeh serve --show foo.py
2022-11-10 15:01:01,348 Starting Bokeh server version 3.0.1 (running on Tornado 6.2)
2022-11-10 15:01:01,574 User authentication hooks NOT provided (default user enabled)
2022-11-10 15:01:01,577 Bokeh app running at: http://localhost:5006/foo
2022-11-10 15:01:01,577 Starting Bokeh server with process id: 30849
BokehDeprecationWarning: tile_providers module was deprecated in Bokeh 3.0.0 and will be removed, use add_tile directly instead.
> /Users/bryan/tmp/foo.py(28)<module>()
-> import pyproj
(Pdb) import sys
(Pdb) print("importlib" in sys.modules and hasattr(sys.modules["importlib"], "metadata"))
False
(Pdb) print("importlib.metadata" in sys.modules)
True
(Pdb)
Okay then seems like the exact same problem as that mailing list post.
If importlib.metadata is already in sys.modules then import importlib.metadata does nothing, so importlib doesn’t get the metadata attribute. It should have got the attribute set when you imported the first time, but then that del statement deleted it.
Just to be clear (for my understanding), the metadata attribute (submodule?) is created dynamically by importlib import, but if it is deleted, then subsequent, explicit import importlib.metadata does not re-create it, because importlib is still cached in sys.modules?
That seems understandable in hindsight, but if I am honest, also seems like a foot-gun.
Well, actually I guess this is just Python 101 and I am just over way overthinking things. I really almost never, ever do import foo.bar without aliasing (or importing from).
When you try to import a submodule, if it succeeds it gets added to sys.modules and then set as an attribute on the parent package (that’s how the attribute access in your own code works). If the import fails, the name gets set to None in sys.modules to avoid trying a known-bad import again. Then future imports pull from sys.modules.
@brettcannon Sure, all of that makes sense. I guess what surprised me here is the mode of failure. If I do del foo.bar, and then subsequently do import foo.bar I guess I would have expected that to fail immediately, rather than silently “work” (if it is not going to be automagic’ed to “actually work”). Like I said I almost always “import as” or “import from” so my intuition just got caught off guard.
The trick here is that del foo.bar isn’t doing anything with modules.
It translates into (approx) foo.__delattr__(bar). So all you’re doing is removing the attribute from foo, but foo.bar is still imported just as it ever was.
