How to properly extend standard `dir()` search with module-level `__dir__`?

Recently I saw PEP 562 which allows to have module-level __dir__ that overrides the standard dir() search on a module and I’m wondering what exactly “standard dir() search” is - I simply want to extend the list already given by dir() and include some additional names in it.
The PEP as an example includes:

def __dir__():
    return sorted(__all__ + deprecated_names)

which won’t really give similar output to standard dir() list.

I’m thinking that this code might be enough - it does give me the output I would expect, but I am not entirely sure if there’s anything else involved in “standard dir() search” that I just simply haven’t encountered in my usage:

def __dir__():
    # Python seems to do the sorting on its own so this can omit sorted()?
    return [*globals().keys(), *deprecated_names]

The sorted(__all__ + deprecated_names) version assumes that __all__
contains all your module’s public names, except for deprecated names. If
you haven’t defined __all__, or if it is out of date, then it will
miss some things.

The [*globals().keys(), *deprecated_names] version grabs everything
from the module namespace, whether you intend them to be listed or not,
including global variables.

What’s the difference? Suppose your module looks like this:

__all__ = ['spam', 'eggs', 'Cheese', 'foo']

import sys

# Public names
spam = 'spam, eggs, spam, spam and spam on toast'
    
def eggs():
    global storage
    storage = 1

class Cheese:
    pass

# Private names
def _private():
    pass

# Incidental variables
for i in range(3):
    print('Hello world')

# Deprecated names
deprecated = ['bar',]

# Magic
def __dir__():
    ...

then the __all__ version will give you a dir() of:

spam, eggs, Cheese, foo, bar

while the version with globals will give you a dir() of:

__all__, sys, spam, eggs, Cheese, _private, i, deprecated, bar, __dir__

plus some additional internal details of modules:

__builtins__, __cached__, __doc__, __file__, __loader__, __name__, 
__package__, __spec__ 

(intentionally not sorting the output but leaving it in input order).

Notice that the global variable “storage” doesn’t show up. That’s
because I haven’t run eggs(). Once I run eggs(), the second version
of dir will change to include it.

In the first case, because your __all__ wrongly includes the obsolete
name “foo”, it still shows up in dir(); in the second case you get a
bunch of stuff you probably don’t want to see, like imports and dunders.

Personally, I strongly dislike the default output of dir() and I run a
monkey-patched version which strips out all those dunder names unless I
explicitly ask for them. If I were to take the time and effort to define
a custom __dir__ for my module, the last thing I would want to do is
fill it with module dunders and private names.

But it’s your module, and you get to choose whatever behaviour makes the
most sense to you.

I know that, what I wanted to ask is if sorted(globals()) is exactly what I would get by using dir(module) if there weren’t a custom __dir__() implementation in the module or if there’s something I could miss in such case. My intention is to get exactly the same output as I would get originally.

[Jakub Kuczys]
“I know that, what I wanted to ask is if sorted(globals()) is exactly
what I would get by using dir(module) if there weren’t a custom
__dir__() implementation in the module or if there’s something I could
miss in such case. My intention is to get exactly the same output as I
would get originally.”

The precise output of dir is not documented, and is subject to
change without a backwards-compatibility guarantee:

Quote:

“Note: Because dir() is supplied primarily as a convenience for use at
an interactive prompt, it tries to supply an interesting set of names
more than it tries to supply a rigorously or consistently defined set of
names, and its detailed behavior may change across releases.”

https://docs.python.org/3/library/functions.html#dir

You can try to match the documented behaviour, or read the source code,
but in my opinion the only guaranteed way to get the exact same output
that dir() would give is to actually call dir().

import builtins
def __dir__():
    return builtins.dir()

I’m curious, if you want to match the exact output of dir, why are you
bothering to write a custom __dir__ function? The easiest way to have
dir(mymodule) give the same output as the standard dir would be to not
write a custom function :slight_smile:

Because I need to add something to it, e.g:

def __getattr__(name):
    if name == "DeprecatedName":
        return globals()["_DeprecatedName"]
    raise AttributeError(...)

def __dir__():
    return dir() + ["DeprecatedName"]

That won’t work though, cause call to dir() without arguments will return the local scope. And also, part of dir()'s implementation probably includes calling module’s __dir__() if it exists which would
probably result in recursion error.

I guess I’ll just stick with what I have right now then, seems to include everything built-in dir includes, thanks.