A common technique for implementing custom changes to python methods/functions is to import and override that functionality. For example,
from pathlib import Path
class MyPath(Path):
def iterdir(self,*args,*kargs):
# make custom changes
This works well in customizing methods of a class. When a module contains functions that one wishes to customize, then this can be accomplished with “from module import *”. For example,
from yaml import *
def load(stream, Loader):
"""
Parse the first YAML document in a stream
and produce the corresponding Python object.
"""
loader = Loader(stream)
try:
#
# add custom functionality here
#
return loader.get_single_data()
finally:
loader.dispose()
However, sometimes a module is built without classes and includes (unfortunately) an __all__ that prevents seamless overriding for custom behavior. For example,
from shutil import *
def copyfile(src, dst, *, follow_symlinks=True):
"""Copy data from src to dst in the most efficient way possible.
If follow_symlinks is not set and src is a symbolic link, a new
symlink will be created instead of copying the file it points to.
"""
#
# add custom functionality here
#
sys.audit("shutil.copyfile", src, dst)
if _samefile(src, dst):
raise SameFileError("{!r} and {!r} are the same file".format(src, dst))
file_size = 0
for i, fn in enumerate([src, dst]):
try:
st = _stat(fn)
except OSError:
# File most likely does not exist
pass
else:
# XXX What about other special files? (sockets, devices...)
if stat.S_ISFIFO(st.st_mode):
fn = fn.path if isinstance(fn, os.DirEntry) else fn
raise SpecialFileError("`%s` is a named pipe" % fn)
if _WINDOWS and i == 0:
file_size = st.st_size
if not follow_symlinks and _islink(src):
os.symlink(os.readlink(src), dst)
else:
with open(src, 'rb') as fsrc:
try:
with open(dst, 'wb') as fdst:
# macOS
if _HAS_FCOPYFILE:
try:
_fastcopy_fcopyfile(fsrc, fdst, posix._COPYFILE_DATA)
return dst
except _GiveupOnFastCopy:
pass
# Linux
elif _USE_CP_SENDFILE:
try:
_fastcopy_sendfile(fsrc, fdst)
return dst
except _GiveupOnFastCopy:
pass
# Windows, see:
# https://github.com/python/cpython/pull/7160#discussion_r195405230
elif _WINDOWS and file_size > 0:
_copyfileobj_readinto(fsrc, fdst, min(file_size, COPY_BUFSIZE))
return dst
copyfileobj(fsrc, fdst)
# Issue 43219, raise a less confusing exception
except IsADirectoryError as e:
if not os.path.exists(dst):
raise FileNotFoundError(f'Directory does not exist: {dst}') from e
else:
raise
return dst
now raises an error. This is because the __all__ in shutil prevents the seamless importing of everything that is used in copyfile(). Hence, adding custom functionality to copyfile() requires editing all the code found in copyfile() (instead of just adding code where you want to customize). This is notably different than in the other cases and feels very non-pythonic. One could painstakingly identify every method/function that is used and manually import each one, for example:
from shutil import _WINDOWS, _fastcopy_sendfile, copyfileobj #etc.
But again, very non-pythonic. One could argue that shutil should be built with classes so that customization is seamlessly possible, but that is not the point of this inquiry. Instead we are wondering how does one cleanly bypass the __all__ method in python, so that seamless customization can be accomplished. Should something like:
from shutil import **
be added to the python language? How would others approach customizing the copyfile() function (by only adding code, not editing the code already in the function) in shutil?
and definitely against the philosophy of 
