Let’s start with a minimal example:
from time import time, sleep
t_start = time()
def log(msg: str, stamp: float = time()):
duration = stamp - t_start
print(f"[{duration:.2f}]", msg)
print("Hello 1")
sleep(1)
print("Hello 2")
sleep(1)
print("Hello 3")
- I’ve made this type of “mistake” several times, I’ve seen other people making this mistake and it usually takes a lot of effort to be diagnosed in a real code base because people usually do not think into this direction.
- I also did an unofficial survey by showing this piece of code to people with moderate experience coding in python (people who coded for a few years, written a few thousand lines of python). And all of them did not see the problem without me pointing it out.
Now, what do you expect it to print? Is it something like below?
[0.00] Hello 1
[1.01] Hello 2
[3.01] Hello 3
Well this is what we actually get:
[0.00] Hello 1
[0.00] Hello 2
[0.00] Hello 3
If you are fairly familiar with either Python or JS, you might already see what’s wrong: the default value for optional argument stamp
is evaluated immediately when the function is declared.
I do understand that this behavior is a feature, not a bug. But I think in some cases, this language feature will cause troubles and make implementations unnecessarily cumbersome.
1. Dynamically updated values
An example for this is already shown above. Let’s look at another case that people might be confused:
count = 0
def log(n=count):
print("count is", n)
for _ in range(10):
count += 1
log()
Will print all zeros.
2. Side Effects
# Suppose this is part of a library
def do_task(prerequisite = print("do some task")): ...
# "do some task" printed upon library imported
def load_yaml_config(file = open("config.yaml")): ...
# "config.yaml" is opened regardless of whether an alternate is provided
# This open() call happens as soon as library is imported
def load_json_config(config = fetch("https://example.com/config.json")): ...
# HTTPS request send even if alternate json file is provided
# This happens as soon as user imports the library,
# even if load_json_config() is never called.
3. Circular References
Sometimes we want a default value to be a not-yet-initialized object. While the not-yet-initialized object also depends on the function itself. This will throw “attribute not found” because argument list is immediately evaluated:
def task1(next_task = task2): ...
def task2(next_task = task1): ...
4. What if anything is meaningful? (e.g. None
has its own meanings)
“Enough,” - you might say - “You can set the default value to None
and check it later”
Well I guess many of you will come up with this solution for my original example:
from time import time, sleep
t_start = time()
def log(msg: str, stamp: float = None):
if stamp is None:
stamp = time()
duration = stamp - t_start
print(f"[{duration:.2f}]", msg)
Well this solves the original problem with the cost of two extra lines of code with some (arguably) ugly indentations. But what if I now ask for one more feature: stamp = None
means do not print stamp?
# The solution will be like this:
class Nothing:
pass
def log(msg: str, stamp: float | None | type[Nothing] = Nothing):
if stamp is Nothing:
# Stamp is not provided, use current time
stamp = time()
if stamp is None:
# User asks to omit stamp
print(msg)
else:
print(f"[{stamp - t0:.2f}]", msg)
# Alternative, but comes with some drawbacks:
# 1. Will loose type hinting to argument stamp
# 2. argument "stamp" is now keyword only
# 3. time() is called each time even when user supplies a stamp
def log(msg: str, **kwargs):
if "stamp" in kwargs and kwargs["stamp"] is None:
# User asks to omit stamp
print(msg)
return
stamp = kwargs.get("stamp", time()) # Extra call to time() here
print(f"[{stamp - t0:.2f}]", msg)
Proposal: adding a “@dynamic” keyword decorator might help
This feature can only be implemented by the interpreter, because normal decorators are invoked after the function has been evaluated, it will be too late to do anything.
For example:
@dynamic
def log(msg: str, stamp: float = time()): ...
# time() is evaluated everytime when log() is called
# with no "stamp" argument supplied.
# Each call to time() will trigger a separate evaluation.
Potential Bonus Feature:
@dynamic
decorator might also help other decorators to access run-time variables.
# Sometimes people would like to access "self" for a decorator on their
# class member-function.
# Currently the only solution is the decorator factory intercepts the
# first argument (`self`) sent to the decorated function.
def context(ctx):
def decorator(fn: callable):
def wrapper(*args, **kwargs):
with ctx:
return fn(*args, **kwargs)
return wraps(fn)(wrapper)
return decorator
class Queue:
def __init__(self):
self._lock = Lock()
...
# attribute "self" is captured from argument list and made available to
# preceding decorator expressions in a temporary locals()
# Scope will be destroyed before evaluating argument list for get()
@dynamic(self)
@context(self._lock)
def get(self):
...