I was responding to Xitop, who was asking if there’s going to be an easier way to find out if a parameter supports lazy object.
Typing is for sure still going to be optional. Before typing came into existence people would go read the docs to find out if a function argument supports multiple types. Typing just makes it easier for people writing code in the IDE to catch issues earlier.
Like many other existing function parameters that support multiple types (such as file for open supporting int, str, Path), parameters supporting both lazy and non-lazy objects can be both documented and typed as such.
5. this - lazy object for signalling, but explicit handling is needed
Progress:
Been using this approach in various places for a while without any issues
Pros/Cons
| Pros Cons
+-------------------------------------------
(1) Implicit Complex
Intuitive Uncertain if can be implemented without Python 4
Nesting Too complex and advanced given current actual needs
(2) Implicit Complexity at parser level
Breaks conventions in undesirable manner
Adds undesirable weight on parser
No attempt at implementation has been made yet
Unlikely to work for extensions
(3) Explicit Verbose
Already exists Inconvenient
Noone uses it despite being available
e.g. why does not `dict.get` have
extra argument for lazy default?
Does not ensure consistent standard
e.g. parameter naming
Extra parameter might be troublesome for some cases
e.g. `assert`
Needs Explicit Handling that
developers need to implement for each case
(4) Explicit Various issues mentioned in
Already Exists https://discuss.python.org/t/builtins-lazy-for-lazy-arguments/86577/67
Just think about issues enabling `dict.get`
to accept lambda as lazy argument
Simply unsuitable to be "one way to do it"
Needs Explicit Handling that
developers need to implement for each case
(5) Explicit Needs Explicit Handling that
Simple developers need to implement for each case
Easily Achievable Relies on mutual agreement to use the same object
Imposes desirable
consistency
Does not have issues
mentioned in
https://discuss.python.org/t/builtins-lazy-for-lazy-arguments/86577/67
Maintains good enough
performance
… although I’ve just found the other thread where annotations are being discussed, and you’re right that my comment is more applicable there. Thanks. I’ve just added a copy to that thread as well.
Sigh. Do we really need multiple threads on variations of lazy evaluation, a topic that’s been discussed to death many times in the past? Apparently we do
I started the thread Lazy evaluation of expressions and thought I was doing the right thing by not hijacking an existing thread with a different, though related, idea. Happy to change discussion to this thread if that is what people prefer?
TLDR The idea is that if you type a variable/argument/return as Lazy and then an expression writing to it is automatically wrapped with Lazy(lambda <expr> and when read it is automatically unwrapped. There is no implementation to try.
The main issue under discussion on the other thread is that assuming it can be implemented, would people want it and would the Core Devs support it or would it be deemed undesirable even if demonstrated to work?
So two questions: should discussion be on this thread and if implemented would the idea be supported?
Let me restate my point - do we need two more proposals on a topic that has been discussed to death over the years? How exactly do these proposals add anything new, that hasn’t been covered in all that time? How do they address all the problems with the idea of lazy evaluation that have been brought up previously?
And yes, it’s hard work to find and review all of those old discussions. But that doesn’t give anyone the right to expect others to find and/or restate the arguments, just to avoid doing that work themselves. This isn’t a game where if no-one remembers a problem that was previously raised then you “win”. Unless you count getting a flawed feature into Python as a win
No, it should stay in its own thread. It has nothing to do with this proposal. But if you want to update my comparison table with new information from your endeavours let me know.
And if anyone feels that I have misjudged something in my comparison table / wants to add more pros/cons let me know as well.
I have been thinking about deferred evaluation since I joined the community and experimented with various ideas and implementations a fair bit. I picked it up at peps/pep-9999.rst at master · DavidMertz/peps · GitHub.
Currently I am 99% certain that complex automatic deferred evaluation has little to no chance of going through due to share complexity, implications and feasibility in contrast with actual needs and subsequent benefit. At least not until Python 4.
(I would be happy to be wrong about this…)
This simple proposal (which requires nothing more than 1 new type) is ironically the result of all my effort.
I’ve read all the past, recent or not, proposals of lazy/deferred evaluations that I could manage to find on this forum (not necessarily every single post, but at the minimum the first several and the last several posts in each discussion), and none of them comes close to the feasibility of this proposal of using a simple callable wrapper class.
It has all the upsides of what we all want from lazy evaluations and barely any downsides.
Below is a summary of what I consider to be its pros and cons compared to other proposals:
Pros:
It requires absolutely no new syntax.
It is dead simple to implement as a built-in.
It is dead simple for existing functions or language constructs (such as assert and f/t-strings) to opt in.
It is universally applicable to functions, language constructs and even dict values, while most other proposals are applicable only to either functions or a particular language construct.
The points of evaluation can be freely and explicitly chosen, leaving no room for ambiguity.
A dedicated lazy type ensures that objects of existing types such as lambda functions or iterators can be easily lazily evaluated too.
Cons: Not as cute or concise as some proposals that require syntax changes or are implicit/magical in nature.
I do hope a core dev can see the value and simplicity in this proposal and sponsor a PEP for it.
You forgot the most important con: builtins.lazy does not do anything for you. It is supposed to be just a wrapper that indicates that some (unspecified!) functions should automatically unwrap and execute the wrapped function. Which ones has not been specified by the OP.
So, all functions in the whole stdlib (actually also third-party libraries) have to decide if they are lazy-aware ore not. In a previous post I have pointed out that the question if a function is lazy-aware is part of the function’s contract.
So, if dict.get was lazy-aware for the default parameter, it would unwrap and execute the given default if it was of type lazy. If it was not lazy-aware, it would return the default value un-executed, even if it was of type lazy. My comment that it would be much simpler if dict.get would allow for a keyword argument callback for a programatically derived default, was dismissed by the OP since it would not scale.
But correct is the opposite:
a optional keyword argument for a callback could be added at any time,
but turning a not-lazy-aware function into a lazy-aware function is not backwards compatible. So, if Python 3.15 introduced builtins.lazy without turning dict.get into a lazy-aware function, later versions of Python cannot do that either.
As I pointed out earlier, while a new separate keyword argument such as lazy_default for a lazy argument also works to some extent, it doesn’t work with positional arguments and template strings such as a log record template:
# nowhere to add a separate lazy argument to
assert validate_sun_rises_from_east(), expensive_verbose_info()
# wasted when logging level > DEBUG
logger.debug("Verbose info: %s", expensive_verbose_info())
Why would it not be backwards compatible? Existing code without lazy arguments will continue to work. And newer code can test for Python versions to determine if it can use lazy arguments for a particular function/language construct or not.
Not all functions/language constructs need to make the decision to support lazy arguments in the same Python release as long as their support is documented for the right version. Just like any other new feature.
Code that assumes that arguments of type lazy are passed through a certain function unchanged will be broken if the function is turned lazy-aware suddenly.
It would not scale in a sense that it would not be applicable solution to say:
def add_lazy(a, b):
if type(a) is Lazy:
a = a()
if type(b) is Lazy:
b = b()
return a + b
Thanks. This is a very good point. And this is the first issue that was raised here that I haven’t thought about as it was never an issue in codebase that I have full control of.
So, if Lazy was in stdlib, but dict.get wasn’t using it, users start passing Lazy object as default arg expecting to get it out unevaluated. Then adapting Lazy to dict.get is backwards incompatible.
Need to think if some strategy is possible to address this.
Old code does not have type lazy so that isn’t a problem. New codes know exactly which Python version has support for lazy so it is not a problem either.
But you’re still requiring the person who’s function you’re using to opt into this contract, and write a clause where the evaluation happens.
This would need to happen for all functions in the standard library, so this is hardly a “cheap” proposal.
If you want it to be applicable to dict values, does that mean even dict.get will need to be modified to be lazy-aware, so that it will evaluate the lazy object stored as values in the dict??
You keep coming back to the logger, so this is apparently an important case for you.
This case is already solvable. Consider the following code. It can be polished further, but this already works:
from collections.abc import Mapping
from typing import Callable
class LazyMsg(Mapping):
__slots__ = ("msg_factory", "key", "msg")
def __init__(self, msg_factory: Callable[[], str], name: str = "LazyMsg"):
self.msg_factory = msg_factory
self.key = name
self.msg = None
def __getitem__(self, _key):
if self.msg is None:
self.msg = self.msg_factory()
return self.msg
def __iter__(self):
yield self.key
def __len__(self) -> int:
return 1
logging.warning("hi %(LazyMsg)s", LazyMsg(lambda: "there"))
def ifelse(a, b, c):
if a:
return b() if type(b) is lazy else b
else:
return c() if type(c) is lazy else c
The point is that I think lazy solution should apply to 99% of cases, otherwise it is not a good solution to be adapted as a general standard. And the above although not very common captures the aspect that IMO lazy solution needs to be able to handle.
Although there is some overlap that callbacks can handle and this idea, these are largely orthogonal. And I think it is a good idea and IMO deserves its own thread if the pool of cases that it solves is big enough to build a case. e.g. “expose dict callbacks to Pure Python”.
Ah I posted the above before reading dg-pb’s response. I get it now. Thanks.
Well then indeed we need to review all the existing language constructs, built-in and stdlib functions, and decide which of them shall become lazy-aware when lazy becomes a feature.
Fortunately I don’t think such a review is going to be overly hard. Functions and language constructs that take potentially expensive arguments and only evaluate them conditionally are candidates. But yes, once we decide which of them to be made lazy-aware, that’ll be it for existing functions.
Exactly, this is the cost. While the benefit is avoiding massive amounts of complexity introduced by something that attempts to do this automatically.
No, this would be adapted only in places that it makes sense. It is the same as some arguments can be None, in the same manner some can be Lazy.
This is not a general paradigm to be applied to all of the Python functions. On the contrary, this is the most minimal solution which solves only problems at points where they exist attempting to limit scope and influence of this as much as possible.
And the clause itself is 2-3 lines of code for 1 argument.
