# Reduce the overhead of functools.lru\_cache for functions with no parameters **URL:** \ **Category:** Ideas\ **Created:** [April 20, 2020, 10:48pm UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956 "2020-04-20T22:48:43Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![orf](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/orf/32/1889_2.png) [@orf](https://discuss.python.org/u/orf)\ **Post date:** [April 20, 2020, 10:48pm UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/1 "2020-04-20T22:48:43Z")
[`functools.lru_cache()`](https://docs.python.org/3/library/functools.html#functools.lru_cache) has two common uses. The first is as it was designed: an LRU cache for a function, with an optional bounded max size. The other is as a replacement for this: ```python _obj = None def get_obj(): global _obj if _obj is None: _obj = create_some_object() return _obj ``` i.e lazy initialization of an object of some kind, with no parameters. You can find a few examples in the Django source code: [https://github.com/django/django/search?q="%40lru\_cache"&unscoped\_q="%40lru\_cache"](https://github.com/django/django/search?q=%22%40lru_cache%22&unscoped_q=%22%40lru_cache%22) I’ve highlited some examples below: - [Example 1](https://github.com/django/django/blob/fa6893e9dbb127d70c1376667c700b71bb987fe0/django/core/management/color.py#L59) - [Example 2](https://github.com/django/django/blob/77aa74cb70dd85497dbade6bc0f394aa41e88c94/django/forms/renderers.py#L19) - [Example 3](https://github.com/django/django/blob/a44d80f88e22eda24dacef48e368895ebea96635/django/utils/version.py#L72) - [Example 4](https://github.com/django/django/blob/89368ab6e358ebe29a0417d65209182238daa245/django/contrib/auth/password_validation.py#L18) There are lots of other examples in the ecosystem outside of Django - it’s a convenient way to lazy intialize objects like Boto3 clients, “default” instances or even ML models. Looking at the source code in `_functoolsmodule.c` I cannot see any special casing for these kind of functions. An `lru_cache` on a function with no parameters is a bit of an oxymoron - there can only ever be one output and it cannot be dependent on any inputs. I’d like to suggest that we add a special case to `lru_cache()` for callables that have no parameters, to avoid the current overhead of using a `PyDict` on every lookup and make it more akin the code sample above - a single PyObject variable that’s stored on the first call and returned on subsequent calls. If there is a generally agreeable reception to this proposal, I’d really like to give this a shot as my first contribution to CPython. ---
**Author:** ![brettcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/brettcannon/32/34895_2.png) [@brettcannon](https://discuss.python.org/u/brettcannon)\ **Post date:** [April 21, 2020, 12:36am UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/2 "2020-04-21T00:36:54Z")
> [@orf](#): > > I’d like to suggest that we add a special case to `lru_cache()` for callables that have no parameters, to avoid the current overhead of using a `PyDict` on every lookup and make it more akin the code sample above - a single PyObject variable that’s stored on the first call and returned on subsequent calls. Can you show there’s enough of a performance improvement on subsequent calls to the functions to warrant the maintenance cost of special-casing for this? ---
**Author:** ![steven.daprano](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steven.daprano/32/1083_2.png) [@steven.daprano](https://discuss.python.org/u/steven.daprano)\ **Post date:** [April 21, 2020, 1:15am UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/3 "2020-04-21T01:15:25Z")
[Tom Forbes] > ```python > _obj = None > def get_obj(): > global _obj > if _obj is None: > _obj = create_some_object() > return _obj > > ``` > > i.e lazy initialization of an object of some kind, with no parameters. Eiffel uses the keyword “ONCE” for this. I would prefer a similar approach, using a specialised decorator, rather than to special-case lru\_cache: ``` @once def get_obj(): obj = Something() obj.extras = None return obj ``` Here’s an untested implementation: ``` def once(func): obj = None @functools.wraps(func) def inner(): nonlocal obj if obj is None: obj = func() return obj return inner ``` ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [April 21, 2020, 7:42am UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/4 "2020-04-21T07:42:16Z")
Agreed, a special `@once` decorator sounds like a better approach here. Although its behaviour is straightforward enough that I usually write it myself if I need it (I don’t even have a library with it in, I just rewrite it each time). And to be honest, I’ve never heard of the idea of using `functools.lru_cache` for this before. Maybe it’s a trick that’s only common in django, which makes me think that django could expose a custom version of this if the performance improvement warrants it? ---
**Author:** ![uranusjr](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/uranusjr/32/103_2.png) [@uranusjr](https://discuss.python.org/u/uranusjr)\ **Post date:** [April 21, 2020, 8:11am UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/5 "2020-04-21T08:11:13Z")
I see this trick in some large, inter-connected projects to reduce startup time (vs. calculate those values on import), and Django is one of them. This is not the only solution to the problem; pip uses classes and cached properties, for example, which makes me wonder whether it would be a better solution to make descriptors work on modules so we can just use `@cached_property` for this… but that’s significantly expanding the scope of the solution. ---
**Author:** ![EpicWink](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/epicwink/32/17968_2.png) [@EpicWink](https://discuss.python.org/u/EpicWink)\ **Post date:** [April 21, 2020, 8:48am UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/6 "2020-04-21T08:48:46Z")
> [@uranusjr](#): > > make descriptors work on modules so we can just use `@cached_property` for this I’ve seen a few projects use the new module ` __getattr__ ` to implement cached properties ---
**Author:** ![orf](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/orf/32/1889_2.png) [@orf](https://discuss.python.org/u/orf)\ **Post date:** [April 21, 2020, 8:50am UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/7 "2020-04-21T08:50:31Z")
> Can you show there’s enough of a performance improvement on subsequent calls to the functions to warrant the maintenance cost of special-casing for this? I believe I can, I’ll try and create a PoC this weekend. Looking at the implementation for `lru_cache` I also think it’s pretty simple to do - I was going to use the [`PyObject` member](https://github.com/python/cpython/blob/master/Modules/_functoolsmodule.c#L779) that’s [currently always a PyDict](https://github.com/python/cpython/blob/master/Modules/_functoolsmodule.c#L1191) as the place to store the returned object. I think most of the savings will be memory from PyDict, but there is also [hashing and PyDict calls](https://github.com/python/cpython/blob/master/Modules/_functoolsmodule.c#L871) that will be removed. > And to be honest, I’ve never heard of the idea of using `functools.lru_cache` for this before. Maybe it’s a trick that’s only common in django, which makes me think that django could expose a custom version of this if the performance improvement warrants it? This pattern is pretty prevalent, far more than just Django. If you [browse through the first few pages of Github code search](https://github.com/search?l=Python&q=%22functools.lru_cache%22&type=Code) you can find quite a few: - [Example](https://github.com/jma127/pcu/blob/6248905351a568be303394ee5d932cc2103d9539/pcu/paths.py#L5) - [Example](https://github.com/blue-mongoose/blueMongoose/blob/af40f02e27adf5c32244177f34b3c81aa0ff5e38/cards/cards.py#L7) - [Example](https://github.com/random-python/nspawn/blob/5a14b5ceceae4d66352b9021143620beda30bb0c/src/main/nspawn/tool/network.py#L12) - [Example](https://github.com/JoanPuig/PyFIT/blob/16b0cd0b9204927cf45d95894b55799ec764a863/FIT/base_types.py#L67) - [Example](https://github.com/Exahilosys/strdata/blob/33c60d238005c70684ffc54c1a17458f4516b312/strdata/pullers/ __init__.py#L32) `@once` might be a semantically better, but it doesn’t exist in the stdlib, and people (myself included) have clearly cottoned on to the idea of using `lru_cache` as a simple, copy-paste free replacement. Plus it’s faster (Using the `once` implementation above) ```auto In [14]: %timeit return_1_once() 98.8 ns ± 4.51 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [15]: %timeit return_1_lru_cache() 56.7 ns ± 1.79 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) ``` ---
**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\ **Post date:** [April 21, 2020, 9:35am UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/8 "2020-04-21T09:35:55Z")
Have you demonstrated those 40 nanoseconds mattered in your application, or is this another case of microoptimization obsession? ---
**Author:** ![orf](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/orf/32/1889_2.png) [@orf](https://discuss.python.org/u/orf)\ **Post date:** [April 21, 2020, 11:12am UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/9 "2020-04-21T11:12:48Z")
It undoubtably doesn’t matter at all, I’m just guessing why people use lru\_cache in the way they currently do. It’s a use case that people currently have, it should be simple to implement and has the upside of being even faster and more memory efficient. If a patch really turns out to be simple enough then it seems like everybody wins. I guess the main contention here is if we should encourage people to use the lru\_cache decorator like that, even implicitly? ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [April 21, 2020, 11:42am UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/10 "2020-04-21T11:42:29Z")
If you can confirm that the maintenance cost of the special case is justified by the performance improvement, this should probably just be submitted as a bpo item and PR (treating it as a special-case optimisation, not trying to promote or emphasise it as a “call once” design pattern, or anything like that). If you can’t justify the cost, I suspect it should just be dropped. > [@orf](#): > > I guess the main contention here is if we should encourage people to use the lru\_cache decorator like that, even implicitly? IMO no. By all means pursue this as an optimisation (if it proves justifiable), but don’t try to make this the “sanctioned” way to do call-once functions. If it’s to be “official”, it should be more discoverable/obvious (i.e., a `@once` decorator), rather than a side-effect of adding a LRU cache. ---
**Author:** ![orf](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/orf/32/1889_2.png) [@orf](https://discuss.python.org/u/orf)\ **Post date:** [April 21, 2020, 12:59pm UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/11 "2020-04-21T12:59:07Z")
Ok, thank you for your guidance. I’ll try and submit a PR with this optimization and see how it goes. I’m keen to also add a `functools.once` decorator, I’ll draft up a proposal and submit it to the python-ideas mailing list (or does this forum supercede that? Im not clear on that, sorry). ---
**Author:** ![steven.daprano](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steven.daprano/32/1083_2.png) [@steven.daprano](https://discuss.python.org/u/steven.daprano)\ **Post date:** [April 21, 2020, 1:19pm UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/12 "2020-04-21T13:19:44Z")
[Tom Forbes] > `@once` might be a semantically better, but it doesn’t exist in the > stdlib, That’s easy to fix, if there is concensus that it is a good thing. I think that this may be a small enough addition that it may not even need a PEP. > Plus it’s faster (Using the `once` implementation above) > > ```auto > > ``` > In [14]: %timeit return\_1\_once() > 98.8 ns ± 4.51 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) > > In [15]: %timeit return\_1\_lru\_cache() > 56.7 ns ± 1.79 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) > ```auto > > ``` o\_O I can’t dispute your numbers, I get similar numbers myself. But … how? lru\_cache does so much work on every call, how is it so fast? Enquiring minds want to know! ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [April 21, 2020, 1:35pm UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/13 "2020-04-21T13:35:03Z")
> [@steven.daprano](#): > > But … how? > > lru\_cache does so much work on every call, how is it so fast? Enquiring minds want to know! The wrapper is written in C. (Which implies that to get a useful level of speed, the wrapper applied by any proposed `once` decorator would need to be in C as well…) @orf are you aware that this is going to be a C code change? ---
**Author:** ![orf](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/orf/32/1889_2.png) [@orf](https://discuss.python.org/u/orf)\ **Post date:** [April 21, 2020, 1:47pm UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/14 "2020-04-21T13:47:42Z")
Yep, and I’m eager to try. I’ll start with the optimization, then take a stab at making a patch for `once()`. I’ll then post to the python-ideas mailing list and see if I can get a consensus on the PR, and see if it needs a PEP or not. ---
**Author:** ![steven.daprano](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steven.daprano/32/1083_2.png) [@steven.daprano](https://discuss.python.org/u/steven.daprano)\ **Post date:** [April 22, 2020, 2:41am UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/15 "2020-04-22T02:41:35Z")
[Paul Moore] > (Which implies that to get a useful level of speed, the wrapper > applied by any proposed `once` decorator would need to be in C as > well…) Sorry, it doesn’t imply that at all. As Serhiy asked, paraphrasing, are you sure that 40ns difference is a bottleneck in your application? It may be that the Python version is “fast enough”. It’s not that I prefer the slower solution because it is slower, but the primary aim here for me is clarity. Also, there ought to be a Python version, even if it is replaced with an accelerated version: ``` def once(): ... try: from _functools import once except ImportError: pass ``` for the benefit of other implementations. ---
**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\ **Post date:** [April 22, 2020, 7:19am UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/16 "2020-04-22T07:19:32Z")
> [@steven.daprano](#): > > Sorry, it doesn’t imply that at all. As Serhiy asked, paraphrasing, are > you sure that 40ns difference is a bottleneck in your application? It > may be that the Python version is “fast enough”. My point is that a pure Python version won’t1 be faster than using a C-accelerated `lru_cache`, and if `once` can’t out-perform `lru_cache` there’s no point (beyond naming2, which can be covered by `once=lru_cache`…) I totally agree that this discussion is all about a micro-optimisation that hasn’t yet been demonstrated to be worth the cost. > [@steven.daprano](#): > > Also, there ought to be a Python version, even if it is replaced with an accelerated version Correct, that’s just standard policy though ([PEP 399](https://www.python.org/dev/peps/pep-0399/)). 1 Conceded, that’s an unproven assertion.I think it’s likely, though 😉 2 I assume when you say “clarity” you mean “of the user’s code”, rather than “of the implementation”. ---
**Author:** ![storchaka](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/storchaka/32/217_2.png) [@storchaka](https://discuss.python.org/u/storchaka)\ **Post date:** [April 23, 2020, 6:38pm UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/17 "2020-04-23T18:38:40Z")
`try/catch NameError` may be faster than `if is None` (especially after implementing a zero-overhead try). I don’t know whether it will be faster than `lru_cache()`. ---
**Author:** ![orf](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/orf/32/1889_2.png) [@orf](https://discuss.python.org/u/orf)\ **Post date:** [April 23, 2020, 7:21pm UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/18 "2020-04-23T19:21:21Z")
I didn’t want to bump the thread unnecessarily, but it occurred to me that thread safety is also a bonus with the lru\_cache vs the pure-python implementations above. These work in a single threaded environment but the moment concurrent threads might call the function you end up with your “once” function being called two or more times. So the “simple” implementations above are not equivalent, and the equivalent with locking is obviously not something nice to repeatedly write. ---
**Author:** ![storchaka](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/storchaka/32/217_2.png) [@storchaka](https://discuss.python.org/u/storchaka)\ **Post date:** [April 24, 2020, 5:05pm UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/19 "2020-04-24T17:05:43Z")
It is easier to implement the variant with locking in Python than in C. There is not a special C API for reentrant locks. `lru_cache()` does not guarantee that the function will be called only once with the same argument. It only guarantees that the internal structure will not be broken and there will be no leaks or crashes even if it will be called multiple times. ---
**Author:** ![orf](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/orf/32/1889_2.png) [@orf](https://discuss.python.org/u/orf)\ **Post date:** [April 26, 2020, 2:13pm UTC](https://discuss.python.org/t/reduce-the-overhead-of-functools-lru-cache-for-functions-with-no-parameters/3956/20 "2020-04-26T14:13:04Z")
I worked on a simple implementation and found that the speed benefits are probably not worth the changes, especially given that using `lru_cache` like this isn’t semantically correct. I’ve started a thread on the python-ideas mailing list about adding `functools.once`: [https://mail.python.org/archives/list/python-ideas@python.org/thread/5OR3LJO7LOL6SC4OOGKFIVNNH4KADBPG/](https://mail.python.org/archives/list/python-ideas@python.org/thread/5OR3LJO7LOL6SC4OOGKFIVNNH4KADBPG/) Thank you to everyone who chipped in here 🙂