Should we remove the `WITH_FREELISTS` macro?

The WITH_FREELISTS macro determines whether or not we use freelists when allocating common objects.
It was added a three years ago in bpo-45522, without much justification IMO.

It complicates the code around freelists making refactoring and optimizations more complicated.
AFAICT it is not used when building or testing for any of the platforms we support.

Is there any reason not to remove it?

4 Likes

Looks like it was for experimenting with mimalloc, which I believe we’ve now switched to? It’s probably worth testing whether the freelists are still necessary at all (that seems to be the suggestion in the issue).

I suspect they are (worth keeping), but it may make refactoring and optimisations even easier if the whole thing is just gone. I think I saw @colesbury working on it recently - he’ll probably have a more informed opinion than mine.

1 Like

I’d like to hear what @vstinner has to say.

Not @vstinner , but I agree with @steve.dower that 1) the freelists are likely to be beneficial in at least some cases (even if your allocator is fast, it still has more work to do than a specialized, type-specific memory reuse mechanism) and 2) it would be good to get benchmark numbers to evaluate the benefit of freelists before we remove the option entirely.

I’m not suggesting removing freelists.
I’m suggesting removing the WITH_FREELISTS macro which, despite its name, is for turning off freelists.

1 Like

I understood this. I was clarifying that the purpose was to allow testing with and without to see which performed better in light of a new allocator. Now that we have the allocator, we can do the testing and choose whether to remove the option or remove the feature.

I don’t think we should remove the option without testing whether it’s worth removing the whole thing.

1 Like

What do you mean by the whole thing?
If you mean freelists, then no, it is not worth removing them. We’ve already done that experiment.

IMO, we shouldn’t be leaving around macros for hypothetical experiments.
The experiments can be done easily enough without the macros, and they are a maintenance burden.

2 Likes

FWIW, I agree with Mark about removing the macro and the --without-freelists configure option, but I don’t feel particularly strongly about it one way or the other.

2 Likes

I agree about removing WITH_FREELISTS macro and configure option as it complicates the freelist code.

3 Likes

I also agree on removing the config option and always use free lists.

2 Likes

I also agree on removing the config option and always use free lists.

Agree. The macro was introduced by this issue. Please take a look if someone need to remind why the macro was introduced.
(Oh Mark already linked…)

One reason for the experimentation was to easily achieve thread safety by using mimalloc’s feature, but I don’t see any benefit after we switched free lists to use the per-thread allocation for the free-threaded build.

So I think that we can remove the related macros and configuration too.

1 Like

If the experiment is done, then go for it.

I only proposed a condition to check to help make the decision, based on the original proposal (as others have since repeated). If it’s been checked already, then you know the answer :slight_smile:

1 Like

Finally, we removed the macro during this year’s sprint!
Thank you for revisiting this macro!

3 Likes