A colleague of mine posted a PR (written by LLM) too help with some OOM issues we are seeing with one of our services running in an K8s node. The PR adds a call to gc.collect(2) that I understand but also a call to malloc_trim(0) that neither of us do. I am trying to understand what exactly this call is doing and if it is in fact needed. Any assistance would be helpful.
For a bit more background - this is the main loop of the service which waits for a message and when it gets one it creates a new object to handle the message. Inside the loop there are no circular references in the objects (although it is possible that the object itself contains circular references.) When there is no message the object created from the previous message persists (as part of this PR we are also setting it to None before the gc.collect(2) call.
(2) but also if you’re facing OOMs, that means you’re running something close to the edge so any little trick might temporarily help keep something under the wire. By the time it comes to adding gc.collect or related calls to an application, while there are situations it helps in, I’m always wary of their being larger issues to track down even if that keeps things running for a while.
Python internally uses its own allocation arenas for “small” <= 512 byte objects so those don’t free up immediately (OS’s aren’t that fine grained anyways), but when freeing larger things or emptying an arena pool that, goes back to the system malloc which can return it to the OS. malloc_trim is merely an advisory trick to tell glibc malloc, for example, to more aggressively unmap such free()'d pages out of the process back to the OS rather than delaying and holding some for potential reuse.
Asking a good LLM should also give an answer like this FWIW. (ex: I asked Claude Fable 5 within the context of a CPython repo to confirm the above before re-stating it in my own words as a check on my own memory of the current state of our implementation)
(3) The specific (including patch level) version of your python build and whether it was an experimental free-threading build or not also matters as the latter --disable-gil build will use its own mimalloc so malloc_trim likely does nothing other than for maybe some third party extension module allocations. version example: 3.14.5 reverted the garbage collector behavior in 3.14.0-3.14.4 to that of 3.13.x due to the new one antagonizing some applications. underlying malloc patterns stay the same, but the python level garbage collector behavior itself changes.
We’re running 3.13.13 for this service so it’s not the changes to the GC (I was aware of those.)
We have used memray and it is showing what we expect, including that the memory each event’'s processing consumed is freed. But when looking at the system level settings, the memory consumption by the python process does not go down when an event finishes processing.
This is a long running process running in kubernetes. We never handle more than 1 event at a time and the largest memory hog event takes <3GB of RAM and the pods are configured for 6GB. There should be plenty of headroom.
I think the word “can” is playing an important role in this sentence. glibccan return it to the OS. From what we can tell, it doesn’t look like it does which is why the LLM suggested adding the call to malloc_trim. We have been running a version of this service in our staging environment that includes calls to gc.collect(2) and malloc_trim(0) after each event finishes. With these changes we have seen the system level memory usage of the python process drop down to the same level it was prior the start of the event.
The middle pod below ran one of those beefy jobs with the malloc_trim code and you can see that the memory usage went back to normal after the spike when the event processing finished.
