Get total live bytes across CPython's allocators

Ideas
1

I’d like to gauge interest in adding a method sys.getallocatedbytes that returns the total bytes currently allocated across CPython’s allocators (the raw domain plus the active object allocator, pymalloc or mimalloc).

Why? This would be useful to get fine-grained measurement of memory, for example to track import overhead.

How?

  • Throughout the program we would keep track of the memory allocated from the raw domain (_PyMem_Raw* methods) by adding/subtracting to a counter when the respective methods are called (using malloc_usable_size() / malloc_size() / _msize() to record the true block size).
  • Then when sys.getallocatedbytes() is called we would sum that value with a walk over the active object allocator’s live allocations — pymalloc’s in-use pools, or mimalloc’s heaps
2

How is this different from tracemalloc.get_traced_memory?

2 Likes
3

My thoughts:

  1. tracemalloc is a lot heavier for both CPU and memory. I don’t know the details of tracemalloc that well, but it seems like every malloc/calloc/realloc/free goes through tracemalloc_alloc, which has significant overhead.
    With sys.getallocatedbytes() there’s just one relaxed atomic add in the _PyMem_Raw* methods. When the call does happen, there is some additional overhead though - a Stop The World call and walking arenas. So the tradeoff would be much less overhead throughout the program, but a non-trivial amount of overhead when the sys.getallocatedbytes() call is made.
  2. tracemalloc records the requested size, whereas getallocatedbytes would record the actual footprint.
  3. sys.getallocatedbytes() is more accurate - with tracemalloc everything before the start() call isn’t tracked and even if you use the environment variable to get it going as early as possible this will still miss some stuff (e.g. core types allocation)
  4. sys.getallocatedbytes() is easier to use - you don’t have to call tracemalloc.start().