Feature or enhancement
MAP_NORESERVE on the mmap module.
Do not reserve swap space for this mapping. When swap space is reserved, one has the guarantee that it is possible to modify the mapping. When swap space is not reserved one might get SIGSEGV upon a write if no physical memory is available. See also the discussion of the file /proc/sys/vm/overcommit_memory in proc(5). In kernels before 2.6, this flag had effect only for private writable mappings.
mmap is a good way to cache data that need to be read often, like dataset files that are read over and over again between epochs.
The problem is when the machine running it has a limited amount of memory, not all the data is required at the same time, so it’s ok for the system to unload it. The more RAM you have, the more “cache hits” you can get, but if the system has not enough memory to hold the whole dataset, it can just unload it and reload when a new page fault happens.
MAP_SHARED, that same piece of memory can be shared between, for example, PyTorch DataLoader workers.
I am not an high performance C computing expert so my expectations about this flag might be wrong. I don’t know if Linux actually releases this memory in a situation of memory pressure but I think it’s the smart thing to do.