Bad alloc Out of memory Ram

I’m priting a complex Python project using multiprocessing library. I have memorey used problems. Is difficult to describe entire project, but I would ask suggestions. Often code stops when I try to copy many big files and read many files in different processes (launched in multiprocessing mode). I would ask if is possible to add a check of availability ram memory and retrying a process when memory is available. For example if copy file stops for bad allocation memory, could be possible to repeat copy command (as other many commands) until process successes, without stopping entire project? Unfortunately I don’t have more ram memory available.

thank you very much
Best regards

I’m priting a complex Python project using multiprocessing library.

Which operating system?

I have memorey used problems.

How do you measure that? e.g. (on Linux):

$ free -h -w

Often code stops when I try to copy many big files and read
many files in different processes (launched in multiprocessing mode).

Error message?

I would ask if is possible to add a check of availability ram memory

On Linux, you can parse /proc/meminfo for information about memory usage.

e.g.:

{y[0]: y[1].strip()
 for x in open("/proc/meminfo").readlines()
 if (y := x.split(":"))}

to learn how to interpret the dictionary: man free.

and retrying a process when memory is available.

If you use subprocess to run the copy (e.g. cp from to) you can loop
for n attempts.

Is your subprocess getting OOM-killed? (OOM = Out Of Memory)