More memory usage of script than expected

Python Help
1

Hi everyone,

I’m runnig into some memory problems when executing my python script. I’m stacking images which are cropped and in another step adjusted. The code works fine but the memory usage is higher than expected. The array will have a size of 77110001500 dtype uint8 after stacking and I’m using about 15 Gb. Size array should be around 1 Gb with sys.getsizeof(). Before my loop I’m creating an empty array with the size of the final array.

In the step of adjusting there is another memory bump of about 1.6 Gb, which is about the expected size for the new array (116610001500 dtype uint8). The required memory doesn’t change in the loop of adjusting the images.

Do you have any idea why it takes so much more space in the first loop? The reason I’m asking is because I plan to execute this on an raspberry pi and I’m limited to 8 Gb memory.

Here are my loops:

    def construct_3d_matrix(self):
        """
        this method crop, scale each layer to neglect distance to camera, concatenate all layers along y-axis
        and scales each concatenated layer (new layer in x-axis and y-axis) to adjust size of pixel
        :return: shape of matrix
        """
        self._get_roi()

        max_intensity = 0
        concatenated_layers = np.empty(shape=[self.num_layers, self.roi['height'], self.roi['width']], dtype=np.uint8)

        for i, layer in enumerate(tqdm.tqdm(self.layers)):
            img = layer.img

            scaled_x_left = int(self.roi['x'] * (1 - self.config.delta_m * layer.layer_num / 2))
            scaled_x_right = int((self.roi['x'] + self.roi['height']) * (1 + self.config.delta_m * layer.layer_num / 2))
            scaled_z_below = int(self.roi['z'] * (1 - self.config.delta_m * layer.layer_num / 2))
            scaled_z_above = int((self.roi['z'] + self.roi['width']) * (1 + self.config.delta_m * layer.layer_num / 2))

            orig_crop = img[scaled_x_left:scaled_x_right, scaled_z_below:scaled_z_above]
            scaled_crop = cv2.resize(orig_crop, dsize=(self.roi['width'], self.roi['height']),
                                     interpolation=cv2.INTER_LANCZOS4)
            #scaled_crop = np.flipud(color.rgb2gray(scaled_crop * ((1 + self.config.delta_m * layer.layer_num) ** 2)))
            scaled_crop = np.flipud(scaled_crop[:,:,0] * ((1 + self.config.delta_m * layer.layer_num) ** 2))


            concatenated_layers[i] = scaled_crop

        max_intensity = np.max(concatenated_layers)
        self._adjust_resolution(concatenated_layers, max_intensity)

        return self.processed_images.shape

    def _adjust_resolution(self, concatenated_layers, max_intensity):
        self.processed_images = np.empty(shape=[int(self.num_layers * self.config.scale), self.roi['height'], self.roi['width']],
                                         dtype=np.uint8)

        if max_intensity < 255:
            for z in tqdm.tqdm(range(self.processed_images.shape[2])):
                layer = concatenated_layers[:, :, z]
                self.processed_images[:, :, z] = cv2.resize(layer, dsize=(layer.shape[1], int(layer.shape[0] * self.config.scale)),
                                                interpolation=cv2.INTER_LANCZOS4)
                self.processed_images[:, :, z] = np.uint8(self.processed_images[:, :, z] / max_intensity * 255)
        else:
            for z in tqdm.tqdm(range(self.processed_images.shape[2])):
                layer = concatenated_layers[:, :, z]
                self.processed_images[:, :, z] = cv2.resize(layer, dsize=(layer.shape[1], int(layer.shape[0] * self.config.scale)),
                                                interpolation=cv2.INTER_LANCZOS4)
2

What OS are you using?
How are you measureing memory use?

Sometimes the obvious numbers are not the real memory used.

3

I’m currently testing on my windows 10 machine and I’m checking memory usage with task manager.

Memory usage is definetly higher than what the array size should be like. I already tested it on my RPi and it freezes in the first loop after about 320 images which is about the available system memory.

I’m wondering why preallocating the memory with concatenated_layers = np.empty() doesn’t seem to do anything since the memory usage is growing linearly with the images loaded during the loop.

4

Do you delete the images after processing them?
If the objects ref count does not go to 0 then the memory will not be released.

5

It should delete the images, this is the code when loading the image:

    @property
    def img(self):
        if self._img is None:
            self._img = io.imread(self.file_path)
        return self._img

It seems they are still stored in the memory. Image size is 210028003 which results in about 13 Gb of required memory.

I also have a Matlab version of this code which does the same but is not object oriented. It uses roughly 4 Gb peak memory.

6

You have not show enough code to show that self._img is ever deleted.
Try running a short version of your code that creates an instances of your class
The immediately delete it.
Now put a loop around that code and see if it leaks.
You can put a sys.stdin.readline() at the end of the test so that process does not exit.
Then you can check the memory size.

7 
from __future__ import annotations

import os
import tqdm
import re
import cv2
import numpy as np
from skimage import color, io

from layer import Layer
from job_config import JobConfig
from typing import Optional

class Job:

    def __init__(self, datadir_path: str,
                 layers: Optional[list[Layer]] = None,
                 num_layers: Optional[int] = None,
                 config: Optional[JobConfig] = None):
        self.processed_images = None
        self.layers = layers
        self.datadir_path = datadir_path
        self.num_layers = num_layers
        self.roi = {}

        if config is None:
            self.config = JobConfig()
        else:
            self.config = config

        if not self.layers and datadir_path:
            self.layers = [Layer(file_path=os.path.join(self.datadir_path, filename)) for filename in
                           sorted(os.listdir(self.datadir_path), key=lambda x: int(re.sub('\D', '', x)))]

        self.shape = self.layers[0].get_shape()

        if not self.num_layers:
            self.num_layers = len(self.layers)

        if not self.layers and not self.datadir_path:
            raise Exception('data source not provided')

    def construct_3d_matrix(self):
        """
        this method crop, scale each layer to neglect distance to camera, concatenate all layers along y-axis
        and scales each concatenated layer (new layer in x-axis and y-axis) to adjust size of pixel
        :return: shape of matrix
        """
        self._get_roi()

        max_intensity = 0
        concatenated_layers = np.empty(shape=[self.num_layers, self.roi['height'], self.roi['width']], dtype=np.uint8)

        for i, layer in enumerate(tqdm.tqdm(self.layers)):
            img = layer.img

            scaled_x_left = int(self.roi['x'] * (1 - self.config.delta_m * layer.layer_num / 2))
            scaled_x_right = int((self.roi['x'] + self.roi['height']) * (1 + self.config.delta_m * layer.layer_num / 2))
            scaled_z_below = int(self.roi['z'] * (1 - self.config.delta_m * layer.layer_num / 2))
            scaled_z_above = int((self.roi['z'] + self.roi['width']) * (1 + self.config.delta_m * layer.layer_num / 2))

            orig_crop = img[scaled_x_left:scaled_x_right, scaled_z_below:scaled_z_above]
            scaled_crop = cv2.resize(orig_crop, dsize=(self.roi['width'], self.roi['height']),
                                     interpolation=cv2.INTER_LANCZOS4)
            scaled_crop = np.flipud(scaled_crop[:,:,0] * ((1 + self.config.delta_m * layer.layer_num) ** 2))

            concatenated_layers[i] = scaled_crop

        max_intensity = np.max(concatenated_layers)
        self._adjust_resolution(concatenated_layers, max_intensity)

        return self.processed_images.shape

    def _get_nozzle_position(self) -> int:
        """
        :return: smallest z coordinate of the nozzle across layers
        """
        nozzle_positions = []
        for layer in tqdm.tqdm(self.layers[0:21]):
            nozzle_positions.append(layer.get_nozzle_pos())
        min_pos = min(nozzle_positions)
        return min_pos

    def _get_roi(self):
        """
        :return: region of interest in form of dict {'x' : x coordinate of bottom left point,
                                                     'z' : z coordinate of bottom left point,
                                                     'width' : size of roi in z direction,
                                                     'height' : size of roi in x direction}
        """
        width = self._get_nozzle_position()
        self.roi['z'] = width - self.config.z_offset
        self.roi['x'] = self.shape[0] // 2 - self.config.x_offset
        self.roi['width'] = self.config.z_offset
        self.roi['height'] = self.config.x_offset * 2
        return self.roi

    def _adjust_resolution(self, concatenated_layers, max_intensity):
        self.processed_images = np.empty(shape=[int(self.num_layers * self.config.scale), self.roi['height'], self.roi['width']],
                                         dtype=np.uint8)

        if max_intensity < 255:
            for z in tqdm.tqdm(range(self.processed_images.shape[2])):
                layer = concatenated_layers[:, :, z]
                self.processed_images[:, :, z] = cv2.resize(layer, dsize=(layer.shape[1], int(layer.shape[0] * self.config.scale)),
                                                interpolation=cv2.INTER_LANCZOS4)
                self.processed_images[:, :, z] = np.uint8(self.processed_images[:, :, z] / max_intensity * 255)
        else:
            for z in tqdm.tqdm(range(self.processed_images.shape[2])):
                layer = concatenated_layers[:, :, z]
                self.processed_images[:, :, z] = cv2.resize(layer, dsize=(layer.shape[1], int(layer.shape[0] * self.config.scale)),
                                                interpolation=cv2.INTER_LANCZOS4)


import skimage
from skimage import io
from skimage import color
import matplotlib.pyplot as plt
import numpy as np
import re
from PIL import Image


class Layer:

    def __init__(self, file_path: str):
        self.file_path = file_path
        self.layer_num = int(re.sub('\D', '', self.file_path))
        self._img = None

    @property
    def img(self):
        if self._img is None:
            self._img = io.imread(self.file_path)
            urn self._img

    def get_shape(self):
        return self.img.shape

    def to_grayscale(self):
        img_grayscale = np.uint8(self.img[:,:,0])
        return img_grayscale

    def get_nozzle_pos(self):
        gray = self.to_grayscale()
        gray_cropped = gray[:, self.INITIAL_OFFSET:]
        coords = np.where(gray_cropped == gray_cropped.max())
        return min(coords[1]) + self.INITIAL_OFFSET - self.NOZZLE_OFFSET

Those are my two files that I use. There is also a third one but it is only for initialisation and declaring some variables. I have tried to manually delete self._img but I only ended up with errors.

8

That lime sets things up for lots of _img to be memory at the same time.

How many Job objects do you have at the same tine?
Each Jobs object that exists will keep all the Layers in memory with there img.

9

I have 771 images which result in 2 arrays with the image data, one for cropping and another one for the adjusted resolution.

Theoretically, it should load one image at the time, crop it and put the cropped image into the array. Afterwards it should be deleted. This process will be repeated for all 771 images.

10

Given the memory issue I think you need to write tests to prove that theory.