Permission denied to a .lock file that doesn't exist

Hi !

I am trying to run a script that imports the following script :

## Created by Pierre 01/04/2021

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import scipy.interpolate as itp
import tables
from pykeops.numpy import LazyTensor as LazyTensor_np

class UMazeLinearizer:
    # A class to define a linearization function of the data.
    # Depending on the maze shape, user might want to change this class
    # to fit to their maze shape.
    def __init__(self, folder):
        with tables.open_file(folder + 'nnBehavior.mat', "a") as f:
            children = [c.name for c in f.list_nodes("/behavior")]
            if "linearizationPoints" in children:
                self.nnPoints =f.root.behavior.linearizationPoints[:]
            else:
                self.nnPoints = [[0.45, 0.40], [0.45, 0.65], [0.45, 0.9], 
                                 [0.7, 0.9], [0.9, 0.9], [0.9, 0.7], [0.9, 0.4]]

            ts = np.arange(0, stop=1, step=1 / np.array(self.nnPoints).shape[0])
            itpObject = itp.make_interp_spline(ts, np.array(self.nnPoints), k=2)
            self.tsProj = np.arange(0, stop=1, step=1 / 100)
            self.mazePoints = itpObject(self.tsProj)
            
    def apply_linearization(self, euclideanData, keops=True):
        if keops:
            return self.pykeops_linearization(euclideanData)
        else:
            projectedPos = np.zeros([euclideanData.shape[0], 2])
            linearFeature = np.zeros([euclideanData.shape[0]])
            for idp in range(euclideanData.shape[0]):
                bestPoint = np.argmin(np.sum(np.square(np.reshape(euclideanData[idp,:], 
                        [1,euclideanData.shape[1]]) - self.mazePoints),axis=1),axis=0)
                projectedPos[idp,:] = self.mazePoints[bestPoint,:]
                linearFeature[idp] = self.tsProj[bestPoint]
            
            return projectedPos,linearFeature

    def pykeops_linearization(self,euclideanData):
        if euclideanData.dtype != self.mazePoints.dtype:
            euclideanData = euclideanData.astype(self.mazePoints.dtype)

        euclidData_lazy = LazyTensor_np(euclideanData[None,:,:])
        mazePoint_lazy = LazyTensor_np(self.mazePoints[:,None,:])

        distance_matrix_lazy = (mazePoint_lazy - euclidData_lazy).square().sum(axis=-1)
        #find the argmin
        bestPoints = distance_matrix_lazy.argmin_reduction(axis=0)
        projectedPos = self.mazePoints[bestPoints[:,0],:]
        linearPos = self.tsProj[bestPoints[:,0]]

        return projectedPos, linearPos

    def verify_linearization(self, ExampleEuclideanData, folder, overwrite=False):
        ## A function to verify and possibly change the linearization.
        # Auxiliary
        def try_linearization(ax, l0s):
            _, linearTrue = self.apply_linearization(euclidData)
            binIndex = [np.where((linearTrue >= projBin[id]) * 
                                 (linearTrue < projBin[id + 1]))[0] for id in
                                 range(len(projBin) - 1)]
            cm = plt.get_cmap("tab20")
            for tpl in enumerate(binIndex):
                id, bId = tpl
                try:
                    l0s[id].remove()
                except:
                    None
                l0s[id] = ax[0].scatter(euclidData[bId, 0], euclidData[bId, 1], 
                                        c=[cm(id)])
            return l0s
        def b1update(n):
            self.nnPoints = [[0.45, 0.40], [0.45, 0.65], [0.45, 0.9], [0.7, 0.9], 
                             [0.9, 0.9], [0.9, 0.7], [0.9, 0.4]]
            # create the interpolating object
            ts = np.arange(0, stop=1, step=1 / np.array(self.nnPoints).shape[0])
            itpObject = itp.make_interp_spline(ts, np.array(self.nnPoints), k=2)
            self.tsProj = np.arange(0, stop=1, step=1 / 100)
            self.mazePoints = itpObject(self.tsProj)
            try:
                self.lPoints.remove()
                fig.canvas.draw()
            except:
                pass
            self.l0s = try_linearization(ax, self.l0s)
            self.lPoints = ax[0].scatter(np.array(self.nnPoints)[:, 0], 
                                         np.array(self.nnPoints)[:, 1], c="black")
            fig.canvas.draw()
        def b2update(n):
            if len(self.nnPoints)>0:
                self.nnPoints = self.nnPoints[0:len(self.nnPoints)-1]
                # create the interpolating object
                ts = np.arange(0, stop=1, step=1 / np.array(self.nnPoints).shape[0])
                itpObject = itp.make_interp_spline(ts, np.array(self.nnPoints), k=2)
                self.tsProj = np.arange(0, stop=1, step=1 / 100)
                self.mazePoints = itpObject(self.tsProj)
                self.lPoints.remove()
                fig.canvas.draw()
                if (len(self.nnPoints)>2):
                    self.l0s = try_linearization(ax, self.l0s)
                else:
                    self.l0s[1] = ax[0].scatter(euclidData[:,0],euclidData[:,1], 
                                                c="blue")
                self.lPoints = ax[0].scatter(np.array(self.nnPoints)[:, 0], 
                                             np.array(self.nnPoints)[:, 1], c="black")
                fig.canvas.draw()
        def b3update(n):
            if len(self.nnPoints) > 0:
                self.nnPoints = []
                self.lPoints.remove()
                self.l0s[1] = ax[0].scatter(euclidData[:,0],euclidData[:,1], c="blue")
                fig.canvas.draw()
        def onclick(event):
            if event.inaxes==self.l0s[1].axes:
                self.nnPoints += [[event.xdata,event.ydata]]
                try: 
                    self.lPoints.remove()
                    fig.canvas.draw()
                except:
                    pass
                if (len(self.nnPoints) > 2):
                    # create the interpolating object
                    ts = np.arange(0, stop=1, step=1 / np.array(self.nnPoints).shape[0])
                    itpObject = itp.make_interp_spline(ts, np.array(self.nnPoints), k=2)
                    self.tsProj = np.arange(0, stop=1, step=1 / 100)
                    self.mazePoints = itpObject(self.tsProj)

                    self.l0s = try_linearization(ax, self.l0s)
                else:
                    self.l0s[1] = ax[0].scatter(euclidData[:, 0], euclidData[:, 1], 
                                                c="blue")
                self.lPoints = ax[0].scatter(np.array(self.nnPoints)[:, 0], 
                                             np.array(self.nnPoints)[:, 1], c="black")
                fig.canvas.draw()
        
        # Check existence of linearized data
        with tables.open_file(folder + 'nnBehavior.mat',"a") as f:
            children = [c.name for c in f.list_nodes("/behavior")]
            if "linearizationPoints" in children:
                print("Linearization points have been created before")
                if overwrite:
                    f.remove_node("/behavior","linearizationPoints")
                    print("Overwriting linearization")
                else:
                    return
            # Body            
            euclidData = ExampleEuclideanData[np.logical_not(np.isnan(
                        np.sum(ExampleEuclideanData, axis=1))), :]
            euclidData = euclidData[1:-1:10,:] #down sample a bit
            projBin = np.arange(0, stop=1.2, step=0.2)
            self.l0s = [None for _ in projBin]
            
            # Figure
            fig = plt.figure()
            gs = plt.GridSpec(3, 2, figure=fig)
            ax =[fig.add_subplot(gs[:,0]),fig.add_subplot(gs[0,1]), 
                 fig.add_subplot(gs[1,1]),fig.add_subplot(gs[2,1])]
            self.l0s = try_linearization(ax, self.l0s)
            self.lPoints = ax[0].scatter(np.array(self.nnPoints)[:, 0], 
                                         np.array(self.nnPoints)[:, 1], c="black")
            ax[0].set_aspect(1)
            b1 = plt.Button(ax[1],"reset",color="grey")            
            b1.on_clicked(b1update)
            b2 = plt.Button(ax[2],"remove last",color="orange")
            b2.on_clicked(b2update)
            b3 = plt.Button(ax[3], "empty", color="red")
            b3.on_clicked(b3update)
            # Next we obtain user click to create a new set of linearization points
            [a.set_aspect(1) for a in ax]
            fig.canvas.mpl_connect('button_press_event', onclick)
            plt.show()
            # create the interpolating object
            ts = np.arange(0, stop=1, step=1 / np.array(self.nnPoints).shape[0])
            itpObject = itp.make_interp_spline(ts, np.array(self.nnPoints), k=2)
            self.tsProj = np.arange(0, stop=1, step=1 / 100)
            self.mazePoints = itpObject(self.tsProj)
            # plot the exact linearization variable:
            _, linearTrue = self.apply_linearization(euclidData)
            cm = plt.get_cmap("Spectral")
            fig,ax = plt.subplots()
            ax.scatter(euclidData[:,0],euclidData[:,1],c=cm(linearTrue))
            ax.set_title("Linearization variable, Spectral colormap")
            plt.show()
            # Save
            f.create_array("/behavior", "linearizationPoints", self.nnPoints)
            f.flush()
            f.close()

However, for the line bestPoints = distance_matrix_lazy.argmin_reduction(axis=0) of the function pykeops_linearization() (see below), I get the following error message :

Exception has occurred: PermissionError
[Errno 13] Permission denied: ‘/home/mobs/.cache/pykeops-1.5-cpython-38//build-483f6db833/pykeops_build2.lock’
File “/home/mobs/Dropbox/Mobs_member/Basile/Code/full_neuroEncoder/neuroEncoders-master/transformData/linearizer.py”, line 51, in pykeops_linearization
bestPoints = distance_matrix_lazy.argmin_reduction(axis=0)
File “/home/mobs/Dropbox/Mobs_member/Basile/Code/full_neuroEncoder/neuroEncoders-master/transformData/linearizer.py”, line 30, in apply_linearization
return self.pykeops_linearization(euclideanData)
File “/home/mobs/Dropbox/Mobs_member/Basile/Code/full_neuroEncoder/neuroEncoders-master/transformData/linearizer.py”, line 61, in try_linearization
_, linearTrue = self.apply_linearization(euclidData)
File “/home/mobs/Dropbox/Mobs_member/Basile/Code/full_neuroEncoder/neuroEncoders-master/transformData/linearizer.py”, line 161, in verify_linearization
self.l0s = try_linearization(ax, self.l0s)
File “/home/mobs/Dropbox/Mobs_member/Basile/Code/full_neuroEncoder/neuroEncoders-master/test.py”, line 88, in
Linearizer.verify_linearization(DataHelper.position/DataHelper.maxPos(),

Here is the function in question :

def pykeops_linearization(self,euclideanData):
        if euclideanData.dtype != self.mazePoints.dtype:
            euclideanData = euclideanData.astype(self.mazePoints.dtype)

        euclidData_lazy = LazyTensor_np(euclideanData[None,:,:])
        mazePoint_lazy = LazyTensor_np(self.mazePoints[:,None,:])

        distance_matrix_lazy = (mazePoint_lazy - euclidData_lazy).square().sum(axis=-1)
        #find the argmin
        bestPoints = distance_matrix_lazy.argmin_reduction(axis=0)
        projectedPos = self.mazePoints[bestPoints[:,0],:]
        linearPos = self.tsProj[bestPoints[:,0]]

        return projectedPos, linearPos

I don’t understand the source of this error, I went to ''/home/mobs/.cache/pykeops-1.5-cpython-38//build-483f6db833' and couldn’t see any file called pykeops_build2.lock. I tried asking chatGPT who told me to run sudo chmod +rw '/home/mobs/.cache/pykeops-1.5-cpython-38/build-483f6db833' to grant permission but it didn’t change anything.

Would you have an idea of what I should do to make that error go away and never come back ? :slight_smile:

However, for the line bestPoints = distance_matrix_lazy.argmin_reduction(axis=0) of the function
pykeops_linearization() (see below), I get the following error
message :

Exception has occurred: PermissionError
[Errno 13] Permission denied: ‘/home/mobs/.cache/pykeops-1.5-cpython-38//build-483f6db833/pykeops_build2.lock’
File “/home/mobs/Dropbox/Mobs_member/Basile/Code/full_neuroEncoder/neuroEncoders-master/transformData/linearizer.py”, line 51, in pykeops_linearization
bestPoints = distance_matrix_lazy.argmin_reduction(axis=0)
File “/home/mobs/Dropbox/Mobs_member/Basile/Code/full_neuroEncoder/neuroEncoders-master/transformData/linearizer.py”, line 30, in apply_linearization
return self.pykeops_linearization(euclideanData)
File “/home/mobs/Dropbox/Mobs_member/Basile/Code/full_neuroEncoder/neuroEncoders-master/transformData/linearizer.py”, line 61, in try_linearization
_, linearTrue = self.apply_linearization(euclidData)
File “/home/mobs/Dropbox/Mobs_member/Basile/Code/full_neuroEncoder/neuroEncoders-master/transformData/linearizer.py”, line 161, in verify_linearization
self.l0s = try_linearization(ax, self.l0s)
File “/home/mobs/Dropbox/Mobs_member/Basile/Code/full_neuroEncoder/neuroEncoders-master/test.py”, line 88, in
Linearizer.verify_linearization(DataHelper.position/DataHelper.maxPos(),

Is this really the entire traceback? I do not see anything in that
which I would expect to fire Permission denied.

I don’t understand the source of this error, I went to
''/home/mobs/.cache/pykeops-1.5-cpython-38//build-483f6db833' and
couldn’t see any file called pykeops_build2.lock.

That’s to be expected: you didn’t have permission to create it!
(Actually, I suppose an existing lock file to which you lack access can
also make this exception, depending on what is being attempted.)

What are the permissions on /home/mobs/.cache/pykeops-1.5-cpython-38//build-483f6db833. try running:

 ls -ld /home/mobs/.cache/pykeops-1.5-cpython-38//build-483f6db833

For example, creating a file requires write permission to the directory
where you want to create the file (because you’re making a new name in
that directory, which is considered writing to the directory).

I tried asking chatGPT who told me to run sudo chmod +rw '/home/mobs/.cache/pykeops-1.5-cpython-38/build-483f6db833' to grant
permission but it didn’t change anything.

Never go to ChatGPT or its ilk for technical advice. Leaving aside that
they make stuff up, they don’t understand your queries. Effectively
they’re regurgitating some text which resembles other text they’ve
seen which looks like an answer to questions which look like yours.

Also: NEVER fiddle permissions without understanding what change you’re
trying to make.

Would you have an idea of what I should do to make that error go away
and never come back ? :slight_smile:

I suspect you have not shown us the full traceback.

Yes I assure you it was the full error. However, in the end it was just because the wrong version of pykeops was installed, even when I uninstalled it and reinstalled it using pip install. I had to run pip install pykeops==2.1 otherwise it would install 1.5, and now it works.

1 Like

It is definitely not the full error traceback. For one thing, the line of code shown at the end is clearly cut off in the middle (the parentheses don’t balance), and it’s missing a fucntion name on the previous line (“line 88, in”).

However, it’s entirely possible that the library has mangled the traceback for you.

A traceback that isn’t caught or logged by anything would look different from this. The first line would say Traceback (most recent call last): (except for syntax errors), and the name and message of the exception would be at the end.

At any rate, please format error messages the same way as code. A Python exception traceback is designed to be displayed in a terminal; it uses indentation at the beginning of lines to show structure, and in more recent versions of Python it may use ^ and ~ to “underline” parts of the code.

1 Like