I have failed making a MRE, which is why I put a question mark.
Essentially, I have a caching library, that uses pickle to load the files from disk.
Reading from disk is implemented the following:
t = []
def READFROM(s,cache,):
if s._filename is None:
return
if os.path.isfile(s._filename):
with open(s._filename,'rb')as f:
_bytes = f.read()
s.t.append(_bytes)
print(f'{len(_bytes)=}')
#pickle.loads(_bytes)
s._object = pickle.loads(_bytes)
print(f'{type(s._object)=} {len(s._object)=}')
The commented out line will be of interest, as it unexpectedly changes the behaviour of pickle.loads() in the succeeding line.
The field t and the prints will be used for debugging.
The file holds a pickled dict.
This is the code example:
from utils.model import stonks
import pickle
stonks.portfolio.AbstractPortfolio.set_cache_file(1)
for i in range(3):
stonks.portfolio.AbstractPortfolio.ledger.READFROM()
print('now reconstructing')
for o in stonks.portfolio.AbstractPortfolio._mode.t:
print(f'{len(pickle.loads(o))=}')
The imported library stonks essentially utilizes the given caching behaviour as a ledger to load known portfolios.
ledger.READFROM() is the READFROM() function defined above.
We will invoke READFROM() multiple times, in essence reloading the cache from the file on disk multiple times.
Then, we will use the t that accumulated the bytes in READFROM() to reconstruct the objects using pickle.loads() to compare the results.
len(_bytes)=1269
type(s._object)=<class 'dict'> len(s._object)=11
len(_bytes)=1269
type(s._object)=<class 'dict'> len(s._object)=0
len(_bytes)=1269
type(s._object)=<class 'dict'> len(s._object)=11
len(_bytes)=1269
type(s._object)=<class 'dict'> len(s._object)=0
now reconstructing
len(pickle.loads(o))=11
len(pickle.loads(o))=11
len(pickle.loads(o))=11
len(pickle.loads(o))=11
We see that the bytes are all read successfully each time, but the object alternatingly is loaded (len==11) and empty (len==0,not failing to load).
But the stored bytes that we now reconstruct with pickle.loads again are all loaded (len==11).
To note, I also checked the 4 stored bytes objects in t, and they all refer to the same bytes object (is keyword check).
Now it gets interesting.
When we uncomment the line from earlier, suddenly each of the 4 times _object is loaded.
When we instead put the uncommented line 1 line lower [after s._object = pickle.loads(_bytes) and before print(f’{type(s._object)=} {len(s._object)=}')], all 4 times _object is empty.
When we put the uncommented line 2 lines lower [after print(f’{type(s._object)=} {len(s._object)=}')], all 4 time _object is loaded.
Thanks for your time and answer