Intermittent File locking collisions

Python Help
1

Greetings

In trying to track down the cause of some file corruption in a shared filesystem (AWS EFS), I have discovered some strange behaviour.

I have used the following setup to execute to processes trying to take a filelock on the same file at the same time:

import fcntl
import struct
import sys
import time

with open("test_file.lock", "w+") as f:
    fd = f.fileno()
    exitcode = 0
    try:
        fcntl.fcntl(fd, fcntl.F_SETLK, struct.pack("hhqqi", 1, 0, 0, 0, 0))
    except OSError as e:
        exitcode = e.errno
    time.sleep(0.01)
    sys.exit(exitcode)

Note: I’m using fcntl.fcntl() instead of fcntl.flock() as I was exploring being able to get the PID of the process that holds the lock on the file.

I then used the following bash script to execute the python script 1000 times in a row:

count=$1
declare -a my_arr
for i in $(seq "$count"); do
  PYTHONUBUFFERED=1 python lock_test2.py
  my_arr+=("$?")
done
printf '%s\n' "${my_arr[@]}"

The weird behaviour that I’m experiencing is due to the time.sleep() before the sys.exit() call.

If I omit the sleep, both processes output exitcode == 0 989/1000 times. If I increase the delay to 1 microsecond, they both “acquire” a lock 454/1000. At 1 millisecond, it’s 95/1000, and if I set it to 10 milliseconds, the lock seems to always be respected.

The issue is, I can’t work out why this is causing a difference in output, given that the sleep is after the lockfile is opened and a lock is taken, not during or before.

My best guess is that it’s something to do with how Python is making the underlying system calls, and that certain low-level operations may not be happening in the order expected by looking at the python script.

Environment details:

  • Python 3.13.13 (main, May 28 2026, 12:11:00) [GCC 16.1.1 20260515 (Red Hat 16.1.1-2)]
  • Operating System: Fedora Linux 44 (Workstation Edition)
  • Kernel: Linux 7.1.13-200.fc44.x86_64
2

But before the file is closed. And notably, you call sys.exit inside the with block, meaning that the call to exit has begun while the file’s still open. Does it make any difference if you pull that out of the with? What if you pull the sleep out as well?

3

Hi Chris,

I just repeated the test twice.

The first time, I kept the sleep inside the with block, and moved the sys.exit() outside the block. Doing that gave largely the same results: 99%/30%/1%/0% overlap in claiming the lock for no delay/1us/1ms/10ms.

Moving the sleep outside the with block, resulted in both processes reporting holding the lock 99% of the time.

My other guess as for what’s happening is the method I’m running both processes in parallel is introducing millisecond delays: I’m using a split-tab terminal to execute the script twice at the same time.

4

@Gmilway AWS EFS appears to use NFS and NFS and network filesystems in general are file lock antagonistic. Do you have some information that the AWS EFS implementation of NFS supports distributed locks? Likewise for your NFS client stack?

5

Okay, that’s good to know. Would have been annoying to debug else. (sys.exit works by raising an exception, so normally that ought to unwind things the same way that returning would; but it would be possible that the lock handler has some sort of issue in its handling of exceptions. Glad that that’s not the case.)

That’s what I would expect, since the lock is now held for much less time.

Hmm. Quick check. Do you know WHAT errno you’re getting when it fails? Is it always consistent?