Should tempfile.NamedTemporaryFile automatically flush the buffer?

Python Help
1

With this kind of code:

text = yaml.dump(secrets)
with tempfile.NamedTemporaryFile(mode="w") as f:
    f.write(text)
    self.backend.copy_in(
        source_path=f.name,
        target_path="/build/secrets.yaml"
    )

I ended up with an empty file in the target system.

I needed to add a flush manually…

text = yaml.dump(secrets)
with tempfile.NamedTemporaryFile(mode="w") as f:
    f.write(text)
    f.flush()
    self.backend.copy_in(
        source_path=f.name,
        target_path="/build/secrets.yaml"
    )

Is there a good reason why I need to do this manually?

2

Yes, tempfile.NamedTemporaryFile() creates a normal file and opens it as usual open(). This means that it has full buffering by default (but this can be changed as with open()).

In your case I would certainly not just flush the buffer but I would close the file before copying it:

text = yaml.dump(secrets)
with tempfile.NamedTemporaryFile(mode="w") as f:
    with f.file as temp_file:
        temp_file.write(text)
    self.backend.copy_in(
        source_path=f.name,
        target_path="/build/secrets.yaml"
    )

The context manager of the file object (f.file) takes care of closing the file.

3

Windows users should note that delete=True is the default behavior. This causes the system to immediately delete the file when it’s closed (i.e. the file is opened with FILE_FLAG_DELETE_ON_CLOSE), in which case the above practice of closing the underlying file object won’t work. That said, the copy_in() call probably wouldn’t work in Windows anyway. The file is opened with delete access, so subsequent opens have to share delete access, which is rare in practice (e.g. open() does not).

Ev2geny has a pull request that adds a new delete_on_close=True parameter. With the proposed behavior of delete=True with delete_on_close=False, the file is only deleted when the context manager exits. One can call the close() method without deleting the file, and, in Windows, the file can be opened again in most cases (provided that subsequent opens that don’t share delete access are closed before the context manager exits).

1 Like