tempfile.TemporaryFile and tempfile.TemporaryDirectory are very useful.
And when used as a context manager, quick and easy.
But when you need to keep the tempdir path around to use, you have to save the object created, and then call its name attribute to get the actual usable file path.
Not a huge deal, but if we made them PathLike, then they could be passed directly into file functions where they are needed.
Seems like one more step to making the standard library fully PathLike compatible.
Or am I missing something that this could break or make a mess of things?
There was a related thread here that petered out:
(how to a refer to another topic here???)
In that, the OP asked that the .name attribute return a Path object – which is a fine idea, but might break too much. In that thread, it was also posed to do what I am suggesting here: make the object PathLike – but that was never closed out.
Ahh, thanks – yes, makes little (no?) sense for TemporaryFile
But it would still be nice for: tempfile.TemporaryDirectory – as there is no such Python object as a “dir-like” – indeed a PathLike is the closest thing we have.
One thing we try to avoid is adding constructs to the standard library that are prone to encouraging resource leaks, and making TemporaryDirectory easier to use without passing it to a with statement first feels like it would be just such a trap (the temporary directory should still get cleaned up eventually when the object is destroyed, but it’s less controlled than doing it with a context manager).
If you need to keep a resource like a TemporaryDirectory instance around, then its preferable to stick it in a contextlib.ExitStack instance (with a known lifecycle), at which point stack.enter_context(tmpdir) returns the path as a string, the same as the as clause in a with statement.
Context managers are great, but I’ve mostly found them suitable for more straightforward “this needs to happen right now” contexts (get it? ).
Maybe it’s because I’m using an OOP structure, but in this case, I need to keep that temporary directory around to be written to for a long time – maybe not until the process quits, but pretty much so, I don’t know how to use a context manger for that use case.
I suppose I could make my cache object a context manager itself, and then wrap it’s creation in a with – but that doesn’t help the problem - I’d still have to write that code.
In short: having an object that will clean up after itself when deleted, and in the worst case, when the process ends, it VERY useful (and HARD to write on your own – my implementation was never robust)
Anyway, yes, it’s better to use a context manager when that’s appropriate, but does that mean we have to make it awkward to use TemporaryDirectory outside of one?
Consenting adults and all that.
Final point: maybe my dev team are idiots, but we have a LOT of code that creates temporary files and never deletes them at all – and this has caused problems with our production server. So while encouraging use of context managers is good, I think encouraging use of self-cleaning-up temporary files is even more important, and making TemporaryDirectory a bit easier to use would accomplish that.
The general idea: store an exit stack instance on the object with resources to manage, add any resources to be cleaned up to the stack as they are allocated, then making the owning object a context manager that calls the stack’s close method.
Rather than adding an entirely new recipe for that, it would make the most sense to modify the existing recipe in contextlib — Utilities for with-statement contexts — Python 3.12.4 documentation to work that way (I wrote the original stdlib recipes back when ExitStack was new, and this is a case where my recommendation has changed, but it didn’t occur to me to update the recipe in the docs).
Back on the original topic of this thread, in an OO use case for temporary directories, my inclination would be to have a read-only property on the owning object to encapsulate Path(self._tmpdir.name).
Hmm – I took a look at contextlib.ExitStack, and I don’t see how I could use it for my case. But even nif so, it’s a fair bit of work for what TemporaryDirectory already does so well.
Yeah, that’s a bit more elegant that what I did – thanks!
def main(...):
with ExitStack() as global_stack:
do_stuff(ctx=global_stack)
if __name__ == __main__:
main()
Now anywhere you want to do something that needs cleaning up you just push it into the global stack and then the closure of that stack is handled at top-level. No with is needed because there is already a top-level with enclosing everything else.
Alternatively the atexit module allows the same but without needing to pass any stack around.
Thanks – won’t help here as the code in question is in a package, and I don’t expect my end users, who are writing the main() function to have to deal with this.
And this package is used in other contexts, like in an engine behind a web service.
However, maybe I could take this approach at the module level – I’ll have to think on that …
I still think TemporaryDirectory could be a PathLike, but it doesn’t seem there’s much suport for that here, and it’s not a big deal.
For a library you need to give your users a context manager that manages this. Then it is up to them to use with with your context manager:
from chris_cool_stuff import data_manager
with data_manager() as foo:
foo.process_stuff()
You could also give them a decorator that they can stick on their main function if that seems nicer:
from chris_cool_stuff import managed_data
@managed_data
def main():
...
Either way you can hide the details of the temporary directory internally but for a library all you can do is provide the user with the pieces. Somewhere they need to put something that defines the scope for when the temporary directory will be created and when it will be released. You can make it easy for them but you can’t make it completely invisible.
in this case, library is maybe not quite the right word – it’s a library, but it’s also almost an application it’s a scripting environment, and one with not-allways sophisticated users.
So I do need a default behavior that’s not-too-bad.
I’ll poke at it some more – I may be able to use a context manager inside the class. that manages the cache – if I can make sure there’s a hook to call when restarting things … hmm…
To close this out – there doesn’t seem to be an appetite for making TemporaryDirectory a PathLike – it’s OK, and maybe good, that folks are required to think carefully about how it’s used when they want to keep the Path around.
import tempfile
class MyFileTypeWriter:
def __init__(self, path_or_fileobj):
if isinstance(path_or_fileobj, os.PathLike):
self.fileobj = open(path_or_fileobj, 'w')
else:
self.fileobj = path_or_fileobj
...
with tempfile.TemporaryFile() as tf:
writer = MyFileTypeWriter(tf)
...
Currently the second branch in __init__ is taken. If we make NamedTemporaryFile path-like, then on Windows the first branch will be taken (because TemporaryFile is an alias to NamedTemporaryFile on that platform), and the open() will raise OSError because the file is already open.
This issue affects zipfile.ZipFile which makes me think it’ll affect plenty of user code too.
yeah, making TemporaryFile PathLike is off the table – for exactly that reason. I didn’t look carefully before posting.
Personally, I think that code should be using ETAFTP – try to use it as a FileLike, if that fails, then try to use it as a PathLike, but oh well, I’m sure there’s a lot of code out there like that.
But TemporaryDirectory doesn’t have that problem.
).