I spend a lot of time on the Python discord, answering questions from newbies, and a common question is “My program is getting FileNotFoundError but I can see the file is RIGHT THERE!” What’s going on is:
their script is trying to open a file that is in the same directory as the script
they’re using a relative file name like open("my-file.txt")
the script’s working directory is not the same as the directory it is in
they don’t understand the concept of “working directory”
So my idea is: the FileNotFoundError exception would include a bit of explanatory text, at least in those cases where the input file name was relative, like this:
>>> open(".bash_history")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: '.bash_history'
current working directory is '/private/tmp'; thus I looked for '/private/tmp/.bash_history'
>>>
That might be enough to explain to the user why they’re getting the exception, and it’d certainly be enough for a helper on discord to explain it.
>>> open('/var/temp/does_not_exist')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: '/var/temp/does_not_exist'
where the full path is in the filename that would be extra confusing. We would need to make sure it had the ability to determine that it is relative for every potential case - not just bare file name.
With that, I think that this could be an improvement for debugging if it handles cases like ./file and ../dir/file properly with full resolution of the file path included.
But then you want to be sure the system call behaves the same way. Which is probably true, but I wouldn’t bet money on their not being a corner case. Like if the current directory has been deleted, or a network disconnected and reconnected, or something more obscure. I realize in this case it doesn’t matter so much if we’re wrong, but in general I wouldn’t try to guess what the OS is doing.
You might consider suggesting such users to try using friendly/friendly-traceback for additional help. Using your example, on my computer right now, with friendly’s console:
[1]: open(".bash_history")
Traceback (most recent call last):
Code block [1], line 1
open(".bash_history")
FileNotFoundError: [Errno 2] No such file or directory: '.bash_history'
[2]: why()
In your program, the name of the file that cannot be found is .bash_history. It was expected to be found in the
C:\Users\Andre\github\friendly directory. I have no additional information for you.
I spend a lot of time on the Python discord, answering questions from
newbies, and a common question is “My program is getting
FileNotFoundError but I can see the file is RIGHT THERE!”
I’ve seen this incredibly common question there too.
So my idea is: the FileNotFoundError exception would include a bit of
explanatory text, at least in those cases where the input file name was
relative, like this:
>>> open(".bash_history")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: '.bash_history'
current working directory is '/private/tmp'; thus I looked for '/private/tmp/.bash_history'
>>>
That might be enough to explain to the user why they’re getting the exception, and it’d certainly be enough for a helper on discord to explain it.
Based on the “can we even compute the full path reliably” over caution later in this thread, how about:
FileNotFoundError: [Errno 2] No such file or directory: '.bash_history' from working directory '/private/tmp'
It does mean looking up the current working directory if the except fires.
Lazy notes for exceptions are a great orthogonal idea and should be proposed separately. This would be very useful to, e.g., gather performance data or runtime state only if an exception ends up being displayed or otherwise queried, but not when it is caught internally.
You have to be careful here. In this case, what if the current directory changes between the exception occurring and when it is displayed? And it’s even worse: What if the process-wide current directory is changed by another thread after the file open fails and before the exception object is created?
You have to be careful here. In this case, what if the current directory changes between the exception occurring and when it is displayed? And it’s even worse: What if the process-wide current directory is changed by another thread after the file open fails and before the exception object is created?
I’m sure there are lots of edge cases, but for most programs, and especially for the people @offby11 was talking about, these edge cases won’t occur. We shouldn’t shy away from presenting useful information to most people because it might be misleading in complex circumstances.
For example, we print source lines in tracebacks even though they might not be the lines as they were originally executed. It can occasionally cause baffling tracebacks, but only rarely, and the information is incredibly helpful. I think the same would be true of current directory information.
Perhaps the FileNotFound exception could capture the information at its creation, further reducing the possibility of misleading details?
Yes, I think that’s the safest thing to do. In this case an error has already occurred, I don’t think reading the current directory will add enough overhead to matter. That said, I guess it possible someone is looking for lots of files and stops as soon as opening one doesn’t throw an error.
How about the functions in os.path? [l]exists(), isdir(), isfile() and islink(). I wouldn’t want these functions to become any slower, so we might need a get_cwd=False argument for os.[l]stat() in that case.
getcwd() is a separate function call. The current working directory can change between calling open() and getcwd(), thus the information will be wrong. This may be okay for application that output an error to the user and third-party library that tries its best to improve error reporting, but not for the core. This one argument would be enough.
Calling getcwd() and converting its result to Python string adds an overhead. It may not matter if the only action is reporting the error to user and quiting or waiting for the user response. But it may be significant if you try to open many files with different names until you find the existing one.
Surely the call to os.getcwd() can be put into FileNotFoundError.__str__() to avoid overhead if the exception isn’t looked at? Granted it’ll be wrong if someone switches directory between failing to open the file and displaying the error but how often is that going to happen? (I’d call all bugs features when mixing relative paths with working directory switches anyway.)
I suppose that suggesting that someone’s working directory is not what they think it is might cause them to try to fix the working directory (by using os.chdir()) instead of the correct fix of using a full path.
Indeed, if we do something we could just suggest to make the path absolute:
FileNotFoundError: [Errno 2] No such file or directory: '.bash_history', consider using an absolute path.
It’s still a bit of overhead to check if the path is relative (if we don’t want to display the hint for absolute paths), but at least it’s resistent to working directory changes.
Having the exception do this for the user would be nice if it were clear how to get the right semantics. But it seems that with a little thought, we realize that there is no one “obviously correct” behavior.
I don’t think that implicitly calling getcwd will ever be wise. But it can be made explicit, e.g. via an API on exceptions which is meant to help beginners.
I’m imagining something like FileNotFoundError.__explain__() (returning a string).
The REPL could be enhanced to call __explain__ automatically, as could sys.excepthook
I hope some kind of note would be added as a result of this thread. The costs are tiny and I’ve seen several of the mentioned newbie questions too.
However, the note should be neutral, IMO. A chdir might be the right fix in some cases - no matter how common or how rare those cases are.
Something like “This file path is relative and thus depending on the current working directory” … I’m not a native speaker. Making the reader of the note aware that the cwd plays a role is the main point.
I’d be surprised if anyone who needs to see this extended error message is someone who is in one of these cases (which, outside of testing, I can’t think of any).