IndexError message should show index and length

Ideas
1

Currently, the IndexError message shown by a sequence (list, tuple, or range) does not show the index and length in question:

$ python3 -c “[0, 1, 2][4]”
Traceback (most recent call last):
File “”, line 1, in 
IndexError: list index out of range

It would be helpful for debugging if the IndexError message showed the index and length.

This is especially useful when indices and lengths are dynamic, so simply pointing to a line of code (like the current message does) is not helpful.

This seems like a convenience with no downsides.

6 Likes
2

Formatting complex error message takes more time than creating a Python string from the C string.

In some cases index and size are not limited, they can be arbitrary large, and converting very large integer (e.g. 2**1000000000) to decimal string will take very long time.

10 Likes
3

Why would the time to create a string be more important, in this situation, than user friendliness?

That argument would rule out any error message involving dynamic values (e.g., KeyError for dicts).

1 Like
4

Because often the string will not be used for anything e.g. if the exception is caught and then discarded.

2 Likes
5

You could just store the index and length in the exception and only format the message if the message attribute is accessed?

7 Likes
6

The old cgitb module used to analyze the stack after an exception and print out var details near the error.

I suppose that could be done for almost all exceptions without affecting the initial raise.

3 Likes
7

Exactly. Like KeyError, where the missing key is passed as args, the proposal should be about making IndexError accept index and length as args with its constructor, which should cost very little. But unlike KeyError, IndexError shall not include index and length when formatted as a string for reasons that Serhiy pointed out. The user can then choose to format an IndexError with index and length included where it makes sense.

2 Likes
8

I think you misunderstood what I’m saying: Can you lazily interpolate when the message attribute is accessed for the first time?

9
6 Likes
10

I see. So you mean to make IndexError more like KeyError, where the constructor takes a key and only formats the message string with key when __str__ is called. Currently IndexError’s constructor takes a preformatted error message, and we can make it append additional information when __str__ is called.

But that approach does not solve the problem that Serhiy pointed out, which is that it may be undesirable for IndexError.__str__ to include index in the formatted message by default because indexcan be too large to be converted into a string in a reasonable amount of time.

To mitigate the problem we can either leave it to the user to include index in the message, or make IndexError.__str__ conditionally include index only if it’s within a certain size limit (e.g. 64 bits).

Thanks for the link. I see that in the prior discussion the core devs believed that including the index that triggers an IndexError in the message is rarely useful to debugging, which I think is indeed usually true.

1 Like
11

Why would that matter? I thought the only problem with interpolation time would be when you’re discarding the exception anyway.

1 Like
12

The problem with including index in the default output of IndexError.__str__ is that a generic error handler that outputs any error as a string would unconditionallly output an IndexError with an overly large index, potentially hanging the program for a long time.

13

Thanks for explaining. In that case, maybe just interpolate numbers up to something that takes about 0.1s to format? (That should still be huge.)

14

Yes, which is why I suggested above the alternative of conditionally including the index in the message only if it fits for example 64 bits.

4 Likes
15

Since a recent Python update we already have the limit to (default) 4300 (decimal) digits on int-to-str formatting. That is different from the old thread that was referenced above, so that argument may weigh lighter than before.
Still the limit can be disabled, but that’s a deliberate user decision.

3 Likes
16

Further discussion is pointless until we have data. We need to try and measure the time it takes to create an exception from a literal string and from a formatted message. If the difference is 1% – okay, this is a noise. If the difference is 100% – this is unacceptable for such common exception.

1 Like
17

The exception may be common, but formatting it on the console to a user that cares about an extra few milliseconds is probably uncommon.

So I think you should also figure out how often the exception is printed, how much longer it takes to print out the text, and in which situations it really matters.

4 Likes
18

Printing a traceback on the console is already slow, printing few more characters will not make difference. In most cases where the performance is important the error message is ignored.

19

When the error message is ignored, this doesn’t matter since you don’t need to interpolate the text until the message attribute is accessed.

4 Likes
20

The same is true for dicts:

$ py -c "{}[2**10000]"
[large number]
$ py -c "{}[2**100000]"
KeyError: <exception str() failed>
5 Likes