Better way to deal with exceptions

Python Help
21

replace the + with chain, replace the *_ with islice

something along the lines of:

x, y, z = islice(chain(arg, (None,)*3), 3)
1 Like
22

Exactly. To clarify further, exceptions should be caught where they occur to ensure proper handling. If you choose to propagate them instead, it may be for debugging purposes.

This is how you can handle multiple nested KeyError exceptions effectively:

def lookup(name=None):
    raise KeyError


try:
    try:
        try:
            try:
                lookup()

            except KeyError:
                print('First KeyError!')
                lookup()

        except KeyError:
            print('Second KeyError!')
            lookup()

    except KeyError:
        print('Third KeyError!')
        lookup()

except KeyError:
    print('Done!')

If you ignore exceptions and cross your fingers, don’t be surprised when important KeyError exceptions are silently suppressed, leaving you unaware of underlying issues:

def lookup(name=None):
    raise KeyError


try:
    lookup(lookup(lookup(lookup())))

except KeyError:
    print('Done?')
23

making a new topic because the old topic got derailed (sigh)

we have a custom exception:

class PropertyError(LookupError):
    """Raised to indicate improper use of a DataProperty.
    """
    pass

(where “improper use” just means it’s not supported by the data source)

we have an abstract API method that raises this exception:

    @abc.abstractmethod
    def get_property_values(self, prop):
        """Returns the values associated with the given property as an iterable.

        If duplicated, earlier values should override later values.

        Args:
            prop (DataProperty): The property.

        Returns:
            The values associated with the given property.

        Raises:
            PropertyError: If the property is not supported by this data
            source.
            LookupError: If the property is supported, but isn't available.

        """
        raise PropertyError

then we have various implementations that processes multiple data sources together into a new data source.

as may or may not be obvious, just because one data source doesn’t support a property, doesn’t mean the other data sources don’t support it. so when you mix data sources that support a property and those that don’t support a property, you need to decide how to handle these various cases.

for example, we might have this:

# ignore sources without the property
for source in self.sources:
  try:
    something = source.get_property_values(prop)
    # some other stuff
  except PropertyError:
    pass

or we might have this:

# require all sources to have property
for source in self.sources:
  something = source.get_property_values(prop)

or we might accidentally write the latter when we want the former.

and, well, these data property processors are themselves data property sources which can be used in other data property processors. that is, we can put a “require property on all” inside “ignore sources without the property”. and this is mostly fine.

unless it’s a bug.

the fact that these are exceptions means they don’t get stopped where we handled them incorrectly. no, they just keep going and propagating into other code in ways we never expected. it’s simply impossible to debug.

in fact, a similar situation is what led to the creation of PEP 479. unfortunately, PEP 479 doesn’t care about exceptions, it cares about generators, even tho generators use exceptions. in our opinion, PEP 479 would’ve been better off if it just deprecated using exceptions for iteration altogether, because it honestly reads like it’s trying to. instead, we get this eternal bolt-on of workarounds for a small issue mostly inherent to exceptions that python is doing its best to avoid properly addressing and avoid taking responsibility for. (we mean, just look at how many times ppl told us we’re holding it wrong in our previous thread, and tell us how that’s not “blaming the user”. but we have no way of getting that addressed so we’re not even gonna bother.) anyway hopefully this thread can stay on-topic and we can have a proper discussion about the issue.

24

As was said in the other thread, the problem seems to be that you’re using a subclass of LookupError when you really just want a custom error. What is the benefit of subclassing LookupError?

What you are describing is exactly what exceptions are supposed to do. If you don’t catch them, they keep going. You’re asking for some way to notice when one of these exceptions leaks out. That’s called catching the exception. That’s exactly what the try statement exists for.

No, this is not the same thing. PEP 479 dealt with a very specific interaction wherein a function call turns into a next() result. This is not that.

Why are you creating a new thread and then making all the exact same complaints? Are you surprised that you’re getting the exact same responses?

3 Likes
25

@moderators Can this just get merged back into Better way to deal with exceptions please?

2 Likes
27

It’s really not Chris that’s the source of confusion here. You still haven’t actually explained what you think a suitable behaviour should be, nor can I really parse what you think the problem is to begin with. I still can’t tell if it’s exceptions that aren’t caught and bubble up the stack (giving a nice stacktrace indicating where you went wrong) or exceptions that are caught overzelously and ignored (a self inflicted issue) that you’re taking issue with.

5 Likes
28

would you say the issue PEP 479 addresses is self-inflicted? why does PEP 479 exist if it’s self-inflicted?

31

You’re being offensive and then blaming the person who takes offense?

33

PEP 479 exists because the way the language used to work was undesirable, where a specific kind of exception meant to implement the iterator protocol was being implicitly handled by the protocol in situations where it shouldn’t. So the people responsible for the language changed its behavior.

What we are trying to say to you is that, the way you described your problem, it doesn’t appear to be your case, because you are explicitly catching your exceptions, and you could adjust the code on your end by simply changing the exception class from which your control flow exceptions inherit.

Maybe we are wrong, but you will have to change the way you are explaining things, or we won’t be able to help you.

3 Likes
34

we would try to explain it, but unfortunately, despite the fact that PEP 479 is related to our issue, even PEP 479 itself fails to recognize that fact.

if all our attempts to bridge that gap so far have failed, why should we keep doing the same thing and expect it to go any different?

35

Trying to sidestep some of the tangents here, but: If you don’t want these conditions to propagate, why are you using exceptions to represent them? Would using some kind of “return code” be an alternative? Can you give an example of a piece of code that currently does one thing that you find problematic, and explain what you behavior you would hope for instead?

1 Like
36

for starters, return codes are unpythonic.

yes they’re a better solution (and why we consider PEP 479 an attempt to retrofit generators to act more like return codes instead of exceptions) but only because python lacks a construct to make exceptions feel like return codes.

if you think about it:

  • who consumes StopIteration? the immediate caller (for)
  • who consumes KeyError? the immediate caller (dict.get (with default))
  • who consumes AttributeError? the immediate caller (getattr (with default), hasattr)

but there’s no concept of, say, “either the immediate caller catches this, or it becomes a RuntimeError”. nor anything comparable. except in generators.

and like, why does this come up in generators? what makes generators different? well they’re not that different from getattr and this is a hill we’re willing to die on:

  • generators call other generators, iterators, etc
  • __getattr__ calls other getattr, uses attributes directly, etc
  • __next__ also calls generators, iterators, etc

tho generators have the convenience of opaque implementation details, so it was easy enough to silently “fix” them (yet not silently enough - still required deprecation period and stuff). but it basically boils down to: generators regularly call code that raises the same exceptions as generators.

37

Okay, but can you explain what you want to do in your code example, which doesn’t involve any generators?

38

we want our code to have the property that, when we call into other data sources (keeping in mind all data sources have to raise the same exceptions, by API contract), we get properties similar to those of generators.

that is, if they raise PropertyError, that should be converted into RuntimeError before being propagated. if we raise PropertyError, that should be propagated directly to the caller.

39

Simply don’t allow them to raise PropertyError. Or, use a private _PropertyError for your own use and allow others to use PropertyError.

40

that doesn’t work with the API contract.

we have a class that takes a list of sources and is itself a source. we need to interoperate with the user both ways.

even if we can change the API contract
 so we swap the exception type, now for interoperability everyone else has to do it too. because the purpose of an API contract is interoperability.

OurDataSource([TheirDataSource(...)]) needs to work, as does TheirDataSource([OurDataSource(...)]).