Assignment method that will trigger RuntimeError if assigned to None

Ideas
1

In the spirit for not passing errors silently. I want have a short hand for expressing that a variable will be something eventhough that is coming from an function that returns an optional output and if not it should trigger a RuntimeError. Such that users would have an incentive to embrace defensive design and eliminating the complexity in the code by removing not realistic execution trajectories.

To take a motivating example:

Developer wants write an update user function and writes the following:

def update_user(user_id, **data_to_update):
    user = User.query.get(user_id)
    user.update(data_to_update)
    db.session.commit()

This fails on the on the type checker because User.query.get(user_id) returns a Optional[User]
Attempting to update a non existing user is something that from the business logic sense newer should happen. So the right thing here would be to trigger a NonImplimentedError. Such that when this for some reason is happening anyway 2 years from now we get it in the error logs.
But since this is so unthinkable at the time of writing a large proportion of developers could tend to the something like following solution to make the static type checker pass:

def update_user(user_id, **data_to_update):
   if user := User.query.get(user_id):
        user.update(data_to_update)
        db.session.commit()

I wish the following was posible

def update_user(user_id, **data_to_update):
    user := User.query.get(user_id)
    user.update(data_to_update)
    db.session.commit()

Which would be translated to:

def update_user(user_id, **data_to_update):
    user = User.query.get(user_id)
    if user is None:
        raise RuntimeError
    user.update(data_to_update)
    db.session.commit()

I know there is an argument to be made that the current version is more explicit. But I think there should also be put emphasis on nudging users to the currect behavior by making it easy.
I think the walrus operator is a good usage for this since it is a combination of assignment and other operations and it is as far as I can see not used for anything else.

EDIT:
Changing NotImplimentedError to RuntimeError thanks Neil Girdhar and Chris Angelico

2

What kind of thing is User.query? Is it a dictionary?

3

I was just using SQLalchemy query.get as an example, which returns the ORM representation of a row with a primary key matching the input if that exists or else None. It could be any function that can return None or an object. Thanks for asking.

4

I don’t think that’s right. NotImplementedError is usually raised by abstract methods. RuntimeError seems more appropriate for something that should never happen, or else ValueError for a bad value.

Yes, another argument you’ll get is the same as with PEP 505: None isn’t special. As soon as you add the condition to satisfy the type-checker, that’s your opportunity to think about what should happen if the condition fails.

5

Yeah, was afraid of that. For anyone who happens upon this thread: If this WERE a plain dictionary, there’s a much MUCH easier option, and that’s to replace thing.get(key) with thing[key], which will raise if it doesn’t exist. That covers a vast number of use-cases, so try that before reaching for something more complicated!

For this sort of situation, unfortunately there’s no easy option, because there are a number of subtly different situations, which need to be handled differently.

  • Is it illogical (but not impossible) for None to be returned? Raise RuntimeError, because this really shouldn’t be happening.
  • Is it a logic error in your caller that results in this? Raise some sort of suitable exception (not NotImplementedError but something that says “This was YOUR fault”).
  • Did you ask for a user ID that simply doesn’t exist? Raise ValueError.
  • Is the omission of user ID a perfectly logical situation, but you just shouldn’t do the thing that needs a user? Guard the code with if user := whatever(): as per your example.
  • Is one of the functions incorrectly type-hinted? Fix the type hints or silence the type checker.
  • Is it actually impossible for the function to return None at this point, but the type checker simply doesn’t know that? Add assert user is not None and the type checker should understand that.

So I don’t think a one-size-fits-all solution will really work here. Trying to “nudg[e] users to the correct behaviour” only works when there’s only one correct behaviour.

6

RuntimeError is better. Thanks for improving my understanding. I will change it in the original comment.
I agree that type checker warnings is the time for implimenting this sort of corner case handling. But what I see in allot of code is unrealistic corner cases are handled in a bad way instead of raising a error. So I wanted to give an easy way to make more people gravitate towards a more defensive design where code fails if it behaves out of spec and the failing happens fast.

7

One of the things I really love about Python is that build in methods almost newer returns None. Dictionaries raise value error if you try to access an element that is not there. min raises an error if you are using an empty list.
Raising errors instead of returning None is in my mind a cornerstone in Python as opposed to other languages. So I want an easy way to continue this philosophy in my code when I’m having function that produces Optional outputs. It just fells more Pythonic, so I think it should be easily enabled by the language. To give people incentive to write more Pythonic code.

Furthermore in code design one aspect of readability is that a curtain part of the code length matches with its importance. Handling cases that are only theoretically possible in the view of the type checker, should not cause mental overhead. I know allot of them can be solved by making extra functions or improving the type annotations. But it often produce much more code and increases the complexity for the reader and the writer.

8

I recall sqlalchemy supported a get exactly one query.
If my recollection is right use that call.

9

So the OP would like a way to turn None when received from some function or method with an Optional result into an exception. Their strawman syntax is an alternate assignment error, in particular using := instead of = would raise if the value is None. This would indeed help static type checking, but the alternate assignment operator just isn’t going to fly, it’s too subtle a signal (especially since := has the meaning of a plain assignment in an expression context).

All this can easily be solved with a simple function,

T = TypeVar("T")
def require(x: T|None) -> T:
    if x is None:
        raise RuntimeError
    return x

Now you can write

    user = require(User.query.get(user_id))  # Infers user as type User

Just make that function part of your personal or project toolbox and you’re done. No language changes needed.

7 Likes
10

We could also write: (bad idea)

assert (user := User.query.get(user_id)) is not None

Unfortunately it raises an AssertionError instead of a RuntimeError

11

No, don’t do this. Assertions may or may not actually be run, so an assert statement with a side effect is a very bad idea.

1 Like
12

I completely agree, it was a very bad idea…

13

So the OP would like a way to turn None when received from some
function or method with an Optional result into an exception.

No.

[…] All this can easily be solved with a simple function, […]
Now you can write

   user = require(User.query.get(user_id))  # Infers user as type User

Just make that function part of your personal or project toolbox and
you’re done. No language changes needed.

I think you’re missing Oeter’s point: to make it easy and automatic
to catch this common situation in an assignment. Let’s take your example
with the SQLAlchemy example call:

 user = User.query.get(user_id)  # Infers user as type User

Peter wants a way to succinctly support
assignment-provided-it-is-not-None without wrapping every expression
in something like your require(expression) cumbersome boiler plate.

Personally, I think the best approach is a linter. Aside from full on
type checkers, one of the linters I use reports functions which return
an explicit value on one path and do an implicit (i.e. return None) on
another path:

 def f(x):
     if x < 0:
         warning("bad x")
     else:
         return sqrt(x)

This function returns None for x<0, but it is almost by accident and
in some code is by accident. pylint will complain:

 Either all return statements in a function should return an 
 expression, or none of them should. (inconsistent-return-statements)

because of the implicit return.

The situation is accidental the flip side of Peter’s objective, which is
to say “this assignment should not receive None`”. Concisely.

The general case would be “this assignment has valid values”, since
None isn’t special (but Very Common, and definitely the return for
bare return).

Personally I think type annotations and a linter (type checker) are the
best way to do that right now:

 user : User
 user = User.query.get(user_id)  # Infers user as type User

where the type for user doesn’t match the return type of
User.query.get, which would be Optional[User].

Cheers,
Cameron Simpson cs@cskk.id.au

14

In Django the .get() method raises if there is no matching thing in the DB. Which is a much better design imo. So yea, I think the problem here isn’t with Python, but with SQLAlchemy.

15

No it isn’t.

  1. Databases are concerned with sets. There is nothing degrading the empty set as “not being correct”. No SQL dialect that I know of consequently fails on returning an empty set.
  2. In a company every refund on a returned item has to be approved by the creditor department. So once a day someone in the department will do a transaction to list all refunds, approve them if they are correct and return them to the department doing returned items if they are incorrect. When there are no refunds, he will probably be greeted by the system telling him “Yay, no refunds today!” Why would this result be coming from a failure?
  3. A failure should result when a required foreign key is missing or does not point to a record in the correlated table. However, this is guarded for in updates and will not be of consequence when retrieving data.

The jab at SQLAlchemy is unwarranted for. It has a rich interface:

… and I probably overlooked something. What is wrong with that?

1 Like
16

There are likely to be similar tools in other database engines, but that’s the first one that came to my mind. It is for exactly this use-case: “give me this thing”, usually querying based on the primary key of a table.

There are plenty of situations where it makes sense to request a single row, and if that row doesn’t exist, it’s an error. Obviously this isn’t ALL database queries, and your second point (refund issuance) is a situation where it’s appropriate to do a regular query and get any number of rows (zero or more) without an error. But there are plenty of use-cases where it’s appropriate to fetch information while assuming foreign key integrity, or assuming something broadly equivalent (maybe it’s an off-database record of the primary key), such that it makes sense for an empty set to be a failure. In fact, you then go on to defend SQLAlchemy with:

So… apparently it does make sense for an empty set to be a failure. I’m not sure why you make such a thing about databases not failing on empty sets if it’s then fine for SQLAlchemy to have a “one” method that does exactly that. Your entire post could be condensed down to “use the .one() method, it does what you want”. Your defense of SQLAlchemy seems fine, but the preceding screed detracts from your credibility :slight_smile:

1 Like
17

method sqlalchemy.orm.Query.one()¶

Return exactly one result or raise an exception.

I already said sqlalchemy has the required API…

18

I should have thought of that. Thank you for correcting me.

I was more into arguing why it should not be a default. In the one() and one_or_none() case, it is actually the application choosing, by selecting one of the functions. I am happy with that.