Gauging sentiment on pattern matching

Committers
21

I voted for I don’t want pattern matching.

I dislike the syntax and semantics expressed in PEP 634. I see the match statement as a DSL contrived to look like Python, and to be used inside of Python, but with very different semantics. When you enter a PEP 634 match statement, the rules of the language change completely, and code that looks like existing Python code does something surprisingly very different. It also adds unprecedented new rules to Python, e.g. you can replace one expression with another in the exact same spot in your code, and if one has dots and the other doesn’t, the semantics of what the statement expresses changes completely. And it changes to yet a third set of semantics if you replace the expression with a single _.

I think the bar for adding new syntax to Python at this point in its life should be set very high. The language is already conceptually pretty large, and every new feature means new concepts one must learn if one is to read an arbitrary blob of someone else’s Python code. The bigger the new syntax, the higher the bar should become, and so the bigger payoff the new syntax has to provide. To me, pattern matching doesn’t seem like it’s anywhere near big enough a win to be worth its enormous new conceptual load.

I can see how the PEP authors arrived at this approach, and I believe them when they say they thought long and hard about it and they really think this is the best solution. Therefore, since I dislike this approach so much, I’m pessimistic that anybody could come up with a syntax for pattern matching in Python that I would like. That’s why I voted for I don’t want pattern matching rather than I want pattern matching, but not as defined in those PEPs. It’s not that I’m against the whole concept of pattern matching, but I now believe it’s impossible to add it to Python today in a way that I would want.

I don’t find this result all that surprising; my admittedly-poorly-informed opinion is that pattern matching as a language feature works well in single-assignment languages, but not so well in languages that permit reassignment. Also, it’s such a large feature that it’s better to add it early in the language’s development, before too much of the available syntactic surface area gets used up (if you follow my meaning).

Finally, I’ll reference something I swear I saw one very smart Pythonista say in an interview: “In Python, we don’t build Domain-Specific Languages, we build Domain-Specific Libraries”. Obviously you can’t add “capture patterns” using a library. But forego that feature and suddenly I suspect most of pattern matching’s other features could be replaced by library code. Compare the with-pattern-matching and without-pattern-matching examples in the Motivation section of PEP 635. Already they’re roughly the same number of lines. The without-pattern-matching code is admittedly more complex. But write a small function or two to externalize that complexity, and you should be able to bridge the complexity gap between the two examples.

13 Likes
22

As a small data point, I am currently writing some code that parses a word-based format (the PyPI changelog data, FWIW). As I was looking at it, my first thought was “this is going to be annoyingly messy and repetitive to write”. But it then occurred to me that it’s an ideal example of pattern matching, and exactly the example used in the tutorial PEP:

match action.split():
    case ["create"]: handle_create()
    case ["add", "Owner", name]: handle_new_owner(name)
    # etc

This is the sort of “simple parsing” I find myself needing very often. So I guess that confirms for me that I’ll find the PEP 634 as it stands will be something I’ll definitely find useful.

Now I have to go back to writing the code “the current way” :slightly_frowning_face:

6 Likes
23

Correct. Python 3.5 added async and await to the language, but they weren’t keywords per se. Instead, they were more like very very special identifiers. The tokenizer had special hard-coded support for them. For example, if it saw the token "async", and the next token was "def", it would return an ASYNC token instead of a NAME token for the "async". It also remembered whether or not it was currently tokenizing an async function so it could return AWAIT tokens instead of NAME tokens containing the string "await".

The asyncio and typing modules were added in 3.4 and 3.5 respectively. Both were “provisional”, which meant they could change in major point releases. The BDFL also declared that both were immune to the “feature freeze” at beta, which meant they were still evolving (sometimes rapidly!) during the betas and even release candidates of their respective .0 releases.

source: Python 3.4 and 3.5 release manager

2 Likes
24

I’d like to say:
“I don’t think pattern matching is a good fit for Python, but I could live with.
However, I think it would be a really bad idea to accept any of the above PEPs.”

I’ve ticked the “I don’t want pattern matching box”

3 Likes
25

I’m inclined to agree with Mark and Larry’s posts, particularly Larry’s point about this being an unnecessary DSL within Python.

This is not to in any way disrespect the work that the authors have put into developing the idea. Assuming that this needs to be dedicated syntax, the PEPs are no doubt the best they can be. But I disagree with the assumption.

As an off-the-cuff counter-example, the below is totally implementable today, and AFAICT the biggest gap is you need to explicitly store the expression in a variable if you don’t want to recalculate it. (The <?> is a capture syntax similar to what many web routing libraries use.)

o = <expression>
if m := match("MyClass(a=100, b=<b>)", o):
    return m.b
elif m := match("int(<x>)", o):
    return m.x

I’ve also ticked the “I don’t want pattern matching” box, assuming it refers to new, dedicated syntax. I could be convinced to support general-purpose syntax that makes pattern matching libraries (hopefully including regex) easier to use, but I’ve seen too many single-purpose tools fail to satisfy even that need to support another one.

4 Likes
26

Like Larry, I’ve come to the conclusion that if the existing proposals are what our best and brightest minds (meant in all sincerity as a compliment) can come up with, then pattern matching is not a good fit for Python’s remaining “syntax space”.

To me it seems like multi-line lambdas: sure they’d be great, and many people would like them, but we’ve never been able to come up with an acceptable fit.

3 Likes
27

Thank you very much for your answers, @brandtbucher.

How would the proposal deal with literals which are defined via a function calls or constants ?

Simply put: the results of those function calls and the values of those constants would need to be put in a qualified namespace.

Ok, so essentially putting a dot into the case target will give us the functionality I was looking for. That’s great.

Since this kind of literal matching is likely going to be the most common use case of the syntax, I think the PEPs should give more attention to this case, not just mention it in an appendix.

The rejections of PEPs 275 and 3103 were our first hint that this would not be the most common use case of the new syntax. It’s clear that the community needs a different feature - one that can’t be replaced by a dictionary or a trivial if - elif - else ladder.

I would not sign this conclusion :slight_smile:

For me, the most common use case of pattern matching is in parsing text or number input, meaning: you take text/numbers and convert it into data structures, events or actions.

That’s why I wrote PEP 275 in those days. Endless if-elif chains are inefficient for this purpose (often enough, you don’t match on patterns, but on constant values) and so are dictionary based branching approaches (Python’s function/method calling is slow). In PEP 275, I would have wanted to take the branching part of the switch statement to the byte code level, allowing for much faster processing without function calls. Ok, but that’s history :slight_smile:

I hardly ever match on types or objects.

The only cases I can remember from the not-so-long-ago past are cases of the visitor pattern on XML/JSON structures, used for either validation, transformation into other data structures or triggering actions. However, in all those cases, the actual matching was done on string values, not on objects. I suppose those could be converted to some kind of object matching based on elementree or dict objects, but the code would not look a lot different or be more readable that way.

I can see an advantage of doing the implicit type check using the PEP approach for certain cases, but if you know that you just tokenized e.g. XML using elementree, it’s clear that you’ll never get anything but strings as input.

Would it be possible to add the syntax only with a from __future__ import pattern_matching_v1 marker to experiment with it for a release or two before making it final ?
This would give the syntax a chance to mature in practice and at the same make it obvious to readers of the code that the syntax may in fact still change in an upcoming release.

The way I see it, that’s what development, alphas, and betas are for. You can even play with it now , if you want to!

I see two issues here: First, alphas and betas are usually not used by anyone who wants to write new code. People who have code and want to check whether it’ll work on the next release will more likely, but not even that is really typical. Second, the few months running in the beta cycle will not be enough to figure out those nits and fix them shortly after, in time for a release. The PEPs are just too complex for this to happen.

So in summary, the use case I’m interested in, is possible, but not the main objective of the PEPs. That’s fine. I’m sure I’ll find a use case for type and object pattern matching going forward :slight_smile: and perhaps we’ll even get the optimized branching which I was after in PEP 275 in a couple of years.

What I’m unsure about is whether the added complexity of having capturing variables balances well with the rest of the pattern matching. It’s a new concept for Python (at least, as far as I can see) and puts more emphasis on the implicit rather than the explicit side of things.

In regular expressions, you have get the values of what you parsed by looking at groups. That’s more explicit and doesn’t mix in implicitly with the locals(), while still working nicely for actual parsing part. Have you considered an approach where the parsed data is made available using a matching object API instead of using capturing ?

1 Like
28

Since explicit is better than implicit, let me also clearly state my sincere thanks and respect for the PEP authors and their efforts. I have no doubt that they worked very hard on this series of PEPs. And I’m always happy to see people so enthusiastic about Python that they write proposals to enhance it.

10 Likes
29

I had the same feeling while looking at this “DSL” feature supported by pattern matching.

Yeah, I didn’t follow up with my thought process for why we have not supported multiline lambda for so long.

1 Like
30

The problem you have with this is the same as with f-strings as a function call: the function needs access to the caller’s scope in order to resolve names.

31

The only problem is finding the steely resolve to do what must be done, Mr. Bond. One can achieve great things with sys._getframe(), its many wonders a joy to behold.

2 Likes
32

PEP 498 has an example of something that won’t work with sys._getframe().

33

Spoilsport! But it was a) intended for the pattern matching discussion, and b) clearly in jest anyway.

2 Likes
34

There’s no name resolution in my example :grin: (but there probably should be, checking type(m).__name__ isn’t ideal).

But you ignored the other part of my post.

I’m far more favorable towards adding general purpose variable capture to a function call. We always seem to come up with plenty of uses for it.

2 Likes
35

I completely agree. But probably no one will like the syntax needed.

1 Like
36

While I’m still working on v3 of PEP 642, one of the outcomes of the review process so far for me is that I’ve shifted from being +1 on PEP 634’s semantics and +0 on its syntax to being -1 on more and more of the syntactic aspects (especially class patterns being positional arg centric rather than attribute centric, and the fact that “NAME as NAME” binds the same reference to two different names).

It means that if I can’t get PEP 642 to a place that both I and other readers genuinely like, I’ll end up with the same view as Larry.

@steve.dower’s counter suggestion does make me wonder if pattern matching libraries might serve as a motivating use case for the long deferred PEP 501, though (that PEP proposed “i-strings”, which would be like f-strings, but without the immediate rendering step - instead of fully rendered strings, they would produce “interpolation templates”, containing both the raw template, and the component values calculated at runtime)

1 Like
37

Summary: half for 634 at least in some form, half not. Since ‘evenly split’ describes my current stance, I am kind of glad I did not vote and break the tie ;-).

One result of the discussions is that I realized that parts of mathematical practice, especially algebra, consists of creating and applying patterns, but I do not remember ever seeing ‘pattern matching’ used to describe this. Programming also involves pattern matching thinking (applying abstract patterns to concrete situations and information structures). So the question is whether we want and can find good enough, new and explicit ‘pattern matching’ syntax. I am not yet sure.