Guide to syntax changes by Python version?

Python Help
21

Thanks. I’m not 100% sure it’s impossible. It wasn’t 2003. Moments before he wrote that code, he read the letter written August 14, 2004. The with statement was introduced in Python 2.5, released September 19, 2006. The with statement’s PEP 343 was created May 13, 2005. So I guess it’s indeed unrealistic, unless code was available nine months earlier. (I’m not willing to trace it back that far :slight_smile: )

1 Like
22

3.11

3.10

  • PEP 634, Structural Pattern Matching: Specification
  • PEP 635, Structural Pattern Matching: Motivation and Rationale
  • PEP 636, Structural Pattern Matching: Tutorial
  • bpo-12782, Parenthesized context managers are now officially allowed.

3.9

  • PEP 584, union operators added to dict;
  • PEP 585, type hinting generics in standard collections;
  • PEP 614, relaxed grammar restrictions on decorators.

3.8

  • PEP 572 Assignment expressions (:=)

  • PEP 570 Positional-only parameters (def quantiles(dist, /, n=4, *, method='exclusive'))

  • f-strings support = (f'{user=} {member_since=}')

  • A continue statement was illegal in the finally clause due to a problem with the implementation. In Python 3.8 this restriction was lifted. (Contributed by Serhiy Storchaka in bpo-32489.)

  • The syntax allowed for keyword names in function calls was further restricted. In particular, f((keyword)=arg) is no longer allowed. It was never intended to permit more than a bare name on the left-hand side of a keyword argument assignment term. (Contributed by Benjamin Peterson in bpo-34641.)

  • Generalized iterable unpacking in yield and return statements no longer requires enclosing parentheses. This brings the yield and return syntax into better agreement with normal assignment syntax:

def parse(family):
        lastname, *members = family.split()
        return lastname.upper(), *members

3.7

  • PEP 563, postponed evaluation of type annotations.

3.6

  • PEP 498, formatted string literals.
  • PEP 515, underscores in numeric literals.
  • PEP 526, syntax for variable annotations.
  • PEP 525, asynchronous generators.
  • PEP 530: asynchronous comprehensions.

3.5

  • PEP 492, coroutines with async and await syntax.
  • PEP 465, a new matrix multiplication operator: a @ b.
  • PEP 448, additional unpacking generalizations.

3.4

  • No new syntax features were added in Python 3.4.

3.3

3.2

  • Previously it was illegal to delete a name from the local namespace if it occurs as a free variable in a nested block:
def outer(x):
    def inner():
        return x
    inner()
    del x

This is now allowed. Remember that the target of an except clause is cleared, so this code which used to work with Python 2.6, raised a SyntaxError with Python 3.1 and now works again:

def f():
    def print_error():
        print(e)
    try:
        something
    except Exception as e:
        print_error()
        # implicit "del e" here

(See bpo-4617.)

3.1

  • The syntax of the with statement now allows multiple context managers in a single statement:
with open('mylog.txt') as infile, open('a.out', 'w') as outfile:
    for line in infile:
        if '<critical>' in line:
            outfile.write(line)

With the new syntax, the contextlib.nested() function is no longer needed and is now deprecated.(Contributed by Georg Brandl and Mattias Brändström; appspot issue 53094.)

3.0

  • Print Is A Function

  • PEP 238: An expression like 1/2 returns a float. Use 1//2 to get the truncating behavior. (The latter syntax has existed for years, at least since Python 2.2.)

  • You can no longer use u"..." literals for Unicode text. However, you must use b"..." literals for binary data.

  • All backslashes in raw string literals are interpreted literally. This means that '\U' and '\u' escapes in raw strings are not treated specially. For example, r'\u20ac' is a string of 6 characters in Python 3.0, whereas in 2.6, ur'\u20ac' was the single “euro” character. (Of course, this change only affects raw string literals; the euro character is '\u20ac' in Python 3.0.)

  • PEP 3131: Non-ASCII letters are now allowed in identifiers. (However, the standard library remains ASCII-only with the exception of contributor names in comments.)

  • PEP 3107: Function argument and return value annotations. This provides a standardized way of annotating a function’s parameters and return value. There are no semantics attached to such annotations except that they can be introspected at runtime using the __annotations__ attribute. The intent is to encourage experimentation through metaclasses, decorators or frameworks.

  • PEP 3102: Keyword-only arguments. Named parameters occurring after *args in the parameter list must be specified using keyword syntax in the call. You can also use a bare * in the parameter list to indicate that you don’t accept a variable-length argument list, but you do have keyword-only arguments.

  • Keyword arguments are allowed after the list of base classes in a class definition. This is used by the new convention for specifying a metaclass (see next section), but can be used for other purposes as well, as long as the metaclass supports it.

  • PEP 3104: nonlocal statement. Using nonlocal x you can now assign directly to a variable in an outer (but non-global) scope. nonlocal is a new reserved word.

  • PEP 3132: Extended Iterable Unpacking. You can now write things like a, b, *rest = some_sequence. And even *rest, a = stuff. The rest object is always a (possibly empty) list; the right-hand side may be any iterable. Example:

(a, *rest, b) = range(5)

This sets a to 0 , b to 4 , and rest to [1, 2, 3].

  • Dictionary comprehensions: {k: v for k, v in stuff} means the same thing as dict(stuff) but is more flexible. (This is PEP 274 vindicated. :slight_smile:
  • Set literals, e.g. {1, 2}. Note that {} is an empty dictionary; use set() for an empty set. Set comprehensions are also supported; e.g., {x for x in stuff} means the same thing as set(stuff) but is more flexible.
  • New octal literals, e.g. 0o720 (already in 2.6). The old octal literals (0720) are gone.
  • New binary literals, e.g. 0b1010 (already in 2.6), and there is a new corresponding built-in function, bin().
  • Bytes literals are introduced with a leading b or B, and there is a new corresponding built-in function, bytes().
  • PEP 3109 and PEP 3134: new raise statement syntax: raise [expr [from expr]]. See below.
  • as and with are now reserved words. (Since 2.6, actually.)
  • True, False, and None are reserved words. (2.6 partially enforced the restrictions on None already.)
  • Change from except exc, var to except exc as var. See PEP 3110.
  • PEP 3115: New Metaclass Syntax. Instead of:
class C:
    __metaclass__ = M
    ...

you must now use:

class C(metaclass=M):
    ...

  • The module-global __metaclass__ variable is no longer supported. (It was a crutch to make it easier to default to new-style classes without deriving every class from object.)

  • List comprehensions no longer support the syntactic form [... for var in item1, item2, ...]. Use [... for var in (item1, item2, ...)] instead. Also note that list comprehensions have different semantics: they are closer to syntactic sugar for a generator expression inside a list() constructor, and in particular the loop control variables are no longer leaked into the surrounding scope.

  • The ellipsis (...) can be used as an atomic expression anywhere. (Previously it was only allowed in slices.) Also, it must now be spelled as .... (Previously it could also be spelled as . . ., by a mere accident of the grammar.)

  • PEP 3113: Tuple parameter unpacking removed. You can no longer write def foo(a, (b, c)): .... Use def foo(a, b_c): b, c = b_c instead.

  • Removed backticks (use repr() instead).

  • Removed <> (use != instead).

  • Removed keyword: exec() is no longer a keyword; it remains as a function. (Fortunately the function syntax was also accepted in 2.x.) Also note that exec() no longer takes a stream argument; instead of exec(f) you can use exec(f.read()).

  • Integer literals no longer support a trailing l or L.

  • String literals no longer support a leading u or U.

  • The from module import * syntax is only allowed at the module level, no longer inside functions.

  • The only acceptable syntax for relative imports is from .[module] import name. All import forms not starting with . are interpreted as absolute imports. (PEP 328)

  • Classic classes are gone.

2.7

  • The syntax for set literals ({1,2,3} is a mutable set).
  • The new "," format specifier described in PEP 378: Format Specifier for Thousands Separator.
  • Dictionary and set comprehensions are another feature backported from 3.x, generalizing list/generator comprehensions to use the literal syntax for sets and dictionaries.
  • The with statement can now use multiple context managers in one statement. Context managers are processed from left to right and each one is treated as beginning a new with statement.
  • Extra parentheses in function definitions are illegal in Python 3.x, meaning that you get a syntax error from def f((x)): pass. In Python3-warning mode, Python 2.7 will now warn about this odd usage. (Noted by James Lingard; bpo-7362.)

2.6

  • Alternate syntax for catching exceptions: except TypeError as exc.
  • PEP 3127 - Integer Literal Support and Syntax (backported)
  • PEP 3129 - Class Decorators
@foo
@bar
class A:
  pass
  • It’s also become legal to provide keyword arguments after a *args argument to a function call.
def f(*args, **kw):
    print args, kw
f(1,2,3, *(4,5,6), keyword=13)
  • Previously this would have been a syntax error. (Contributed by Amaury Forgeot d’Arc; bpo-3473.)

2.5

2.4

  • PEP 318 [Decorators for Functions and Methods](https://PEP 318: Decorators for Functions and Methods)
  • None is now a constant; code that binds a new value to the name None is now a syntax error. (Contributed by Raymond Hettinger.)

2.2 - 2.3

2.1

x = 1
def f():
    # The next line is a syntax error
    exec 'x=2'
    def g():
        return x

2.0

# Syntax error
[ x,y for x in seq1 for y in seq2]
# Correct
[ (x,y) for x in seq1 for y in seq2]
  • The full list of supported assignment operators is += , -= , *= , /= , %= , **= , &= , |= , ^= , >>= , and <<= .
  • The print statement can now have its output directed to a file-like object by following the print with >> file 
print >> sys.stderr, "Warning: action field not supplied"
  • f(*args, **kw) is a shorter and clearer way to achieve the same effect. This syntax is symmetrical with the syntax for defining functions:
  • Modules can now be renamed on importing them, using the syntax import module as name or from module import name as othername . The patch was submitted by Thomas Wouters.
8 Likes
23

@nikdissv-Forever, your list is missing PEP 646, which introduces syntax changes in 3.11

24

Thank you @nikdissv-Forever ! This is such a great document. Have you memorialized it somewhere more permanently addressable?

If not, do you have any objection if I copy it (with credit) to a web page or repository location?

25

I don’t mind.

26

I often answer such questions for myself using these:
https://stromberg.dnsalias.org/~strombrg/cpythons/
https://stromberg.dnsalias.org/~strombrg/pythons/

The first link builds historical versions of CPython going back to CPython 0.9. Sadly, this works much better on Linux than on Mac; on Linux it can go clear back to 0.9 (from 3.11), but on Mac it errors out on anything that wants to be built 32 bit. I doubt it will work at all on Windows. It installs them all under /usr/local/cpython-*

The second URL runs a command or script under each python interpreter found under /usr/local, and summarizes the result. It does CPython versions, but also Pypy and Micropython versions, as well as the less common Jython and Tauthon.

This can empirically figure out differences between Python versions - you just supply the second link, pythons, with a command or script that can tell the difference.

Then you just need to be concerned with things like, EG, Python’s Ordered Dicts - because they became ordered in 3.6, but were only guaranteed ordered in 3.7 - the pythons script cannot help you with such changes. That is, in 3.6 this ordering was an implementation detail, while in 3.7 it became a feature of the language - but pythons will tell you the behavior changed in 3.6, and nothing more.

HTH

27

Perhaps there is enough interest in such a list that it could be added as an appendix to the language reference, or as a separate document. The main issue of an official doc is whether it would get updated. Alternatively, the reference could mention a wiki page.

2 Likes
28

I would suggest to write a scraping tool (or any other type of extraction) that manages to parse the related changes out of the What’s New articles.

If that creates a reStructuredText document you can publish it with ReadTheDocs.org just like probably most of the documentation of Python open source projects.

That as an open source project may manage to stay alive. I’m sure there are more than just a handful people interested in this topic.

29

Might scraping the docs (or the PDFs) be an option? Is the phrase “New in version 3.10”, New in version 3.4: The create_module() method of loaders, … sufficiently consistent?
Regards =dn

30

This thread is gold! Thank you @nikdissv-Forever for the amazing document.

1 Like
31

Thanks Chris; I used 2 to 3 and got through almost 200 lines this time. I think I can get this code working and am learning Python as I progress.
Peter.

32

I have reproduced Nikita’s document at: sdt-artifacts/Nikita_Denissov_discuss.md at main · atlantistechnology/sdt-artifacts · GitHub. This is described and linked to at sdt-artifacts/README.md at main · atlantistechnology/sdt-artifacts · GitHub. But all of this is used for GitHub Pages, and hence at https://www.sdt.dev/ (once the served site updates, which seems to take 1-24 hours to happen).