What’s New In Python 3.10¶
- Editor:
Pablo Galindo Salgado
This article explains the new features in Python 3.10, compared to 3.9. Python 3.10 was released on October 4, 2021. For full details, see the changelog.
Summary – Release highlights¶
New syntax features:
PEP 634, Structural Pattern Matching: Specification
PEP 635, Structural Pattern Matching: Motivation and Rationale
PEP 636, Structural Pattern Matching: Tutorial
bpo-12782, Parenthesized context managers are now officially allowed.
New features in the standard library:
PEP 618, Add Optional Length-Checking To zip.
Interpreter improvements:
PEP 626, Precise line numbers for debugging and other tools.
New typing features:
PEP 604, Allow writing union types as X | Y
PEP 612, Parameter Specification Variables
PEP 613, Explicit Type Aliases
PEP 647, User-Defined Type Guards
Important deprecations, removals or restrictions:
New Features¶
Parenthesized context managers¶
Using enclosing parentheses for continuation across multiple lines in context managers is now supported. This allows formatting a long collection of context managers in multiple lines in a similar way as it was previously possible with import statements. For instance, all these examples are now valid:
with (CtxManager() as example):
...
with (
CtxManager1(),
CtxManager2()
):
...
with (CtxManager1() as example,
CtxManager2()):
...
with (CtxManager1(),
CtxManager2() as example):
...
with (
CtxManager1() as example1,
CtxManager2() as example2
):
...
it is also possible to use a trailing comma at the end of the enclosed group:
with (
CtxManager1() as example1,
CtxManager2() as example2,
CtxManager3() as example3,
):
...
This new syntax uses the non LL(1) capacities of the new parser. Check PEP 617 for more details.
(Contributed by Guido van Rossum, Pablo Galindo and Lysandros Nikolaou in bpo-12782 and bpo-40334.)
Better error messages¶
SyntaxErrors¶
When parsing code that contains unclosed parentheses or brackets the interpreter now includes the location of the unclosed bracket of parentheses instead of displaying SyntaxError: unexpected EOF while parsing or pointing to some incorrect location. For instance, consider the following code (notice the unclosed ‘{‘):
expected = {9: 1, 18: 2, 19: 2, 27: 3, 28: 3, 29: 3, 36: 4, 37: 4,
38: 4, 39: 4, 45: 5, 46: 5, 47: 5, 48: 5, 49: 5, 54: 6,
some_other_code = foo()
Previous versions of the interpreter reported confusing places as the location of the syntax error:
File "example.py", line 3
some_other_code = foo()
^
SyntaxError: invalid syntax
but in Python 3.10 a more informative error is emitted:
File "example.py", line 1
expected = {9: 1, 18: 2, 19: 2, 27: 3, 28: 3, 29: 3, 36: 4, 37: 4,
^
SyntaxError: '{' was never closed
In a similar way, errors involving unclosed string literals (single and triple quoted) now point to the start of the string instead of reporting EOF/EOL.
These improvements are inspired by previous work in the PyPy interpreter.
(Contributed by Pablo Galindo in bpo-42864 and Batuhan Taskaya in bpo-40176.)
SyntaxError exceptions raised by the interpreter will now highlight the
full error range of the expression that constitutes the syntax error itself,
instead of just where the problem is detected. In this way, instead of displaying
(before Python 3.10):
>>> foo(x, z for z in range(10), t, w)
File "<stdin>", line 1
foo(x, z for z in range(10), t, w)
^
SyntaxError: Generator expression must be parenthesized
now Python 3.10 will display the exception as:
>>> foo(x, z for z in range(10), t, w)
File "<stdin>", line 1
foo(x, z for z in range(10), t, w)
^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized
This improvement was contributed by Pablo Galindo in bpo-43914.
A considerable amount of new specialized messages for SyntaxError exceptions
have been incorporated. Some of the most notable ones are as follows:
Missing
:before blocks:>>> if rocket.position > event_horizon File "<stdin>", line 1 if rocket.position > event_horizon ^ SyntaxError: expected ':'
(Contributed by Pablo Galindo in bpo-42997.)
Unparenthesised tuples in comprehensions targets:
>>> {x,y for x,y in zip('abcd', '1234')} File "<stdin>", line 1 {x,y for x,y in zip('abcd', '1234')} ^ SyntaxError: did you forget parentheses around the comprehension target?
(Contributed by Pablo Galindo in bpo-43017.)
Missing commas in collection literals and between expressions:
>>> items = { ... x: 1, ... y: 2 ... z: 3, File "<stdin>", line 3 y: 2 ^ SyntaxError: invalid syntax. Perhaps you forgot a comma?
(Contributed by Pablo Galindo in bpo-43822.)
Multiple Exception types without parentheses:
>>> try: ... build_dyson_sphere() ... except NotEnoughScienceError, NotEnoughResourcesError: File "<stdin>", line 3 except NotEnoughScienceError, NotEnoughResourcesError: ^ SyntaxError: multiple exception types must be parenthesized
(Contributed by Pablo Galindo in bpo-43149.)
Missing
:and values in dictionary literals:>>> values = { ... x: 1, ... y: 2, ... z: ... } File "<stdin>", line 4 z: ^ SyntaxError: expression expected after dictionary key and ':' >>> values = {x:1, y:2, z w:3} File "<stdin>", line 1 values = {x:1, y:2, z w:3} ^ SyntaxError: ':' expected after dictionary key
(Contributed by Pablo Galindo in bpo-43823.)
tryblocks withoutexceptorfinallyblocks:>>> try: ... x = 2 ... something = 3 File "<stdin>", line 3 something = 3 ^^^^^^^^^ SyntaxError: expected 'except' or 'finally' block
(Contributed by Pablo Galindo in bpo-44305.)
Usage of
=instead of==in comparisons:>>> if rocket.position = event_horizon: File "<stdin>", line 1 if rocket.position = event_horizon: ^ SyntaxError: cannot assign to attribute here. Maybe you meant '==' instead of '='?
(Contributed by Pablo Galindo in bpo-43797.)
Usage of
*in f-strings:>>> f"Black holes {*all_black_holes} and revelations" File "<stdin>", line 1 (*all_black_holes) ^ SyntaxError: f-string: cannot use starred expression here
(Contributed by Pablo Galindo in bpo-41064.)
IndentationErrors¶
Many IndentationError exceptions now have more context regarding what kind of block
was expecting an indentation, including the location of the statement:
>>> def foo():
... if lel:
... x = 2
File "<stdin>", line 3
x = 2
^
IndentationError: expected an indented block after 'if' statement in line 2
AttributeErrors¶
When printing AttributeError, PyErr_Display() will offer
suggestions of similar attribute names in the object that the exception was
raised from:
>>> collections.namedtoplo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'collections' has no attribute 'namedtoplo'. Did you mean: namedtuple?
(Contributed by Pablo Galindo in bpo-38530.)
Warning
Notice this won’t work if PyErr_Display() is not called to display the error
which can happen if some other custom error display function is used. This is a common
scenario in some REPLs like IPython.
NameErrors¶
When printing NameError raised by the interpreter, PyErr_Display()
will offer suggestions of similar variable names in the function that the exception
was raised from:
>>> schwarzschild_black_hole = None
>>> schwarschild_black_hole
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'schwarschild_black_hole' is not defined. Did you mean: schwarzschild_black_hole?
(Contributed by Pablo Galindo in bpo-38530.)
Warning
Notice this won’t work if PyErr_Display() is not called to display the error,
which can happen if some other custom error display function is used. This is a common
scenario in some REPLs like IPython.
PEP 626: Precise line numbers for debugging and other tools¶
PEP 626 brings more precise and reliable line numbers for debugging, profiling and coverage tools. Tracing events, with the correct line number, are generated for all lines of code executed and only for lines of code that are executed.
The f_lineno attribute of frame objects will always contain the
expected line number.
The co_lnotab attribute of
code objects is deprecated and
will be removed in 3.12.
Code that needs to convert from offset to line number should use the new
co_lines() method instead.
PEP 634: Structural Pattern Matching¶
Structural pattern matching has been added in the form of a match statement and case statements of patterns with associated actions. Patterns consist of sequences, mappings, primitive data types as well as class instances. Pattern matching enables programs to extract information from complex data types, branch on the structure of data, and apply specific actions based on different forms of data.
Syntax and operations¶
The generic syntax of pattern matching is:
match subject:
case <pattern_1>:
<action_1>
case <pattern_2>:
<action_2>
case <pattern_3>:
<action_3>
case _:
<action_wildcard>
A match statement takes an expression and compares its value to successive patterns given as one or more case blocks. Specifically, pattern matching operates by:
using data with type and shape (the
subject)evaluating the
subjectin thematchstatementcomparing the subject with each pattern in a
casestatement from top to bottom until a match is confirmed.executing the action associated with the pattern of the confirmed match
If an exact match is not confirmed, the last case, a wildcard
_, if provided, will be used as the matching case. If an exact match is not confirmed and a wildcard case does not exist, the entire match block is a no-op.
Declarative approach¶
Readers may be aware of pattern matching through the simple example of matching a subject (data object) to a literal (pattern) with the switch statement found in C, Java or JavaScript (and many other languages). Often the switch statement is used for comparison of an object/expression with case statements containing literals.
More powerful examples of pattern matching can be found in languages such as Scala and Elixir. With structural pattern matching, the approach is “declarative” and explicitly states the conditions (the patterns) for data to match.
While an “imperative” series of instructions using nested “if” statements could be used to accomplish something similar to structural pattern matching, it is less clear than the “declarative” approach. Instead the “declarative” approach states the conditions to meet for a match and is more readable through its explicit patterns. While structural pattern matching can be used in its simplest form comparing a variable to a literal in a case statement, its true value for Python lies in its handling of the subject’s type and shape.
Simple pattern: match to a literal¶
Let’s look at this example as pattern matching in its simplest form: a value,
the subject, being matched to several literals, the patterns. In the example
below, status is the subject of the match statement. The patterns are
each of the case statements, where literals represent request status codes.
The associated action to the case is executed after a match:
def http_error(status):
match status:
case 400:
return "Bad request"
case 404:
return "Not found"
case 418:
return "I'm a teapot"
case _:
return "Something's wrong with the internet"
If the above function is passed a status of 418, “I’m a teapot” is returned.
If the above function is passed a status of 500, the case statement with
_ will match as a wildcard, and “Something’s wrong with the internet” is
returned.
Note the last block: the variable name, _, acts as a wildcard and insures
the subject will always match. The use of _ is optional.
You can combine several literals in a single pattern using | (“or”):
case 401 | 403 | 404:
return "Not allowed"
Behavior without the wildcard¶
If we modify the above example by removing the last case block, the example becomes:
def http_error(status):
match status:
case 400:
return "Bad request"
case 404:
return "Not found"
case 418:
return "I'm a teapot"
Without the use of _ in a case statement, a match may not exist. If no
match exists, the behavior is a no-op. For example, if status of 500 is
passed, a no-op occurs.
Patterns with a literal and variable¶
Patterns can look like unpacking assignments, and a pattern may be used to bind variables. In this example, a data point can be unpacked to its x-coordinate and y-coordinate:
# point is an (x, y) tuple
match point:
case (0, 0):
print("Origin")
case (0, y):
print(f"Y={y}")
case (x, 0):
print(f"X={x}")
case (x, y):
print(f"X={x}, Y={y}")
case _:
raise ValueError("Not a point")
The first pattern has two literals, (0, 0), and may be thought of as an
extension of the literal pattern shown above. The next two patterns combine a
literal and a variable, and the variable binds a value from the subject
(point). The fourth pattern captures two values, which makes it
conceptually similar to the unpacking assignment (x, y) = point.
Patterns and classes¶
If you are using classes to structure your data, you can use as a pattern the class name followed by an argument list resembling a constructor. This pattern has the ability to capture instance attributes into variables:
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def location(point):
match point:
case Point(x=0, y=0):
print("Origin is the point's location.")
case Point(x=0, y=y):
print(f"Y={y} and the point is on the y-axis.")
case Point(x=x, y=0):
print(f"X={x} and the point is on the x-axis.")
case Point():
print("The point is located somewhere else on the plane.")
case _:
print("Not a point")
Patterns with positional parameters¶
You can use positional parameters with some builtin classes that provide an
ordering for their attributes (e.g. dataclasses). You can also define a specific
position for attributes in patterns by setting the __match_args__ special
attribute in your classes. If it’s set to (“x”, “y”), the following patterns
are all equivalent (and all bind the y attribute to the var variable):
Point(1, var)
Point(1, y=var)
Point(x=1, y=var)
Point(y=var, x=1)
Nested patterns¶
Patterns can be arbitrarily nested. For example, if our data is a short list of points, it could be matched like this:
match points:
case []:
print("No points in the list.")
case [Point(0, 0)]:
print("The origin is the only point in the list.")
case [Point(x, y)]:
print(f"A single point {x}, {y} is in the list.")
case [Point(0, y1), Point(0, y2)]:
print(f"Two points on the Y axis at {y1}, {y2} are in the list.")
case _:
print("Something else is found in the list.")
Complex patterns and the wildcard¶
To this point, the examples have used _ alone in the last case statement.
A wildcard can be used in more complex patterns, such as ('error', code, _).
For example:
match test_variable:
case ('warning', code, 40):
print("A warning has been received.")
case ('error', code, _):
print(f"An error {code} occurred.")
In the above case, test_variable will match for (‘error’, code, 100) and
(‘error’, code, 800).
Guard¶
We can add an if clause to a pattern, known as a “guard”. If the
guard is false, match goes on to try the next case block. Note
that value capture happens before the guard is evaluated:
match point:
case Point(x, y) if x == y:
print(f"The point is located on the diagonal Y=X at {x}.")
case Point(x, y):
print(f"Point is not on the diagonal.")
Other Key Features¶
Several other key features:
Like unpacking assignments, tuple and list patterns have exactly the same meaning and actually match arbitrary sequences. Technically, the subject must be a sequence. Therefore, an important exception is that patterns don’t match iterators. Also, to prevent a common mistake, sequence patterns don’t match strings.
Sequence patterns support wildcards:
[x, y, *rest]and(x, y, *rest)work similar to wildcards in unpacking assignments. The name after*may also be_, so(x, y, *_)matches a sequence of at least two items without binding the remaining items.Mapping patterns:
{"bandwidth": b, "latency": l}captures the"bandwidth"and"latency"values from a dict. Unlike sequence patterns, extra keys are ignored. A wildcard**restis also supported. (But**_would be redundant, so is not allowed.)Subpatterns may be captured using the
askeyword:case (Point(x1, y1), Point(x2, y2) as p2): ...
This binds x1, y1, x2, y2 like you would expect without the
asclause, and p2 to the entire second item of the subject.Most literals are compared by equality. However, the singletons
True,FalseandNoneare compared by identity.Named constants may be used in patterns. These named constants must be dotted names to prevent the constant from being interpreted as a capture variable:
from enum import Enum class Color(Enum): RED = 0 GREEN = 1 BLUE = 2 color = Color.GREEN match color: case Color.RED: print("I see red!") case Color.GREEN: print("Grass is green") case Color.BLUE: print("I'm feeling the blues :(")
For the full specification see PEP 634. Motivation and rationale are in PEP 635, and a longer tutorial is in PEP 636.
Optional EncodingWarning and encoding="locale" option¶
The default encoding of TextIOWrapper and open() is
platform and locale dependent. Since UTF-8 is used on most Unix
platforms, omitting encoding option when opening UTF-8 files
(e.g. JSON, YAML, TOML, Markdown) is a very common bug. For example:
# BUG: "rb" mode or encoding="utf-8" should be used.
with open("data.json") as f:
data = json.load(f)
To find this type of bug, an optional EncodingWarning is added.
It is emitted when sys.flags.warn_default_encoding
is true and locale-specific default encoding is used.
-X warn_default_encoding option and PYTHONWARNDEFAULTENCODING
are added to enable the warning.
See Text Encoding for more information.
Other Language Changes¶
The
inttype has a new methodint.bit_count(), returning the number of ones in the binary expansion of a given integer, also known as the population count. (Contributed by Niklas Fiekas in bpo-29882.)The views returned by
dict.keys(),dict.values()anddict.items()now all have amappingattribute that gives atypes.MappingProxyTypeobject wrapping the original dictionary. (Contributed by Dennis Sweeney in bpo-40890.)PEP 618: The
zip()function now has an optionalstrictflag, used to require that all the iterables have an equal length.Builtin and extension functions that take integer arguments no longer accept
Decimals,Fractions and other objects that can be converted to integers only with a loss (e.g. that have the__int__()method but do not have the__index__()method). (Contributed by Serhiy Storchaka in bpo-37999.)If
object.__ipow__()returnsNotImplemented, the operator will correctly fall back toobject.__pow__()andobject.__rpow__()as expected. (Contributed by Alex Shkop in bpo-38302.)Assignment expressions can now be used unparenthesized within set literals and set comprehensions, as well as in sequence indexes (but not slices).
Functions have a new
__builtins__attribute which is used to look for builtin symbols when a function is executed, instead of looking into__globals__['__builtins__']. The attribute is initialized from__globals__["__builtins__"]if it exists, else from the current builtins. (Contributed by Mark Shannon in bpo-42990.)Two new builtin functions –
aiter()andanext()have been added to provide asynchronous counterparts toiter()andnext(), respectively. (Contributed by Joshua Bronson, Daniel Pope, and Justin Wang in bpo-31861.)Static methods (
@staticmethod) and class methods (@classmethod) now inherit the method attributes (__module__,__name__,__qualname__,__doc__,__annotations__) and have a new__wrapped__attribute. Moreover, static methods are now callable as regular functions. (Contributed by Victor Stinner in bpo-43682.)Annotations for complex targets (everything beside
simple nametargets defined by PEP 526) no longer cause any runtime effects withfrom __future__ import annotations. (Contributed by Batuhan Taskaya in bpo-42737.)Class and module objects now lazy-create empty annotations dicts on demand. The annotations dicts are stored in the object’s
__dict__for backwards compatibility. This improves the best practices for working with__annotations__; for more information, please see Annotations Best Practices. (Contributed by Larry Hastings in bpo-43901.)Annotations consist of
yield,yield from,awaitor named expressions are now forbidden underfrom __future__ import annotationsdue to their side effects. (Contributed by Batuhan Taskaya in bpo-42725.)Usage of unbound variables,
super()and other expressions that might alter the processing of symbol table as annotations are now rendered effectless underfrom __future__ import annotations. (Contributed by Batuhan Taskaya in bpo-42725.)Hashes of NaN values of both
floattype anddecimal.Decimaltype now depend on object identity. Formerly, they always hashed to0even though NaN values are not equal to one another. This caused potentially quadratic runtime behavior due to excessive hash collisions when creating dictionaries and sets containing multiple NaNs. (Contributed by Raymond Hettinger in bpo-43475.)A
SyntaxError(instead of aNameError) will be raised when deleting the__debug__constant. (Contributed by Donghee Na in bpo-45000.)SyntaxErrorexceptions now haveend_linenoandend_offsetattributes. They will beNoneif not determined. (Contributed by Pablo Galindo in bpo-43914.)
New Modules¶
None.
Improved Modules¶
asyncio¶
Add missing connect_accepted_socket()
method.
(Contributed by Alex Grönholm in bpo-41332.)
argparse¶
Misleading phrase “optional arguments” was replaced with “options” in argparse help. Some tests might require adaptation if they rely on exact output match. (Contributed by Raymond Hettinger in bpo-9694.)
array¶
The index() method of array.array now has
optional start and stop parameters.
(Contributed by Anders Lorentsen and Zackery Spytz in bpo-31956.)
asynchat, asyncore, smtpd¶
These modules have been marked as deprecated in their module documentation
since Python 3.6. An import-time DeprecationWarning has now been
added to all three of these modules.
base64¶
Add base64.b32hexencode() and base64.b32hexdecode() to support the
Base32 Encoding with Extended Hex Alphabet.
bdb¶
Add clearBreakpoints() to reset all set breakpoints.
(Contributed by Irit Katriel in bpo-24160.)
bisect¶
Added the possibility of providing a key function to the APIs in the bisect
module. (Contributed by Raymond Hettinger in bpo-4356.)
codecs¶
Add a codecs.unregister() function to unregister a codec search function.
(Contributed by Hai Shi in bpo-41842.)
collections.abc¶
The __args__ of the parameterized generic for
collections.abc.Callable are now consistent with typing.Callable.
collections.abc.Callable generic now flattens type parameters, similar
to what typing.Callable currently does. This means that
collections.abc.Callable[[int, str], str] will have __args__ of
(int, str, str); previously this was ([int, str], str). To allow this
change, types.GenericAlias can now be subclassed, and a subclass will
be returned when subscripting the collections.abc.Callable type. Note
that a TypeError may be raised for invalid forms of parameterizing
collections.abc.Callable which may have passed silently in Python 3.9.
(Contributed by Ken Jin in bpo-42195.)
contextlib¶
Add a contextlib.aclosing() context manager to safely close async generators
and objects representing asynchronously released resources.
(Contributed by Joongi Kim and John Belmonte in bpo-41229.)
Add asynchronous context manager support to contextlib.nullcontext().
(Contributed by Tom Gringauz in bpo-41543.)
Add AsyncContextDecorator, for supporting usage of async
context managers as decorators.
curses¶
The extended color functions added in ncurses 6.1 will be used transparently
by curses.color_content(), curses.init_color(),
curses.init_pair(), and curses.pair_content(). A new function,
curses.has_extended_color_support(), indicates whether extended color
support is provided by the underlying ncurses library.
(Contributed by Jeffrey Kintscher and Hans Petter Jansson in bpo-36982.)
The BUTTON5_* constants are now exposed in the curses module if
they are provided by the underlying curses library.
(Contributed by Zackery Spytz in bpo-39273.)
dataclasses¶
__slots__¶
Added slots parameter in dataclasses.dataclass() decorator.
(Contributed by Yurii Karabas in bpo-42269)
Keyword-only fields¶
dataclasses now supports fields that are keyword-only in the generated __init__ method. There are a number of ways of specifying keyword-only fields.
You can say that every field is keyword-only:
from dataclasses import dataclass
@dataclass(kw_only=True)
class Birthday:
name: str
birthday: datetime.date
Both name and birthday are keyword-only parameters to the
generated __init__ method.
You can specify keyword-only on a per-field basis:
from dataclasses import dataclass, field
@dataclass
class Birthday:
name: str
birthday: datetime.date = field(kw_only=True)
Here only birthday is keyword-only. If you set kw_only on
individual fields, be aware that there are rules about re-ordering
fields due to keyword-only fields needing to follow non-keyword-only
fields. See the full dataclasses documentation for details.
You can also specify that all fields following a KW_ONLY marker are keyword-only. This will probably be the most common usage:
from dataclasses import dataclass, KW_ONLY
@dataclass
class Point:
x: float
y: float
_: KW_ONLY
z: float = 0.0
t: float = 0.0
Here, z and t are keyword-only parameters, while x and
y are not.
(Contributed by Eric V. Smith in bpo-43532.)
distutils¶
The entire distutils package is deprecated, to be removed in Python
3.12. Its functionality for specifying package builds has already been
completely replaced by third-party packages setuptools and
packaging, and most other commonly used APIs are available elsewhere
in the standard library (such as platform, shutil,
subprocess or sysconfig). There are no plans to migrate
any other functionality from distutils, and applications that are
using other functions should plan to make private copies of the code.
Refer to PEP 632 for discussion.
The bdist_wininst command deprecated in Python 3.8 has been removed.
The bdist_wheel command is now recommended to distribute binary packages
on Windows.
(Contributed by Victor Stinner in bpo-42802.)
doctest¶
When a module does not define __loader__, fall back to __spec__.loader.
(Contributed by Brett Cannon in bpo-42133.)
encodings¶
encodings.normalize_encoding() now ignores non-ASCII characters.
(Contributed by Hai Shi in bpo-39337.)
enum¶
Enum __repr__() now returns enum_name.member_name and