The Python language is designed to make complex tasks simple, so updates iterate relatively quickly and require us to keep up with them!
The release of a new version is always accompanied by new features and functions, and the first thing we need to understand and attribute these points before upgrading the version, so that we may use them flexibly in our programming later. Can’t wait, ready to go, so let’s start now!

1. PEP 604
New Type Union Operator
In previous versions when you wanted to declare a type to contain multiple types, you needed to use Union[] to contain multiple types, now you can just use the | symbol to do it.
-
Old Version
-
New Version
-
can be used in
isinstanceandissubclass.
2. PEP 613
TypeAlias
Sometimes we want to customize a type, so we can create an alias (Alias) to do so. However, for the type checker (Type Checker), it can’t tell if it’s a type alias or a normal assignment, and now we can easily tell by introducing TypeAlias.
-
The previous writing style
1Board = List[Tuple[str, str]] -
The new version of the writing style
3. PEP 647
User-Defined Type Guards
In contemporary static checking tools (such as typescript, mypy, etc.) there is a feature called Type narrowing. This is when a parameter type could have matched more than one type, but under certain conditions the type range can be narrowed down to a smaller range of type(s).
|
|
-
Problems identified - Deficiencies exist
A more accurate knowledge of the object’s type is very friendly to
mypy, and the conclusions of the check will be more accurate; type narrowing can be problematic in some scenarios.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25In : def is_str(obj: object) -> bool: ...: return isinstance(obj, str) ...: ...: ...: def to_list(obj: object) -> list[str]: ...: if is_str(obj): ...: return list(obj) ...: return [] ...: In : def is_str(obj: object) -> bool: ...: return isinstance(obj, str) ...: ...: ...: def to_list(obj: object) -> list[str]: ...: if is_str(obj): ...: return list(obj) ...: return [] ...: In : to_list('aaa') Out: ['a', 'a', 'a'] In : to_list(111) Out: []This code is simpler than the one mentioned in
PEP, they are both correct codes and the type comments are problematic. But runningmypyprompts an error message. In the2functionsobjis used because it is not sure of the object type, soobjectis used, and in factto_listwill only handleobjas typestr. Originallyif is_str(obj)would have narrowed the type, but because it was split into functions,isinstancedidn’t succeed in narrowing it here.1 2 3 4 5 6➜ mypy wrong_to_list.py wrong_to_list.py:7: error: No overload variant of "list" matches argument type "object" wrong_to_list.py:7: note: Possible overload variants: wrong_to_list.py:7: note: def [_T] list(self) -> List[_T] wrong_to_list.py:7: note: def [_T] list(self, Iterable[_T]) -> List[_T] Found 1 error in 1 file (checked 1 source file) -
User-defined
Type Guardsare provided in the new version to resolveOriginally the type of the return value was
bool, now we specify it asTypeGuard[str]so thatmypycan understand its type. Actually, to put it another way, you can readTypeGuard[str]as an alias forboolwith a type declaration, so please understand this carefully.
4. PEP 612
Parameter Specification Variables
The type system in Python has limited support for types of Callable (such as functions), it can only specify the type of the Callable, but it cannot be propagated for arguments to function calls.
-
This problem exists mainly in the usage of decorators
The argument value received by
joinis supposed to be a list of strings, butmypydoes not validate this lastprint(join([1, 2]))correctly. Because the type ofargsandkwargsin theinnerfunction in thelogdecorator isAny, this causes the type of the arguments chosen for the call to be unverified, and frankly, it can be written any way.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17from collections.abc import Callable from typing import Any, TypeVar R = TypeVar('R') def log(func: Callable[..., R]) -> Callable[..., R]: def inner(*args: Any, **kwargs: Any) -> R: print('In') return func(*args, **kwargs) return inner @log def join(items: list[str]): return ','.join(items) print(join(['1', '2'])) # Correct usage print(join([1, 2])) # Wrong usage, mypy should prompt type error -
Newer versions can use
ParamSpecto resolveBy using
typing.ParamSpec, the argument type ofinneris passed directly throughP.argsandP.kwargsfor validation purposes.typing.ParamSpechelps us to facilitate [referencing] positional and keyword arguments, and thisPEPaddition of `typing. is to provide an ability to add, remove or convert parameters of another callable object.The more common added arguments I can think of are the [injected] type of decorators. The
joinfunction has2arguments, but since the first argumentloggeris [injected] in thewith_loggerdecorator, you only need to pass the value of theitemsargument when you use it.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21import logging from collections.abc import Callable from typing import TypeVar, ParamSpec, Concatenate logging.basicConfig(level=logging.NOTSET) R = TypeVar('R') P = ParamSpec('P') def with_logger(func: Callable[Concatenate[logging.Logger, P], R]) -> Callable[P, R]: def inner(*args: P.args, **kwargs: P.kwargs) -> R: logger = logging.getLogger(func.__name__) return func(logger, *args, **kwargs) return inner @with_logger def join(logger: logging.Logger, items: list[str]): logger.info('Info') return ','.join(items) print(join(['1', '2'])) print(join([1, 2]))In addition to adding, removing and converting parameters can also be done with
Concatenate, see another example of removing parameters. With theremove_firstdecorator, the first argument passed in is ignored, soadd(1, 2, 3)is actually computingadd(2, 3). Be careful to understand where thisConcatenateis, if it is added, thenConcatenateis added to the type declaration of theCallableof the decorator argument, if it is removed, it is added to the type declaration of the returnedCallable.
5. PEP 618
Parameters of the zip function
I believe that every Python developer with much working experience has experienced this trap, when the elements within the argument are of different lengths, the part with the longer length is ignored, without any hint, when in fact it was originally mentioned in the documentation.
But for most developers no one really pays attention to this documentation, the problem is very implicit and requires developers to understand the zip problem. The newer versions of strict=True will report an error when the length of the elements within the argument is different.
|
|
6. PEP 626
Better error messages
There are some examples of error messages listed in the official that were not obvious and friendly enough in the previous version, this version deals with them centrally. Here they are: Same SyntaxError Isn’t the hint in the new version easier to understand? Not only does it suggest the type of error, it also gives hints.
-
The old way of writing
-
New writing style
-
The old way of writing
-
New writing style
7. PEP 634-636
Match-Case
Many Python core developers believe that Python does not need to add switch-case syntax, because the same effect can be achieved with if/elif/else. This new syntax is called Structural Pattern Matching in Chinese, and there are three PEPs to introduce it because there are a lot of new features
- PEP 634: Introduces the
matchsyntax and supported patterns. - PEP 635: Explains the reason for the syntax being designed this way
- PEP 636: a tutorial introducing the concepts, syntax and semantics
match is followed by the variable to be matched, case is followed by a different condition, and then the statement to be executed if the condition is met. The last case is underlined to indicate a default match, so if the previous condition does not match, it will be executed in this case, which is equivalent to the previous else.
But the match-case syntax can do much more than the switch-case syntax in C / Go languages, it is actually the match-case syntax in Scala / Erlang languages, which supports complex pattern matching, and I will demonstrate the flexibility of this new syntax and pythonic in detail with several pattern examples.
-
[1] Literal mode
The above example is a literal pattern that uses the basic data structures that come with
Python, such as strings, numbers, booleans, andNone. -
[2] Capture mode
Can match the assignment target of a single expression. If
greetingis not null, it will be assigned toname, but note that ifgreetingis null aNameErrororUnboundLocalErrorerror will be thrown, sincenamehas not been defined before.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25In : capture('Alex') Hi Alex! In : capture('Santa') Hi Santa! Match In : capture('') Hello! --------------------------------------------------------------------------- UnboundLocalError Traceback (most recent call last) Input In [4], in <cell line: 1>() ----> 1 capture('') Input In [1], in capture(greeting) 1 def capture(greeting): 2 match greeting: 3 case "": 4 print("Hello!") 5 case name: 6 print(f"Hi {name}!") ----> 7 if name == "Santa": 8 print('Match') UnboundLocalError: local variable 'name' referenced before assignment -
[3] Sequence mode
The results can be used in
matchin either list or tuple format. You can also use thefirst, *rest = seqpattern to unpack as described in PEP 3132 - Extended Iterable Unpacking.The first element of this
matchcondition needs to be1, otherwise the match fails. The firstcaseuses a list and unpacking, the secondcaseuses a tuple, which is actually the same semantics as a list, and the third is still a list.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28In : def sequence(collection): ...: match collection: ...: case 1, [x, *others]: ...: print(f"Got 1 and a nested sequence: {x=}, {others=}") ...: case (1, x): ...: print(f"Got 1 and {x}") ...: case [x, y, z]: ...: print(f"{x=}, {y=}, {z=}") ...: In : sequence([1]) In : sequence([1, 2]) Got 1 and 2 In : sequence([1, 2, 3]) x=1, y=2, z=3 In : sequence([1, [2, 3]]) Got 1 and a nested sequence: x=2, others=[3] In : sequence([1, [2, 3, 4]]) Got 1 and a nested sequence: x=2, others=[3, 4] In : sequence([2, 3]) In : sequence((1, 2)) Got 1 and 2If the pattern after
caseis a single item, you can remove the parentheses and write it that way. However, note thatcase 1, [x, *others]cannot be written without the parentheses, and the logic of the decomposition will change if the parentheses are removed. -
[4] Wildcard pattern
Use the single underscore
_to match any result, but not bound (not assigned to a variable or variables), the finalcase _is the wildcard pattern, but of course there can be multiple matches, and the sequence pattern mentioned earlier also supports_. The use of wildcards requires attention to the logical order, putting the small range in front and the large range in the back to prevent it from not meeting expectations.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21In : def sequence2(collection): ...: match collection: ...: case ["a", *_, "z"]: ...: print('matches any sequence of length two or more that starts with "a" and ends with "z".') ...: case (_, _, *_): ...: print('matches any sequence of length two or more.') ...: case [*_]: ...: print('matches a sequence of any length.') ...: In : sequence2(['a', 2, 3, 'z']) matches any sequence of length two or more that starts with "a" and ends with "z". In : sequence2(['a', 2, 3, 'b']) matches any sequence of length two or more. In : sequence2(['a', 'b']) matches any sequence of length two or more. In : sequence2(['a']) matches a sequence of any length. -
[5] Constant value mode
This pattern, mainly matches constants or enumeration values of the
enummodule.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34In : class Color(Enum): ...: RED = 1 ...: GREEN = 2 ...: BLUE = 3 ...: In : class NewColor: ...: YELLOW = 4 ...: In : def constant_value(color): ...: match color: ...: case Color.RED: ...: print('Red') ...: case NewColor.YELLOW: ...: print('Yellow') ...: case new_color: ...: print(new_color) ...: In : constant_value(Color.RED) # Match the first case Red In : constant_value(NewColor.YELLOW) # Match the second case Yellow In : constant_value(Color.GREEN) # Match the third case Color.GREEN In : constant_value(4) # The constant values match the second case as well Yellow In : constant_value(10) # Other constants 10Note here that since
casehas a binding effect, you cannot use constants likeYELLOWdirectly, which is wrong. -
[6] Mapping mode
It’s actually
casefollowed by support for using a dictionary to do the matching.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18In : def mapping(config): ...: match config: ...: case {'sub': sub_config, **rest}: ...: print(f'Sub: {sub_config}') ...: print(f'OTHERS: {rest}') ...: case {'route': route}: ...: print(f'ROUTE: {route}') ...: In : mapping({}) In : mapping({'route': '/auth/login'}) # Match the first case ROUTE: /auth/login # Match dictionaries with sub keys, values are bound to sub_config and the rest of the dictionary is bound to rest In : mapping({'route': '/auth/login', 'sub': {'a': 1}}) # Match the second case Sub: {'a': 1} OTHERS: {'route': '/auth/login'} -
[7] Class mode
After
casesupports any object to do matching, this is because for matching the position needs to be determined , so the position parameter needs to be used to identify it.Another solution to this custom class that doesn’t use positional arguments for matching is to use
__match_args__to return an array of positional arguments, like this. HerePoint2uses the standard library’sdataclasses.dataclassdecorator, which will provide the__match_args__property, so it can be used directly.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29In : class Point: ...: __match_args__ = ('x', 'y') ...: ...: def __init__(self, x, y): ...: self.x = x ...: self.y = y ...: In : from dataclasses import dataclass In : @dataclass ...: class Point2: ...: x: int ...: y: int ...: In : def class_pattern(obj): ...: match obj: ...: case Point(x, y): ...: print(f'Point({x=},{y=})') ...: case Point2(x, y): ...: print(f'Point2({x=},{y=})') ...: In : class_pattern(Point(1, 2)) Point(x=1,y=2) In : class_pattern(Point2(1, 2)) Point2(x=1,y=2) -
[8] Combination (OR) mode
Multiple literals can be combined to represent an or relation using
|, and|can exist more than one within acasecondition to represent multiple or relations.1 2 3 4 5 6 7 8 9 10 11 12def or_pattern(obj): match obj: case 0 | 1 | 2: # 0,1,2 three numbers match print('small number') case list() | set(): # List or set matching print('list or set') case str() | bytes(): # String or bytes match print('str or bytes') case Point(x, y) | Point2(x, y): # Borrowing from the previous 2 classes, one of them can match print(f'{x=},{y=}') case [x] | x: # list and only one element or single value matches the print(f'{x=}')Note that the
[x]incase [x] | xis not triggered because of the matching order, andxcannot be a set, string,byte, etc., because it will not be matched in the previous condition. Also, there is nocasesyntax inPythonfor theANDrelationship.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29In : or_pattern(1) small number In : or_pattern(2) small number In : or_pattern([1]) list or set In : or_pattern({1, 2}) list or set In : or_pattern('sss') str or bytes In : or_pattern(b'sd') str or bytes In : or_pattern(Point(1, 2)) x=1,y=2 In : or_pattern(Point2(1, 2)) x=1,y=2 In : or_pattern(4) x=4 In : or_pattern({}) x={} -
[9] AS mode
The
ASpattern was actually a walrus pattern in the early days, but it was later discussed that the use of theaskeyword would give the syntax an edge. It should be noted that[0, int() as i]in this case is a subpattern, that is, it contains patterns within patterns:[0, int() as i]is the sequence pattern matched bycase, and whereint() as iis the subpattern, it is theASpattern.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27In : def as_pattern(obj): ...: match obj: ...: case str() as s: ...: print(f'Got str: {s=}') ...: case [0, int() as i]: ...: print(f'Got int: {i=}') ...: case [tuple() as tu]: ...: print(f'Got tuple: {tu=}') ...: case list() | set() | dict() as iterable: ...: print(f'Got iterable: {iterable=}') ...: ...: In : as_pattern('sss') Got str: s='sss' In : as_pattern([0, 1]) Got int: i=1 In : as_pattern([(1,)]) Got tuple: tu=(1,) In : as_pattern([1, 2, 3]) Got iterable: iterable=[1, 2, 3] In : as_pattern({'a': 1}) Got iterable: iterable={'a': 1} -
[10] Add conditions to the schema
The pattern also supports adding an
ifjudgment (calledguard), which allows the match to be further judged, equivalent to achieving some degree ofANDeffect.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21In : def go(obj): ...: match obj: ...: case ['go', direction] if direction in ['east', 'north']: ...: print('Right way') ...: case direction if direction == 'west': ...: print('Wrong way') ...: case ['go', _] | _: ...: print('Other way') ...: In : go(['go', 'east']) # Matching condition 1 Right way In : go('west') # Matching condition 2 Wrong way In : go('north') # Match default conditions Other way In : go(['go', 'west']) # Match default conditions Other way
8. BPO 12782
New Context Manager syntax
In previous versions, sometimes we wanted to put multiple contexts inside a with on a long line, and a common way to do this was to use a backslash, which would align open. But writing it this way caused code checking tools like pep8 and black to report an error: they all thought that backslashes should not be used to explicitly continue a line.
-
The previous writing style
-
New writing style
Now you can add a bracket to these context managers and it’s perfect, in fact you can use
withto enclose multiple context managers (Context Manager) in brackets.
9. Note
Introduce the new features of the module and analyze their use and usage.
- [1] itertools.pairwise
Its a way to return iterators in the form of elements of an iterable object in order, with two elements placed next to each other in a group. The package more-itertools was available in previous versions and has now been added to the standard library.
- [2] contextlib.aclosing
The standard contextlib.closing decorator has been introduced before, and contextlib.closing is actually just a version of async. As officially described, its role is to implement the following logic.
A good habit with Python is to close the file handle when the operation is done, and the built-in open with with is a good practice. It works by calling ff.__exit__ to automatically close the file handle when the block is finished.
But note that not all open methods natively support with (or provide __exit__ and __enter__ methods), and of course there are other types of operations that need to be made sure to be closed (even if an exception is thrown). In fact, the with syntax is generally supported by standard libraries or well-known projects, but may not be available when writing your own or corporate projects, for example, I define a class like this.
|
|
So, you can wrap it with contextlib.closing. And contextlib.closing is actually a version of asyncio.
-
[3] contextlib.AsyncContextDecorator
contextlibContext.Decoratorwas added inPython 3.2, which is the basis forcontextlib.contextmanager, as expressed in the class name, and which serves to allow the context manager to be used as a decorator.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17# The most basic context manager without parameters class mycontext: def __enter__(self): print('Starting') return self def __exit__(self, *exc): print('Finishing') return False # You can use the with syntax like this In : with mycontext(): ...: print('Inner') ...: Starting Inner FinishingWhat if you want to change the management of this context to a decorator? Just make
mycontextinherit from the base classcontextlib.ContextDecorator, isn’t that convenient? Socontextlib.AsyncContextDecoratoris actually the version ofasyncio.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19In : class mycontext(contextlib.ContextDecorator): # Note the inheritance here ...: def __enter__(self): ...: print('Starting') ...: return self ...: ...: def __exit__(self, *exc): ...: print('Finishing') ...: return False ...: In : @mycontext() ...: def p(): ...: print('Inner') ...: In : p() Starting Inner Finishing
10. Reference
- https://www.escapelife.site/posts/584a5e6a.html
- # What’s New In Python 3.10
- Python 3.10 adds four new features related to the type system
- https://peps.python.org/pep-0604/
- https://peps.python.org/pep-0612/
- https://peps.python.org/pep-0647/
- https://peps.python.org/pep-0613/
- Python 3.10 Match-Case Syntax Explained