Changelog¶

gh-139700: Check consistency of the zip64 end of central directory record. Support records with “zip64 extensible data” if there are no bytes prepended to the ZIP file.

gh-139400: xml.parsers.expat: Make sure that parent Expat parsers are only garbage-collected once they are no longer referenced by subparsers created by ExternalEntityParserCreate(). Patch by Sebastian Pipping.

gh-135661: Fix parsing start and end tags in html.parser.HTMLParser according to the HTML5 standard.

• Whitespaces no longer accepted between </ and the tag name. E.g. </ script> does not end the script section.

• Vertical tabulation (\v) and non-ASCII whitespaces no longer recognized as whitespaces. The only whitespaces are \t\n\r\f and space.

• Null character (U+0000) no longer ends the tag name.

• Attributes and slashes after the tag name in end tags are now ignored, instead of terminating after the first > in quoted attribute value. E.g. </script/foo=">"/>.

• Multiple slashes and whitespaces between the last attribute and closing > are now ignored in both start and end tags. E.g. <a foo=bar/ //>.

• Multiple = between attribute name and value are no longer collapsed. E.g. <a foo==bar> produces attribute “foo” with value “=bar”.

gh-135661: Fix CDATA section parsing in html.parser.HTMLParser according to the HTML5 standard: ] ]> and ]] > no longer end the CDATA section. Add private method _set_support_cdata() which can be used to specify how to parse <[CDATA[ — as a CDATA section in foreign content (SVG or MathML) or as a bogus comment in the HTML namespace.

gh-102555: Fix comment parsing in html.parser.HTMLParser according to the HTML5 standard. --!> now ends the comment. -- > no longer ends the comment. Support abnormally ended empty comments <--> and <--->.

gh-135462: Fix quadratic complexity in processing specially crafted input in html.parser.HTMLParser. End-of-file errors are now handled according to the HTML5 specs – comments and declarations are automatically closed, tags are ignored.

gh-118350: Fix support of escapable raw text mode (elements “textarea” and “title”) in html.parser.HTMLParser.

gh-86155: html.parser.HTMLParser.close() no longer loses data when the <script> tag is not closed. Patch by Waylan Limberg.

gh-139312: Upgrade bundled libexpat to 2.7.3

gh-138998: Update bundled libexpat to 2.7.2

gh-130577: tarfile now validates archives to ensure member offsets are non-negative. (Contributed by Alexander Enrique Urieles Nieto in gh-130577.)

gh-135374: Update the bundled copy of setuptools to 79.0.1.

gh-128840: Fix parsing long IPv6 addresses with embedded IPv4 address.

gh-134062: ipaddress: fix collisions in __hash__() for IPv4Network and IPv6Network objects.

gh-123409: Fix ipaddress.IPv6Address.reverse_pointer output according to RFC 3596, §2.5. Patch by Bénédikt Tran.

bpo-43633: Improve the textual representation of IPv4-mapped IPv6 addresses (RFC 4291 Sections 2.2, 2.5.5.2) in ipaddress. Patch by Oleksandr Pavliuk.

gh-131809: Update bundled libexpat to 2.7.1

gh-131261: Upgrade to libexpat 2.7.0

gh-105704: When using urllib.parse.urlsplit() and urllib.parse.urlparse() host parsing would not reject domain names containing square brackets ([ and ]). Square brackets are only valid for IPv6 and IPvFuture hosts according to RFC 3986 Section 3.2.2.

gh-121284: Fix bug in the folding of rfc2047 encoded-words when flattening an email message using a modern email policy. Previously when an encoded-word was too long for a line, it would be decoded, split across lines, and re-encoded. But commas and other special characters in the original text could be left unencoded and unquoted. This could theoretically be used to spoof header lines using a carefully constructed encoded-word if the resulting rendered email was transmitted or re-parsed.

gh-80222: Fix bug in the folding of quoted strings when flattening an email message using a modern email policy. Previously when a quoted string was folded so that it spanned more than one line, the surrounding quotes and internal escapes would be omitted. This could theoretically be used to spoof header lines using a carefully constructed quoted string if the resulting rendered email was transmitted or re-parsed.

gh-119511: Fix a potential denial of service in the imaplib module. When connecting to a malicious server, it could cause an arbitrary amount of memory to be allocated. On many systems this is harmless as unused virtual memory is only a mapping, but if this hit a virtual address size limit it could lead to a MemoryError or other process crash. On unusual systems or builds where all allocated memory is touched and backed by actual ram or storage it could’ve consumed resources doing so until similarly crashing.

gh-127257: In ssl, system call failures that OpenSSL reports using ERR_LIB_SYS are now raised as OSError.

gh-121277: Writers of CPython’s documentation can now use next as the version for the versionchanged, versionadded, deprecated directives.

gh-125041: Re-enable skipped tests for zlib on the s390x architecture: only skip checks of the compressed bytes, which can be different between zlib’s software implementation and the hardware-accelerated implementation.

gh-109396: Fix test_socket.test_hmac_sha1() in FIPS mode. Use a longer key: FIPS mode requires at least of at least 112 bits. The previous key was only 32 bits. Patch by Victor Stinner.

gh-126623: Upgrade libexpat to 2.6.4

gh-122792: Changed IPv4-mapped ipaddress.IPv6Address to consistently use the mapped IPv4 address value for deciding properties. Properties which have their behavior fixed are is_multicast, is_reserved, is_link_local, is_global, and is_unspecified.

gh-124651: Properly quote template strings in venv activation scripts.

gh-103848: Add checks to ensure that [ bracketed ] hosts found by urllib.parse.urlsplit() are of IPv6 or IPvFuture format.

gh-119690: Fixes data type confusion in audit events raised by _winapi.CreateFile and _winapi.CreateNamedPipe.

gh-116773: Fix instances of <_overlapped.Overlapped object at 0xXXX> still has pending operation at deallocation, the process may crash.

gh-112769: The tests now correctly compare zlib version when zlib.ZLIB_RUNTIME_VERSION contains non-integer suffixes. For example zlib-ng defines the version as 1.3.0.zlib-ng.

gh-117187: Fix XML tests for vanilla Expat <2.6.0.

gh-100454: Fix SSL tests CI for OpenSSL 3.1+

gh-112275: A deadlock involving pystate.c’s HEAD_LOCK in posixmodule.c at fork is now fixed. Patch by ChuBoning based on previous Python 3.12 fix by Victor Stinner.

gh-115398: Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:

• xml.etree.ElementTree.XMLParser.flush()

• xml.etree.ElementTree.XMLPullParser.flush()

• xml.parsers.expat.xmlparser.GetReparseDeferralEnabled()

• xml.parsers.expat.xmlparser.SetReparseDeferralEnabled()

• xml.sax.expatreader.ExpatParser.flush()

gh-115399: Update bundled libexpat to 2.6.0

gh-114572: ssl.SSLContext.cert_store_stats() and ssl.SSLContext.get_ca_certs() now correctly lock access to the certificate store, when the ssl.SSLContext is shared across multiple threads.

gh-113659: Skip .pth files with names starting with a dot or hidden file attribute.

gh-102388: Fix a bug where iso2022_jp_3 and iso2022_jp_2004 codecs read out of bounds

gh-115197: urllib.request no longer resolves the hostname before checking it against the system’s proxy bypass list on macOS and Windows.

gh-115133: Fix tests for XMLPullParser with Expat 2.6.0.

gh-81194: Fix a crash in socket.if_indextoname() with specific value (UINT_MAX). Fix an integer overflow in socket.if_indextoname() on 64-bit non-Windows platforms.

gh-109858: Protect zipfile from “quoted-overlap” zipbomb. It now raises BadZipFile when try to read an entry that overlaps with other entry or central directory.

gh-91133: Fix a bug in tempfile.TemporaryDirectory cleanup, which now no longer dereferences symlinks when working around file system permission errors.

gh-115399: Document CVE-2023-52425 of Expat <2.6.0 under “XML vulnerabilities”.

gh-111239: Update Windows builds to use zlib v1.3.1.

gh-109991: Windows builds now use OpenSSL 1.1.1w. Note that OpenSSL 1.1 has reached its end of life and no future fixes will be made, and this version of Python is no longer receiving maintenance fixes and will not be updated to OpenSSL 3.0.

gh-109991: Update GitHub CI workflows to use OpenSSL 3.0.11 and multissltests to use 1.1.1w, 3.0.11, and 3.1.3.

gh-108310: Fixed an issue where instances of ssl.SSLSocket were vulnerable to a bypass of the TLS handshake and included protections (like certificate verification) and treating sent unencrypted data as if it were post-handshake TLS encrypted data. Security issue reported as CVE-2023-40217 by Aapo Oksman. Patch by Gregory P. Smith.

gh-107845: tarfile.data_filter() now takes the location of symlinks into account when determining their target, so it will no longer reject some valid tarballs with LinkOutsideDestinationError.

gh-107565: Update multissltests and GitHub CI workflows to use OpenSSL 1.1.1v, 3.0.10, and 3.1.2.

gh-99612: Fix PyUnicode_DecodeUTF8Stateful() for ASCII-only data: *consumed was not set.

gh-103142: The version of OpenSSL used in our binary builds has been upgraded to 1.1.1u to address several CVEs.

gh-99889: Fixed a security in flaw in uu.decode() that could allow for directory traversal based on the input if no out_file was specified.

gh-104049: Do not expose the local on-disk location in directory indexes produced by http.client.SimpleHTTPRequestHandler.

gh-102153: urllib.parse.urlsplit() now strips leading C0 control and space characters following the specification for URLs defined by WHATWG in response to CVE-2023-24329. Patch by Illia Volochii.

gh-103935: Use io.open_code() for files to be executed instead of raw open()

gh-102953: The extraction methods in tarfile, and shutil.unpack_archive(), have a new a filter argument that allows limiting tar features than may be surprising or dangerous, such as creating files outside the destination directory. See Extraction filters for details.

gh-89412: Add missing documentation for the end_lineno and end_offset attributes of the traceback.TracebackException class.

gh-103262: Fixes Windows installer build to work with latest compilers.

gh-101727: Updated the OpenSSL version used in Windows and macOS binary release builds to 1.1.1t to address CVE-2023-0286, CVE-2022-4303, and CVE-2022-4303 per the OpenSSL 2023-02-07 security advisory.

gh-101283: subprocess.Popen now uses a safer approach to find cmd.exe when launching with shell=True. Patch by Eryk Sun, based on a patch by Oleg Iarygin.

gh-102416: Do not memoize incorrectly automatically generated loop rules in the parser. Patch by Pablo Galindo.

gh-102356: Fix a bug that caused a crash when deallocating deeply nested filter objects. Patch by Marta Gómez Macías.

gh-102397: Fix segfault from race condition in signal handling during garbage collection. Patch by Kumar Aditya.

gh-102126: Fix deadlock at shutdown when clearing thread states if any finalizer tries to acquire the runtime head lock. Patch by Kumar Aditya.

gh-102027: Fix SSE2 and SSE3 detection in _blake2 internal module. Patch by Max Bachmann.

gh-101967: Fix possible segfault in positional_only_passed_as_keyword function, when new list created.

gh-101765: Fix SystemError / segmentation fault in iter __reduce__ when internal access of builtins.__dict__ keys mutates the iter object.

gh-102947: Improve traceback when dataclasses.fields() is called on a non-dataclass. Patch by Alex Waygood

gh-101979: Fix a bug where parentheses in the metavar argument to argparse.ArgumentParser.add_argument() were dropped. Patch by Yeojin Kim.

gh-102179: Fix os.dup2() error message for negative fds.

gh-101961: For the binary mode, fileinput.hookcompressed() doesn’t set the encoding value even if the value is None. Patch by Gihwan Kim.

gh-101936: The default value of fp becomes io.BytesIO if HTTPError is initialized without a designated fp parameter. Patch by Long Vo.

gh-101566: In zipfile, apply fix for extractall on the underlying zipfile after being wrapped in Path.

gh-101997: Upgrade pip wheel bundled with ensurepip (pip 23.0.1)

gh-101892: Callable iterators no longer raise SystemError when the callable object exhausts the iterator but forgets to either return a sentinel value or raise StopIteration.

gh-97786: Fix potential undefined behaviour in corner cases of floating-point-to-time conversions.

gh-101517: Fixed bug where bdb looks up the source line with linecache with a lineno=None, which causes it to fail with an unhandled exception.

gh-101673: Fix a pdb bug where ll clears the changes to local variables.

gh-96931: Fix incorrect results from ssl.SSLSocket.shared_ciphers()

gh-88233: Correctly preserve “extra” fields in zipfile regardless of their ordering relative to a zip64 “extra.”

gh-95495: When built against OpenSSL 3.0, the ssl module had a bug where it reported unauthenticated EOFs (i.e. without close_notify) as a clean TLS-level EOF. It now raises SSLEOFError, matching the behavior in previous versions of OpenSSL. The options attribute on SSLContext also no longer includes OP_IGNORE_UNEXPECTED_EOF by default. This option may be set to specify the previous OpenSSL 3.0 behavior.

gh-94440: Fix a concurrent.futures.process bug where ProcessPoolExecutor shutdown could hang after a future has been quickly submitted and canceled.

gh-103112: Add docstring to http.client.HTTPResponse.read() to fix pydoc output.

gh-85417: Update cmath documentation to clarify behaviour on branch cuts.

gh-97725: Fix asyncio.Task.print_stack() description for file=None. Patch by Oleg Iarygin.

gh-102980: Improve test coverage on pdb.

gh-102537: Adjust the error handling strategy in test_zoneinfo.TzPathTest.python_tzpath_context. Patch by Paul Ganssle.

gh-101377: Improved test_locale_calendar_formatweekday of calendar.

gh-102711: Fix -Wstrict-prototypes compiler warnings.

gh-101759: Update Windows installer to SQLite 3.40.1.

gh-101614: Correctly handle extensions built against debug binaries that reference python3_d.dll.

gh-103207: Add instructions to the macOS installer welcome display on how to workaround the macOS 13 Ventura “The installer encountered an error” failure.

gh-101759: Update macOS installer to SQLite 3.40.1.

gh-87235: On macOS python3 /dev/fd/9 9</path/to/script.py failed for any script longer than a couple of bytes.

gh-101400: Fix wrong lineno in exception message on continue or break which are not in a loop. Patch by Dong-hee Na.

gh-101372: Fix is_normalized() to properly handle the UCD 3.2.0 cases. Patch by Dong-hee Na.

gh-101046: Fix a possible memory leak in the parser when raising MemoryError. Patch by Pablo Galindo

gh-100942: Fixed segfault in property.getter/setter/deleter that occurred when a property subclass overrode the __new__ method to return a non-property instance.

gh-100892: Fix race while iterating over thread states in clearing threading.local. Patch by Kumar Aditya.

gh-100776: Fix misleading default value in input()’s __text_signature__.

gh-100374: Fix incorrect result and delay in socket.getfqdn(). Patch by Dominic Socular.

gh-100050: Honor existing errors obtained when searching for mismatching parentheses in the tokenizer. Patch by Pablo Galindo