Security
gh-115398: Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:
xml.etree.ElementTree.XMLParser.flush()
xml.etree.ElementTree.XMLPullParser.flush()
xml.parsers.expat.xmlparser.GetReparseDeferralEnabled()
xml.parsers.expat.xmlparser.SetReparseDeferralEnabled()
xml.sax.expatreader.ExpatParser.flush()
gh-115399: Update bundled libexpat to 2.6.0
gh-115243: Fix possible crashes in collections.deque.index() when the deque is concurrently modified.
gh-114572: ssl.SSLContext.cert_store_stats() and ssl.SSLContext.get_ca_certs() now correctly lock access to the certificate store, when the ssl.SSLContext is shared across multiple threads.
Core and Builtins
gh-109120: Added handle of incorrect star expressions, e.g f(3, *). Patch by Grigoryev Semyon
gh-99108: Updated the hashlib built-in HACL* project C code from upstream that we use for many implementations when they are not present via OpenSSL in a given build. This also avoids the rare potential for a C symbol name one definition rule linking issue.
gh-116735: For INSTRUMENTED_CALL_FUNCTION_EX, set arg0 to sys.monitoring.MISSING instead of None for CALL event.
gh-113964: Starting new threads and process creation through os.fork() are now only prevented once all non-daemon threads exit.
gh-116604: Respect the status of the garbage collector when indirect calls are made via PyErr_CheckSignals() and the evaluation breaker. Patch by Pablo Galindo
gh-116626: Ensure INSTRUMENTED_CALL_FUNCTION_EX always emits CALL
gh-116296: Fix possible refleak in object.__reduce__() internal error handling.
gh-116034: Fix location of the error on a failed assertion.
gh-115823: Properly calculate error ranges in the parser when raising SyntaxError exceptions caused by invalid byte sequences. Patch by Pablo Galindo
gh-112087: For an empty reverse iterator for list will be reduced to reversed(). Patch by Donghee Na.
gh-115154: Fix a bug that was causing the tokenize.untokenize() function to handle unicode named literals incorrectly. Patch by Pablo Galindo
gh-114828: Fix compilation crashes in uncommon code examples using super() inside a comprehension in a class body.
gh-115011: Setters for members with an unsigned integer type now support the same range of valid values for objects that has a __index__() method as for int.
gh-112215: Change the C recursion limits to more closely reflect the underlying platform limits.
gh-96497: Fix incorrect resolution of mangled class variables used in assignment expressions in comprehensions.
Library
gh-117467: Preserve mailbox ownership when rewriting in mailbox.mbox.flush(). Patch by Tony Mountifield.
gh-117310: Fixed an unlikely early & extra Py_DECREF triggered crash in ssl when creating a new _ssl._SSLContext if CPython was built implausibly such that the default cipher list is empty or the SSL library it was linked against reports a failure from its C SSL_CTX_set_cipher_list() API.
gh-117178: Fix regression in lazy loading of self-referential modules, introduced in gh-114781.
gh-117084: Fix zipfile extraction for directory entries with the name containing backslashes on Windows.
gh-117110: Fix a bug that prevents subclasses of typing.Any to be instantiated with arguments. Patch by Chris Fu.
gh-90872: On Windows, subprocess.Popen.wait() no longer calls WaitForSingleObject() with a negative timeout: pass 0 ms if the timeout is negative. Patch by Victor Stinner.
gh-116957: configparser: Don’t leave ConfigParser values in an invalid state (stored as a list instead of a str) after an earlier read raised DuplicateSectionError or DuplicateOptionError.
gh-90095: Ignore empty lines and comments in .pdbrc
gh-116764: Restore support of None and other false values in urllib.parse functions parse_qs() and parse_qsl(). Also, they now raise a TypeError for non-zero integers and non-empty sequences.
gh-116811: In PathFinder.invalidate_caches, delegate to MetadataPathFinder.invalidate_caches.
gh-116600: Fix repr() for global Flag members.
gh-116484: Change automatically generated tkinter.Checkbutton widget names to avoid collisions with automatically generated tkinter.ttk.Checkbutton widget names within the same parent widget.
gh-116401: Fix blocking os.fwalk() and shutil.rmtree() on opening named pipe.
gh-116143: Fix a race in pydoc _start_server, eliminating a window in which _start_server can return a thread that is “serving” but without a docserver set.
gh-116325: typing: raise SyntaxError instead of AttributeError on forward references as empty strings.
gh-90535: Fix support of interval values > 1 in logging.TimedRotatingFileHandler for when='MIDNIGHT' and when='Wx'.
gh-115978: Disable preadv(), readv(), pwritev(), and writev() on WASI.
Under wasmtime for WASI 0.2, these functions don’t pass test_posix (
https://github.com/bytecodealliance/wasmtime/issues/7830).
gh-88352: Fix the computation of the next rollover time in the logging.TimedRotatingFileHandler handler. computeRollover() now always returns a timestamp larger than the specified time and works correctly during the DST change. doRollover() no longer overwrite the already rolled over file, saving from data loss when run at midnight or during repeated time at the DST change.
gh-87115: Set __main__.__spec__ to None when running a script with pdb
gh-76511: Fix UnicodeEncodeError in email.Message.as_string() that results when a message that claims to be in the ascii character set actually has non-ascii characters. Non-ascii characters are now replaced with the U+FFFD replacement character, like in the replace error handler.
gh-116040: [Enum] fix by-value calls when second value is falsey; e.g. Cardinal(1, 0)
gh-75988: Fixed unittest.mock.create_autospec() to pass the call through to the wrapped object to return the real result.
gh-115881: Fix issue where ast.parse() would incorrectly flag conditional context managers (such as with (x() if y else z()): ...) as invalid syntax if feature_version=(3,
was passed. This reverts changes to the grammar made as part of gh-94949.
gh-115886: Fix silent truncation of the name with an embedded null character in multiprocessing.shared_memory.SharedMemory.
gh-115809: Improve algorithm for computing which rolled-over log files to delete in logging.TimedRotatingFileHandler. It is now reliable for handlers without namer and with arbitrary deterministic namer that leaves the datetime part in the file name unmodified.
gh-74668: urllib.parse functions parse_qs() and parse_qsl() now support bytes arguments containing raw and percent-encoded non-ASCII data.
gh-67044: csv.writer() now always quotes or escapes '\r' and '\n', regardless of lineterminator value.
gh-115712: csv.writer() now quotes empty fields if delimiter is a space and skipinitialspace is true and raises exception if quoting is not possible.
gh-112364: Fixed ast.unparse() to handle format_spec with ", ' or \\. Patched by Frank Hoffmann.
gh-111358: Fix a bug in asyncio.BaseEventLoop.shutdown_default_executor() to ensure the timeout passed to the coroutine behaves as expected.
gh-115618: Fix improper decreasing the reference count for None argument in property methods getter(), setter() and deleter().
gh-115570: A DeprecationWarning is no longer omitted on access to the __doc__ attributes of the deprecated typing.io and typing.re pseudo-modules.
gh-112006: Fix inspect.unwrap() for types with the __wrapper__ data descriptor.
gh-101293: Support callables with the __call__() method and types with __new__() and __init__() methods set to class methods, static methods, bound methods, partial functions, and other types of methods and descriptors in inspect.Signature.from_callable().
gh-115392: Fix a bug in doctest where incorrect line numbers would be reported for decorated functions.
gh-114563: Fix several format() bugs when using the C implementation of Decimal: * memory leak in some rare cases when using the z format option (coerce negative 0) * incorrect output when applying the z format option to type F (fixed-point with capital NAN / INF) * incorrect output when applying the # format option (alternate form)
gh-115197: urllib.request no longer resolves the hostname before checking it against the system’s proxy bypass list on macOS and Windows.
gh-115165: Most exceptions are now ignored when attempting to set the __orig_class__ attribute on objects returned when calling typing generic aliases (including generic aliases created using typing.Annotated). Previously only AttributeError was ignored. Patch by Dave Shawley.
gh-115133: Fix tests for XMLPullParser with Expat 2.6.0.
gh-115059: io.BufferedRandom.read1() now flushes the underlying write buffer.
gh-79382: Trailing ** no longer allows to match files and non-existing paths in recursive glob().
gh-114071: Support tuple subclasses using auto() for enum member value.
gh-114763: Protect modules loaded with importlib.util.LazyLoader from race conditions when multiple threads try to access attributes before the loading is complete.
gh-97959: Fix rendering class methods, bound methods, method and function aliases in pydoc. Class methods no longer have “method of builtins.type instance” note. Corresponding notes are now added for class and unbound methods. Method and function aliases now have references to the module or the class where the origin was defined if it differs from the current. Bound methods are now listed in the static methods section. Methods of builtin classes are now supported as well as methods of Python classes.
gh-112281: Allow creating union of types for typing.Annotated with unhashable metadata.
gh-111775: Fix importlib.resources.simple.ResourceHandle.open() for text mode, added missed stream argument.
gh-90095: Make .pdbrc and -c work with any valid pdb commands.
gh-107155: Fix incorrect output of help(x) where x is a lambda function, which has an __annotations__ dictionary attribute with a "return" key.
gh-105866: Fixed _get_slots bug which caused error when defining dataclasses with slots and a weakref_slot.
gh-60346: Fix ArgumentParser inconsistent with parse_known_args.
gh-100985: Update HTTPSConnection to consistently wrap IPv6 Addresses when using a proxy.
gh-100884: email: fix misfolding of comma in address-lists over multiple lines in combination with unicode encoding.
gh-95782: Fix io.BufferedReader.tell(), io.BufferedReader.seek(), _pyio.BufferedReader.tell(), io.BufferedRandom.tell(), io.BufferedRandom.seek() and _pyio.BufferedRandom.tell() being able to return negative offsets.
gh-96310: Fix a traceback in argparse when all options in a mutually exclusive group are suppressed.
gh-93205: Fixed a bug in logging.handlers.TimedRotatingFileHandler where multiple rotating handler instances pointing to files with the same name but different extensions would conflict and not delete the correct files.
bpo-44865: Add missing call to localization function in argparse.
bpo-43952: Fix multiprocessing.connection.Listener.accept() to accept empty bytes as authkey. Not accepting empty bytes as key causes it to hang indefinitely.
bpo-42125: linecache: get module name from __spec__ if available. This allows getting source code for the __main__ module when a custom loader is used.
gh-66543: Make mimetypes.guess_type() properly parsing of URLs with only a host name, URLs containing fragment or query, and filenames with only a UNC sharepoint on Windows. Based on patch by Dong-hee Na.
bpo-33775: Add ‘default’ and ‘version’ help text for localization in argparse.
Documentation
gh-115399: Document CVE-2023-52425 of Expat <2.6.0 under “XML vulnerabilities”.
gh-115233: Fix an example for LoggerAdapter in the Logging Cookbook.
Tests
gh-83434: Disable JUnit XML output (--junit-xml=FILE command line option) in regrtest when hunting for reference leaks (-R option). Patch by Victor Stinner.
gh-117187: Fix XML tests for vanilla Expat <2.6.0.
gh-116333: Tests of TLS related things (error codes, etc) were updated to be more lenient about specific error message strings and behaviors as seen in the BoringSSL and AWS-LC forks of OpenSSL.
gh-115979: Update test_importlib so that it passes under WASI SDK 21.
gh-112536: Add –tsan to test.regrtest for running TSAN tests in reasonable execution times. Patch by Donghee Na.
gh-116307: Added import helper isolated_modules as CleanImport does not remove modules imported during the context. Use it in importlib.resources tests to avoid leaving mod around to impede importlib.metadata tests.
gh-115720: Leak tests (-R, --huntrleaks) now show a summary of the number of leaks found in each iteration.
gh-115122: Add --bisect option to regrtest test runner: run failed tests with test.bisect_cmd to identify failing tests. Patch by Victor Stinner.
gh-115596: Fix ProgramPriorityTests in test_os permanently changing the process priority.
Build
gh-116313: Get WASI builds to work under wasmtime 18 w/ WASI 0.2/preview2 primitives.
gh-116117: Backport libb2’s PR #42 to fix compiling CPython on 32-bit Windows with clang-cl.
gh-115167: Avoid vendoring vcruntime140_threads.dll when building with Visual Studio 2022 version 17.8.
gh-112536: Add support for thread sanitizer (TSAN)
Windows
gh-117267: Ensure DirEntry.stat().st_ctime behaves consistently with os.stat() during the deprecation period of st_ctime by containing the same value as st_birthtime. After the deprecation period, st_ctime will be the metadata change time (or unavailable through DirEntry), and only st_birthtime will contain the creation time.
gh-116773: Fix instances of <_overlapped.Overlapped object at 0xXXX> still has pending operation at deallocation, the process may crash.
gh-91227: Fix the asyncio ProactorEventLoop implementation so that sending a datagram to an address that is not listening does not prevent receiving any more datagrams.
gh-115554: The installer now has more strict rules about updating the Python Launcher for Windows. In general, most users only have a single launcher installed and will see no difference. When multiple launchers have been installed, the option to install the launcher is disabled until all but one have been removed. Downgrading the launcher (which was never allowed) is now more obviously blocked.
gh-115543: Python Launcher for Windows can now detect Python 3.13 when installed from the Microsoft Store, and will install Python 3.12 by default when PYLAUNCHER_ALLOW_INSTALL is set.
gh-115049: Fixes py.exe launcher failing when run as users without user profiles.
gh-115009: Update Windows installer to use SQLite 3.45.1.
IDLE
gh-88516: On macOS show a proxy icon in the title bar of editor windows to match platform behaviour.
Tools/Demos
gh-113516: Don’t set LDSHARED when building for WASI.
C API
gh-117021: Fix integer overflow in PyLong_AsPid() on non-Windows 64-bit platforms.
gh-116869: Make the C API compatible with -Werror=declaration-after-statement compiler flag again. Patch by Victor Stinner.
