Merge remote-tracking branch 'upstream' into delete-global

Author: brijkapadia

Doc/c-api/threads.rst

@@ -242,6 +242,61 @@ a thread state that was previously attached for the current thread.
 .. seealso::
    :pep:`788`
 
+.. _c-api-reuse-thread-state:
+
+Reusing a thread state across repeated calls
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Creating and destroying a :c:type:`PyThreadState` is not free, and is more
+expensive on a :term:`free-threaded build`.  A foreign thread that calls into
+the interpreter many times -- for example, a worker thread in a native thread
+pool -- should avoid creating a fresh thread state on every entry and
+destroying it on every exit.  Instead, set up one thread state when the thread
+starts (or lazily on its first call into Python), attach and detach it around
+each call, and tear it down once when the thread exits.
+
+Manage the thread state explicitly with :c:func:`PyThreadState_New`, attaching
+and detaching it with :c:func:`PyEval_RestoreThread` and
+:c:func:`PyEval_SaveThread`.  This happens in three distinct phases, at
+different points in the thread's life.
+
+When the thread starts, create one thread state for it.  ``interp`` is the
+target interpreter, captured by the code that created this thread while it held
+an attached thread state (for example via :c:func:`PyInterpreterState_Get`)::
+
+   PyThreadState *tstate = PyThreadState_New(interp);
+
+Then, on each call into Python -- which may happen many times over the thread's
+life -- attach the thread state, make the Python C API calls that require it,
+and detach again so the thread does not hold the GIL while off doing non-Python
+work::
+
+   PyEval_RestoreThread(tstate);
+   result = CallSomeFunction();  /* your Python C API calls go here */
+   PyEval_SaveThread();
+
+When the thread is finished calling into Python, destroy the thread state once::
+
+   PyEval_RestoreThread(tstate);
+   PyThreadState_Clear(tstate);
+   PyThreadState_DeleteCurrent();
+
+The general-purpose entry points for calling in from a foreign thread --
+:c:func:`PyThreadState_Ensure` and the older :c:func:`PyGILState_Ensure` -- do
+*not* guarantee a persistent thread state: their thread-state lifetime is
+deliberately implementation-defined, so a matched acquire/release pair may
+create and destroy a thread state each time.  Use :c:func:`PyThreadState_New`,
+as shown here, whenever you specifically want to reuse one thread state across
+calls.
+
+The code that created the foreign thread must arrange for the shutdown sequence
+to run before the thread exits, and before :c:func:`Py_FinalizeEx` is called.
+If interpreter finalization begins first, the shutdown
+:c:func:`PyEval_RestoreThread` call will hang the thread rather than return (see
+:ref:`cautions-regarding-runtime-finalization`).  If the thread exits without
+running the shutdown sequence, the thread state is leaked for the remainder of
+the process.
+
 .. _c-api-attach-detach:
 
 Attaching/detaching thread states

Doc/c-api/type.rst

@@ -639,7 +639,7 @@ but need extra remarks for use as slots:
    in the following situations:
 
    - The base is not variable-sized (its
-     :c:member:`~PyTypeObject.tp_itemsize`).
+     :c:member:`~PyTypeObject.tp_itemsize` is zero).
    - The requested :c:member:`PyType_Spec.basicsize` is positive,
      suggesting that the memory layout of the base class is known.
    - The requested :c:member:`PyType_Spec.basicsize` is zero,

Doc/deprecations/pending-removal-in-3.16.rst

@@ -33,12 +33,12 @@ Pending removal in Python 3.16
   * :mod:`asyncio` policy system is deprecated and will be removed in Python 3.16.
     In particular, the following classes and functions are deprecated:
 
-    * :class:`asyncio.AbstractEventLoopPolicy`
-    * :class:`asyncio.DefaultEventLoopPolicy`
-    * :class:`asyncio.WindowsSelectorEventLoopPolicy`
-    * :class:`asyncio.WindowsProactorEventLoopPolicy`
-    * :func:`asyncio.get_event_loop_policy`
-    * :func:`asyncio.set_event_loop_policy`
+    * :class:`!asyncio.AbstractEventLoopPolicy`
+    * :class:`!asyncio.DefaultEventLoopPolicy`
+    * :class:`!asyncio.WindowsSelectorEventLoopPolicy`
+    * :class:`!asyncio.WindowsProactorEventLoopPolicy`
+    * :func:`!asyncio.get_event_loop_policy`
+    * :func:`!asyncio.set_event_loop_policy`
 
     Users should use :func:`asyncio.run` or :class:`asyncio.Runner` with
     *loop_factory* to use the desired event loop implementation.

Doc/deprecations/pending-removal-in-3.21.rst

@@ -17,3 +17,9 @@ Pending removal in Python 3.21
     are not generated by the parser or accepted by the code generator.
   * The ``dims`` property of ``ast.Tuple`` will be removed in Python 3.21. Use
     the ``ast.Tuple.elts`` property instead.
+
+* :mod:`struct`:
+
+  * Soft-deprecated since Python 3.15, using ``'F'`` and ``'D'`` type codes are now
+    deprecated.  These codes will be removed in Python 3.21.  Use instead
+    two-letter forms ``'Zf'`` and ``'Zd'``.

Doc/deprecations/soft-deprecations.rst

@@ -19,8 +19,3 @@ There are no plans to remove :term:`soft deprecated` APIs.
 
   (Contributed by Gregory P. Smith in :gh:`86519` and
   Hugo van Kemenade in :gh:`148100`.)
-
-* Using ``'F'`` and ``'D'`` format type codes of the :mod:`struct` module
-  now are :term:`soft deprecated` in favor of two-letter forms ``'Zf'``
-  and ``'Zd'``.
-  (Contributed by Sergey B Kirpichev in :gh:`121249`.)

Doc/faq/design.rst

@@ -263,6 +263,8 @@ In general, structured switch statements execute one block of code
 when an expression has a particular value or set of values.
 Since Python 3.10 one can easily match literal values, or constants
 within a namespace, with a ``match ... case`` statement.
+See :ref:`the specification <match>` and :ref:`the tutorial <tut-match>`
+for more information about :keyword:`match` statements.
 An older alternative is a sequence of ``if... elif... elif... else``.
 
 For cases where you need to choose from a very large number of possibilities,

Doc/howto/curses.rst

@@ -52,8 +52,8 @@ code, all the functions described here will probably be available.  The older
 versions of curses carried by some proprietary Unixes may not support
 everything, though.
 
-The Windows version of Python doesn't include the :mod:`curses`
-module.  A ported version called :pypi:`UniCurses` is available.
+The Windows version of Python doesn't include the :mod:`curses` module.
+The third-party :pypi:`windows-curses` package provides the same interface on Windows.
 
 
 The Python curses module

Doc/howto/mro.rst

@@ -10,7 +10,7 @@ The Python 2.3 Method Resolution Order
    The Method Resolution Order discussed here was *introduced* in Python 2.3,
    but it is still used in later versions -- including Python 3.
 
-By `Michele Simionato <https://www.phyast.pitt.edu/~micheles/>`__.
+By `Michele Simionato <https://github.com/micheles>`__.
 
 :Abstract:
 

Doc/library/asyncio-eventloop.rst

@@ -48,10 +48,10 @@ an event loop:
    running event loop.
 
    If there is no running event loop set, the function will return
-   the result of the ``get_event_loop_policy().get_event_loop()`` call.
+   the loop set by :func:`set_event_loop`, or raise a :exc:`RuntimeError`
+   if no loop has been set.
 
-   Because this function has rather complex behavior (especially
-   when custom event loop policies are in use), using the
+   Because this function has rather complex behavior, using the
    :func:`get_running_loop` function is preferred to :func:`get_event_loop`
    in coroutines and callbacks.
 
@@ -62,13 +62,6 @@ an event loop:
    .. versionchanged:: 3.14
       Raises a :exc:`RuntimeError` if there is no current event loop.
 
-   .. note::
-
-      The :mod:`!asyncio` policy system is deprecated and will be removed
-      in Python 3.16; from there on, this function will return the current
-      running event loop if present else it will return the
-      loop set by :func:`set_event_loop`.
-
 .. function:: set_event_loop(loop)
 
    Set *loop* as the current event loop for the current OS thread.
@@ -77,10 +70,6 @@ an event loop:
 
    Create and return a new event loop object.
 
-Note that the behaviour of :func:`get_event_loop`, :func:`set_event_loop`,
-and :func:`new_event_loop` functions can be altered by
-:ref:`setting a custom event loop policy <asyncio-policies>`.
-
 
 .. rubric:: Contents
 

Doc/library/asyncio-llapi-index.rst

@@ -19,10 +19,10 @@ Obtaining the Event Loop
       - The **preferred** function to get the running event loop.
 
     * - :func:`asyncio.get_event_loop`
-      - Get an event loop instance (running or current via the current policy).
+      - Get the running event loop or the event loop set for the current thread.
 
     * - :func:`asyncio.set_event_loop`
-      - Set the event loop as current via the current policy.
+      - Set the event loop for the current thread.
 
     * - :func:`asyncio.new_event_loop`
       - Create a new event loop.
@@ -497,27 +497,3 @@ Protocol classes can implement the following **callback methods**:
       - Called when the child process has exited. It can be called before
         :meth:`~SubprocessProtocol.pipe_data_received` and
         :meth:`~SubprocessProtocol.pipe_connection_lost` methods.
-
-
-Event Loop Policies
-===================
-
-Policies is a low-level mechanism to alter the behavior of
-functions like :func:`asyncio.get_event_loop`.  See also
-the main :ref:`policies section <asyncio-policies>` for more
-details.
-
-
-.. rubric:: Accessing Policies
-.. list-table::
-    :widths: 50 50
-    :class: full-width-table
-
-    * - :meth:`asyncio.get_event_loop_policy`
-      - Return the current process-wide policy.
-
-    * - :meth:`asyncio.set_event_loop_policy`
-      - Set a new process-wide policy.
-
-    * - :class:`AbstractEventLoopPolicy`
-      - Base class for policy objects.