Reading code: CPython's asyncio

I’m going to write more async code in Python soon. Since it’s always nice to know how things work under the hood, I took a closer look at CPython’s asyncio lib. This is not a coherent text, but rather a table of contents with short summaries and pointers into the code¹. I simplify by looking only at a subset of the API and skipping a lot of parameters and corner cases to make this more readable. Maybe it can serve as a starting point if you’re interested in reading the code.

• runners.py: high-level API that takes care of running a coroutine
  • def run: convenience method for using the Runner
  • class Runner: context manager that creates event loop and takes care of closing it
    • def run: calls run_until_complete with given coroutine on event loop
• base_events.py: base implementation of an event loop
  • class BaseEventLoop: base implementation of AbstractEventLoop
    • self._ready: a simple collections.deque() queue
    • self._scheduled: heap queue (priority queue) for timer handles
    • def _call_soon: appends to self._ready queue, called by Task to run __step as callback
    • def call_at: pushes to heap queue
    • def create_task: creates a new Task object, which (in its __init__ method) puts the first computation step of its coroutine on the event loop’s ready queue
    • def run_forever: run loop until stop() is called
      • def _run_once: runs one iteration of event loop, appends ready scheduled tasks to ready queue, pops handle from queue and calls its _run() method
    • def stop: sets self._stopping = True, which is the break condition in run_forever()
• tasks.py
  • class Task: wrapper around a coroutine, suspends execution of coroutine if waiting on Future, inherits from Future
    • def __init__: calls def call_soon on event loop with self.__step() as callback function
    • def __step: runs one step of the coroutine, used as a callback that is wrapped in a Handle and added to the _ready queue
      • def __step_run_and_handle_result: uses send and throw methods of the coroutine (PEP 343), passes the next __step to the event loop’s call_soon method
    • def set_result: Task inherits from Future, but results (and exceptions) are set by the __step method on the super() object and this method raises an exception, so that the user does not try to set the result manually
  • def sleep: coroutine that completes after given time
    • def __sleep0: bare yield: when awaited, event loop does one more iteration until it jumps out of __sleep0
  • def gather: coroutines/tasks are wrapped in a _GatheringFuture, which allows to cancel all children
    • def _done_callback: error handling, collecting results
• futures.py: defines Future class, not thread-safe
  • def __await__: tells Task to wait for completion if not done or returns def result()
  • def cancel: cancel future and schedule callbacks previously registered by add_done_callback()
  • def add_done_callback: appends a done callback
  • def set_result: set result of Future
• events.py: coroutine wrapper classes and some interfaces
  • class Handle: wrapper around a callback function
  • class TimerHandle: inherits from Handle, can be sorted by scheduled time