Introduction to Python's asyncio.TaskGroup

Python 3.11 introduced asyncio.TaskGroup, bringing structured concurrency to the standard library. With TaskGroup, if any subtask fails with an unhandled exception, the group automatically cancels all other running tasks and raises an ExceptionGroup. This clean fail-fast behavior simplifies managing concurrent asynchronous tasks like LLM API calls, batch web requests, or database operations.

However, developers often encounter a confusing issue: instead of failing fast and raising an exception group immediately, the entire TaskGroup hangs until tasks finish their retry delays or complete unexpectedly. In this article, we'll explore why this happens and how to write safe retry logic that plays nicely with structured concurrency.

Why Does TaskGroup Hang?

To understand why asyncio.TaskGroup hangs, we need to look at how structured concurrency works under the hood. When a task inside a TaskGroup encounters an exception or triggers cancellation, two main rules apply:

  1. TaskGroup waits for all child tasks to terminate: Before leaving the async with TaskGroup() block and raising an ExceptionGroup, the task group must wait for every single managed task to reach a finished state (either completed, failed, or cancelled).
  2. Cancellation is requested, not forced: When a task fails, TaskGroup sends an asyncio.CancelledError to all other tasks. However, Python cannot forcefully terminate an async task mid-execution. A task must yield control (usually at an await point) to receive and process the cancellation request.

Deconstructing the Problematic Code

Consider what happens in a retry loop like the one below when an error occurs:

async def call_llm(prompt: str):
    try:
        await asyncio.sleep(0.1)
        if prompt == "bad-prompt":
            raise ValueError(f"API rejected: {prompt}")
        return f"response to {prompt}"
    except Exception as e:
        print(f"retrying {prompt} after {e}")
        await asyncio.sleep(5)  # Backoff before retry
        return f"response to {prompt}"

Two issues happen depending on where the error occurs:

  • Swallowing the Original Error: The bad-prompt task catches its own ValueError in the except Exception block. Because it catches the error internally, TaskGroup never learns that bad-prompt failed! Instead, bad-prompt sleeps for 5 seconds and returns a dummy string, hiding the failure completely.
  • Swallowing or Delaying Cancellation: If a task does raise an unhandled exception to the TaskGroup, the group cancels sibling tasks by throwing asyncio.CancelledError into them. While asyncio.CancelledError inherits from BaseException (since Python 3.8) to prevent except Exception from swallowing it, any except BaseException or async operations inside teardown/cleanup blocks can delay task termination, causing TaskGroup to wait.

How to Write Safe Retry Workers for TaskGroup

To ensure your concurrent workers allow TaskGroup to cancel cleanly and fail fast, follow these best practices.

1. Catch Only Specific Exceptions

Never catch generic Exception or BaseException in retry handlers. Only catch expected, transient operational errors (e.g., HTTP status errors, connection timeouts).

import asyncio
import httpx

async def call_llm_safe(prompt: str):
    # Catch specific network/API exceptions only
    try:
        # Imagine this calls an actual LLM client
        if prompt == "bad-prompt":
            raise httpx.HTTPStatusError("400 Bad Request", request=None, response=None)
        return f"response to {prompt}"
    except httpx.HTTPStatusError as e:
        # Handle or re-raise if non-retryable
        raise e

2. Always Propagate CancelledError Explicitly

If you must use broad exception handling for unexpected errors, make sure asyncio.CancelledError is explicitly caught and re-raised immediately:

async def worker(prompt: str):
    try:
        return await perform_request(prompt)
    except asyncio.CancelledError:
        # Always allow cancellation to propagate immediately!
        raise
    except Exception as e:
        # Handle transient application errors here
        print(f"Error processing {prompt}: {e}")
        raise

3. Use an Async-Aware Retry Library Like Tenacity

Instead of manually writing try/except retry loops with asyncio.sleep, use a robust library like tenacity, which handles asyncio.CancelledError correctly by default:

from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type

@retry(
    stop=stop_after_attempt(3),
    wait=wait_exponential(multiplier=1, min=1, max=10),
    retry=retry_if_exception_type(httpx.RequestError)  # Don't retry on CancelledError or unexpected errors
)
async def call_llm_tenacity(prompt: str):
    # Perform API call here
    pass

Summary

When asyncio.TaskGroup appears to hang, it is usually waiting for child tasks that have caught cancellation, caught their own internal exceptions, or engaged in lengthy await delays inside exception handlers. By catching specific exception types, allowing asyncio.CancelledError to propagate without delay, and using robust retry policies, you can build responsive, resilient asynchronous applications in Python.