Master async programming in Python. Learn event loops, coroutines, tasks, aiohttp for concurrent HTTP requests, and when async actually makes sense vs. threads or multiprocessing.
Async Python confuses people because it looks like threading but works nothing like it. There is no parallelism, no second thread, no preemption — just a single thread that suspends a task whenever it would wait and runs another in the meantime. Once that mental model clicks, asyncio stops being mysterious. This tutorial builds that model from the ground up, then covers the patterns, pitfalls, and real-world uses that make async genuinely faster — and the cases where it does not help at all.
Most web work is not CPU-bound; it is spent waiting — for a database, an external API, a disk, a network socket. A synchronous program waiting on a slow API call sits idle, doing nothing, until the response arrives. If you are making fifty such calls one after another, you wait for the sum of all fifty. Async lets that single thread start all fifty, and while each is waiting, work on the others, so the total time is roughly the slowest single call rather than the sum. This is the entire value proposition: async makes I/O-bound work concurrent without threads. For CPU-bound work — heavy computation — it does nothing, because there is no waiting to overlap.
At the heart of asyncio is the event loop: a single-threaded scheduler that runs tasks until each one hits a point where it must wait, at which moment the task yields control back to the loop, which runs another ready task. When a task's awaited operation completes, the loop resumes it. There is only ever one thing executing at a time — concurrency comes from interleaving many tasks during their wait periods, not from running them simultaneously. Internalizing that "one thread, cooperative switching at await points" is the single insight that makes all of async make sense; everything else is mechanics built on top of it.
A function defined with async def is a coroutine — calling it does not run it, it returns a coroutine object that runs only when awaited or scheduled on the loop. The await keyword is where the magic happens: it suspends the current coroutine until the awaited thing completes, handing control back to the event loop so other tasks can run in the meantime.
import asyncio, aiohttp
async def fetch(session, url):
async with session.get(url) as resp:
return await resp.text()
Every await is a potential switch point. Between awaits, your code runs uninterrupted; at an await, the loop may run something else. That cooperative model is why async is predictable in a way threads are not.
Awaiting coroutines one by one is still sequential — you only get concurrency when multiple operations are in flight at once. asyncio.gather schedules many coroutines together and waits for them all, which is where the speedup appears:
async def fetch_all(urls):
async with aiohttp.ClientSession() as session:
return await asyncio.gather(*(fetch(session, u) for u in urls))
Fifty URLs that would take fifty sequential round trips now overlap, completing in about the time of the slowest one. This is the canonical async win: fan out independent I/O operations, then collect the results together. For finer control over many tasks, asyncio.TaskGroup (in modern Python) offers structured concurrency with cleaner error handling.
Since Python 3.11, asyncio.TaskGroup is the preferred way to run a known set of tasks together. Every task created inside the async with block is guaranteed to have finished (successfully, with an error, or cancelled) when the block exits, so no task can outlive the code that started it. If any task raises, the group cancels the remaining ones and re-raises everything as an ExceptionGroup, which you handle with except*:
async def load_dashboard(session, user_id):
try:
async with asyncio.TaskGroup() as tg:
profile = tg.create_task(fetch_json(session, f"/users/{user_id}"))
orders = tg.create_task(fetch_json(session, f"/orders?user={user_id}"))
alerts = tg.create_task(fetch_json(session, f"/alerts?user={user_id}"))
except* aiohttp.ClientResponseError as eg:
for exc in eg.exceptions:
log.warning("upstream failed: %s %s", exc.status, exc.request_info.url)
raise
return {
"profile": profile.result(),
"orders": orders.result(),
"alerts": alerts.result(),
}
Two details matter here. First, results are read from the task objects after the block exits, when they are guaranteed to be done. Second, except* can match several exception types from the same group in separate clauses, and any unmatched exceptions propagate. gather is still perfectly fine for a flat list of homogeneous calls where you want results in input order; TaskGroup earns its place when tasks are heterogeneous or when an orphaned task would be a bug.
Because there is one thread and switching is cooperative, any code that blocks without awaiting freezes everything. A synchronous database call, a time.sleep, a CPU-heavy loop, or a blocking library call holds the single thread and stalls every other task until it returns. This is the number-one async bug, and it is insidious because the program still works — it is just mysteriously not concurrent. The rule is absolute: never call blocking code directly in a coroutine. Use the async equivalent (asyncio.sleep, an async HTTP client, an async database driver), or push the blocking work to a thread pool with run_in_executor or asyncio.to_thread so it does not hold the loop.
Blocking calls rarely announce themselves. The most reliable detector is asyncio's debug mode, enabled with asyncio.run(main(), debug=True), the environment variable PYTHONASYNCIODEBUG=1, or Python's development mode (python -X dev). In debug mode the loop logs a warning for any callback or task step that runs longer than loop.slow_callback_duration (0.1 seconds by default) and names the offending coroutine, which turns "everything is slow" into a specific line of code.
import asyncio, logging, hashlib
logging.basicConfig(level=logging.DEBUG)
def legacy_lookup(customer_id): # synchronous SDK call, ~300 ms
return slow_vendor_sdk.get_customer(customer_id)
async def handler(customer_id):
# Wrong: blocks the loop for the whole call
# data = legacy_lookup(customer_id)
# Right: runs in the default thread pool, loop stays responsive
data = await asyncio.to_thread(legacy_lookup, customer_id)
return data
async def main():
loop = asyncio.get_running_loop()
loop.slow_callback_duration = 0.05 # be stricter than the default
await asyncio.gather(*(handler(i) for i in range(20)))
asyncio.run(main(), debug=True)
asyncio.to_thread is a convenience over loop.run_in_executor(None, ...) that also propagates contextvars, so request IDs and logging context survive the hop into the thread. The default executor is a ThreadPoolExecutor with min(32, os.cpu_count() + 4) workers; if you offload many slow blocking calls at once, they queue behind that limit, so a fan-out of two hundred to_thread calls is not two hundred concurrent calls. When a blocking dependency sits on your hot path, the durable fix is an async-native client; threads are the bridge while you get there.
A frequent misunderstanding is expecting async to speed up computation. It cannot. Async overlaps waiting, and a CPU-bound task does not wait — it occupies the single thread fully while it computes, blocking every other task exactly like any other non-awaiting code. For parallel computation you need multiple cores, which means multiprocessing or a process pool, not asyncio. The clean division: I/O-bound and high-concurrency work is for async; CPU-bound work is for processes. Mixing them up — reaching for async to make a number-crunching job faster — leads to disappointment and confusing code that is no quicker than the synchronous version.
The most common real-world use of async is making many outbound HTTP calls, and a synchronous client used in async code is the classic loop-blocking mistake. Use an async client — aiohttp or httpx in async mode — which awaits at every network operation so the loop stays free. Reuse a single session across requests so connections are pooled rather than reopened, and bound concurrency with a semaphore so you do not open a thousand sockets at once and overwhelm the target or yourself. Aggregating data from many APIs, calling microservices, scraping — these are where async pays off most visibly, turning minutes of sequential waiting into seconds of overlapped calls.
Unbounded concurrency is its own problem: firing ten thousand requests at once will exhaust file descriptors, trip rate limits, or knock over the service you are calling. An asyncio.Semaphore caps how many operations run simultaneously, letting you say "at most twenty in flight" while still processing thousands in total:
sem = asyncio.Semaphore(20)
async def fetch_limited(session, url):
async with sem:
return await fetch(session, url)
This is essential, real-world hygiene. Concurrency is a resource you must bound, not maximize blindly — the goal is enough parallelism to overlap waits, not so much that you cause failures elsewhere.
When many tasks run together, failures need deliberate handling. By default gather raises on the first exception, abandoning the others; pass return_exceptions=True to collect results and exceptions together so one failure does not lose every other result. TaskGroup takes a stricter stance, cancelling siblings when one fails, which is often what you want for an all-or-nothing operation. Decide per use case: a batch where partial success is fine wants gathered exceptions; a transaction-like group wants cancellation. Either way, never let an exception in one concurrent task silently vanish — unhandled task exceptions are a common, hard-to-debug async failure.
Django supports async views, middleware, and an async ORM interface, but the integration has sharp edges worth understanding. You cannot call the synchronous ORM directly from an async view without wrapping it in sync_to_async, or you block the loop; conversely, calling async code from sync context needs async_to_sync. Async views shine for I/O-bound work — calling several external services, handling many slow connections — but offer nothing for ordinary database-bound views, which are usually faster left synchronous. Adopt async in Django where the work is genuinely I/O-concurrent, and keep the rest synchronous; forcing everything async adds complexity without benefit and reintroduces the loop-blocking trap at every ORM call.
The case where async Django clearly wins is a view that aggregates several slow upstream services. Running the calls concurrently makes the response time track the slowest call instead of their sum, and Django's async ORM methods (aget, acreate, acount, aexists, async for over a queryset) let simple queries run without an explicit sync_to_async wrapper:
# views.py
import asyncio
import httpx
from django.http import JsonResponse, Http404
from .models import Project
async def project_status(request, pk):
user = await request.auser() # request.user would query synchronously
try:
project = await Project.objects.aget(pk=pk, owner=user)
except Project.DoesNotExist:
raise Http404
async with httpx.AsyncClient(timeout=5.0) as client:
async with asyncio.TaskGroup() as tg:
ci = tg.create_task(client.get(f"https://ci.example.com/api/{project.slug}"))
uptime = tg.create_task(client.get(f"https://status.example.com/{project.slug}"))
open_issues = await project.issues.filter(state="open").acount()
return JsonResponse({
"ci": ci.result().json(),
"uptime": uptime.result().json(),
"open_issues": open_issues,
})
Several caveats apply. The async ORM methods still execute the query in a thread under the hood, so they keep the loop free but do not make the query itself faster. Transactions do not work in async code: transaction.atomic must wrap synchronous code, so put multi-statement writes in a sync function and call it through sync_to_async. Evaluating request.user triggers a synchronous session and database lookup, which is why the view uses await request.auser(), added in Django 5.0. Finally, you only get the concurrency benefit when the project runs under an ASGI server such as Uvicorn, Daphne or Hypercorn; under WSGI, Django runs each async view in its own one-off event loop, which works but adds overhead rather than removing it.
Async generators (async def with yield, consumed with async for) let you stream data as it arrives rather than buffering it all first. This is the backbone of streaming responses — paginating through an API, processing a large download, or streaming tokens from an LLM to a browser as they generate. Each yielded item is produced and consumed without waiting for the whole sequence, and awaits inside the generator keep the loop free between items. Streaming with async generators is how you handle data that is too large or too slow to materialize at once, delivering it incrementally and responsively.
Async bugs have their own flavor. A program that runs but is slow often has a hidden blocking call holding the loop; enable asyncio's debug mode to get warnings when a coroutine blocks too long. "Coroutine was never awaited" warnings mean you called an async function without awaiting or scheduling it, so it never ran. Forgotten await keywords are the most common mistake, producing code that silently does nothing. And unhandled exceptions in fire-and-forget tasks disappear unless you await the task or attach a callback. Knowing these signatures turns baffling async behavior into recognizable, fixable patterns.
Async is a tool with a specific purpose, not a universal upgrade. Use it when you have many concurrent I/O-bound operations — numerous outbound calls, many simultaneous connections, streaming. Skip it for simple request-response views, CPU-bound work, and code where the added complexity buys no concurrency. A synchronous Django app serving database-backed pages is perfectly fast and far simpler than an async rewrite that gains nothing. The mark of good judgment here is reaching for async precisely where overlapping waits matters, and leaving the rest synchronous. The complexity of async is only worth paying when the concurrency it buys is real.
A distinction that trips people up: awaiting a coroutine directly runs it to completion before moving on, while wrapping it in a task with asyncio.create_task schedules it to run concurrently with other code. If you want several things happening at once, you need tasks (or gather, which creates them for you), not a sequence of direct awaits. A common bug is expecting concurrency from code that simply awaits coroutines one after another — it runs sequentially, no faster than synchronous code. Understanding that concurrency comes from scheduling multiple tasks on the loop, not from the async keyword alone, is essential to actually getting the speedup async promises.
Async code can be cancelled, and handling it well matters for robustness. A task can be cancelled — by a timeout, by a parent shutting down — which raises a cancellation exception inside it at the next await point, and your code should clean up resources gracefully when that happens. asyncio.timeout (or wait_for) bounds how long an operation may take, cancelling it if it overruns, which is essential for not hanging forever on a slow external call. Treating cancellation as a normal part of an async task's lifecycle, rather than an edge case, is what makes long-running async services shut down cleanly and recover from stuck operations.
Cancellation arrives as asyncio.CancelledError raised at the task's current await. Since Python 3.8 it inherits from BaseException, so a broad except Exception no longer swallows it by accident — but a bare except: or except BaseException still does, and a task that swallows its cancellation keeps running and breaks timeouts and TaskGroup shutdown. Clean up, then re-raise:
async def sync_account(client, account_id):
lock = await acquire_remote_lock(client, account_id)
try:
async with asyncio.timeout(30):
await push_changes(client, account_id)
except TimeoutError:
log.warning("sync for %s timed out", account_id)
raise
except asyncio.CancelledError:
log.info("sync for %s cancelled, releasing lock", account_id)
raise # never swallow it
finally:
# shield so the release is not itself cancelled half-way
await asyncio.shield(release_remote_lock(client, lock))
asyncio.timeout converts the internal cancellation into a plain TimeoutError when the deadline passes, so callers catch TimeoutError rather than CancelledError. It can also be rescheduled: async with asyncio.timeout(None) as cm followed by cm.reschedule(asyncio.get_running_loop().time() + 5) sets a deadline once you know how long you have. asyncio.shield protects the inner operation from being cancelled by the outer one, but the outer await still raises, so use it for short, must-finish cleanup — not as a way to make a task uncancellable.
Resources that must be opened and closed — connections, sessions, locks — use async context managers with async with, ensuring cleanup happens even when an await inside raises. This is how you manage an HTTP session, a database connection, or a pool correctly in async code, guaranteeing the resource is released on every path. Forgetting to use proper async resource management leads to leaked connections and exhausted pools, a common source of async production issues. The async with construct is the async counterpart to the ordinary context manager, and using it consistently for every acquirable resource is what keeps an async application from slowly leaking the very resources it depends on.
Real applications are rarely all-async, and the boundaries between async and synchronous code need care. Calling blocking synchronous code from async without offloading it to a thread blocks the loop; calling async code from sync requires running it on an event loop properly. Tools exist on both sides — thread executors for running sync work from async, and helpers for running async from sync — but the key is to be deliberate about every crossing. Many async bugs live exactly at these boundaries, where a synchronous call sneaks into an async path and silently destroys the concurrency. Mapping where your code crosses between the two worlds, and handling each crossing correctly, keeps the async parts actually asynchronous.
Single-threaded does not mean race-free. Code between two awaits is atomic with respect to other tasks, but any sequence that spans an await can be interleaved. The classic bug is check-then-act across an await:
_cache = {}
_locks = {}
async def get_token(client, tenant):
# Without the lock, 50 concurrent requests for a cold tenant all see
# a miss and all call the auth server ("cache stampede").
lock = _locks.setdefault(tenant, asyncio.Lock())
async with lock:
token = _cache.get(tenant)
if token is None or token.expired():
token = await client.fetch_token(tenant) # await = switch point
_cache[tenant] = token
return token
The rule of thumb: if you read shared state, await, then write shared state based on what you read, another task may have changed it in between, and you need an asyncio.Lock (or a design that avoids the shared write). asyncio's primitives — Lock, Event, Condition, Semaphore, Queue — are not thread-safe; they coordinate tasks on one loop. If a thread from to_thread needs to hand a result back to the loop, use loop.call_soon_threadsafe or asyncio.run_coroutine_threadsafe, never direct calls into loop objects from the thread.
Async Python is one thread cooperatively switching between tasks at every await, which makes I/O-bound work concurrent without threads — and does nothing for CPU-bound work, which belongs in processes. The event loop runs tasks until they wait, then runs others, so the cardinal rule is to never block it: use async libraries and push blocking calls to a thread pool. Get concurrency from gather or TaskGroup, bound it with a semaphore, handle errors deliberately so failures in one task do not vanish or sink the rest, and stream with async generators. In Django, adopt async only for genuinely I/O-concurrent views and wrap every ORM call across the sync boundary. Used where overlapping waits actually matters, async turns minutes of sequential I/O into seconds; used everywhere by reflex, it just adds complexity. The skill is knowing which situation you are in.