RecallStack
Login

Asynchronous Programming

HARD3 min readby AdminJune 20, 2026History
0.0|0 ratingsLog in to rate

Asynchronous programming is a concurrency model that allows a single thread to run multiple tasks concurrently by releasing control to the event loop when waiti...

#ai-engineering#course#asynchronous

Definition

Asynchronous programming is a concurrency model that allows a single thread to run multiple tasks concurrently by releasing control to the event loop when waiting for I/O operations (like network requests or disk reads) to complete. In Python, this is built using asyncio, async, and await.

Why It Exists

LLM calls are highly latency-heavy, often taking several seconds. In synchronous systems, if you fetch 10 separate prompt outputs sequentially, your system blocks for 10 x 2s = 20 seconds. Using asyncio, all 10 calls run concurrently, returning in approximately 2 seconds total.

How It Works

python
1
2
3
4
5
6
7
8
9
10
11
SYNCHRONOUS CALLS (Blocks Thread)
[Request 1] ────► [Wait 2s] ────► [Request 2] ────► [Wait 2s] ────► (Total: 4s)

ASYNCHRONOUS CALLS (Concurrently managed by Event Loop)
               ┌──► [Request 1 (Wait 2s)] ──┐
[Event Loop] ──┼──► [Request 2 (Wait 2s)] ──┼──► [Resumes processing as results return] (Total: ~2s)
               └──► Releases thread context ┘

1. **Event Loop**: A loop running in a single thread that monitors and schedules active asynchronous tasks.
2. **Coroutines**: Functions defined with `async def`. When called, they return a coroutine object instead of executing immediately.
3. **Awaiting**: Placing `await` before a coroutine yields control back to the event loop, letting other tasks execute while waiting for I/O.

Real-World Example

Example

Perplexity triggers multiple web searches and model retrievals simultaneously when answering a user question. It uses asynchronous requests to query Google, Bing, and internal indexes concurrently, compiling the results as they arrive.

Python Example

Example

Here is a production template that demonstrates fetching response completions from multiple prompt options concurrently to save processing latency:

python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
import asyncio
import time
from typing import List

async def fetch_completion_async(prompt_id: int, latency: float) -> str:
    """Simulates an asynchronous API request to an LLM provider."""
    print(f"Starting request {prompt_id} (simulated latency {latency}s)...")
    # Release control to event loop. Do NOT use time.sleep() here as it is blocking.
    await asyncio.sleep(latency)
    print(f"Finished request {prompt_id}!")
    return f"Response {prompt_id}"

async def main():
    start_time = time.time()
    
    # Define three concurrent model calls
    tasks = [
        fetch_completion_async(1, 1.5),
        fetch_completion_async(2, 2.0),
        fetch_completion_async(3, 1.0)
    ]
    
    # Run all tasks concurrently
    results: List[str] = await asyncio.gather(*tasks)
    
    end_time = time.time()
    print(f"Results: {results}")
    print(f"Total concurrent execution time: {end_time - start_time:.2f} seconds")

if __name__ == "__main__":
    # Run the event loop
    asyncio.run(main())

Interview Questions

  • Beginner: What keyword must be used to define a coroutine, and what keyword is used to pause it?
  • Intermediate: What is the difference between concurrency and parallelism, and where does asyncio fit in?
  • Advanced: Why will using time.sleep() inside an async def function block the entire application? What should you use instead?

Interview Answers

  • Beginner: You define a coroutine using the async def syntax, and you pause its execution to wait for a result using the await keyword.
  • Intermediate: Concurrency is about dealing with lots of things at once (structuring tasks to run in overlapping timeframes, typically on a single thread). Parallelism is about doing lots of things at once (running tasks on separate CPU cores). asyncio provides concurrency for I/O-bound tasks.
  • Advanced: time.sleep() is a synchronous, blocking function that freezes the executing OS thread, preventing the event loop from switching tasks. Instead, you must use await asyncio.sleep(), which is non-blocking and yields control back to the event loop.

Common Mistakes

  • Forgetting to await: Writing result = fetch_completion_async(1, 2.0) without await. This returns a coroutine object instead of the actual string result.

Assignment

Write an asynchronous script that executes three functions concurrently. Function 1 sleeps for 0.5s, Function 2 sleeps for 1.0s, and Function 3 sleeps for 1.5s. Print the total execution time, verifying it is ~1.5 seconds.

Discussion

Join the discussion! Sign in to leave comments and ask questions.

Loading discussion...