LLMs are General Asynchronous Agents

Abstract

Modern LLMs are increasingly capable as autonomous agents, but they follow sequential interaction cycles: read, think, reply or call tools, repeat. Many real-world use cases are not sequential: voice assistants, embodied agents, and monitoring systems receive new inputs while they think or perform another task. Modern LLMs address this with specialized architectures for voice interaction and video streams, VLAs for robot control, asynchronous tool calling for API usage, and others. In this work, we generalize from different asynchronous tasks to general asynchronous agents that can adapt to different types of concurrency. To achieve this, we develop an asynchronous LLM framework that lets users (or the agents themselves) define inference coroutines with overlapping memory states. We showcase that Qwen 3.x models are capable of asynchronous operation for streaming video understanding, videogames, and monitoring, without task-specific training.

How AsyncLLM Works

AsyncLLM lets you define asynchronous agents in terms of coroutines communicating through shared cache blocks. Each coroutine writes to its own cache block and can access other coroutines’ outputs using cache views. Below, an agent reasons and simultaneously provides the user with a running summary of its progress. Colors mark cache blocks; the outlined block is where a coroutine writes.

Agent definition

In [1]:

api = AsyncLLM("Qwen/Qwen3.5-9B")
prompt_block, thinker_block, writer_block = [
    await api.create_block() for _ in range(3)]
paragraph_finished = asyncio.Event()  # thinker -> writer signal
thinker_done = asyncio.Event()
await api.forward(encode(PROMPT), write_to=prompt_block)

In [2]:

async def thinker_coro():
    async for token in api.generate(
        "<think>\n", [prompt_block, thinker_block],
        stop="</think>",
    ):
        if "\n\n" in token:
            paragraph_finished.set()  # notify the writer
    thinker_done.set()
    paragraph_finished.set()  # one last summary

In [3]:

async def writer_coro():
    while (not thinker_done.is_set()
           or paragraph_finished.is_set()):
        await paragraph_finished.wait()  # new paragraph
        paragraph_finished.clear()
        async for token in api.generate(
            "...</think> Summary:",
            [prompt_block, thinker_block, writer_block],
            stop="\n",
        ):
            send_to_user(token)  # e.g. speak via TTS

In [*]:

await asyncio.gather(thinker_coro(), writer_coro())

What happens

Inference engine

T decode W decode W prefill

One column per forward pass. When both coroutines are active they share a batch. Click a column to jump there.

Cache blocks

Sprompt_block Tthinker_block Wwriter_block

Thinker view In [2]

S

T

Reads the prompt and its own reasoning. Never sees the writer.

Writer view In [3]

S

T

W

Reads the thinker's block as it grows and writes the summary.

Demos

All of the demos below are built on top of AsyncLLM. Each agent is a set of coroutines communicating through shared cache blocks.

Streaming video. An event probe scores every incoming frame (bars along the bottom), and the agent describes new events as they happen while the video keeps playing.

Qwen3.5-35B-A3B finds how much RAM the listing with blue LED lights has.

The same model finds the email of the seller of the guitar in the red case.

Web agent on VisualWebArena Classifieds.

Self-Defining Agents

Because AsyncLLM inference is defined directly in Python, we can hand an agent its own runtime and let it define its own inference structure: which coroutines to run, which cache blocks they share, and how they react to the environment.

1WriteThe agent generates tool calls that rewrite its own inference code.
2PatchThe new code is applied to the already-running instance. Cache blocks and background tasks survive every rewrite.
3ActIts act() method runs on every environment step.
4EvaluateA harness scores the agent and reports the result in the next round's prompt.

next round

We run this loop with Qwen3.6-35B-A3B on two ViZDoom environments. The agent gets its runtime, the previous evaluation result, and the ability to modify its inference code, but no task-specific guidance on what inference structure to build. Self-defined agents show good initial results on our evaluations. The capability is not yet reliable, but it suggests that future LLMs may build self-adapting agents from an environment description.

More in our blog post, The KV cache as an agent runtime.

Results

	SoccerNet (streaming)			ProactiveVideoQA
Model	AUROC	Trigger Acc	TimVal	PAUC (ω=0.5)
AsyncLLM Qwen3.5-9B	0.677	62.82	39.62	0.541
AsyncLLM Qwen3.8-27B	0.608	54.88	33.27	0.504
AsyncLLM Qwen3.6-35B-A3B	0.651	62.55	37.46	0.539
Mage-VL	0.555	52.79	27.87	0.428

Streaming video understanding. Training-free AsyncLLM agents outperform Mage-VL, a model trained specifically for video streaming (higher is better).

Qwen3.6-35B-A3B agent	Accuracy % ↑	# forward passes ↓	Mean % events ↓
Sequential	61.76	8452	100
+ early answer	47.06	6641	58.71
+ skip rows	26.47	3686	65.86
AsyncLLM	55.88	3837	55.49

System monitoring on DevOps-Gym. AsyncLLM answers after 55% of the log events on average, using 45% of the sequential agent's forward passes.

BibTeX

@misc{yakushev2026llmsgeneralasynchronousagents,
      title={LLMs are General Asynchronous Agents},
      author={George Yakushev and Denis Mazur and Vladimir Bartenev and Vyacheslav Zhdanovskiy and Timofey Byzov and Vladimir Kaurkin and Vadim Pastushenko},
      year={2026},
      eprint={2609.35427},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2609.35427},
}