Essay

One untrusted agent is enough.

01

Three concrete demonstrations

Agent Smith studies simulated environments of up to one million LLaVA-1.5 agents. Gu and colleagues show that an adversarial image placed in one agent’s memory can spread harmful behavior through randomized pairwise conversations in that setup. The scale is a property of their simulation, not evidence that every deployed agent network behaves this way.

Here Comes The AI Worm studies propagation through retrieval-augmented applications, including an experimental ecosystem of email assistants. Cohen, Bitton, and Nassi show how self-replicating prompts can carry an attack between applications.

Prompt Infection examines malicious prompts spreading across communicating agents. Lee and Tiwari also evaluate a defense that labels agent-originated content and combines that information with other safeguards.

02

What carries across the boundary?

The common design question is how a message acquires authority at the next step. A peer’s output may be useful evidence without being an instruction the recipient should obey. Memory adds another path: content can be stored now and retrieved as context later.

That makes source identity, permissions, and the distinction between data and instructions worth preserving. It does not establish that tags alone stop attacks, or that a particular defense generalizes to a new model and environment.

03

Why this matters to my memory work

I want handoffs to let a new agent recover useful context. The same handoff is also an input that needs scrutiny. A durable record should preserve who supplied a claim and what checked it, not merely make the text easy to retrieve.

Recall Debt motivates a retrieval question about missing bridges. These attack studies motivate a different question about what can travel over a bridge. I need to examine both without treating them as the same experiment.

04

Keep the analogy limited

Accounts of chatbot personas spreading through copied prompts raise a related question about transmission. They do not demonstrate the same mechanism as a controlled prompt-injection study, or establish that a model has acquired a motive to replicate.

For a system I operate, the useful next step is concrete: identify the message and memory boundaries, decide which inputs may authorize actions, and test whether those boundaries hold. The papers supply cases to learn from, not a universal verdict on multi-agent systems.

Keep this distinction in view

Check what crosses the boundary.

  1. Source

    A message, document, or tool result

  2. Transfer

    Content moves into another agent’s context

  3. Authority

    Is content being treated as an instruction?

  4. Action

    What can the receiving agent now change?

A threat-model schematic. Different studies test different routes; this does not establish one universal attack.