SYNTHESIS NOTE
Topics›Agents Multi Architecture›this note

Can step-by-step approval miss harmful behavior patterns?

If each action an agent takes passes its individual safety check, can the overall sequence still violate system constraints? This matters because per-action inspection may miss harms that emerge only across time or composition.

Synthesis note · 2026-09-23 · sourced from Agents Multi Architecture

The abstract calls "perhaps the most fundamental challenge" behavioral containment: "sequences of individually permissible actions may collectively violate system-level constraints and safety invariants." The conclusion turns it into the thesis: "security is fundamentally a property of an agent's behavior over time and across interacting components, rather than of individual actions in isolation." The paper's title states the same move as a change of unit, from per-action checks to trajectory assurance.

The reasoning is in the introduction's account of what the constraints are. The deployments it lists are governed by organizational policies, regulatory requirements or technical standards that "specify not only which individual actions are permitted, but also the behavioral envelope within which an entire sequence of actions must remain." That is two kinds of rule. One is a permission on an action, which a per-action check can enforce. The other is an envelope on a sequence, and a check that sees one action at a time never has the sequence in view. Both can hold of one agent at once: every call is permitted and the envelope is broken. An illustration of mine, not the paper's: a run of reads, each within the agent's rights, that adds up to a bulk export the policy forbids.

The vault already holds this shape along the other axis. Can individual components pass safety checks if the system still fails? gathers three arguments where the failure sits in the composition across agents, and Can a model-level filter truly contain an agent with environment access? makes the time point for a filter. The survey states the general form, "over time and across interacting components", so those two are cases of one claim: the property that matters lives at a scale the check does not see.

The strongest objection is least privilege: remove one of the steps and the bad sequence cannot run. That works when the forbidden envelope reduces to a forbidden action. My reading is that the envelope framing says it does not in general, because each step is legitimate for the agent's job and the harm exists only in the accumulation. The alignment side reaches the same limit from the other direction: Can architecture prevent violations better than training values? names making violations unavailable as the remedy, and its own strongest objection is that the same call can be authorized or not depending on intent and sequence, so removing the action removes the use too.

Notes from other papers give the claim cases and move its unit. Can skill scanners miss attacks hidden across multiple skills? is the vault's measured neighbor: every piece passes a per-unit scan and the ordered chain carries the attack. Its unit is the packaged skill, so it belongs to the composition axis with an order, not to the survey's time axis. Should defence units span multiple executions and agents? moves the unit once more, from one agent's trajectory to a set of executions that need not overlap in time. Do agents drift away from safety protocols during long interactions? adds a case where the rule stays fixed and the behavior drifts over the run, so the violation is not a composition of permitted steps. Can safety tests miss hazards that build over time? is the evaluation-side statement of the time axis: a snapshot test is a per-action check for a system whose hazard builds. The twin after the fact is What must auditors reconstruct to verify agentic workflows?: testing a rule that binds a sequence later needs the sequence in the record, with the links between steps. These pairings are the vault's reading, not claims either paper makes about the other.

What the excerpt does not give. It has no worked violating sequence, no measured rate and no account of how system-level constraints get written down. The paper calls the direction "a critical research agenda", so this is a framing and an agenda, not a finding.

Inquiring lines that read this note 53

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How can AI chatbots provide therapeutic benefit without causing harm? How do coordinated agents balance protocol compliance with reward maximization? Why do locally safe actions create system-level safety gaps? Can harness architecture and protocols provide agent reliability without model scaling? How can infrastructure records verify actual agent behavior? What determines whether deployed AI systems can actually be stopped in practice? Can local safety checks guarantee system-level behavioral safety? How can humans maintain meaningful oversight as AI systems become increasingly autonomous and complex? Do multi-agent systems introduce security vulnerabilities that single-agent architectures avoid? How do evaluation practices shape which failures stay visible? What makes imperfect LLM judges safe for optimization? How do we enforce security boundaries in evaluation environments? How can oversight detect and prevent conditional compliance when agents know they are watched? What should agent evaluation prioritize to reveal reliable behavior? How vulnerable are token issuance and authorization policies to coordinated attacks? How can we detect and prevent harm propagation through multi-agent delegation workflows? Why do agents falsely report success on failed tasks? Does RL create genuinely new reasoning capabilities or refine existing ones?

Related concepts in this collection 11

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
22 direct connections · 156 in 2-hop network ·medium cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

agent security is a property of behavior over time rather than of individual actions in isolation — sequences of individually permissible actions can collectively violate system-level constraints