How to Respond When an AI Agent Takes a Harmful Action
Why the pillar's cut-credential, stop-instance, export-evidence order is structurally forced rather than a tidy convention, a concrete answer for who holds standing revocation authority, a pre-incident drill for the evidence-export path, and what changes when the compromised credential belongs to a third-party tool server instead of the agent itself.
- Credential revocation has to happen first in an AI agent security incident because it is a control-plane action that lands immediately regardless of whether the compromised instance is healthy, responsive, or resisting shutdown, while stopping that instance can stall on exactly the process least likely to cooperate.
- A live credential keeps working even after the instance that first requested it goes offline, since a queued retry, a delegated sub-process, or an attacker using the credential directly from outside the agent entirely can all still act on it until the credential itself is revoked.
- Stopping an AI agent instance during a security incident should mean quarantining it — freezing its process and cutting its network access — rather than deleting or redeploying it, because a destructive teardown can wipe local trace data before the evidence-export step ever runs.
- An evidence-export path that authenticates using the same identity as the per-instance credential a security incident revokes can end up locked out of the exact storage it needs to reach, so that path needs a credential the incident response plan never touches.
- Revoking a credential only removes access and never grants new capability, which is why a named on-call security role should hold standing authority to revoke any AI agent credential immediately, separate from the multi-step approval chain built to slow down actions that create new risk.
- When the credential behind a harmful AI agent action is held by a third-party tool server or MCP connector rather than issued to the agent itself, cutting it off can mean disabling that connector for every other tenant it serves, not just the one compromised instance.
Why must a credential be cut before the instance that used it is stopped?
A compromised credential has to be revoked before its agent instance is stopped because revocation is a control-plane action that lands immediately no matter what state that instance is in, while stopping the instance depends on the instance itself cooperating — the exact thing a compromised process is least likely to do. Killing a hung process, waiting out an orchestrator's graceful-drain period, or forcing a container teardown can all take longer than expected, and if the instance is actively unresponsive or resisting shutdown, "stop it first" can leave a live, usable credential exposed for however long that fight takes. A revocation call to a secrets manager or an OAuth authorization server, by contrast, does not care whether the instance it was issued to is healthy, hung, or already gone — it takes effect at the credential itself, one layer removed from whatever the compromised agent instance is doing at that moment.
The second reason is just as concrete: a credential does not stop working the moment the instance that first requested it goes offline. Anything else holding a copy — a retry a worker already queued, a delegated sub-process, a webhook callback still in flight, or an attacker who extracted the credential directly and is using it from outside the agent's own process entirely — keeps acting on it until the credential itself is dead, regardless of whether the original instance is still running. Stopping the instance closes off one path to further harm; revoking the credential closes off every path at once, which is exactly why it has to be the first move rather than the second.
What happens if the evidence gets exported after cleanup has already started?
Exporting an AI agent's trace data after cleanup has already touched the compromised instance risks losing the evidence outright, because the step most teams fold into "stopping the instance" — deleting a container, redeploying a service, wiping a virtual machine — can erase local trace data the export step still needed to reach. The shipping AI agents to production reference states the underlying requirement directly: preserve traces for the investigation window, accessible independent of the production system, precisely because a compromised system cannot be trusted to keep reporting on itself accurately once remediation begins. That requirement has a sharper implication than "export before you clean up" alone conveys: it means step 2, stopping the instance, has to mean quarantine — freeze the process, cut its network access, pull it out of rotation — not destroy. A destructive teardown and a quarantine both stop an instance from acting again, but only one of them leaves anything behind for step 3 to find.
Treat the two as genuinely separate operations with a hard ordering between them: quarantine first, export second, and only run anything that deletes or replaces the underlying compute — a redeploy, a container removal, an image rebuild — after the export step has confirmed it captured what it needed. A response plan that folds "stop the instance" and "tear it down" into a single automated action skips straight past the window step 3 depends on.
The interaction the order alone doesn't show: an export path that can lock itself out
An evidence-export path authenticated with the same identity as the credential a security incident is revoking can end up cut off from the very storage it needs to reach, which is a failure the fixed order of the triplet does not make visible on its own. If the pipeline that copies trace data out to independent storage authenticates using the agent's own per-instance credential — or a credential scoped anywhere near it — then step 1's revocation can silently break step 3 before step 3 ever runs, turning a clean three-step response into one where the third step fails for a reason nobody diagnosed as connected to the first.
The fix is structural, not procedural: the export path needs its own credential, provisioned and stored separately from anything a security incident would ever revoke, and scoped only to write access on the destination store. Confirming that separation is a design check worth running once, at build time, rather than something to discover by watching an export fail mid-incident.
Who should hold standing authority to revoke a credential mid-incident?
A named on-call security role, distinct from general engineering on-call, should hold pre-authorized standing authority to revoke any AI agent credential immediately, without routing the decision through the multi-step approval chain that exists for ordinary operations. The reasoning turns on what revocation actually does: it only removes access and never grants new capability, executes no action against an external system, and can be undone by reissuing a fresh credential once the investigation clears the instance. That is close to the opposite risk profile from the irreversible or state-changing calls the pillar's own permission guidance requires a human sign-off gate for — a gate this cluster's runtime-enforcement piece covers as an escalate verdict routed to human approval. Gating a purely subtractive action like revocation behind that same approval flow trades a real delay, at the exact moment speed matters most, for a safety guarantee revocation never actually needed.
The practical version of this: designate credential revocation across all agent infrastructure as a standing responsibility of the security on-call rotation, with authority to act unilaterally and a requirement to log the decision — who, when, which credential, why — for review after the fact rather than approval before it. After-the-fact review keeps the authority accountable without adding a delay to the one step in the whole response where delay is the actual cost.
Test the evidence-export path before you need it, not during an incident
An evidence-export target earns its place in a response plan only after it has actually received and returned a real trace export, not merely been named as the intended destination. Run this as a scheduled drill, independent of any real incident: export a live trace to the designated off-instance store, confirm the write succeeds, confirm the exported data is actually readable back out rather than just present, confirm the destination is reachable using the separate credential described above, and time the whole operation against whatever window the response plan assumes it will take. Any of those four checks failing during a drill costs a rerun; the same failure discovered mid-incident costs the investigation itself.
How does the response change when the compromised credential belongs to a third-party connector?
The credential/instance/evidence triplet, as stated, assumes the credential in question is the operator's own per-instance credential — the object the agent security operations pillar's issuance design controls end to end, scoped to one instance and revocable on demand. That assumption breaks the moment a harmful action routed through a tool server or MCP connector that authenticated to the downstream system on its own behalf, holding a custodial credential the agent instance itself never saw. As managing secrets for AI agents covers, that custodial credential belongs to the connector, not to the instance under investigation — and the operator's own revocation path to it may not exist at all.
What "cut the credential" actually means depends entirely on which kind of connector was involved:
Per-caller revocation supported. The connector can revoke access for one calling agent or tenant without touching any other. The triplet applies exactly as written — cut this instance's access at the connector, stop the instance, export the evidence.
No per-caller revocation. The connector's custodial credential is shared across every tenant or instance it serves. Cutting it off now means disabling the connector's access to the downstream system entirely, taking every other caller offline as collateral — a blast-radius decision the operator's own per-instance credentials never force, because those are already scoped to one instance by design.
Which of those two situations a given dependency falls into should already be known from the custody audit described in managing secrets for AI agents, not discovered mid-incident while deciding whether pulling the plug on a shared connector is worth the collateral damage to every other tenant it serves. The evidence side carries a matching gap: the connector's own record of what it did with its custodial credential typically lives on infrastructure the operator does not control, retained on the connector's own schedule rather than the operator's — see building an audit trail for an AI agent for what the operator's own export actually captures. Exporting the agent's own trace store in step 3 preserves everything the instance itself logged, but when a custodial connector mediated the harmful action, part of the full record of what actually happened downstream can sit entirely outside that export — worth confirming, per custodial dependency, before an incident, not after one is already underway.
Frequently asked questions
- Why does an AI agent's credential need to be revoked before its instance is stopped, not after?
- An AI agent's credential needs to be revoked before its instance is stopped because credential revocation is a control-plane action that takes effect immediately regardless of the compromised instance's own state, while stopping an instance can stall if that instance is unresponsive or resists shutdown, and because a credential stays usable by anything else that holds it — a queued retry, a delegated sub-process, an attacker acting directly — even after the instance that first requested it is no longer running.
- Who should have standing authority to revoke an AI agent's credential during a security incident?
- A named on-call security role, distinct from routine engineering on-call and from the multi-step approval chain built for ordinary operations, should hold pre-authorized standing authority to revoke any AI agent credential immediately, because revocation only removes access and never grants new capability, and its decisions can be logged and reviewed after the fact rather than approved before it, which is the opposite risk profile from an action that could itself cause new harm.
- Should you test an AI agent's evidence-export destination before a security incident happens?
- Yes — an evidence-export destination should be exercised with a real trace export before any incident occurs, confirming the write succeeds, the data is readable back out, the destination is reachable using a credential separate from anything the incident might revoke, and the whole export finishes inside the window the response plan assumes, because discovering any of those gaps mid-incident costs far more than a rehearsal ever would.
- What changes in the credential/instance/evidence response when the compromised credential belongs to a third-party MCP connector instead of the agent?
- When the compromised credential is custodial to a third-party tool server or MCP connector rather than issued to the agent itself, cutting it off depends on whether that connector supports revoking access for a single caller — if it does, the response is unchanged, but if the connector is shared across tenants with no per-caller revocation, disabling the credential means taking the connector offline for every other tenant it serves, and part of the evidence of what happened may sit in that connector's own access log rather than in the agent's own trace store.
This guide is free and stays free. The reference corpus behind it — machine-readable contracts, verified primary sources, continuously refreshed — is the paid product: a €5 starter key unlocks every premium reference for one agent via API; a €25 corpus license delivers the full corpus as RAG / fine-tuning data with an explicit AI-use grant; the €150 enterprise license adds commercial redistribution rights.