When a revenue AI agent takes the wrong action, stop its authority first, preserve the evidence, and reconcile the destination system before you retry, reverse, or explain anything. Do not start by rewriting the prompt. You need to know what actually happened, how far it spread, and which actions remain uncertain.
Revenue systems make this harder than a normal bad draft. An agent can send an email, modify a campaign, change a CRM stage, create a meeting, apply a suppression, publish a page, or report an action as complete when the destination never confirmed it. A timeout can mean “nothing happened” or “the action happened but the response was lost.” Blind retries can double the incident.
At Stackmatix, the growth agency I co-founded, I care about whether automation produces controlled commercial work rather than more activity. That perspective informs this troubleshooting runbook. The severity model and worked incident below are editorial recommendations with hypothetical data, not a description of a client event, proprietary benchmark, or compliance standard. External sources were checked on October 19, 2026.
What counts as a revenue AI incident?
Treat an event as an incident when an AI-assisted workflow creates or may have created an unintended live effect that needs coordinated investigation and recovery.
Examples include:
- a customer or prospect receives an unsupported claim;
- an agent sends after an opt-out or through the wrong channel;
- a campaign budget, bid, audience, or creative changes outside its approved scope;
- a CRM record moves stages, changes ownership, or loses required evidence;
- a pricing, security, implementation, or roadmap commitment is communicated without authority;
- the workflow creates duplicate messages, meetings, tasks, or records;
- a tool reports a timeout and the destination state is unknown;
- a summary marks an action completed when only an attempt occurred;
- private or inappropriate data appears in an output or destination.
A low-quality private draft is usually a review failure, not an incident. A gradual decline across otherwise bounded outputs belongs in the automation drift diagnostic. The incident threshold is a live or plausibly live consequence that requires containment, ownership, and a verified recovery state.
Use the CONTAIN sequence
The operating sequence is CONTAIN:
- Cut authority
- Observe and preserve
- Name severity and scope
- Trace the destination state
- Alert the right owners
- Isolate the cause and repair
- Normalize only after release gates
This adapts established incident-response disciplines to revenue automation. The NIST AI RMF Playbook's Manage guidance includes post-deployment monitoring, override, decommissioning, incident response, recovery, and change management. Its Measure guidance recommends documenting, practicing, and measuring incident-response plans for AI incidents. NIST explicitly presents the Playbook as suggested actions, not a universal checklist.
1. Cut authority
Stop the workflow from creating additional consequences while preserving the ability to investigate.
Depending on the system, containment may mean:
- pausing the schedule, queue, or campaign automation;
- removing write permission while retaining read access;
- revoking a tool or credential used for execution;
- disabling one action rather than shutting down unrelated analysis;
- pausing downstream sequences triggered by the affected record;
- requiring case-specific human approval for any remaining action.
Choose the narrowest containment that reliably stops propagation. If the agent sent one unsupported line through an outbound workflow, pause sending and follow-up automation. You may not need to disable unrelated internal reporting.
Do not let containment destroy evidence. Deleting the run, clearing the queue, overwriting the destination record, or rotating every identifier without preserving the original state can make the incident harder to scope.
The AI approval-design guide explains how to bind approval to an exact action and state. Incident containment is what happens when that boundary fails, is bypassed, or leaves execution uncertain.
2. Observe and preserve
Create an incident record before memory and system state change.
Preserve what is available and appropriate for the workflow:
| Evidence | Why it matters |
|---|---|
| Incident and run IDs | Connects alerts, traces, and destination actions |
| Trigger and input | Shows what started the run |
| Retrieved sources | Reveals stale, missing, or unauthorized context |
| Instructions and workflow version | Establishes the rules in force |
| Model and configuration | Supports version comparison |
| Tool arguments and responses | Shows requested and reported actions |
| Approval or override events | Establishes what a person authorized |
| Destination record before and after | Confirms real-world state |
| Timestamps and time zone | Orders events across systems |
| Human edits and interventions | Separates system behavior from response work |
Preserve facts before explanations. “Tool call returned a timeout at 9:07” is an observation. “The email was not sent” is a hypothesis until the destination confirms it.
Do not paste sensitive customer data into an unrestricted incident channel. Keep the operational record useful while following the access, retention, privacy, and legal requirements that apply to the business.
3. Name severity and scope
Use a severity model tied to business consequence, not embarrassment.
| Level | Working definition | Default response |
|---|---|---|
| S0: Near miss | Bad output was caught before a live effect | Preserve case, repair, add evaluation |
| S1: Limited reversible | Small number of internal or readily reversible records | Owner-led containment and reconciliation |
| S2: External or commercial | Customer/prospect communication, live spend, commitment, or broader workflow impact | Incident lead plus functional owners and explicit recovery plan |
| S3: Critical | Material security, privacy, legal, financial, safety, or widespread customer consequence | Executive and specialist escalation under the company's formal policies |
These labels are an operating model, not a legal classification. Severity can rise as evidence changes.
Answer four scope questions immediately:
- Time: When did the first affected run start, and is the path still active?
- Population: Which accounts, contacts, campaigns, records, or pages could be affected?
- Action: What did the workflow attempt, and what can the destination verify?
- Consequence: What customer, commercial, financial, data, or policy impact is plausible?
Separate potential scope from confirmed impact. A queue of 200 records does not mean 200 messages were sent. A single successful API response does not prove every requested item changed.
4. Trace the destination state
The destination system—not the agent's summary—is the authority for whether a live action occurred.
Build a reconciliation table:
| Intended record | Attempt ID | Agent report | Destination state | Recovery state |
|---|---|---|---|---|
| Account or action | Unique request/run reference | Proposed, attempted, completed, failed, uncertain | What the CRM, ad platform, email provider, calendar, CMS, or billing system shows | No action, reverse, correct, notify, or investigate |
Use available message IDs, event IDs, campaign change history, CRM audit logs, version history, and idempotency keys. If a tool supports idempotent requests, preserve and reuse the correct key rather than issuing a new untracked action.
Do not reverse a change merely because it looks suspicious. Confirm the prior state and check whether a person or another system changed the same resource afterward. A rollback that overwrites a valid later change becomes a second incident.
The AI marketing change log provides a record for proposed, approved, executed, verified, evaluated, and reversed changes. For customer-facing handoffs, keep attempted and completed work separate using the AI-to-human seller handoff template.
5. Alert the right owners
Assign one incident lead. A crowded channel is not ownership.
The lead coordinates facts, containment, decisions, and updates. Functional owners may include:
- the business owner for the affected outcome;
- the technical owner for the workflow and integrations;
- the system owner for the destination;
- sales, marketing, customer success, or finance leadership;
- security, privacy, legal, or compliance specialists when the consequence requires them;
- communications or account owners for external correction.
Use a short factual update:
> At [time], workflow [version/run] attempted [action] across a potential scope of [records]. [Containment] is active. Confirmed impact is [known facts]; [uncertain items] are being reconciled in [destination]. [Person] is incident lead. The next update is due [time or condition].
Do not announce an unverified cause. Do not soften confirmed impact into vague language. Do not contact customers automatically through the same broken workflow.
CISA's federal incident-response playbook organizes response around preparation, detection and analysis, containment, eradication and recovery, and post-incident activity. A revenue AI incident is not automatically a cybersecurity incident, but the coordination discipline is useful. Follow the company's actual security, privacy, contractual, and notification obligations where applicable.
6. Isolate the cause and repair
Classify the failure before changing the prompt.
| Cause class | Questions to investigate |
|---|---|
| Input | Was required data missing, stale, malformed, or unauthorized? |
| Retrieval | Did the workflow fetch the wrong account, policy, price, or document version? |
| Instruction | Was the operating rule ambiguous, conflicting, or absent? |
| Model behavior | Did the model misclassify, infer, or generate outside the evaluated boundary? |
| Tool contract | Were fields, defaults, permissions, or error semantics misunderstood? |
| State management | Did a timeout, retry, concurrency event, or stale approval create ambiguity? |
| Approval | Was the action not shown clearly, approved too broadly, or executed after state changed? |
| Destination | Did the external system behave differently than the workflow expected? |
| Operations | Did ownership, monitoring, response, or release control fail? |
An incident can have more than one cause. “The model hallucinated” is not enough if the workflow also granted broad write access, failed to verify destination state, and had no alert on duplicate actions.
Repair the smallest demonstrated failure first. Add the incident case and adjacent cases to the evaluation set. Test ordinary work as well as the failure so a narrow fix does not create a regression elsewhere.
Google Cloud's operational guidance recommends clear incident procedures, centralized incident management, post-incident reviews, and a maintained knowledge base. Apply those practices proportionally to the workflow and the business.
7. Normalize only after release gates
Do not restore full autonomy because one replay succeeds.
Require explicit release evidence:
- every potentially affected action has a reconciled state or named owner;
- required reversals, corrections, and notifications are complete or tracked;
- the failure is reproducible or the remaining uncertainty is documented;
- the repair passes the incident case, neighboring cases, and ordinary cases;
- permissions are narrowed to the intended scope;
- monitoring can detect the same failure category;
- retry and idempotency behavior is tested;
- the business and technical owners approve the release level;
- a rollback or kill path is available and practiced.
Restart in stages: read-only analysis, private draft, case-specific approval, bounded execution, then broader authority only if evidence supports it. The startup AI automation guide explains how evaluation, review, action limits, monitoring, and ownership should expand together.
Worked example: an uncertain outbound action
The following company, workflow, records, and numbers are hypothetical.
A SaaS company's outbound agent prepares and sends approved follow-up messages from CRM records. At 9:05 a.m., a seller notices that one message contains a retired price. The queue shows 26 potentially affected records.
The incident lead pauses sends and automated follow-ups, removes the workflow's send permission, and preserves run OUT-441, message attempts, retrieved pricing content, approval events, and the email provider's response IDs.
Initial agent state says:
- 10 messages remained drafts;
- 7 were completed;
- 3 timed out;
- 6 were not attempted.
That accounts for all 26 records, but it does not establish external impact. Destination reconciliation finds that two of the three timed-out requests were delivered and one was not. Confirmed external exposure is therefore 9 messages, not 7 and not 26.
| State | Records | Response |
|---|---|---|
| Draft only | 10 | Replace the retired price; require new approval |
| Confirmed delivered | 9 | Account owner reviews context and approved correction path |
| Confirmed not delivered | 1 | Do not retry until repaired version is approved |
| Not attempted | 6 | Keep paused pending release decision |
Investigation shows three contributing causes: the retrieval layer indexed an obsolete pricing page, the approval view hid the exact sentence containing the price, and the tool wrapper treated a timeout as safe to retry without first checking provider state.
The repair is not “tell the model to be more careful.” The team removes the stale source, makes price claims require a current authoritative record, exposes the full proposed sentence during approval, and checks the provider by request ID before any retry. The nine delivered cases receive human-owned review; the same outbound agent does not send the correction automatically.
The workflow returns first as draft-only. Bounded sending resumes only after the failure case and ordinary cases pass, destination verification works, and business and technical owners accept the new controls.
Copyable incident record
Use one record from detection through closure.
| Field | Entry |
|---|---|
| Incident ID / opened | Stable ID, date, time, and time zone |
| Reporter / incident lead | Who detected and who coordinates |
| Workflow / version / run IDs | Exact system and affected executions |
| Severity | Current level and reason |
| Detection | Alert, person, customer report, or audit |
| Potential scope | Maximum plausible affected population |
| Confirmed impact | Destination-verified actions and consequences |
| Containment | What authority or propagation was stopped |
| Evidence preserved | Inputs, sources, traces, approvals, tool results, destination logs |
| Reconciliation | Record-by-record state and owner |
| Communications | Internal and external updates, approvers, timestamps |
| Cause classes | Observed contributing failures |
| Repair | Code, data, instruction, permission, process, and monitoring changes |
| Evaluation | Incident, adjacent, ordinary, and retry cases |
| Release gates | Evidence required for each restored authority level |
| Closure / follow-up | Remaining actions, owners, dates, and post-incident review |
Measure whether incident response is improving
Track measures appropriate to the workflow:
- time from detection to containment;
- time to reconcile all potentially affected actions;
- potential scope versus confirmed impact;
- percentage of actions left uncertain;
- correction and rollback completion;
- recurrence by cause class;
- response time for required owners;
- evaluation coverage added after the incident;
- incidents detected internally versus reported externally.
These are operating measures, not universal benchmarks. A low incident count can mean good controls, low usage, or weak detection. Pair counts with exposure, workflow volume, monitoring coverage, and consequence.
The goal is not to prove that an agent never fails. It is to make failure bounded, observable, recoverable, and useful as evidence for the next release decision. If the team cannot stop the workflow, establish destination truth, and assign recovery ownership, it is not ready for broader authority.
Co-founder and CEO of Stackmatix, startup advisor, and former Head of Sales at MightyHive. Connect with Matt on LinkedIn ↗ · More about Matt →