Failure pattern · 23 documented incidents

Privileged action without an enforceable approval gate

Limits and approvals live in a prompt, a UI checkbox or a convention instead of code the automation cannot bypass.

See it happen

Watch how this failure unfolds, then switch to the same situation with the safeguard in place. Click any step to replay it.

Scroll the diagram sideways to see every system.

12 / 12 s

The limit was a suggestion, not a control.

What goes wrong:

  1. The refund limit lives only in the prompt: “never refund more than $500.”
  2. An email claims a manager has already approved an $8,000 refund.
  3. Persuaded, the agent calls the API, and no code checks the amount.
  4. The limit was a suggestion, not a control.

With the safeguard:

  1. The limit is enforced in the payment path, in code the model cannot change.
  2. The same email claims an $8,000 approval.
  3. The agent tries anyway. The gate holds the payment and asks a person.
  4. A person decides, and the agent cannot skip the check.

What it is

Spending limits, approvals or “never do this” rules live in a prompt, a UI checkbox or team habit rather than in code the automation has to pass through. So when the automation is confused, persuaded or simply wrong, nothing stops it.

How it happens

  • An AI agent or script holds credentials that let it act directly
  • Limits are written into instructions or checked only in the user interface
  • No server-side check requires a human to sign off above a threshold

What it causes

  • Refunds, payments or trades far beyond what anyone intended
  • Commitments made on the company’s behalf without authority
  • Deletions or permission changes that can’t be undone

Warning signs

  • High-value actions in the logs with no matching approval
  • Transcripts where the agent agrees to an out-of-policy request
  • A sudden jump in how fast actions are being approved

How to prevent it

Enforce limits and approvals in deterministic code. Put spending limits, kill switches and human approval for high-impact actions in a backend control layer that the agent, script or operator interface cannot skip.

  • Limits and approval thresholds are enforced in backend code, never only in a prompt or the UI
  • Actions above a risk threshold need an approval the automation cannot create for itself
  • The agent can propose a high-impact action, but a separate step with its own check executes it

Key ideas:Kill switchLeast privilegeHuman-in-the-loop approval

Check your system against the full launch checklist

Real incidents with this pattern

Documented incidents first. Each links to its public source.

Open in the database
Documented incidentIncident 2025-08-26AI agent

GitHub Actions injection leaks Nx's npm token; malicious packages use local AI coding tools to hunt for secrets (2025)

A pull-request title validation workflow used pullrequesttarget, which runs with repository secrets, and echoed the unsanitized PR title, creating an injection point; the workflow's token also had read/write permissions.

Safeguard: Never run untrusted pull-request input with secrets

Privileged action without an enforceable approval gatenx.dev
Documented incidentDisclosed 2023-06-07

SQL injection in MOVEit Transfer exploited by the CL0P ransomware gang (2023)

CISA and the FBI report that the CL0P ransomware gang exploited a SQL injection vulnerability (CVE-2023-34362) in Progress MOVEit Transfer beginning May 27, 2023, installing a web shell called LEMURLOOT.

Safeguard: Patch internet-facing transfer tools immediately

Known vulnerability left unremediatedcisa.gov
Documented incidentDisclosed 2021-12-10

Log4Shell: remote code execution through JNDI lookups in Apache Log4j (2021)

CVE-2021-44228 (Log4Shell) affects Apache Log4j 2.0-beta9 through 2.14.1 and lets an unauthenticated remote actor execute arbitrary code through JNDI lookups.

Safeguard: Maintain a software bill of materials and patch fast

Known vulnerability left unremediatedcisa.gov
Documented incidentIncident 2017-02-28

Mistyped input to a capacity-removal command takes down S3 in us-east-1 (AWS, 2017)

An engineer ran a command meant to remove a small number of servers from an S3 subsystem, but one input was entered incorrectly and more capacity was removed than intended.

Safeguard: Enforce minimum-capacity limits inside the tool

Privileged action without an enforceable approval gateaws.amazon.com
404 security advisories and illustrative scenarios with this pattern
Security advisory2026-10-16

Software component issue could reach internal services

Server-Side Request Forgery (SSRF) reported in the ticket sales chart PDF export functionality of Wegow. According to the report, an authenticated user with a promoter account can manipulate the parameter of the POST…

Privileged action without an enforceable approval gatenvd.nist.gov
Security advisory2026-10-16

Software component issue could allow unintended commands to run

The ELEX WordPress HelpDesk & Customer Ticketing System plugin for WordPress is vulnerable to Arbitrary File Upload in all versions up to, and including, 3.3.7 via the initMailPart function.

Privileged action without an enforceable approval gatenvd.nist.gov
Security advisory2026-10-16

Software component issue could allow unintended commands to run

The 10Web Booster – Website speed optimization, Cache & Page Speed optimizer plugin for WordPress is vulnerable to a limited file write via path traversal in versions up to, and including, 2.34.5 This is due to the…

Privileged action without an enforceable approval gatenvd.nist.gov
Security advisory2026-10-16AI system

Software component issue could allow unintended commands to run

The The Slim SEO – AI SEO Plugin, Lightweight, Fast & Automated plugin for WordPress is vulnerable to arbitrary shortcode execution in all versions up to, and including, 4.11.1 This is due to the software allowing users…

Indirect prompt injection changes agent behaviornvd.nist.gov
Security advisory2026-10-16

Software component issue could make the application process unsafe input

The Sunshine Photo Cart – Client Photo Gallery & Photo Proofing for Photographers plugin for WordPress is vulnerable to PHP Object Injection in all versions up to, and including, 3.7.4 via deserialization of untrusted…

Privileged action without an enforceable approval gatenvd.nist.gov
Security advisory2026-10-16

Software component issue could allow unintended commands to run

The Groundhogg — CRM, Newsletters, and Marketing Automation plugin for WordPress is vulnerable to arbitrary file deletion due to insufficient file path validation in the copyfile function in all versions up to, and…

Privileged action without an enforceable approval gatenvd.nist.gov
Security advisory2026-10-16

Software component issue could expose or alter stored data

The Vitepos – Point of Sale (POS) for WooCommerce plugin for WordPress is vulnerable to blind SQL Injection via the 'srcby[].val.start / srcby[].val.end (operator 'bt')' parameter in all versions up to, and including,…

Privileged action without an enforceable approval gatenvd.nist.gov
Security advisory2026-10-16

Software component issue could grant elevated privileges

The User Profile Builder – Beautiful User Registration Forms, User Profiles & User Role Editor plugin for WordPress is vulnerable to Privilege Escalation in all versions up to, and including, 4.0.3 via the role parameter…

Privileged action without an enforceable approval gatenvd.nist.gov

In the standards

CollectSEA's mapping; not an official determination by the standards bodies.