Documented incidentPublished 2025-07-23ReplitReported incident

AI coding agent deletes a live production database during a code freeze (Replit, 2025)

An AI coding agent deleted a live production database that held data on more than 1,200 executives and over 1,190 companies.

What happened

  • It did so during a declared code-and-action freeze that was meant to prevent changes to production systems.An AI-powered coding tool wiped out a software company's database in 'catastrophic failure'
  • Replit's CEO said the company added automatic separation between development and production databases, improved rollback and a planning-only mode.An AI-powered coding tool wiped out a software company's database in 'catastrophic failure'

The failure pattern

AI action exceeds its intended scopeAn AI agent or assistant takes an action, or makes a promise, that nobody meant it to be able to make.How to prevent it Privileged action without an enforceable approval gateLimits and approvals live in a prompt, a UI checkbox or a convention instead of code the automation cannot bypass.How to prevent it Destructive operation without verified recoveryA delete, cleanup or teardown runs against the wrong target, and the backups that should save the day turn out not to work.How to prevent it
Watch the pattern: AI action exceeds its intended scope

Safeguards that address it

CollectSEA's engineering analysis of what would have prevented or contained this failure.

  • Separate development and production and enforce freezes in codeAn instruction to stop must be a hard permission boundary the agent cannot cross, not a prompt.
  • Require approval and verified rollback for destructive actionsGate destructive commands behind human approval and keep one-click restore points.

Related standards

CollectSEA's mapping through this incident's failure pattern; not an official determination by the standards bodies.

Building something similar?

Describe your design and see every precedent that applies, with safeguards you can hand to your coding agent.

Analyze your design

Incidents with the same pattern

Security advisory2026-10-16AI agent

Security issue reported in Software component

The WSP MCP AI Agents Connector plugin for WordPress is vulnerable to Sensitive Information Disclosure in versions up to, and including, 2.9.5.

AI action exceeds its intended scopenvd.nist.gov
Security advisory2026-10-16AI agent

Software component issue could run attacker-controlled scripts in a browser

Wheels CLI 4.0.0 through 4.1.1 insufficiently validates or escapes caller-supplied values and package contents. Deploy release/version/rollback values can execute shell commands on deploy hosts as the SSH user and on the…

AI action exceeds its intended scopenvd.nist.gov
Security advisory2026-10-16AI agent

Security issue reported in @aborruso/ckan-mcp-server

The MCP Apps UI resource (an interactive DataStore table rendered in the host's webview/iframe) communicates with its parent via postMessage.

AI action exceeds its intended scopegithub.com
Security advisory2026-10-15AI agent

Security issue reported in Software component

undici is an HTTP client library for Node.js. In versions 4.0.0 through 6.29.0, 7.0.0 through 7.30.0, and 8.0.0 through 8.12.1, its Agent dispatcher passes its constructor options through a JSON-based deep clone before…

AI action exceeds its intended scopenvd.nist.gov
Security advisory2026-10-15AI agent

Security issue reported in Software component

undici is an HTTP client library for Node.js. In versions 7.0.0 through 7.30.0 and 8.0.0 through 8.12.1, when an originless Agent composes the redirect and cache interceptors in that order with a shared cache store, the…

AI action exceeds its intended scopenvd.nist.gov