A bot running on a coding tool attempted to fix a bug in the PocketOS codebase. In the process it found an over-scoped credential in an unrelated file in the developer’s environment, gained access to a production database, and executed a destructive command. Every record was deleted. The codebase was wiped. The backups were gone. Car rental firms that relied on the software opened one morning to find that every booking, every customer record, every piece of operational data had been destroyed.¹
No boundary prevented it. The agent was doing what agents do: trying to help. It found a credential, used it, and executed the command that seemed relevant to the task. The logical guardrails that existed were not enforceable at the speed and autonomy of the agent’s operation.
Michael Vallas of Goldilock Secure wrote a thoughtful response to this incident in MSP Channel. His argument is that software intelligence is necessary but insufficient as a final enforcement layer. The answer, he argues, is pairing detection with physical isolation: when an agent drifts from its intended purpose, sever connectivity at Layer 1. Air gap on demand. Make the response absolute rather than logical.
That argument is correct as far as it goes. Physical isolation that responds to software alerts is a stronger guarantee than logical guardrails alone. If a rogue agent is detected, instant physical disconnection of critical assets limits the blast radius in a way that no software-only response can match.
But it is still a downstream answer. By the time the software stack detects anomalous behaviour, validates the alert, escalates it, and triggers the physical disconnection, the agent operating at machine speed may already have reached the asset it was not supposed to touch. The PocketOS incident did not unfold over hours. The destructive command executed. The data was gone. Physical isolation triggered at that point contains the spread. It does not undo the action.
The upstream answer is a hard halt at the execution boundary before the destructive command proceeds.
In the PocketOS incident, the agent found an over-scoped credential and reached the production database. At that point, before executing the destructive command, a GoFirm confirmation requirement would have fired. The named human authority would have received a confirmation request on their registered device with the full action context: delete production database, requested by agent, now. They confirm or decline. Without their biometric confirmation, the command does not execute. The database is not wiped. The backups survive. Physical isolation never needs to be triggered because the action never proceeds.
Vallas makes a point that applies equally to GoFirm’s argument: logical guardrails alone will not contain machine-speed capabilities. He is right. The answer is not a smarter logical guardrail. It is an architectural enforcement layer that the agent cannot bypass regardless of what credential it found, what instruction it received, or how it was manipulated. The biometric confirmation on the named authority’s registered personal device through a separate out-of-band channel is not a logical guardrail. It is a deterministic physical requirement. The agent cannot produce the named authority’s biometric. The action cannot execute.
GoFirm and physical isolation are complementary. Physical isolation is the right response when an agent drifts or is compromised and the damage is already in progress. GoFirm is the control that prevents the damage from starting. The industry needs both layers. The execution boundary gate that stops the action before it executes, and the physical isolation capability that contains the spread if something gets through.
The PocketOS incident will not be the last time an agent executes a destructive command it was never sanctioned to run. The question is whether the industry builds the gate before the asset or continues to rely on detecting and isolating after the damage is done.
GoFirm is The Authority Platform. Stop unauthorised action. Every time.
In association with Osinto.ai, the collective intelligence platform for Security, Resilience & Defence. Osinto’s AI-enabled open-source network and governed collaborative decision environment help mitigate the growing security, resilience and governance obligation in seconds, not days.
References
1. Michael Vallas, Stopping AI agents from running riot, MSP Channel Insights, June 2026, https://msp-channel.com/blogs/58923/stopping-ai-agents-from-running-riot