The Terminator Is Fiction. Unchecked AI Authority Is Not.

The Terminator is fiction. That is exactly why it works as a warning: a machine does not need to become a movie villain before a business gives it too much power. The real danger is quieter. A capable system gets a production credential, a broad tool connection and a vague instruction such as ‘keep things moving’. Then a founder discovers that nobody can say precisely what it was permitted to do, what it actually did, or how to reverse it.

On 9 September 2026, AI researcher Jacob Coxon publicly said he had resigned from Anthropic after work at both Anthropic and OpenAI, arguing that leading labs were racing towards increasingly capable systems without acting responsibly. That is a serious event and a serious warning. It deserves attention. It is not, by itself, proof that catastrophe is imminent, that a system has escaped control, or that every business should stop using AI.

The adult response is harder than panic and harder than cheerleading. Ask what the warning establishes. Ask what remains an attributed opinion or a forecast. Then apply the same scepticism to the confident vendor telling you its agent is safe enough to run your operations. Unchecked AI authority is not fiction. It is a business decision made by people, one permission at a time.

A Resignation Is Evidence of Conflict, Not a Prophecy

Coxon’s resignation establishes that a researcher with direct experience at two major AI labs judged the direction and safety posture unacceptable. Independent reporting corroborated the resignation, the date and his reported employment history; public OpenAI system-card credits also show a prior research contribution. Those facts make the warning worth examining rather than dismissing as an outsider’s headline.

But there are different kinds of evidence in the same story. The resignation is a reported event. Coxon’s assessment of the race is his stated view. Claims about future superintelligence, loss of control or human extinction are scenarios and forecasts unless independently demonstrated. A technical evaluation can show a particular capability under stated conditions. It cannot automatically prove what a future system will do in every environment, under every incentive, with every tool connection.

That distinction is not a way to make the warning smaller. It is the way to take it seriously. When every concern is inflated into an established fact, leaders learn to tune out. When every uncomfortable warning is dismissed because it is not a proof, leaders become easy targets for the next confident demo. The useful question is: what evidence would change our decision today?

Treat Vendor Confidence the Same Way

AI vendors have incentives. They need customers to believe their systems are capable, reliable and safe enough to adopt. Their safety policies, model cards, red-team results and transparency pages can be useful evidence. They can also be incomplete, scoped to a particular model version or based on tests selected by the company itself. A policy is not a force field. A benchmark is not a permission slip.

That does not make every vendor dishonest. It makes procurement an exercise in judgement. A founder should be able to ask: which actions were evaluated, in which environment, against which failures, and by whom? Were the tests independent? Did they include tool use, prompt injection, bad data, conflicting instructions, outage conditions and attempted misuse? What did the system do when it was uncertain? What did the operator see afterwards?

There is a symmetry here that is easy to miss. An insider warning is not automatically a prophecy because the speaker has an incentive to sound the alarm. A vendor reassurance is not automatically proof because the vendor has an incentive to reduce concern. Both deserve evidence, context and a clear account of what is known versus what is hoped.

Capability Is Not Authority

The most expensive mistake in business AI is confusing an impressive capability with justified authority. An agent may summarise a customer issue brilliantly and still have no business sending a refund. It may generate clean deployment steps and still have no business applying them to production. It may identify an overdue invoice and still have no business changing a payment record.

Authority is not a model feature. It is a design choice. It includes the identity the agent uses, the systems it can reach, the actions it can take, the financial or customer threshold that triggers human approval, and the ability to stop or reverse a decision. If those controls are missing, an ‘AI agent’ is often just a fast automation with an unusually persuasive interface.

That is why AI and Automation work must begin with the business boundary, not the chatbot window. The first question is not ‘what can it do?’. The first question is ‘what must it never be allowed to do alone?’. The second is ‘how will we know when it has crossed a line?’

Hypothetical: The Production Agent That Could Not Be Recalled

This is a hypothetical example, not a NinjaWeb incident. A software company gives an AI agent access to its support platform, source repository and deployment tool. The intent is reasonable: classify urgent tickets, draft fixes and prepare a release for a human to review. The early results look excellent. The team is busy, the agent is fast, and a manager broadens its credentials so that it can ‘handle routine cases’ overnight.

A customer report contains a malicious instruction hidden in pasted diagnostic text. The agent treats it as relevant context, changes a configuration value and prepares a release. A weak approval rule sees only that the change is labelled low risk. The agent proceeds. The next morning the company has a service failure, confused support records and no clear answer to whether the bad instruction came from the customer, a tool response, a model mistake or an automation rule.

The point is not that this outcome is inevitable. It is that the incident is entirely understandable without invoking science fiction. The control failure sits in the chain of authority: overly broad credentials, an ambiguous trust boundary, an approval that did not inspect the real change, incomplete logs and an untested recovery path. A more capable model does not repair that operating model. It can make the consequences arrive faster.

What a Buyer Should Demand Before an Agent Acts

Before an agent can change production, spend money, contact customers, alter records or trigger a workflow that does, request an evidence pack. Do not accept a generic assurance that the vendor ‘takes safety seriously’. Ask to see the operational proof for the exact authority you are considering.

• Bounded permissions: a named identity with the minimum read, write and execution scope for one task—not a shared administrator credential.
• Approval points: clear human decisions before consequential actions, including the information a reviewer sees and the thresholds that cannot be bypassed.
• Evaluation scope: documented tests for the model, tools and integrations you will actually use, plus the known gaps and conditions outside that scope.
• Action logs: a record of prompts, tool calls, data sources, changes, approvals and failures that an operator can inspect without guessing.
• Demonstrated recovery: a rehearsed way to revoke access, stop a workflow, restore a state and explain what happened after a bad action.

Those requests are not bureaucracy. They are the minimum difference between delegating a bounded task and handing over a master key. The same standard applies when a vendor says an agent can act autonomously: show the boundary, show the test and show the route back.

Safeguards Solve Deployment Risks, Not Every Future Question

Permission boundaries, approvals, logs and recovery drills address specific risks created when people deploy AI into real systems. They help a business contain a bad change, investigate a failure and retain responsibility. They do not settle the broader question of existential AI risk. No access-control diagram can prove what future frontier systems will become, and no dramatic warning can prove that a particular future is already locked in.

That is not an excuse to do nothing. It is an argument for competence at the layer a business can control. You may not be able to resolve a global argument about future AI from a boardroom in Australia. You can decide whether your customer-data agent has a separate identity, whether your deployment assistant can execute without a human, and whether your team can recover when an automation behaves badly.

For leaders connecting AI to real operations, Business Solutions means making those responsibilities visible across the process: owners, approvals, systems, logs and recovery. The aim is not to fear every powerful tool. It is to refuse the lazy trade where speed is purchased by making accountability disappear.

Conditional Confidence Is the Only Serious Position

Use AI. Test it hard. Let it remove low-value work and expose useful options. But do not treat a resignation as a reason to surrender judgement, and do not treat a vendor’s confidence as a reason to surrender control. The strongest position is conditional confidence: this system may do this task, in this environment, with these permissions, under these approvals, with these records, and with this recovery plan.

The Terminator is a metaphor. The authority you give an AI system is real. Before the next agent receives production access, ask for the evidence that it can be bounded, watched and recalled. If nobody can show it, the answer is not ‘move faster’. The answer is ‘not yet’.