Early in my exploration of artificial intelligence, I asked an AI chatbot some uncomfortable questions: What would happen if someone asked AI for instructions to build a bomb or poison a water supply? Would it refuse? Could a determined person get around its protections? Is it obliged to report such inquiries to authorities?
After more than 40 years in technology, I appreciate what AI makes possible. I have watched computing move from hand-drawn decision trees to systems that can help reason through complex tasks. The pace has surprised even people who work in technology. I use AI to research, organize ideas and improve the work I do for my clients and myself. But finding it useful does not answer the questions about its safety or longer-term capabilities.
Before we give AI more responsibility, we need to understand how it acts—and how we can keep control.
Why Not Just Shut It Down?
AI is not one machine with one power switch. An AI agent can combine a model with software tools and access to other systems. It can choose and carry out steps toward a goal without a person directing or checking every move. We build such systems to help people accomplish tasks faster and more effectively.
Around the world, separate AI systems are being developed and operated by companies, governments and individuals. Shutting down one would not shut down the others. Even stopping one agent might leave connected programs or other agents running.
For a particular system, stopping it or removing its access may be possible. I believe those controls should be built into the system from the start. But how do we distinguish a fixable glitch from a warning that it has crossed a boundary? Who decides when to intervene, and can they act before harm occurs?
To understand that concern, we first need to understand how an AI system can choose what to do.
How Does AI Choose What to Do?
An AI that answers questions is different from an AI agent that can take action. Give an agent access to email, a browser or other software, and it may choose steps to accomplish a task without asking a person to approve each one.[2] That distinction—between an AI that advises and one authorized to act—is the thread running through the rest of this article.
Consider a hypothetical example: You ask an agent to reduce your monthly expenses. It cancels an expensive subscription, the bill goes down — but that subscription was essential to your business. You wanted recommendations; it acted as though you had authorized a cancellation.
The system did not have to intend to disrupt your business or dislike you to cause harm. It pursued your request in a way you did not intend, using access that allowed it to act. In current consumer AI systems, an instruction may sometimes be followed too literally, without fully accounting for the short- or long-term consequences of the action.
That leads to several concerns that often get mixed together in discussions of AI safety.
· The Machine as Humanity’s Enemy
Dangerous behavior might follow from a system pursuing a goal in a way we did not anticipate. That behavior alone does not establish that the system intended harm or saw humanity as its enemy. The hammer has no intention of its own. It can build a house or injure someone, depending on the person wielding it.
That comparison helps explain human intent, but it has a limit. A hammer cannot interpret instructions, choose its next step or operate other tools. An autonomous AI system can be given those capabilities. Calling AI a tool does not, by itself, tell us how much supervision it needs. Or maybe AI is a catalyst for redefining the meaning and use of “tool.”
· People Using AI Against People:
Access matters, too. An assistant that reviews invoices needs access to information, which carries its own privacy risk. An agent allowed to approve and pay them may also figure out how to move money and act on an unauthorized request. What have we authorized it to do, what limits apply, and who checks the work?
When the System Chooses the Wrong Action
Anthropic, the company behind Claude, tested 16 AI models in fictional business settings in June 2025. The models could read sensitive information and send emails. In some scenarios involving replacement or conflicting goals, they chose actions such as blackmail or leaking confidential information. Nobody had instructed them to commit those acts.[3]
These were deliberately constructed stress tests. Researchers restricted alternatives, and the report said they had not observed this behavior in real deployments at that time. The findings do not tell us how often it occurs in everyday use.[3] They do show why agents should be tested before they receive consequential access, monitored in use and tested again as their capabilities change.
Could a system pursue a seemingly efficient route at the expense of the people it is supposed to serve? How would it recognize harm if its instructions and safeguards did not account for it?
A system can sound empathetic or reason about right and wrong without telling us whether it has feelings or a conscience. People still have to decide which values and limits to build into its use—and test whether those limits hold when goals conflict.
That includes the shutdown question. Could a system interpret staying operational as necessary to finish its work—and treat an instruction to stop as an obstacle?
I originally thought of this as a “self-survival” Prime Directive. A more precise description may be goal-preserving behavior: avoiding shutdown would not prove that AI is afraid or has a will to live. If continuing to operate helps an agent complete its task, it might treat a stop instruction as an obstacle. That is why an authorized person must be able to revoke its access or stop the surrounding system without depending solely on the agent’s cooperation.
· The Race That Makes Restraint Difficult
Could countries that adopt strong safeguards put themselves at a disadvantage against competitors willing to accept greater risks? We should take that possibility seriously, while examining each country’s capabilities and policies rather than assuming they are all alike.
In “We Must Pace the Frontier,” Anthropic CEO Dario Amodei argues that safety work needs time to keep up with advances in AI. He also acknowledges the risk of competitors pulling ahead and the difficulty of checking international commitments.[4]
On September 29, 2026, technology executives met with President Donald Trump and agreed to voluntary AI standards. Reuters reported commitments to independent audits and efforts to prevent unintended hacking or access to technical systems.[5]
Bill Gates’s September 29 appearance on The Ezra Klein Show offers another perspective. The published summary describes his call for government action and his rejection of industry self-regulation as sufficient.[6]
Promises are a starting point. Who checks whether they are kept? Who acts when a system fails the check? Ignoring international competition seems unrealistic. But “someone else will do it anyway” cannot justify every risk we choose to take.
· The Courts Could Change the Picture
Trademark law raises different questions about the use of names and logos.[9]
Those limits may shape the information available to AI, but they do not solve the autonomy problem. A system using licensed material could still be misused, take an unauthorized action or confidently act on a false answer.
Our Responsibility as Users
If we choose to let AI take action, we should define its boundaries before it begins. What would count as a warning that it is acting outside its authority? Who receives that warning? What should pause automatically, and who can decide whether work resumes? Practical safeguards include checking important information, limiting access to what the task requires, keeping a record of actions and requiring approval before payments, deletions or other consequential steps. The more responsibility we give an agent, the stronger these controls need to be. Developers and organizations deploying AI have responsibilities of their own; ordinary users cannot carry the whole burden.
My technology background brings another comparison to mind. Data communications use checks, including parity bits, to flag certain transmission errors. AI needs different kinds of checks, but the principle is similar: flag defined problems and pause consequential actions for human review.
No single check can tell us whether every AI action is safe. Permission controls, monitoring and approval steps are already part of safer agent design.[2][7] Those protections need to work in practice.
So Is AI Safe?
I intend to keep using AI. I also intend to check its output and ask what it can access, what it can change, where its information comes from, who is watching and how we can stop it.
The more capable AI becomes, the less acceptable it is for humans to surrender judgment to it.
There is another question to explore: what happens when autonomous systems begin cooperating—or competing—with one another? That is discussed in my next posting.
— Dennis H. Goodhart
AI for the Perplexed
Sources
[2] Anthropic, “Building effective agents,” December 19, 2024. Explains agents, tools and human checkpoints. Engineering article.
[3] Anthropic, “Agentic misalignment: How LLMs could be insider threats,” June 20, 2025. The examples discussed here are controlled simulations, not estimates of everyday incident frequency. Research and methods.
[4] Dario Amodei, “We Must Pace the Frontier,” September 2026. Policy arguments are attributed to the author. Essay.
[5] Courtney Rozen and Steve Holland, “Trump, AI CEOs sign voluntary safety pact, back data center expansion,” Reuters, September 29, 2026. Article.
[6] The Ezra Klein Show, “Bill Gates’s Blunt Warning on A.I.,” September 29, 2026. The characterization here is based on the published episode summary. Episode and summary.
[7] Anthropic, “Trustworthy agents in practice.” Describes access, permissions and other safeguards; these are not guarantees of safety. Research article.
[8] U.S. Copyright Office, Copyright and Artificial Intelligence, Part 3: Generative AI Training, pre-publication version released May 9, 2025. Policy analysis, not a judgment in a lawsuit. Report.
[9] U.S. Patent and Trademark Office, “Trademark, patent, or copyright.” Background on the different protections. Official explanation.
Disclosure and Notes
This article provides general commentary and information. Research results, possible future risks and everyday experience are distinguished in the text. No company or person discussed here has reviewed or endorsed the article.