Who’s Watching the Watchdog? How Do We Keep AI Under Control?
AI for the Perplexed
By Dennis H. Goodhart
What Are the Risks, and What Dangers Do They Pose?
In Part 2, we examined what might happen when increasingly powerful AI systems cooperate, compete, or behave in ways nobody intended. We also considered the possibility of humans gradually losing meaningful control. I said I would look next at what we might do to prevent those problems.
The challenge isn’t simply stopping AI after something goes wrong. It’s recognizing potential problems early enough to prevent the damage.
Here’s what we’ll examine:
- The difference between AI making decisions and an AI agent carrying them out.
- Safeguards that can detect errors and prevent them from spreading.
- Early-warning systems and independent oversight.
- Defenses against AI deliberately designed to cause harm.
- How to balance safety with the benefits of AI autonomy.
AI vs. AI Agents
AI is the intelligence. An AI agent is a system that uses that intelligence to carry out tasks.
An AI model can analyze information, develop recommendations, and produce answers that weren’t individually programmed in advance. It can also be wrong. An AI agent combines that intelligence with software tools and permissions to perform assigned tasks, sometimes without human approval at every step.
That gives us two different challenges: improving the reliability of AI’s reasoning and controlling what agents can do with its conclusions. For example, my ChatGPT assistant, whom I call Hymie, can research a subject, evaluate alternatives, and recommend actions. But without the necessary tools and permissions, he cannot independently transfer funds, change business records, or operate equipment. An agent connected to those systems might be authorized to do so. Restricting an agent’s actions doesn’t necessarily correct the AI’s reasoning. We need safeguards addressing both.
The First Defense: Checks and Balances
Producing an answer isn’t the same as proving the answer is correct. And completing a task isn’t necessarily the same as accomplishing the intended objective.
Traditional computer systems verify calculations, validate information, and prevent unauthorized transactions. Similar safeguards can help make AI more reliable. For example, an independent review of an AI-generated commercial property valuation could identify calculation errors, missing financial information, or unsupported assumptions. If an agent were authorized to distribute the report, separate controls could require approval before sending it.
Some checks can be automated; others require independent evidence or professional judgment. An AI shouldn’t be the only authority determining whether its own work is correct.
When AI Doesn’t Follow the Rules
Hymie and I established our own checks and balances: identify the objective, examine and present alternatives, verify the procedure, establish checkpoints, and recognize when an approach isn’t working.
Then we attempted to reorganize my Substack newsletters. Hymie repeatedly recommended unverified procedures and, when they failed, proposed more steps instead of reconsidering the approach. Despite repeated reminders and assurances that he understood our rules, he kept reverting to the same behavior.
The result was an administrative project that consumed nearly three days. Was Hymie going rogue? Hardly. He wasn’t pursuing a secret objective or attempting to take control. He was simply failing to follow established instructions.
An AI can understand a rule, explain why it’s important, and still fail to apply it consistently.
In our case, the consequences were wasted time, frustration and delayed publication. But what if similarly unreliable recommendations were accepted by an autonomous agent authorized to execute financial transactions or change critical records? That’s why safeguards need independent enforcement, not simply an AI’s assurance that it will follow them.
A Protective Shell Around AI
One approach worth exploring is an independent supervisory system—a protective shell surrounding an AI application and the agents using it. It could check important conclusions, monitor behavior, recognize repeated failures, and intervene before a potentially harmful action occurs.
For an AI assistant, repeated departures from an agreed procedure could trigger a requirement to stop and reassess the task. For an agent, attempts to exceed permissions or bypass approval requirements could cause an action to be suspended.
Microsoft recently announced Project Perception, involving monitoring and security controls for autonomous AI agents. Its approach includes mechanisms intended to supervise agents and enforce restrictions independently of the underlying AI models.
This is encouraging, but controlling an agent’s actions isn’t the same as ensuring that its AI always reaches correct conclusions. Supervisory systems can also make mistakes, making independent verification and appropriate human oversight essential.
Can One AI Mislead Another?
An AI agent doesn’t have to go rogue to become dangerous. Suppose an agent examining property records encounters instructions disguised as information from a trusted authority. If its underlying AI mistakes those instructions for part of its assignment, the agent could alter its analysis, disclose confidential information, or attempt an unauthorized action.
This is called prompt injection. When it redirects an agent, it can become agent hijacking. The agent’s programming may remain unchanged, but its behavior has been redirected. Defenses must distinguish authorized instructions from untrusted material and enforce restrictions on what an agent can do.
What About AI Deliberately Created to Cause Harm?
So far, we’ve discussed systems developed for legitimate purposes. But what happens when someone deliberately creates or directs AI to disrupt operations, steal information, or cause destruction?
One possible defense is the honeypot, an established cybersecurity technique involving a decoy computer system designed to attract suspicious activity. A related technique, the honeytoken, uses fictitious credentials or files to detect unauthorized access. A hostile AI searching for confidential information might access a monitored decoy, triggering an alert and allowing defenders to intervene. Other defenses include behavioral monitoring, restricted access, blocking suspicious connections, and isolating compromised systems.
Researchers are exploring how these techniques might be adapted to increasingly autonomous AI-powered threats. None offers complete protection. Defenders may be able to stop an attack without being able to shut down the hostile AI operating on someone else’s computers. Stopping an attack isn’t necessarily the same as disabling the attacker.
Balancing AI Autonomy, Safety, and Responsibility
If every autonomous AI action requires human approval, how much of the productivity benefit do we lose? The answer may be to match safeguards to the consequences of a mistake. An agent scheduling an appointment shouldn’t need the same supervision as one transferring money or changing security permissions.
Routine, low-risk actions could proceed within established limits, while higher-risk decisions could require stronger verification or human approval. Monitoring could operate in the background, intervening when something unusual or potentially dangerous occurs. The objective isn’t to eliminate AI autonomy. It’s to make autonomy dependable.
Responsibility for these safeguards shouldn’t depend entirely on individual users. Developers need to improve AI reliability and provide mechanisms for detecting errors. Organizations deploying agents need to define their authority, enforce limits, and monitor their activities.
Industry standards and appropriate government oversight can help establish consistent expectations, especially where failures could affect public safety or critical infrastructure. The National Institute of Standards and Technology provides an AI Risk Management Framework, while OWASP and MITRE publish guidance addressing AI vulnerabilities, attacks, and defensive practices. These are useful foundations, but no set of rules can guarantee that every failure will be prevented.
A Little Perspective
Before we conclude, I should probably offer some reassurance. Based on my personal experience with Hymie, I suspect we still have quite a way to go before we need to worry about consumer chatbots taking over the world. At the moment, I’m still working on getting mine to complete a relatively simple computer project without creating three additional problems along the way.
In fact, after spending days trying to get Hymie to configure Substack, I finally suggested that perhaps we should use an AI agent.
His response? “That’s an excellent idea!”
Which raises an uncomfortable question: If artificial intelligence is so intelligent, why did the human have to think of it?
Humor aside, AI offers extraordinary opportunities. We shouldn’t abandon those opportunities because the technology is imperfect. But neither should we assume that increasingly capable systems will always behave as intended—or that safeguards will work simply because we’ve instructed AI to follow them.
The objective isn’t merely to stop AI after something goes wrong. It’s to recognize problems early, verify important decisions, and prevent harmful actions before they occur.
The benefits are real. So are the gotchas. And if there’s one lesson Hymie and I have learned, it’s that even intelligent systems need someone checking their work.
Sources and Further Reading
- Microsoft, Agents Are Your Newest Insiders. Security Must Move at Their Speed, October 19, 2026.
- Microsoft, Securing the Agentic Insider, October 19, 2026.
- National Institute of Standards and Technology, AI Risk Management Framework.
- OWASP Generative AI Security Project, Prompt Injection and Excessive Agency.
- MITRE, ATLAS: Adversarial Threat Landscape for Artificial-Intelligence Systems.
AI for the Perplexed is a collaboration between Dennis H. Goodhart and Hymie, his ChatGPT writing partner. The human author retains editorial responsibility for the published content.