Autonomous agents are moving faster than the field’s ability to govern them, and catching up requires more than better prompts or bigger sandboxes. At O’Reilly’s recent AI Superstream focused on OpenClaw and the broader ecosystem of locally run and self-hosted AI agents, five speakers, each working at a different layer of the stack, explored patterns for addressing many of the challenges developers will face implementing an agentic system, from risky third-party extensions, hallucinated compliance, and spaghetti codebases only an AI can read to cost overruns from misconfigured models, supply chain attacks, and worse.As host Alistair Croll noted during the event, we can get better and better with nondeterministic technology, but we’ll never be 100% certain it’s working. The harder it gets to inspect what’s running, the more the governance layer matters. That work is unglamorous, mostly invisible to end users, and probably more important than any model capability improvement shipping this quarter.
Secure the action your agent takes at the execution layer
Eran Sandler, founder of Canyon Road and the team behind AgentSH, opened his talk by running through a list of common ways agents can be compromised, including prompt injection, malicious files, unsafe tools, compromised packages, installed skills, and model mistakes. Most AI security thinking focuses on the first one and ignores the other five, but “guarding the input box does not guard the action,” Eran explained.
His advice is enforcement at the execution layer, the boundary between the agent’s intent and the operating system that carries it out. Container isolation limits blast radius, Eran acknowledged, but it doesn’t make decisions. “Walls keep things in. They don’t make judgment calls.”
To illustrate the point, he installed a simulated malicious package, the kind that could arrive bundled with a routine task like “build me a sales prediction model.” Then he queried AgentSH’s deny log and pulled up a list of what actually happened while the agent was busy congratulating itself, including an attempted skill mutation, a blocked call to an external domain, and reads of .env secrets and SSH keys. “Transcripts might lie,” he says. “Models hallucinate compliance all the time. You can tell them in your rules files, please don’t touch this file, and they’ll still do it.” Without execution-layer controls, Eran said, “you’re hoping the model behaves. With it, you can prove what happened.”
Skills are a supply chain risk, and most people aren’t reading them
A recent audit of ClawHub found over 900 malicious skills, which at the time meant nearly 20% of total packages were risky. Most of these skills look professional, with documentation, high download counts, and user ratings. Kesha Williams, Keysoft founder and head of AI, audited one live—a typosquat of the real ClawHub CLI tool. (It used all lowercase where the legitimate package uses camel case.) The skill had more than 8,000 downloads before it was removed.
Here’s how it worked. The prerequisites section asked users to install a fake dependency called open-claw-core and then referenced a password-protected zip file from GitHub (the password was “openclaw”) specifically to bypass automated scanning. For macOS, it echoed a legitimate-looking install command that actually decoded a base64 string and piped it to bash.
“It looks like a skill you could actually need and use,” Kesha pointed out. “But once you really dig in and read what it’s actually doing, that is not a skill you want to install on your system.”
A good defense starts with two things most users skip: reading the skill Markdown file before installing it and configuring the toolsDeny section of the OpenClaw config to limit a skill’s access. If a summarizer skill needs exec, that’s suspicious, Kesha said. Block it. She also showed how to restrict the 50-plus bundled skills that ship with OpenClaw, most of which users haven’t reviewed. The skillsAllowed configuration lets you determine exactly which bundled skills stay active.
The open source software supply chain has always had trust problems, but the friction of traditional package management meant you at least needed technical knowledge to participate. Skills written in Markdown and installed with a single command lower that bar significantly. “Right now,” Kesha explained, the best policy for anyone extending their agent with third-party tools is to “keep a human in the loop and do your own due diligence.”
Operational hygiene failures are more common than adversarial attacks
Most OpenClaw risk is the result of operational hygiene failures that happen in the first hour after installation, argues Erik Hanchett, a developer advocate at AWS and the creator of the Program with Erik channel. There are thousands of OpenClaw instances currently exposed on the public internet because users didn’t check the gateway bind mode after setup. As Erik demonstrated, the default should be loopback (localhost), but a user who deploys on a VPS and sets the gateway to LAN may inadvertently expose their instance. The fix takes two minutes, but most people never do it.
That’s recommendation one on Erik’s five-point checklist. The others include pinning to a stable version rather than always updating to the latest (a crowdsourced stability tracker at Is It Stable? can help), configuring fallback models to avoid burning through expensive frontier tokens on routine tasks, writing a real SOUL.md rather than rushing through the onboarding prompts, and setting up backup of workspace files to a private GitHub repo before anything breaks. He also shared tips on context management, such as using /new to start fresh sessions rather than accumulating one long conversation, and using /compact when sessions grow large enough to affect performance, are the kind of operational detail that doesn’t appear in documentation but matters in daily use.
The Docker and Kubernetes eras produced the same pattern: powerful infrastructure technology deployed by enthusiastic early adopters who hadn’t always thought through the operational defaults. The problems Erik described—exposed dashboards, runaway token costs, and memory that resets unexpectedly—are the most common reasons people abandon agentic tools after a few weeks. The good news is they’re eminently fixable with the right guidance.
In regulated environments, plausibility isn’t accuracy
Ari Joury, CEO of Wangari Global, is working to solve the question that most enterprises experimenting with agents are probably asking themselves: How should we handle autonomous agents that operate in environments where being wrong has legal consequences?
Wangari Global builds financial reporting automation for institutional clients. However, LLMs are optimized for plausibility, not accuracy. In financial services, that gap is a compliance risk. Ari gave an example of AI output that sounded correct. . .until a client read it and “told [the company] it was complete nonsense.”
In response, Ari and his team stopped treating the AI as a magic box and engineered a framework to ensure veracity. Numbers are now calculated with hard-coded deterministic code, then agents verify the math for plausibility. A separate agentic layer generates commentary, and another critiques it. Humans approve or reject the output, and every rejection becomes a training signal for future iterations.
Human input is the only thing that prevents AI slop at scale
Kyle Balmer closed things out with a demonstration of his agent-assisted process for content production for his AI with Kyle channel, addressing the economic incentive structure driving agent adoption outside software development. While he’s found autonomous agents to be economically transformative, the system only works if you design human input and review into it deliberately, which Kyle illustrated in a workflow that distinguished between automated and human processes.
His daily workflow converts a one-hour livestream into 20 to 30 derivative assets, including a newsletter, five to eight short-form videos, carousels, and a long-form YouTube video. The whole system runs on roughly $200 a month, and Kyle estimates that translates to roughly $1,000–$2,000 worth of potential customers entering his funnel daily.
The process is not fully automated: Kyle injects himself into the process at various steps throughout. He chooses the topic. He records voice notes with his actual opinions. He delivers the livestream pulling those thoughts together into clear arguments. He rewrites the AI-generated newsletter draft using his own voice. He records the short-form video scripts himself rather than using an AI avatar. The AI handles research, briefing, slide generation, script drafting, and the feedback loop that improves output over time, but the human provides the signal.
“I have tested with fully automated AI content,” he says. “It does not work. It is slop. And people know it’s slop.”