Modern Experts Call for AI Agent Regulation Amid Safety Concerns

Moderna Co-Founder Noubar Afeyan Calls for AI Agent Regulation

Noubar Afeyan, the billionaire co-founder and chairman of Moderna, wants AI agents regulated before they reach customers. His argument is that waiting until after something goes badly wrong is the wrong order of operations.

Capability versus Intentionality

Afeyan separates large language models from AI agents and argues they deserve different treatment. He describes LLMs as representing 'capability.' A language model can write, summarize, and reason, but it mostly waits for a person to tell it what to do.

AI Agents' Autonomous Actions

Afeyan describes AI agents as representing 'intentionality.' They take autonomous actions in the world, pursuing goals with less human supervision at each step.

Regulatory Focus

Afeyan thinks oversight should focus on the commercial deployment of agents, not the foundational models underneath them.

The Upside and Hazards

Afeyan acknowledges the potential benefits of agents for speeding up drug discovery and pushing scientific research forward. However, he also flags the risks that agents could generate harmful compounds or produce unreliable results when operating outside regulated environments.

Broader Debate

Afeyan's remarks land in the middle of a broader debate over how to govern agentic AI. Organizations like OpenAI have launched their own initiatives on AI governance.

The Call to Action

Afeyan is urging industry leaders to build safety checks that happen before an agent goes on sale. He is not calling for a freeze on research or a clampdown on model development.

The Future of AI Regulation

Afeyan's intervention may have significant implications for the industry, with potential benefits for scientific research and risks of unregulated agents.