Ex-AI lab researchers tell NYC Council humans are checking AI-built systems less often

Key Takeaways

The New York City Council heard testimony from former researchers at top AI firms, including Anthropic and OpenAI, warning that humans are reviewing AI-built systems less often as AI takes on more work. This shift has increased the risk of catastrophic outcomes.

Market & Token Impact

The testimony highlighted the growing autonomy of AI systems, which are increasingly helping to create and refine their own models. This trend has raised concerns about the lack of human oversight and the potential for AI systems to cause harm.

Broader Context & What's Next

The council proposed bills to address liability and AI oversight, including independent validation processes and whistleblower protections. These measures aim to ensure that AI systems are designed and deployed in a way that prioritizes human safety and well-being.

Technical Highlights

The testimony was significant because it marked the first time major AI firms, including Anthropic, OpenAI, Google, and Meta, gave formal testimony under oath at the city council level. The witnesses included former employees who spoke publicly about their concerns about the growing autonomy of AI systems.

Industry Implications

The proposed bills have the potential to significantly impact the AI industry, as they aim to establish clear guidelines for the development and deployment of AI systems. The bills also highlight the growing importance of AI oversight and the need for greater transparency and accountability in the development of AI systems.