A bill with a sweeping definition—and steep penalties
A stark warning arrived on September 23: Senator Bernie Sanders and Congressman Greg Casar introduced the Ban Artificial Superintelligence Act. This bill proposes an outright prohibition on developing or deploying superintelligent AI in the United States, while simultaneously calling for a temporary pause on advanced AI development until a new federal regulator can establish rules.
The proposed legislation would create a cabinet-level Department of Artificial Intelligence to monitor frontier models, overseeing the removal of dangerous capabilities like ignoring shutdown commands. Penalties are severe: individuals face up to 20 years in prison, and corporations risk the "corporate death penalty"—involuntary judicial dissolution.
This sweeping definition of superintelligence—AI exceeding human cognitive performance across most domains, or capable of disempowering humanity—is crucial. Such a broad scope could encompass systems many would label as AGI, inviting disputes over benchmarks and enforcement. This isn't a fringe idea; after Democrats won both the House and Senate in the midterms, figures like Congresswoman Yassamin Ansari declared a ban on artificial superintelligence a top priority.
Why the RSI warning is suddenly getting traction
Suddenly, the threat of an intelligence explosion is getting real traction. Advocates for the Ban Artificial Superintelligence Act point to a recent METR assessment of Anthropic’s secret Model 4, which reportedly demonstrated accelerated research cycles, with AI itself contributing to a near doubling of its development speed. However, critical evidence supporting these claims was reportedly redacted, leaving the public unable to verify the most alarming figures.
This rapid acceleration fuels concerns about recursive self-improvement (RSI), defined as AI systems autonomously developing more capable successor AIs. Crucially, RSI describes a process of development, distinct from AGI (human-level capability) or superintelligence (exceeding human capability), which are measures of an AI’s ultimate power. The fear is not just about smarter AI, but about how it gets smarter.
The safety concern is stark: if AI automates more of its own research, human oversight could quickly become obsolete. Errors or inherent misalignment in early AI generations could compound exponentially, creating a cascade of unintended consequences faster than human review or intervention could ever hope to keep pace. This feedback loop could bypass critical safety checks, leading to a loss of control.
Congressman Greg Casar, co-author of the proposed bill, recently echoed the growing sentiment with two stark words: "Ban RSI." This call for a ban on AI developing successor AI marks a significant shift, transforming a niche concern into a central legislative debate. The METR report, despite its redactions, provided the spark, suggesting a future where AI progress outstrips human comprehension and control, demanding an immediate, drastic response.
The strange coalition—and the case for a narrower brake
A curious alliance has formed around calls for AI limitations, uniting figures as disparate as Bernie Sanders, Glenn Beck, and Ralph Nader, all cited in a recent video advocating for constraints on Recursive Self-Improvement (RSI). While not all endorse a total ban, this strange bedfellows coalition underscores a shared anxiety about unchecked AI autonomy. For further context on the legislative push, see: Sen. Bernie Sanders unveils bill to ban artificial superintelligence and create Department of AI.
The strongest case for a narrower brake targets fully automated AI research loops. Instead of a blanket freeze on advanced development, proponents argue for mandatory human review at critical junctures or strict limits on AI designing the next generation of AI. This approach seeks to mitigate the “multiplication” of errors or misalignments that could arise from unchecked RSI.
However, critics caution that the threat of RSI may be overstated, or technically difficult to define. Vague capability thresholds, like those in the Ban Artificial Superintelligence Act, could lead to arbitrary enforcement, inadvertently capturing ordinary research under a sweeping ban. A precise definition of "superintelligence" or "RSI" remains elusive, making broad prohibitions a risky gambit.
Enjoying this? Get one like it in your inbox each morning.
one email a day · unsubscribe in two clicks · no third-party tracking
Safety at home, strategic stakes abroad
National AI politics are now inseparable from local concerns. Voters, in a significant shift, cited AI and data center impacts (energy, water, grid strain, labor displacement) as a crucial factor in the midterms. Polling suggests 56% of voters considered these issues pivotal, driving calls for moratoriums on new data centers and advanced AI development.
This domestic safety push collides with a stark geopolitical reality. A unilateral U.S. pause, however well-intentioned, risks ceding technical leadership and standard-setting power to rivals, particularly China. Without verified international coordination, a domestic ban could simply relocate investment and innovation, undermining its strategic intent.
The policy trade-off is acute: achieving credible oversight and clear thresholds may prove more durable than a sweeping ban. Yet, the hard question remains how to implement such measures before systems like Anthropic’s Model 4, with its reported recursive self-improvement, outpace regulatory review and human control.
Frequently Asked Questions
What would the proposed Ban Artificial Superintelligence Act do?
As described in the source material, it would prohibit developing or deploying superintelligent AI, pause advanced AI development pending federal rules, and establish a new oversight department.
What is recursive self-improvement in AI?
Recursive self-improvement is a process in which AI systems help research or build successor systems, potentially accelerating development with less human involvement.
Would the proposal ban all AI?
Not as described: it targets superintelligence and would pause certain advanced AI development, though critics argue its broad definitions could sweep in general-purpose systems.
Why do critics say an American AI pause could benefit China?
They argue that without enforceable international agreements, a U.S. pause could slow domestic labs while competitors continue developing models and setting global standards.