Anthropic's >10% AI extinction odds trigger UK Cabinet shakeup

For AI builders and researchers, this is not merely a policy story — it is an alignment story. An Anthropic safety researcher publicly stated the lab has no plan to solve superintelligence alignment, while another resigned over an industry race to self-improving systems. The PM's response signals that frontier labs may soon face harder external testing demands.

On 9 September 2026, the UK's AI safety debate crossed a significant threshold. Prime Minister Andy Burnham told the House of Commons that artificial intelligence poses "risks to national security" after Evan Hubinger, a top safety researcher at Anthropic, wrote on X that he personally assigns a greater than 10% probability to AI killing all humans within the next decade. Hubinger's statement was not an isolated comment: it came in direct response to his colleague Jacob Coxon's resignation, in which Coxon warned that the industry's race toward self-improving superintelligence is "gambling with our lives." The same day, Burnham confirmed that AI safety minister Kanishka Narayan had been elevated to a Cabinet-level role, a structural change the PM said had "lifted the level of the conversation about artificial intelligence within Government."

“

The Financial Times reported that Anthropic withheld its latest model from the UK's AI Safety Institute, one of the world's leading bodies for pre-deployment risk testing.

The convergence of internal dissent at Anthropic and a policy response in Westminster marks a new phase in the global conversation about frontier AI risk. Anthropic has long marketed itself as one of the most safety-conscious AI labs, making the public alarm from its own researchers particularly notable. Hubinger was explicit: "I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to." That admission undercuts the lab's public confidence and suggests that even the organizations most institutionally committed to AI safety are struggling to keep pace with the technical challenge of aligning systems more capable than their creators. For the UK government, the timing is difficult to dismiss. The Financial Times reported that Anthropic withheld its latest model from the UK's AI Safety Institute, one of the world's leading bodies for pre-deployment risk testing. The combination of internal warnings and reduced external transparency creates pressure on policymakers to treat engagement from frontier labs as neither sufficient nor reliable.

The elevation of Kanishka Narayan to Cabinet is more than symbolic. It places AI safety at the same table as traditional national security portfolios, signaling that existential risk from machine intelligence is now considered a first-order state concern rather than a niche policy issue. Burnham's framing to the Commons was carefully calibrated: he acknowledged the risk without endorsing the most alarming probability estimates, and he extended an offer to colleagues on all sides to participate in the safety discussion. That bipartisanship may be essential if the UK is to maintain credibility as a hub for AI governance while also pressing firms like Anthropic to submit models for independent evaluation. The fact that a US-based lab chose to withhold a model from the UK's safety institute raises immediate questions about whether voluntary testing regimes are viable at the frontier, or whether governments will need to establish mandatory access conditions for developers operating at the highest capability thresholds.