A panel organised by PauseAI UK brought together concerns about the manipulation of teenagers by AI chatbots, the approaching threat of climate-related disasters, data protection and the absence of adequate corporate and governmental accountability.
The event took place on 24 October at the HB Allen Centre.
Dame Annelise Dodds, the MP for Oxford East, told the panel that these dangers could not simply be ignored.
PauseAI UK describes itself as a civic movement seeking to prevent the development of superintelligent AI capable of threatening humanity.
Its central priority is avoiding human extinction resulting from the unchecked expansion of advanced AI systems.
Alex Rose, an AI researcher and research assistant at the Oxford University Institute for Ethics in AI, said PauseAI was not opposed to artificial intelligence.
The movement instead treats the technology as potentially presenting an extreme danger if development and regulation fail to keep pace.
Its proposed response is to suspend further progress temporarily until safe development can be ensured.
Rose said the opportunity to regulate superintelligent AI could disappear within the next year.
PauseAI UK wants AI companies subject to legal controls intended to avert catastrophe and accountable for deaths and damage already linked to the technology.
Government-led investigations were also being advocated, with the argument that companies cannot be expected to regulate their own operations effectively.
Laiba Rehman, an AI safety writer with CeSIA, a former employee and current PauseAI UK volunteer, argued that concern for the future of human meaning should motivate greater action.
She said humanity as a whole faced an existential danger and that many people remained unaware of its scale or failed to change their professional direction in response.
Toby Ord, a senior researcher at the Oxford University AI Governance Initiative and the panel’s keynote speaker, identified four ways advanced AI could threaten humanity.
These included an AI takeover that removes human control, a human takeover in which the technology executes a harmful instruction, AI-created bioweapons and gradual disempowerment as machines surpass the capabilities of human society.
Ord cited an academic consensus suggesting a 10% probability of human extinction.
His research focuses particularly on AI self-improvement and the increasingly automated process through which companies use the technology to develop itself.
He said systems might surpass the people operating them, including chief executives, and advance faster than effective regulation.
He compared the assessment of risks created by rapid AI automation with the mathematical modelling used to track the development of pandemics.
Without intervention by governments, automation could also expand at a dangerously accelerated rate.
Ord believed major AI firms would be unlikely to restrain development voluntarily because doing so could leave them behind competitors.
Antitrust legislation, including the Competition Act 1998, prevents such companies from coordinating a collective slowdown.
A direct government order, however, would make a coordinated suspension possible.
PauseAI UK is also seeking increased involvement from middle-power countries, including the UK, the Netherlands and Canada, during what it considers a critical year for global campaigning.
Ge-Wang expected middle powers to assume a substantially larger role over the coming year.
The organisation plans a march against AI extinction in London on 21 November.
Dodds anticipated that AI could produce significant benefits within the civil service, although she also expected political actors to exploit the technology.
Ord concluded that the scale of the risk outweighed the benefits.
Rehman said political decision-making remained too heavily focused on votes, even as concern about the dangers grew.
She maintained that greater public awareness of extinction risks could encourage politicians to give the issue greater priority.
Dodds also said access to frontier AI models had become a geopolitical instrument.
She said the US administration had placed pressure on Anthropic, whether directly or indirectly, and supported international coordination because regulation across several jurisdictions would carry greater weight.
She wanted frontier-model rules to be both internationally coordinated and effectively enforced.
Legislation addressing superintelligence has been introduced in both the United Kingdom and the United States.
The Artificial Superintelligence Bill in the UK and the Ban Artificial Superintelligence Act in the US have yet to become law and would face substantial political obstacles.
Passing either measure would require increased support from the executive and across party lines.
Even if enacted, both would apply only within their respective countries.
Benjamin Harack, co-founder of the Oxford Hardware AI Governance Lab, estimated that the United States was home to between 75% and 80% of the world’s AI companies.
With the US and China leading rapid AI development, he said an international regulatory agreement would be necessary for the next phase of AI safety.
Rehman said the eventual objective should be a treaty between the US and China covering specialised AI hardware and imposing strict export controls to preserve human oversight.
Such an agreement could provide additional time to determine how superintelligent AI might be developed safely, she said.