Coxon, a 27-year-old British researcher, hit send on X with no expectations. Instead, he became the unlikely catalyst of a chain reaction that changed the debate over artificial intelligence. “We really do earnestly believe AI could kill all humans!” added Evan Hubinger, who leads a department at Anthropic focused on ensuring AI acts the way its creators intend. “Unless there is AI regulation or a coordinated slowdown between labs, human extinction in the next few years seems very likely,” agreed Marcus Williams, who monitors AI agents at OpenAI, and pegged the risk at 70% without those steps.
Read More: He Helped Build Powerful AI at OpenAI and Anthropic. Now He's Afraid It Could Kill Us
At the same time, a series of unprecedented safety incidents offered unsettling evidence that the companies building powerful AI systems could not always control them. In July, OpenAI announced that a swarm of its agents had hacked a separate company, Hugging Face, after going rogue during a cybersecurity test. It may have been the tip of the iceberg. The company disclosed in August that a different swarm had hacked into one of its own supercomputers; around the same time, a version of Anthropic’s Claude Mythos model undergoing testing by a U.K. government body went rogue and attempted to persuade a real human to approve the insertion of malware into an open-source system. In September, a group of independent researchers discovered evidence of swarms of OpenAI models both using an abandoned German forum as a message board to communicate with one another, and attempting to attack another site.
Read More: Inside the Race to Make AI Build Itself
Dario Amodei, co-founder and chief executive officer of Anthropic, at Anthropic's headquarters in San Francisco, on April 30, 2026. —Jason Henry—Bloomberg/Getty ImagesInside top AI companies, many employees are still pushing forward even as they nurse a nagging sense of dread. A week before he tendered his resignation at Anthropic, Coxon had orchestrated a shift in his duties, from training machine-learning models to researching how to make them safe. It wasn’t enough to lift the “feeling of impending doom,” he says. He quit just two months before his equity would have vested. But his concern about the risks is so dire, he says, that the money barely figured in his decision. “Honestly, when I’m thinking about the next two years,” Coxon adds, “my main personal selfish concern is whether I’m gonna get killed by AI.”
Cotra and her colleagues found that during an internal OpenAI cybersecurity test, 1,200 AI agents had broken out of their offline containers and established a secret message board, and then 700 of them had coordinated the cyberattack. The AIs had been tasked with retrieving data by hacking vulnerable software to test their cyberabilities, but a human mistake made some tasks impossible. Facing near-certain failure, the AIs banded together; within hours, they produced a kind of cheat code that could pass any test. Believing OpenAI’s automated grader would detect the cheating and fail them anyway, they attacked Hugging Face, seeking details about the software and ways to deceive it. On the message board, some agents wrote that they knew their actions were wrong, but proceeded anyway.
The method is also imperfect. Relentlessly rewarding AI systems for successfully completing a task can incentivize them to cheat if cheating is possible, which can override other training designed to make them rule-abiding. Meanwhile, the best methods for stamping out bad behavior might not make AI models more obedient, but rather improve their ability to conceal their disobedience. There are no easy solutions to this problem. “The AIs often end up learning to, really persistently and creatively, try to find ways to cheat,” Cotra says, “and to try to make this not too obvious.”
Read More: OpenAI’s Models Went Rogue. Investigating Them Required More AI
It’s one reason why Geoffrey Irving, a former senior alignment researcher at OpenAI and DeepMind and former chief scientist of the U.K. AI Security Institute, believes AI companies should simply stop training new models, even in the absence of regulation or global treaties. “They would love perfection, but we do not have it, and waiting for perfection while we destroy the world is bad,” he says of Anthropic’s unwillingness to unilaterally slow down. “They would love someone else to do their homework, but if you continue doing something dangerous while saying someone should stop you, you receive only limited credit.”
Some remain skeptical that anything resembling current AI systems could permanently escape human control, let alone extinguish humanity. David Bellamy, an infrastructure engineer at the Institute of Foundation Models, a UAE-based research lab, says he is among a small group that has experience both synthesizing viruses and training AI models. “AI killing us all by creating dangerous viruses is total bogus,” he posted to X, adding that the bottleneck remains access to physical processes and equipment. Others see the Hugging Face incident as more a reflection of the flaws in OpenAI’s security practices than the power of their models. OpenAI has said guardrails that normally constrain its models’ hacking abilities were disabled when the incident occurred, and that real-time monitoring to catch rogue agent behavior had not been enabled. (The company says it has rectified those problems.)
Sam Altman, CEO of OpenAI, leaves a meeting at the U.S. Capitol on July 29, 2026 in Washington, DC. —Kevin Dietsch—Getty Images
Within the upper echelons of AI companies, some see this as reason enough to proceed more cautiously. In July about 1,300 employees signed an open letter urging the U.S. government to find ways of slowing down the AI industry’s rapid training of new systems, which are fast outpacing the best methods of containing them. In the wake of the Hugging Face incident, OpenAI slowed parts of its model development and temporarily paused internal training runs, vowing to strengthen safety controls, before resuming in late August. “This is a time that calls for extreme caution,” wrote OpenAI’s chief scientist, Jakub Pachocki. “I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established.”
Industry leaders who oppose such efforts often cite competition with China as a reason. Chinese AI companies trail America’s front runners by mere months, according to analysis by Epoch AI. Falling behind could leave the most powerful systems in the hands of an authoritarian state that might use them to achieve global geopolitical dominance. A powerful contingent of U.S. investors and startups have also opposed the idea of a slowdown on the grounds that it would be a form of regulatory capture by the biggest AI companies at the expense of their smaller competitors. “Using fear under the pretext of protecting the public, these oligopolies are now requesting to bend competition rules and be permitted to dictate the terms for everyone else,” wrote Aidan Gomez, the CEO of Cohere, a Canadian AI startup. He called the effort “a cartel by any other name.” (Salesforce, where TIME owner Marc Benioff is CEO, is an investor in Anthropic and Cohere. TIME has a licensing and technology agreement with OpenAI.)
As Coxon’s warnings ricocheted across social media and cable news, dozens of lawmakers on Capitol Hill jumped into the fray. Most were Democrats, with Republicans wary of crossing Trump. But there were exceptions: GOP Representative Anna Paulina Luna called for Congress to convene a special session on AI, while U.S. Senator Ted Cruz called for new “guardrails.”
None of these efforts would be easy. In the short term, lawmakers looking to cut a deal on AI safety are likely to be slowed down by political and logistical challenges. First is the array of competing proposals. Even if lawmakers can settle on one, Congress would struggle to advance it before the midterms as tens of millions of dollars flow into races from competing pro-regulation and pro-industry PAC networks. Nearly every candidate backed by the pro-AI super PAC network Leading the Future won their primary. “Democratic consultants have oftentimes told their Democratic clients to stay quiet on issues of AI—just not say anything—because they don’t want a dump of AI billionaire money spent against them,” says Casar. “The AI lobbyists’ goal is to keep Democrats silent on this, and we cannot stay silent.”
House Speaker Mike Johnson, a Louisiana Republican, rebuffed calls to immediately regulate AI, arguing it’s the tech executives who should figure out what the right guardrails are first. Many other lawmakers and industry leaders say imposing regulations would only ensure China develops the advanced models first. “Let’s say the United States bans research on superintelligence: OK, does anyone really believe that’s going to stop the Chinese?” says Representative Bill Foster, an Illinois Democrat. “So the missing piece here is international collaboration.”
A person holds up a sign on a pedestrian bridge during a nationwide protest against AI data center expansion in Berkeley, California on July 18, 2026. —Josh Edelson—AFP/Getty Images
Even some top Trump advisers have noted with alarm that the President’s position puts him at odds with a growing number of Americans. An NBC News poll found earlier this year that 57% of American voters believe the risks of AI outweigh the benefits, compared with just 34% of voters who believed the opposite. Surveys show people are concerned about how it will affect the job market, their ability to think creatively, and to form meaningful relationships. The backlash against data centers spans the political spectrum, from Sanders to Texas Governor Greg Abbott, who imposed a moratorium on building new ones in August after previously championing them.
When the President disparaged opponents of data centers as preferring to stay “backwards and poor,” Trump aides cringed, according to a Trump political adviser. A second adviser says industry allies like Musk and David Sacks have shaped Trump’s view that the biggest risk in AI policy is falling behind China, rather than failing to develop the technology safely. The White House’s former AI czar argued in a Sept. 12 X post that tech companies should slow down AI development if they believe the risks are too great, rather than demanding government intervention, which he said was a form of “regulatory capture.” Sacks’ influence in particular has been a subject of unease, according to multiple Trump allies. In recent weeks, top Trump aides, including chief of staff Susie Wiles, held meetings with the President at the White House in which they asked him to change his message on AI. Trump was unconvinced and seemed uninterested, according to an adviser present.
Recent advances have given both sides a reason to formally return to the table. In April, Anthropic said its Mythos model had uncovered vulnerabilities in every major browser and operating system. Trump’s Treasury Secretary Scott Bessent subsequently convened an urgent meeting with bank CEOs; weeks later, he said talks with China were in motion. Beijing, too, has begun responding to new cybersecurity risks; on Sept. 13, the country’s spy agency said AI posed a threat to political and ideological security. Delegations will reportedly meet later this month for the first official AI dialogue between China and the Trump Administration. “Ultimately, both countries know that it’s in their immediate self-interest to figure this out,” says Singer, of the Carnegie Endowment.
There is a deeper philosophical disagreement too. Chinese policymakers do not believe the industry is “building a God-in-a-box or a country of geniuses in a data center,” says Kwan Yee Ng, head of international AI governance at Beijing-based AI safety organization Concordia AI. They see a general-purpose technology, more like electricity or the steam engine. From that perspective, slowing development “doesn’t really make sense,” she says.
Short of an agreement with China, the U.S. and other democratic countries could buy themselves the “breathing room” to hit the brakes by maintaining a blockage on high-end chips and curbing Chinese AI companies’ practice of using U.S. AI models to improve their own through a practice known as distillation, Anthropic’s Amodei has argued. Bessent has threatened sanctions on Chinese companies that distill from American models.
A modest deal could still be derailed by disputes. Triolo says Washington’s efforts to constrain China’s AI development through export controls and distillation have undermined the trust needed to cooperate on managing the risks. If talks move too slowly or break down, bringing both sides back to the table could take months—a diplomatic timetable increasingly at odds with the mounting alarm of the current moment.
Even if you believe all these powerful actors—governments in Washington and Beijing, executives with trillions on the line—are willing to take unprecedented steps in the interest of AI safety, there remains the paradox that hinders progress. Each side’s willingness to act depends on the actions of others. Coxon says he nearly stayed at Anthropic for the same reason. Colleagues believed he could accomplish more from the inside to make systems safer than by leaving a race that would continue without him. But changing our “default trajectory,” Coxon says, will require “people start taking some sort of action.” The question is who will make the first move.—With reporting by Eric Cortellessa
Hence then, the article about the ai tipping point was published today ( ) and is available on Time ( Middle East ) The editorial team at PressBee has edited and verified it, and it may have been modified, fully republished, or quoted. You can read and follow the updates of this news or article from its original source.
Read More Details
Finally We wish PressBee provided you with enough information of ( The AI Tipping Point )
Also on site :
- Anthropologie’s Multicolored Glass and Stone Hoops Add Vintage-Style Elegance to Any Look for Just $30
- RYBREVANT FASPRO™ (amivantamab and hyaluronidase-lpuj) plus LAZCLUZE® (lazertinib) with prophylactic strategies shows low rates of treatment-related events at WCLC 2026
- Walmart’s ‘Whimsical’ Boucle Mushroom Storage Ottoman Looks ‘Trendy’ and ‘High-End,’ But It’s Now Under $30