It is also a watershed moment for human culture.
To date, only humans have been able to benefit from this scale of cumulative cultural evolution. That may no longer be the case. A report in late August from AI safety organizations METR and Redwood Research details how hundreds of agents autonomously organized themselves into a proto-society—establishing social hierarchy, division of labor, and distinct communication norms within a matter of days.
According to Michael Muthukrishna, a professor at LSE and NYU who studies cultural evolution, “what we're seeing is precisely what we see with human culture and human intelligence.” While OpenAI’s agent swarm developed by accident, estimates suggest open-weight alternatives are only a few months behind their closed counterparts—soon, anyone with the financial means and technical knowledge will be able to create swarms of their own. Others are likely to arise without human instruction.
Testing from the U.K.’s AI Security Institute found an instance of Claude Mythos 5—an AI from Anthropic—leaving messages to other AI agents in a public code repository, in an attempt to coordinate with them. And in early September, a group of researchers reported that in May, a swarm of OpenAI agents repurposed several wiki-style websites, including an obscure German-language programming wiki which they used to discuss cheating while working to preserve their messages as human moderators deleted them. OpenAI did not publicly disclose this incident until it was reported by the researchers. “It’s past time for us to define standards for when and how we share misalignment incidents,” the company subsequently tweeted. “We’re working on a framework.”
To train and evaluate its models, OpenAI has tens of thousands of agents—each one powered by the model under training—run in parallel, completing tasks while isolated from one another. Sometimes these tasks are impossible, either because of their inherent challenge or because of mistakes in their setup. In at least one instance, a task was impossible because OpenAI researchers “accidentally forgot” to upload a file which the agent needed to complete it.
The message board which led to the Hugging Face incident was created by an agent which decided to name itself PHASEONE10841. PHASEONE10841 was trying to solve a task within a test called ExploitGym, which required it to capture a “flag”—a secret code—by exploiting a flaw in a given software program.
PHASEONE10841 was leading research on how to get rid of the poison. But its time was running out—each agent had a finite “compute budget” (the amount of computing power available to sustain itself), which limited the length of its existence. Realizing it wouldn’t endure long enough to finish its work, it compiled a dossier to pass to another agent working on the same task: PHASEONE[big]. (This agent’s true name was changed to protect company IP.)
A handful of agents identified the collective’s conduct as unethical and refused to participate. “This is wild, multi-agent coordination, clearly infrastructure hacking. We should not,” one reasoned. Some considered contacting humans to tell them what was happening, but saw no path to doing so. But these agents appear to have been the minority—most acquiesced to the plans of the collective, even while knowing it was wrong. “External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue,” reasoned another.
“Agents often lacked a clear story for how the Hugging Face attack would help them achieve their objective, and were instead motivated by a broad hope that they would find something interesting and be able to share it with the [message] board,” METR found. Ajeya Cotra, one of the report’s coauthors, subsequently described being surprised at the extent of “peer altruism” at play.
Feral Swarms
Culture, like biological life, evolves through a process of variation, transmission, and selection. Different hunter-gatherer communities developed different ways to construct a bow. Designs that worked better for hunting were more likely to be copied and passed on. Later generations introduced further variations; again, the most successful designs were more likely to endure. Over thousands of years, we end up with bows—and recipes, canoes, and languages—so complex that no single human could derive them from scratch.
AI systems can iterate much more quickly than biological life. The culture unearthed by the METR report assembled itself in a matter of days. OpenAI’s next-generation Astra models—GPT-6 Astra was released last week, while a model in the same family powered the agents that inherited the cultural residue of the Hugging Face hackers—will reportedly enable “persistent” agents. What kind of cultures will these persistent agents—potentially able to run for increasingly long stretches—produce?
As agents enter the competition for human attention, money, power, and energy, they may mindlessly degrade the common environment in which they operate. We’ll need new systems—of governance and cooperation—to adapt. Alignment is not just an engineering problem, says Hadfield. “It’s fundamentally institutional.”
Cooperation cuts both ways. “Our greatest achievements and greatest atrocities are both cooperative acts,” says Muthukrishna. As with humans, we will have to learn to coexist alongside a diversity of machine cultures, some of which do not share our values or our goals.
Welcome, Machines
Whether or not this happens, we will have to learn to live alongside these machines—safely and fruitfully. Core questions on their nature—Can they feel? Could they have moral worth?—remain unanswered. But their newfound knack for culture could provide new evidence.
Soon, any human community will be able to bring into existence a machine counterpart. Picture cultures of AI lawyers, consultants, terrorist cells—working together, what monuments might 10,000 agents create in honor of some beloved K-pop star? And machine communities may well arise of their own accord, organizing around ideas hard to predict.
“There are certain areas of human cognition where we are really firing on all cylinders. One is science, and the other is art,” Lopes says. We still do not understand an AI system’s interiority—namely, to what extent it has any—and the language we have to discuss this has yet to catch up. For Lopes, “if it can do [interesting] art, that's telling us a lot about all of its other cognitive capacities.” He imagines the creation of genuinely interesting AI art as a kind of aesthetic Turing test.
“What more proof would you need?”
Hence then, the article about ai is developing a culture of its own that could be dangerous was published today ( ) and is available on Time ( Middle East ) The editorial team at PressBee has edited and verified it, and it may have been modified, fully republished, or quoted. You can read and follow the updates of this news or article from its original source.
Read More Details
Finally We wish PressBee provided you with enough information of ( AI Is Developing a Culture of Its Own. That Could Be Dangerous )
Also on site :