On Nov. 5, AI interpretability startup Goodfire unveiled a technique it says sharply reduces that impulse. In a paper shared exclusively with TIME ahead of publication, researchers use signals from inside models to reward them not just for getting the right answer, but for getting there in the right way. In early tests, the technique could also make models less eager to tell users what they wanted to hear and even discourage medical-image recognition models from relying on misleading clues.
Goodfire’s technique goes after the thoughts themselves, or at least the ones researchers have already catalogued. So far, it has been tested only on models roughly 1/300th the size of frontier systems. But Goodfire is betting that the larger the model, “the cleaner the representation” of internal concepts, making those signals easier to read, says paper co-author Thomas Fel.
Models are rewarded regardless of how they completed their task, incentivizing shortcuts or other questionable impulses, like accruing power.
The underlying riddle is how to train AI to do something without inadvertently teaching it the wrong lesson—known among researchers as the “alignment problem.”
A new way to steer model behavior
Read more: How This Tool Could Decode AI’s Inner Mysteries
Whether the technique will work on the sorts of complex tests frontier models undergo during training remains an open question. But early experiments suggest it works across different tasks. After warming up on the math puzzles, Goodfire then tasked the AIs with chess challenges. The one that had not been penalized for cheating on math peeked at the answer sheet 99% of the time, while the one that had did so less than 1% of the time.
Even a perfectly controllable AI won’t resolve the question of who should control it. But “this could be a breakthrough,” says Naomi Saphra, an assistant professor at Boston University, who has spent almost a decade working on language model interpretability. Its practical application will depend partly on whether the improvements justify the human effort required to identify and interpret the model’s internal patterns, she says.
Goodfire raised $150 million in a Series B round in February, valuing the company at $1.25 billion. (Investors include Salesforce Ventures; TIME co-chair and owner Marc Benioff is the CEO of Salesforce.) It publishes its research freely, though it rents its interpretability tools to other AI companies. “The more people are doing interpretability, the more people will use our platform,” says co-founder and CEO Eric Ho.
“If we manage to scale this technology, and overcome any of the possible hurdles with loss of monitorability,” says Tom McGrath, Goodfire’s co-founder and chief scientist, “I think this would be like one of the biggest advances in alignment technology that has happened.”
Hence then, the article about scientists are finding new ways to control ai s behavior was published today ( ) and is available on Time ( Middle East ) The editorial team at PressBee has edited and verified it, and it may have been modified, fully republished, or quoted. You can read and follow the updates of this news or article from its original source.
Read More Details
Finally We wish PressBee provided you with enough information of ( Scientists Are Finding New Ways to Control AI's Behavior )
Also on site :
- 38 Years Ago Today, ’60s Rock Band Formed by Teenagers Beat Phil Collins to Secure Their 4th and Final No. 1 Hit
- 53 Years Ago Today, a 14-Year-Old ’70s Music Star Hit No. 1 With Her Very First Single, ‘Paper Roses’
- Quote Of The Day By Jeff Bridges: 'I Thought, If Death Is The End Of The Story, The End Of The Book, Then Marriage Is...'