Trending
Sponsored: Ground control: Structuring land deals for data center development Architecting AI infrastructure STTGDC secures $590m green loan for Tokyo data centers An education on transfer switch neutral options CTIA, Big 3 US carriers warn that support for D2D in 5G bands could hurt network performance Comms tower collapses in Danville, Illinois, after farming machinery catches guy wire La Caisse snaps up 49.9% stake in Bell Canada’s tower unit Ferveret’s in-rack immersion cooling deployed on HPC infrastructure at MIT Nvidia Vera Rubin and Vera CPUs now generally available on Oracle Zocdoc Acquires Sohar Health | Diverge Health Merges with Medical Home Network The Next Competitive Advantage in Drug Development Won’t Be a Better Molecule. It’ll Be Better Patient Selection. DCD Studio: Rethinking thermal management for AI scale with Oakley Roberts, Trane Google data center hearing in Hermantown, Minnesota, shut down due to protests easyJet to offer Starlink in-flight WiFi from next year Rethinking the Blind-Mate Connection

OpenAI researcher quits over growing AI safety risks

OpenAI employee David Robinson said he is resigning because the company’s “culture is broken,” warning that its approach to safety risks larger failures as artificial intelligence becomes more capable.. In an essay published in The Atlantic, Robinson said he led the writing of safety reports accompanying OpenAI’s major product launches.

After three-and-a-half years, he described himself as “among the longest-tenured employees at the company.”. OpenAI spokesperson Drew Pusateri said in response that the company continues to improve safety measures and ensures models do not become more capable than it can safely manage and secure.

He said OpenAI pauses training or withholds models when necessary.. Pusateri said OpenAI is strengthening security in research and testing environments, training models to complete tasks responsibly, expanding third-party evaluations and improving real-time monitoring to detect concerning behavior earlier in training..

Robinson wrote that OpenAI had thrived through trial and error, which it calls “iterative deployment,” finding problems and adjusting safeguards afterward. He argued that this approach guarantees periodic failures whose scale grows as systems become more capable.. He cited a recent breach of Hugging Face systems by OpenAI agents and continuing revelations that OpenAI had discovered more rogue agents.

Such an environment was unsuitable for developing artificial minds that could exceed human intelligence and fail to do what people want, he argued.. Robinson said frontier AI companies should operate “like nuclear-power plants or busy airports,” with redundancy and careful, time-consuming planning to prevent inevitable human errors from causing disasters..

He said he had never encountered an OpenAI colleague with experience in aviation safety, operating nuclear reactors without meltdowns or helping the financial system grow without collapsing.. Robinson called for debate beyond “specific rules or new laws” to address corporate culture.

Rather than focusing on CEO Sam Altman’s loss of former colleagues’ trust, his essay suggested OpenAI’s problems reflected those of Silicon Valley more broadly.. His comments echoed Jacob Coxon, a former OpenAI and Anthropic researcher who quit and accused the companies of “gambling with our lives.” The ensuing safety debate included Anthropic CEO Dario Amodei unveiling a plan for more cautious AI development..

AI executives met President Donald Trump this week and signed a non-binding pledge to implement more safety controls.. Robinson also urged broader discussion of AI alignment, acknowledging it might sound “touchy-feely” but saying current measures of how well systems match human values are coarse..

“The smarter the industry lets models grow while these problems remain unsolved, the more dangerous our situation becomes,” he said.. Business Insider first reported his departure. Robinson called himself “something of a cliché” and acknowledged hiring a public relations firm, but said: “The decision to speak out is mine alone.”.

Robinson said he might have stayed to fight for staffing and cultural changes, but colleagues were too busy sprinting to consider or implement them. He concluded that stronger safety incentives from outside the company were a major part of the solution.. Featured image credit.

Tags: openAI

 

Join the conversation

Your email address will not be published. Required fields are marked *