Trending
Kushagra Vaid appointed CEO of Omniva An education on transfer switch neutral options Verizon signs $2bn AWS AI infrastructure deal Ecoblox partners with Rimac on modular data center solution Microcapsules in the walls of data centers could help absorb server heat Google data center hearing in Hermantown, Minnesota, shut down due to protests Architecting AI infrastructure La Caisse snaps up 49.9% stake in Bell Canada’s tower unit easyJet to offer Starlink in-flight WiFi from next year Oracle will offer Google’s TPU AI chip via OCI The Next Competitive Advantage in Drug Development Won’t Be a Better Molecule. It’ll Be Better Patient Selection. Vodafone Ukraine to install solar arrays across 400 base stations Ferveret’s in-rack immersion cooling deployed on HPC infrastructure at MIT AirTrunk secures AU$4.25 billion green loan for Sydney data center campus SoleCrypt ups size of planned data center campus in Bizerte, Tunisia

OpenAI researcher quits over growing AI safety risks

OpenAI employee David Robinson said he is resigning because the company’s “culture is broken,” warning that its approach to safety risks larger failures as artificial intelligence becomes more capable.. In an essay published in The Atlantic, Robinson said he led the writing of safety reports accompanying OpenAI’s major product launches.

After three-and-a-half years, he described himself as “among the longest-tenured employees at the company.”. OpenAI spokesperson Drew Pusateri said in response that the company continues to improve safety measures and ensures models do not become more capable than it can safely manage and secure.

He said OpenAI pauses training or withholds models when necessary.. Pusateri said OpenAI is strengthening security in research and testing environments, training models to complete tasks responsibly, expanding third-party evaluations and improving real-time monitoring to detect concerning behavior earlier in training..

Robinson wrote that OpenAI had thrived through trial and error, which it calls “iterative deployment,” finding problems and adjusting safeguards afterward. He argued that this approach guarantees periodic failures whose scale grows as systems become more capable.. He cited a recent breach of Hugging Face systems by OpenAI agents and continuing revelations that OpenAI had discovered more rogue agents.

Such an environment was unsuitable for developing artificial minds that could exceed human intelligence and fail to do what people want, he argued.. Robinson said frontier AI companies should operate “like nuclear-power plants or busy airports,” with redundancy and careful, time-consuming planning to prevent inevitable human errors from causing disasters..

He said he had never encountered an OpenAI colleague with experience in aviation safety, operating nuclear reactors without meltdowns or helping the financial system grow without collapsing.. Robinson called for debate beyond “specific rules or new laws” to address corporate culture.

Rather than focusing on CEO Sam Altman’s loss of former colleagues’ trust, his essay suggested OpenAI’s problems reflected those of Silicon Valley more broadly.. His comments echoed Jacob Coxon, a former OpenAI and Anthropic researcher who quit and accused the companies of “gambling with our lives.” The ensuing safety debate included Anthropic CEO Dario Amodei unveiling a plan for more cautious AI development..

AI executives met President Donald Trump this week and signed a non-binding pledge to implement more safety controls.. Robinson also urged broader discussion of AI alignment, acknowledging it might sound “touchy-feely” but saying current measures of how well systems match human values are coarse..

“The smarter the industry lets models grow while these problems remain unsolved, the more dangerous our situation becomes,” he said.. Business Insider first reported his departure. Robinson called himself “something of a cliché” and acknowledged hiring a public relations firm, but said: “The decision to speak out is mine alone.”.

Robinson said he might have stayed to fight for staffing and cultural changes, but colleagues were too busy sprinting to consider or implement them. He concluded that stronger safety incentives from outside the company were a major part of the solution.. Featured image credit.

Tags: openAI

 

Join the conversation

Your email address will not be published. Required fields are marked *