OpenAI has a new safety team — it’s run by Sam Altman

Illustration: The VergeOpenAI is forming a new safety team, and it’s led by CEO Sam Altman, along with board members Adam D’Angelo and Nicole Seligman. The committee will make recommendations on “critical safety and security decisions for OpenAI projects...

Troov Troov
May 28, 2024 - 21:02
0 35
OpenAI has a new safety team — it’s run by Sam Altman

OpenAI is forming a new safety team, and it’s led by CEO Sam Altman, along with board members Adam D’Angelo and Nicole Seligman. The committee will make recommendations on “critical safety and security decisions for OpenAI projects and operations” — a concern several key AI researchers shared when leaving the company this month.

For its first task, the new team will “evaluate and further develop OpenAI’s processes and safeguards.” It will then present its findings to OpenAI’s board, which all three of the safety team’s leaders have a seat on. The board will then decide how to implement the safety team’s recommendations.

Its formation follows the departure of OpenAI co-founder and chief scientist Ilya Sutskever, who supported the board’s attempted coup to dethrone Altman last year. He also co-led OpenAI’s Superalignment team, which was created to “steer and control AI systems much smarter than us.”

The Superalignment team’s other co-leader, Jan Leike, announced his departure shortly after Sutskever left. In a post on X, Leike said safety at OpenAI has “taken a backseat to shiny products.” OpenAI has since dissolved the Superalignment team, according to Wired. Last week, OpenAI policy researcher Gretchen Krueger announced her resignation, citing similar safety concerns.

Along with the new safety board, OpenAI announced that it’s testing a new AI model, but it didn’t confirm whether it’s GPT-5.

Earlier this month, OpenAI revealed its new voice for ChatGPT, called Sky, which sounds eerily similar to Scarlett Johansson (something Altman even alluded to on X). However, Johansson then confirmed that she refused Altman’s offers to provide a voice for ChatGPT. Altman later said that Open AI “never intended” to make Sky sound like Johansson and that he reached out to Johansson after the company cast the voice actor. The entire affair left AI fans and critics alike concerned.

Other members of OpenAI’s new safety team include head of preparedness Aleksander Madry, safety head Lilian Weng, head of alignment science John Schulman, security head Matt Knight, and chief scientist Jakub Pachocki. But with two board members — and Altman himself — heading up the new safety board, it doesn’t seem like OpenAI is actually addressing its former workers’ concerns.