Former OpenAI Safety Leader Joins Rival Anthropic
In a significant shift within the AI industry, Jan Leike, former co-head of OpenAI’s Superalignment team, has joined rival AI firm Anthropic. This move follows a series of resignations and criticisms directed at OpenAI’s evolving priorities, highlighting the increasing tension between commercial interests and AI safety.
Jan Leike, a prominent figure in AI safety research, announced his departure from OpenAI earlier this month. His resignation came on the heels of Ilya Sutskever’s exit, OpenAI’s co-founder and former chief scientist, which led to the dissolution of the Superalignment team. The team was dedicated to addressing long-term risks associated with superintelligent AI models. In his farewell statement, Leike expressed his dissatisfaction with OpenAI’s shifting focus, stating, “Safety culture and processes have taken a backseat to shiny products.”
Leike has since joined Anthropic, an AI company founded by former OpenAI employees, including Dario Amodei, who previously served as OpenAI’s VP of research. Anthropic positions itself as a safety-centric AI research firm, committed to the cautious and ethical development of AI technologies. Leike’s role at Anthropic will involve leading a new “superalignment” team, focusing on scalable oversight, weak-to-strong generalization, and automated alignment research.
Anthropic has gained substantial backing, including a recent $4 billion investment from Amazon. This funding underscores the company’s commitment to advancing AI research while maintaining a strong emphasis on safety.
The formation of the new superalignment team at Anthropic mirrors the mission of OpenAI’s disbanded group, which aimed to solve the core technical challenges of controlling superintelligent AI within the next four years. Despite its ambitious goals, the team at OpenAI often found itself constrained by a lack of support and resources from the company’s leadership.
Leike’s transition to Anthropic signifies a broader shift within the AI industry, as researchers and organizations grapple with the balance between innovation and safety. The urgency of addressing AI safety has grown since the introduction of ChatGPT by OpenAI in late 2022, which spurred a rapid expansion of generative AI products and investments. Industry experts have raised concerns that the rush to commercialize powerful AI technologies may outpace the development of adequate safety measures.
In response to these concerns, OpenAI announced the establishment of a new safety and security committee led by senior executives, including CEO Sam Altman. This committee is tasked with recommending safety and security decisions for OpenAI projects and operations to the company’s board. However, this move comes amid criticisms that OpenAI has deprioritized safety in favor of commercial pursuits.
Anthropic, on the other hand, has consistently positioned itself as a company prioritizing safety. The company employs “constitutional AI,” which integrates ethical principles into AI model training by incorporating public input and adhering to standards like the United Nations Universal Declaration of Human Rights. This approach aims to ensure that AI systems are designed to align with human values and ethical considerations.
As AI continues to evolve, the industry must navigate the complex interplay between innovation and safety. The departures of key figures like Leike and Sutskever from OpenAI to Anthropic highlight the growing divide within the AI community regarding the prioritization of safety. Anthropic’s approach and its recent high-profile hires signal a strong commitment to addressing these challenges and ensuring that AI technologies are developed with a focus on long-term societal benefits.
