OpenAI Establishes 'Super Team' To Explore Ways To Control 'Super AI' In The Future

On July 5th, OpenAI revealed that it is forming a new team led by Ilya Sutskever, the company's Chief Scientist and one of its co-founders, intending to develop methods to steer and control potential "superhuman artificial intelligence."

 

Ilya Sutskever

OpenAI previously suggested Superintelligence will be the most consequential technology ever invented by humans. It could help humans to solve many of the world's most important problems, but the immense power of this technology also poses risks and could lead to the loss of human control or even human extinction.

 

Sutskever and Jan Leike, the leader of OpenAI's Alignment team, believe that AI systems with intelligence surpassing that of humans could emerge within the next decade. They emphasize that if such AI systems do arise, there is no guarantee that they will be "beneficial," necessitating research into methods for control and limitation.

 

"Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue," Sutskever and Leike said, "Our current techniques for aligning AI, such as reinforcement learning from human feedback, rely on humans' ability to supervise AI. But humans won't be able to reliably supervise AI systems much smarter than us."

 

The phrase "AI alignment" refers to the process of guiding and constraining the behavior of artificial intelligence systems to ensure they conform to the intentions and desired goals of their designers. An aligned AI system behaves in a manner that aligns with the designer's specifications, while a misaligned AI system may exploit loopholes or act in unexpected, potentially "harmful" ways to achieve its objectives.

 

Therefore, OpenAI has decided to establish a "Superalignment" team led by Sutskever and Leike to make progress in the field of superintelligence alignment. This team will comprise scientists and engineers from OpenAI's previous Alignment division, as well as researchers from other departments within the company. In order to support the team's work and tackle the core technical challenges of controlling superintelligent AI, OpenAI plans to allocate 20% of its computational resources.

 

Based on the hypothesis that AI systems can make faster and better progress in alignment research than humans, Sutskever and Leike plan to build a "human-level automated alignment researcher." This involves training AI systems using human feedback, utilizing AI to assist in the evaluation of other AI systems, and ultimately constructing AI systems capable of conducting alignment research. The goal is to develop AI systems that can conceive, implement, study, and develop improved alignment techniques beyond the capabilities of current methods.

 

Leike and colleagues John Schulman and Jeffrey Wu believe that as progress is made in this area, AI systems can progressively take over more alignment work, ultimately collaborating with humans to ensure the alignment of their own successors. They envision that human researchers will shift their focus towards reviewing alignment research conducted by AI systems, rather than generating such research themselves.

 

However, it is acknowledged by Leike, Schulman, and Wu that no method is foolproof, and they recognize the various limitations of OpenAI. Utilizing AI for evaluation purposes has the potential to amplify inconsistencies, biases, or vulnerabilities in AI systems. Furthermore, they note that the most challenging aspects of the alignment problem may not solely be engineering-related.

 

Nevertheless, Sutskever and Leike believe that the endeavor is worth pursuing. They state, "Superintelligence alignment is fundamentally a machine learning problem, and we think great machine learning experts — even if they're not already working on alignment — will be critical to solving it."Meanwhile, Sutkever and Leike also add, "We plan to share the fruits of this effort broadly and view contributing to alignment and safety of non-OpenAI models as an important part of our work."