OpenAI scientists shocked TED conference: Let AI models think for 20 seconds, boosting performance by 100,000 times!
OpenAI scientist Noam Brown proposed at the TED conference that AI models can improve performance by thinking for 20 seconds, equivalent to expanding by 100,000 times and training by 100,000 times. He emphasized that "System 2 thinking" is key, improving reasoning abilities through self-play and reinforcement learning. Brown pointed out that the significant advancement of AI in the past 5 years has mainly relied on scale, but existing models need to transition to a slower, more cautious reasoning approach to solve complex problems
Early this morning, according to the well-known tech media Venturebeat, Noam Brown, a senior research scientist at OpenAI and the creator of Pluribus AI, presented a shocking theory at the TED AI conference in San Francisco. He proposed that the performance improvement brought by allowing AI models to think for 20 seconds is equivalent to scaling up the model by 100,000 times and training it for 100,000 times longer.
Initially, Brown was also shocked by this result and even wrote multiple papers to verify its authenticity. He found that "System 2 thinking" is the key to significantly improving the performance of AI models. The latest o1 model released by OpenAI also incorporates this technical concept and has achieved outstanding performance improvements.
In his speech, Brown stated that the significant advancements in AI over the past 5 years can be summarized in one word - scale. However, the cutting-edge AI models today are still based on the Transformer architecture introduced in 2017, with the main differences lying in data scale and computational power.
It is now time for a paradigm shift in training and inference. AI models need to move beyond simple data preprocessing and enter a "System 2 thinking" mode, solving super complex problems in a slower, more cautious, and more human-like reasoning form.
Introduction to System 2 Thinking
"System 2 thinking" is a psychological concept that describes the deep thinking mode humans adopt when dealing with complex problems. This concept was initially proposed by psychologist Daniel Kahneman in his work "Thinking, Fast and Slow" to explain the two different thinking modes of the human brain.
In Kahneman's theory, System 1 thinking is fast, intuitive, and automatic. It handles everyday, familiar tasks, such as recognizing familiar faces or understanding simple sentences.
This thinking mode does not require conscious thought and relies on our intuition and experience, but it can sometimes lead to errors because it does not involve deep logical reasoning.
System 2 thinking, on the other hand, is slow, logical, and effortful. It involves careful thinking, calculation, and reasoning. When faced with complex, novel, or analytically demanding problems, System 2 thinking is activated. This thinking mode requires us to focus our attention, consume more cognitive resources, but it can help us make more accurate and thoughtful decisions Brown directly applied this concept to the field of AI, proposing a revolutionary idea: by simulating human System 2 thinking, AI models can significantly improve performance without increasing a large amount of data or computational resources.
Taking his developed poker AI Libratus that defeated humans as an example, by allowing the AI to think for 20 seconds per hand, it can achieve the same performance improvement as expanding the model by 100,000 times. The core of this method lies in enabling AI models to conduct more in-depth analysis and reasoning before making decisions, rather than relying solely on massive data and computations.
Similarly, OpenAI's latest release, the o1 model, also introduces System 2 thinking, capable of deep reasoning and mimicking the gradual problem-solving process of humans. Through reinforcement learning training methods such as self-play, it enhances reasoning abilities.
For instance, in the International Mathematical Olympiad qualification exam, the o1 model achieved an accuracy rate of 83% in accurately reasoning complex mathematical formulas with System 2 thinking, far surpassing GPT-4o's 13%. This is crucial for industries such as finance, healthcare, scientific research, and coding that demand rigorous data requirements.
Therefore, System 2 thinking has many benefits for enhancing the capabilities of large models, enabling them to better adapt to new, unseen tasks and environments. When facing errors, uncertainties, and exceptional circumstances, System 2 thinking can also help large models become more robust by encouraging them to adopt more cautious and conservative strategies. In terms of human-computer interaction, simulating System 2 thinking can assist large models in better understanding and predicting the needs and intentions of human users, thereby improving the interactive experience.
AIGC Open Community, original title: "OpenAI Scientists Shock TED Conference: AI Models Think for 20 Seconds, Boost Performance by 10,000 Times!"
