Crazy benchmarking against OpenAI, Zhipu AI is ready to fight to the death

Editor | Yang Yong

Caption | IC Photo

Recently, the AI giant model "unicorn" Zhipu AI completed a new round of financing led by companies in the Zhongguancun Science City with an investment valuation of 20 billion yuan, raising tens of billions of yuan. It is reported that this round of financing is the third round of financing completed by Zhipu AI since 2024.

In June 2024, it was reported that Prosperity7, a fund managed by the venture capital department of the Middle Eastern oil giant Saudi Aramco, invested $400 million in Zhipu AI, directly pushing Zhipu AI's valuation into the "20 billion yuan club". The 20 billion yuan valuation is widely regarded as the benchmark for entering the top tier of the industry.

As of now, Zhipu AI's shareholder list includes well-known investment institutions such as Hillhouse Capital, Qiming Venture Partners, and Sequoia Capital, as well as the presence of internet giants such as Meituan's Strategic Investment Department, Ant Group, Alibaba, and Tencent Investment. In addition, there are also state-owned forces such as the Social Security Fund Zhongguancun Fund, Beijing Artificial Intelligence Industry Fund, and Zhongguancun Science City.

Established in June 2019, Zhipu AI was transformed from the technical achievements of the Knowledge Engineering Laboratory (KEG) of Tsinghua University. It is currently the only fully domestic and self-developed large model enterprise in China. As early as 2020, it started the research and development of the GLM pre-training architecture and trained a 10 billion parameter model GLM-10B; in 2022, it cooperated to develop a 130 billion level ultra-large-scale pre-training universal model GLM-130B; in 2023, Zhipu AI launched the hundred billion open-source base dialogue model GLM series, and in January 2024, it launched GLM-4. Subsequently, Zhipu AI released the GLM-4-9B open-source model in June, and launched the video generation tool Qingying in July.

"Catching up with OpenAI" and "Benchmarking OpenAI is the goal of Zhipu AI" are slogans repeatedly mentioned by Zhipu AI CEO Zhang Peng when sharing externally. However, the current competition of large models is no longer a debate of existence or non-existence, but a competition of implementation. With the support of many capital forces, can Zhipu AI, known as the "Chinese OpenAI," successfully break through in the competition of large models?

I. OpenAI is still the leader

Since its establishment, Zhipu AI has always regarded OpenAI as a target to catch up with. As of now, Zhipu AI has developed a complete set of model products that benchmark OpenAI, including the AI efficiency assistant Zhipu Qingyan, the high-efficiency code model CodeGeeX, the multimodal understanding model CogVLM, and the text-image model CogView, etc However, although Zhipu AI claims to be China's first open-source large model, it is not easy to compare with OpenAI.

For example, at the first Zhipu DevDay held in January 2024, Zhipu AI released a new generation language model GLM-4. Although GLM-4's overall performance has increased by 60% compared to the previous generation, claiming to be "on par with GPT-4", it actually only reaches about 90% of GPT-4's level.

Zhang Peng also admitted that compared to foreign large models, domestic large models started later, coupled with limitations in high-performance computing power, differences in data quality, etc., there is a certain gap in scale and core capabilities between domestic large models and the world's advanced level, with this gap being approximately one year.

Firstly, from a technical perspective, OpenAI focuses more on generality, portability, and scalability. Its GPT series models can be applied in multiple scenarios and have a high degree of customization. In contrast, Zhipu AI's technical route is "large model + small model", adapting to different scenarios and tasks through pre-training and fine-tuning of large models. This technical route can improve the model's generalization ability and application scope, but it also faces issues such as high model complexity, large computational requirements, and long training times.

Secondly, OpenAI's GPT series models are large in scale, capable of handling large amounts of natural language data, thereby achieving better model performance. In comparison, Zhipu AI's model scale may be smaller, with limited data processing capabilities, which may affect its model performance and generalization ability.

Furthermore, in terms of data resources, OpenAI has a large amount of natural language data resources for training and optimizing its models, while Zhipu AI's data resources may be relatively limited, restricting the effectiveness and performance of its model training.

The difference between the two is most evident in the number of users. In November 2022, ChatGPT under OpenAI surpassed one million users within five days of its launch, and in January 2023, its monthly active users exceeded 100 million, becoming the fastest-growing consumer application in history. In contrast, as of November 2023, the daily active user range of Zhipu AI's Zhipu Qingyan is only between 100,000 and 400,000.

In fact, the gap between Zhipu AI and OpenAI is widening. On September 13, OpenAI released the o1 series models, including the o1 preview and o1-mini. In a series of benchmark tests, o1 has made significant improvements compared to GPT-4o, even "comparable to human experts" in benchmark tests for physics, biology, and chemistry.

For example, in the International Mathematical Olympiad (IMO), GPT-4o scored 13.4%, while o1 scored as high as 83.3%; in Codeforces programming competitions, o1 achieved an excellent score of 89%, while GPT-4o's accuracy was only 11%. Additionally, in the GPQA-diamond test, human experts had an accuracy of 69.7, while o1 reached 78% It can be seen that Zhipu AI is still far from OpenAI. Although Zhipu AI has achieved remarkable results, facing the new o1 series model launched by OpenAI, Zhipu AI undoubtedly needs to work even harder.

2. Escalating Price Wars

Since May 2024, the price war in the large model field has been going on for more than four months, leading to more and more large model companies being drawn into the price war vortex.

This price war started with the AI company DeepSeek under the private equity giant Illusion Quantitative. On May 6th, DeepSeek announced the open source of the second-generation MoE large model DeepSeek-V2, priced at nearly 1% of GPT-4-Turbo, with one million tokens priced at only 1 yuan.

Zhipu AI quickly followed suit. On May 11th, Zhipu AI announced that the personal version GLM-3Turbo calling price was reduced from 5 yuan/million tokens to 1 yuan/million tokens. At the Zhipu AI Open Day event held on June 5th, Zhipu AI once again announced a price reduction for the entire model matrix. Among them, the prices of GLM-4-Air and GLM-3-Turbo were reduced to 0.6 yuan/million tokens, the Embedding-2 model dropped to 0.3 yuan/million tokens, and the price of the GLM-4-Flash model dropped to as low as 0.06 yuan/million tokens.

ByteDance also joined the price war, claiming to be below the industry average by 99.3%, announcing that the main model of Bean Bag (Bean Bag General Model Pro) was priced at 0.0008 yuan/thousand tokens in the enterprise market, while the usual price for similar models on the market is 0.12 yuan/thousand tokens, making the Bean Bag model 150 times cheaper.

Subsequently, Alibaba, Tencent, Baidu, and iFlytek all announced price reductions for large models. For example, Alibaba Cloud reduced the input price of Qwen-Long to 0.0005 yuan/thousand tokens, and the output price was directly reduced by 90% to 0.002 yuan/thousand tokens; Baidu Intelligent Cloud announced that the two main models of the Wenxin large model, ENIRE Speed and ENIRE Lite, are open for free.

OpenAI is also a major player in the price war, with the price of GPT-4o being halved compared to GPT-4-Turbo, marking the fourth price reduction since early 2023. According to OpenAI's expectations, its large models will continue to be reduced by 50%-75% annually.

It is worth mentioning that although the continuous decline in the pricing of large models is expected to accelerate commercialization, price wars often mean that large model companies need to make concessions in terms of pricing. For Zhipu AI, whose profit-making ability is already limited, if it continues to engage in ongoing price wars, it may further reduce profits, making profitability even more difficult to achieve In contrast, the impact of price wars on OpenAI may be relatively small. As early as December 2023, OpenAI CEO Sam Altman revealed that the current monthly revenue of OpenAI has reached the billion-dollar level, with annualized revenue likely to exceed 1.5 billion. Third-party institutions also predict that OpenAI's revenue in 2024 is likely to more than double that of 2023, with optimistic estimates reaching 5 billion dollars.

It can be foreseen that with the combination of price wars and technological gaps, Zhifu AI may not have an easy time in 2024. Zhang Peng also admitted that the challenges facing Zhifu AI in 2024 are very daunting: on one hand, in 2024, OpenAI will make new breakthroughs in super cognition and super alignment, requiring Zhifu AI to continuously iterate its technology to keep up with the world's leading pace; on the other hand, in 2024, large models will experience a wave of commercialization, increasing the commercial competition pressure on Zhifu AI.

III. Accelerating Ecological Investments

Product layout and investment layout are the two main lines for Zhifu AI to achieve commercialization.

Zhang Peng has publicly explained Zhifu AI's investment strategy: "We hope to build a large model ecosystem, where we work hand in hand with partners to make the ecosystem bigger and bigger. This is our longer-term commercial goal." When discussing the business vision for 2024, Zhang Peng stated, "Making large models truly land and down-to-earth is our important task."

In 2024, Zhifu AI will launch an open-source large model fund, which includes three "1000s": Zhifu AI will provide 1,000 computing cards to the large model open-source community to support open-source development; provide 10 million RMB in cash to support open-source projects related to large models; and provide 100 billion free API tokens to outstanding open-source developers. Zhang Peng stated that the purpose of the large model open-source fund is to promote significant progress in large model research and development, and to promote the overall prosperity of the open-source ecosystem for large models.

Facing global large model entrepreneurs, Zhifu AI will upgrade the "Z Plan" and launch a 1 billion RMB large model entrepreneurship fund in collaboration with ecosystem partners to support original innovation in large models, covering directions such as large model algorithms, underlying operators, chip optimization, industry large models, and super applications.

In August 2024, during the Zhifu AI "Z Plan" enterprise roadshow event, Zhang Peng officially announced the launch of the AGI Ecological Fund: Z Fund in collaboration with ecosystem partners to support more early-stage projects with potential in the large model track. On September 3rd, humanoid robot manufacturer Dongyi Technology completed a tens of millions RMB angel round financing, with the lead investor being Z Fund, which is also the first external investment of Z Fund. As of now, Zhifu AI has invested in 11 enterprises such as "ListenMind Intelligence", "Power Law Intelligence", "Shudao Zhisuan", and "Shengshu Technology" in the AI model layer, intelligent legal service products, software and information technology services, and generative AI application providers Objectively speaking, in the case of insufficient industry chain support, investing in the layout of the entire industry chain is a breakthrough method. However, the more crucial breakthrough lies in how to create products that expand user imagination and how to turn these products into productivity. This will be the next key question for Aito AI