The arrival of the domestic AI large-scale model "Android Moment"! Alibaba Cloud Tongyi Qianwen is free, open source, and commercially available.
The general model Qwen-7B and the chat model Qwen-7B-Chat of Tongyi Qianwen have been launched on the Moda Community. Qwen-7B is a foundational model that supports multiple languages such as Chinese and English, with pre-training data exceeding 22 trillion tokens. Qwen-7B-Chat is a large language model for Chinese and English conversations based on Qwen-7B, achieving alignment with human cognition.
During the battle of large models, following Meta overseas, Alibaba has become another technology giant driving the trend of large-scale artificial intelligence (AI) models with the "Android Moment".
According to Beijing Business Daily, on Thursday, August 3rd, Alibaba Cloud open-sourced the Qwen-7B and Qwen-7B-Chat models with 7 billion parameters, including the general-purpose model Qwen-7B and the dialogue model Qwen-7B-Chat. These two models have been launched on the domestic "Model-as-a-Service" open platform, Moda Community, which is open source, free, and available for commercial use.
The open-source code supports the quantization of Qwen-7B and Qwen-7B-Chat, allowing users to deploy and run the models on consumer-grade graphics cards. Users can directly download the models from the Moda Community or access and invoke Qwen-7B and Qwen-7B-Chat through Alibaba Cloud's Lingji platform, which provides services including model training, inference, deployment, and fine-tuning.
The Moda Community has already posted threads specifically introducing the installation, creative space experience, model inference, and best practices for model training of the aforementioned Qwen-7B models, along with model links and screenshots of the download situation.
Public information shows that Qwen-7B is pre-trained on more than 22 trillion tokens of deduplicated and filtered data, serving as a foundational model that supports multiple languages such as Chinese and English, with a context window length of up to 8k. It includes high-quality data in Chinese, English, multiple languages, code, mathematics, etc., covering the entire web text, encyclopedias, books, code, mathematics, and various verticals.
One of the benchmark evaluations for assessing English proficiency, MMLU, shows that Qwen-7B outperforms other similar open-source pre-trained models both domestically and internationally in terms of English evaluation. It also demonstrates strong competitiveness compared to larger-scale versions of models. In terms of Chinese evaluation, Qwen-7B achieved the highest score among existing models of the same scale on the C-Eval validation set, and even exhibits strong competitiveness compared to larger-scale models.
The following is a comparison of the MMLU 5-shot accuracy results for Qwen-7B.
Based on Qwen-7B, Alibaba Cloud has developed the AI assistant Qwen-7B-Chat using alignment mechanisms. It is a large-scale language model for Chinese and English dialogues based on Transformers, which has achieved alignment with human cognition. The pre-training data is diverse and covers a wide range, including a large amount of web text, professional books, code, etc.
Whether on the C-Eval validation set or the MMLU evaluation set, the zero-shot accuracy of the Qwen-7B-Chat model performs relatively well among similar alignment models. Below is a comparison of zero-shot accuracy results on the C-Eval test set.
After the open sourcing of Tongyi Qianwen, Alibaba Cloud became the first large-scale technology company in China to join the ranks of large model open source. In July of this year, Microsoft announced its collaboration with Meta to release a commercially available version of the open source AI model, Llama 2, which provides alternative products to OpenAI and Google models. Also in July, Zhidu AI and Tsinghua KEG Laboratory announced that China's top open source large model, ChatGLM2-6B, is available for free commercial use.
As mentioned in a previous article by Wall Street News, the benefits of open source models include higher user acceptance and more data input for AI processing. The more data LLM has, the more powerful its functionality becomes. In addition, open source models enable researchers and developers to discover and address vulnerabilities while improving technology and security.
At the Alibaba Cloud Summit in April 2023, Alibaba announced the opening of Tongyi Qianwen to enterprises, allowing them to leverage the capabilities of Tongyi Qianwen to train their own large models.
Zhou Jingren, Chief Technology Officer (CTO) of Alibaba Cloud Intelligence Group, explained at the time that in the future, enterprises on Alibaba Cloud will be able to access the full capabilities of Tongyi Qianwen and train their own enterprise models based on their industry knowledge and application scenarios. For example, each enterprise can have its own intelligent customer service, intelligent shopping guide, intelligent voice assistant, copywriting assistant, AI designer, and autonomous driving model.
Zhang Yong, CEO of Alibaba Group and CEO of Alibaba Cloud Intelligence Group, stated at the time that all Alibaba products will be integrated with the Tongyi Qianwen large model in the future.
Zhang Yong said that in the AI era, all products deserve to be redesigned with large models. Based on this belief, Alibaba Cloud also hopes to help more enterprises adopt large models, so that every enterprise can have its own exclusive large model with industry-specific capabilities.
