Z.AI Founder Tang Jie Discusses Scaling Law: Trillion-Parameter Models Were a Detour for the Industry; The Next Key Step Lies in Post-Training

Tang Jie, founder of Z.AI, posted on the X platform on the 19th, systematically elaborating on his latest understanding of Scaling Law and using GLM-5.3 as a "control variable experiment" to support this judgment. The release of this post directly responded to external doubts about why Z.AI had not trained a new base model.

In the article, Tang Jie pointed out that parameter count is never the sole dimension for evaluating model capabilities; data scale, computing power allocation, and inference deployment conditions are equally indispensable. He stated that GLM-5.3 shares the same base, architecture, total parameters (753B), and activated parameters (40B) as its predecessor, GLM-5.2. The only difference was the addition of one month of long-horizon environment training and reinforcement learning (RL) post-training—resulting in capability improvements that were "not minor." In the AA benchmark index, GLM-5.3 scored 60, a 7-point increase over GLM-5.2's 53.

The core of this experimental conclusion is that when the parameter scale of a base model is sufficient to carry adequate knowledge, additional capability improvements often come from "turning other knobs"—especially during the post-training phase—rather than continuing to stack parameters. This judgment has direct reference value for how the entire AI industry currently allocates training resources.

Trillion Parameters: A Detour Taken by the Industry

Tang Jie outlined the evolution of Scaling Law in his article. In 2020, Kaplan and others concluded that parameter scale should grow faster than data growth, at a ratio of approximately 2.7:1. This conclusion drove the concentrated emergence of ultra-large-scale models such as GPT-3, Gopher, and MT-NLG, with trillion-parameter models once seen as an inevitable path for industry progress.

However, in 2022, Hoffmann and others re-conducted experiments on more than 400 models and found that the optimal allocation of computing power should be close to 20 tokens per parameter. Furthermore, as computing power continues to expand, parameter count and data volume should grow at roughly the same rate, rather than the gap between them widening continuously. This is the widely cited "Chinchilla Scaling Law."

Tang Jie stated bluntly in his post that early fitting errors were constantly amplified as the magnitude of computing power increased, leading to the largest models at the time having the most imbalanced computing power allocation:

"Looking back, the trillion-parameter phase was a detour taken by the entire field together, before everyone turned back."

Inference Costs and MoE: The Optimal Solution Keeps Shifting

After Chinchilla, the optimal solution did not remain fixed. Tang Jie pointed out that Chinchilla optimized for models that are "trained once, evaluated once," whereas today's mainstream models may be called billions of times daily, making inference costs far exceed one-time training costs over their full lifecycle.

When inference costs are included in the objective function, the optimal solution shifts toward "smaller models, trained longer." He cited Llama-2-7B and Gemma-2-9B as examples, where the number of tokens per parameter is approximately 290 and 889, respectively, both falling into the category of deliberate "over-training."

The popularity of Mixture of Experts (MoE) architecture has further complicated the issue. Tang Jie distinguished two dimensions that must be discussed separately in the context of MoE: Total parameter count determines how much knowledge, facts, and long-tail information a model can store; activated parameters and the effective depth of each forward computation determine the model's long-range reasoning capabilities.

He cited research findings from Roberts et al. in 2025, which showed that the optimal number of tokens per parameter varies by task—memory tasks tend to favor more parameters, while reasoning tasks favor more data. Under the condition of fixed tokens per parameter, continuing to increase total parameters reduces reasoning capabilities, whereas activating more experts can steadily improve reasoning performance. Based on this, Tang Jie pointed out that core capabilities for tasks requiring twenty-step causal chain reasoning, such as vulnerability identification, do not reside in total parameter count.

GLM-5.3: A Control Variable Experiment

Based on the above judgments, Tang Jie positioned GLM-5.3 as an intentional "control variable experiment." Compared to GLM-5.2, GLM-5.3 kept the base, architecture, total parameters, and activated parameters completely unchanged; the only variable was the addition of one month of long-horizon environment Scaling and reinforcement learning post-training.

He admitted that the choice of post-training as the "knob" to turn was because it currently offers the greatest room for improvement, not because other dimensions have no potential left—"This does not mean that other knobs have been turned to their limits." He explicitly stated that base model scale, pre-training data, and computing power investment per forward computation are still under consideration, and the next steps may involve adjustments at the mid-training and pre-training levels.

This approach directly responds to previous external questions about why Z.AI did not train a new base model. In Tang Jie's framework, not all knobs need to be adjusted simultaneously; the key lies in determining "which one is most worth turning next." Currently, the coding capabilities of GLM-5.3 have been rated by multiple developers as the current best among domestic models.

Risk Warning and Disclaimer

The market carries risks; investment requires caution. This article does not constitute personal investment advice, nor does it take into account the specific investment objectives, financial status, or needs of individual users. Users should consider whether any opinions, views, or conclusions in this article align with their specific circumstances. Investment decisions made based on this content are the sole responsibility of the investor.