The Distillation Storm: The Tech Race AI Companies Don't Want to Talk About
No one is willing to talk about it publicly, but everyone is quietly paying attention to it.
Some believe it is a form of dishonorable theft. Others believe it is something that a few leading players are stigmatizing for their own benefit, and that it is merely an optimization technique.
Over the past few months, many threads in the AI field have converged on the same node—distillation.
Related changes and events include: open-source models approaching the strongest closed-source models; 77 US companies signing an open letter opposing hasty restrictions on open-source models; Anthropic twice accusing Chinese companies of using large numbers of fraudulent accounts to extract data; a $1.5 billion copyright infringement lawsuit settlement; Zhang Yiming's positive response about "not distilling" at the ByteDance Seed all-hands meeting...
This technology, which has existed for years, has been repeatedly mentioned, discussed, misunderstood, and misrepresented in 2026. What exactly is distillation? How is large-scale distillation achieved? Can distillation become a barrier for a model development team? And what is its cost?
We interviewed nearly ten researchers and practitioners in the model field from different companies, and combined public research and technical reports to restore the past, present, and the additional changes that distillation is bringing.
The Starting Point of Distillation: For Compression, Not for Becoming Stronger
Distillation is not plagiarism, nor is it stealing software code, and it cannot directly obtain another model's weights and complete training data.
The type of distillation currently under controversy—the one that makes models stronger—is, technically, a method of obtaining high-quality data: repeatedly asking a stronger "teacher model" questions, getting answers, and then using these "question-answer" data pairs to train another "student model" so that the latter achieves similar performance.
The idea of distillation has been around for a long time. In 2015, Geoffrey Hinton, who had just joined Google Brain, together with Jeff Dean (then head of Google Brain) and young researcher Oriol Vinyals, published "Distilling the Knowledge in a Neural Network," which for the first time consolidated earlier ideas such as Model Compression (2006) into the concept of distillation.
At that time, it was still nearly two and a half years before Google proposed the Transformer (the architectural foundation of today's large language models). Hinton and his team applied the idea of distillation to image recognition models: the method was to have the student model learn the probability distribution output by the teacher model. For example, identifying a cat as a cat: in deep learning, this is a statistical process: 0.7 cat, 0.2 fox, 0.1 dog → it is a cat.
The student model can see this set of probability distributions output by the teacher, which is learning the "logits"—the so-called "soft distillation" (logits are a set of raw scores, which after Softmax transformation become probability distributions).
Soft distillation is usually "white-box distillation": because it requires the teacher model's output probabilities to be fully open to the student model.
This type of distillation typically occurs within the same organization. Its starting point is not to become stronger, but to "compress"—using a smaller-parameter model to approximate the capabilities of a larger-parameter model. This may sacrifice some performance, but it makes inference faster and cheaper.
Even today, "compression" remains one of the most typical uses of distillation.
For example, in the field of autonomous driving, a more powerful cloud-based large model is first developed, and then through distillation, pruning, and other methods, it is turned into a smaller model that can run on the vehicle's chip. Companies like Li Auto and XPeng have such practices.
A more recent example is DeepSeek-R1 from early 2025. At that time, DeepSeek also released six smaller distilled models, all of which used the 671-billion-parameter R1 itself as the teacher model. Among the student models, four were based on Alibaba's Qwen2.5, and two were based on Meta's Llama 3, with parameter sizes ranging from 1.5 billion to 70 billion.
DeepSeek first had R1 generate about 600,000 "question-reasoning process-answer" reasoning data items and about 200,000 non-reasoning data items, and then used these data to supervised fine-tune (SFT) the six smaller base models during the post-training phase. These models all gained stronger reasoning capabilities.
"LatePost" reported that after the 2026 Spring Festival, one of the core authors of R1, Guo Daya, joined ByteDance Seed.
"Industrial-Scale Distillation Attacks"
For compression purposes, distillation is a neutral technical method. But by February 2026, Google and Anthropic repeatedly published articles using the terms "distillation" and "attack" together, directly accusing certain companies of using distillation for unfair competition.
Google considers this an IP theft:
Over the past year, "distillation attacks" have increased as a means of intellectual property theft.
>
——GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use
Anthropic, in February and June respectively, stated that DeepSeek, Moonshot AI (Kimi), MiniMax, and Alibaba Qwen, through a total of approximately 50,000 fraudulent accounts, had conducted over 44.8 million interactions with Claude in an attempt to extract Claude's capabilities. (February: Detecting and preventing distillation attacks; June: Anthropic's letter to the US Senate.) The mentioned companies did not make any public response.
How has large-scale distillation of leading closed-source models become possible? The root of the change still lies in technology, and three threads can be observed:
In 2016, Yoon Kim, then a PhD student at Harvard, and his advisor Sasha Rush proposed sequence-level knowledge distillation, applying this method (originally used in image recognition) to the language task of translation.
Sequence-level distillation no longer requires learning the probability distribution of each step of the model's output. Instead, it lets the teacher model first generate high-quality translations, and then lets the student model learn the complete "source-text-target-text" sequence pairs.
The original purpose of this technique was to compress large translation models and improve decoding speed, but it also brought about an effect: distillation no longer needed to know the step-by-step probability distribution of the teacher model's output; it could be achieved by only looking at the "final answer."
This is "hard distillation," which can be performed in a black-box manner, i.e., by directly calling the API to obtain the teacher model's answers.
Sasha Rush later joined the AI coding company Cursor in March 2025. In June of this year, Cursor was acquired by SpaceXAI (the new name after the merger of SpaceX and xAI) for $60 billion, and the team has been integrated.
In September 2024, amidst discussions about the Scaling Law hitting a wall, OpenAI released the o1 reasoning model. o1 brought two changes:
Both of these changes amplify the effect of distillation: First, large-scale distillation is a method mainly used in the post-training phase. As the importance of post-training increases, the return on investment in distillation also increases. Second, test-time computation causes the model to produce not only final answers but also chains of thought and reasoning traces (including thought chains, tool calls, search processes, and error correction processes, etc.)—long reasoning processes. These outputs can be used as raw data for distillation.
Four months later, DeepSeek's distillation process, detailed in the R1 technical report, provided a more thorough demonstration of how distillation makes models stronger. R1 had several specific practices and findings:
First, the distillation process mainly used the "question-reasoning process-answer" data pairs generated by R1. This is currently the ideal state of distillation, and its effect is better than using only "question-answer" data pairs.
DeepSeek, which is relatively open, directly displayed the complete chain of thought when releasing R1, "hoping to help the community distill better small models."
In contrast, leading closed-source model companies—OpenAI, Anthropic, and Google DeepMind—have always hidden the complete chain of thought and reasoning traces from users.
Therefore, when practitioners mention distillation, they often say: "So-and-so has cracked so-and-so's chain of thought."
This Monday (August 10), researchers from the University of Tübingen and other institutions published "Stealing Reasoning Traces from Proprietary LLM APIs," demonstrating methods they discovered for recovering reasoning traces.
The research webpage: stolen-thoughts.com
In fact, this has long been an open secret—even if closed-source model companies deliberately hide them, chains of thought and reasoning traces can be recovered through technical means.
This is essentially because they are all part of the model's output, products of the model's usage phase. As long as you use a model, the chain of thought and reasoning process will be generated, leaving traces. This still falls under black-box distillation and hard distillation.
Another finding of R1 regarding distillation is: during the post-training phase, the improvement brought by direct distillation is greater than letting the model perform reinforcement learning on its own.
DeepSeek conducted a controlled experiment using Qwen2.5-32B:
We have demonstrated that the reasoning patterns of large models can be distilled into small models, and these distilled reasoning patterns outperform those learned by small models through reinforcement learning.
——R1 Technical Report
Although this is a result under a specific experiment and may not be generalizable to all situations, it is still a very attractive finding. Because long-step reinforcement learning training is inherently more difficult than supervised fine-tuning, it imposes more requirements on infrastructure, is often slower, and has higher computational costs.
DeepSeek has made public a relatively economical, efficient, and deterministic method for improving the reasoning capabilities of smaller or weaker models.
Several practitioners said that in some recent practices, during the post-training phase, only supervised fine-tuning is mainly performed, with little or no reinforcement learning, and good results can still be achieved.
Also, from the entire year since DeepSeek-R1 until now, Anthropic, OpenAI, and Google have stated that they have detected an increasing number of "distillation attacks."
During the same period, more exploration of post-training has also promoted another type of distillation that is not controversial: on-policy distillation, which is currently mainly used for capability merging in post-training.
The difference between on-policy and off-policy distillation lies in who generates the data. The previously mentioned distillation where a model learns from another closed-source model is mostly off-policy distillation: the data is generated by the teacher model. In on-policy distillation, the student model generates reasoning and answers, and the teacher model provides feedback. The feedback can be the probability distribution of each generated token (white-box), or it can be an evaluation of the reasoning trace and answer (black-box).
From the second half of 2025 to the present, Alibaba's Qwen, Thinking Machines Lab, and Xiaomi's MiMo have all contributed practices and improvements to on-policy distillation.
In its MiMo V2-Flash technical report released early this year, Xiaomi introduced the MOPD (Multi-Teacher On-Policy Distillation) method, and in June it published a separate article ("Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training"): first, separate teacher models are trained for mathematics, code, tool calling, and other directions; then the student model generates its own traces and receives feedback from different teachers according to the task. It aims to solve a new problem in post-training: if multiple capabilities are directly mixed together for reinforcement learning, they often interfere with each other, causing a seesaw effect.
In the technical reports of DeepSeek-V4 and Kimi K3, both stated that they used the MOPD approach to merge multiple expert models during the post-training phase.
The third thread of scaling distillation is that distillation itself is becoming more automated as AI capabilities grow stronger.
Going back to the key of distillation—the "question-reasoning process-answer" data pairs.
First, the questioning step can be shifted from humans to AI: since 2022, many studies such as Self-Instruct have been addressing the problem of insufficient and expensive high-quality questions.
In practice, companies can first filter high-quality real questions from authorized user behavior, then amplify these real questions by generating more AI-generated questions, asking the teacher model more frequently, and obtaining more answers. This is like having some yeast (real data) and then using it to ferment a larger dough.
The entire "question-reasoning process-answer" data pairs can also be rewritten and amplified in this way.
This is essentially a fundamental mindset in the AI field today: using AI and models to automate and accelerate AI itself.
Throughout the distillation pipeline, there are many specific steps where AI can be used, such as filtering high-quality questions from massive real questions, evaluating "what constitutes high-quality data," improving the diversity of synthetic data... Some steps can be done with models, and some can be accelerated and optimized by using increasingly powerful AI coding capabilities to build and improve various systems faster.
It can be said that from 2025 to the present, various methods and practices for large-scale distillation have gradually matured. Its effectiveness and necessity have become increasingly apparent as the importance of post-training has risen.
Discussions around "distillation" have also gradually moved beyond the AI tech circle, becoming simplified, misunderstood, and even distorted in dissemination. Distillation is no longer a purely technical issue; it has moved to the center of the storm.
Misunderstandings About Distillation: It Is Neither a Silver Bullet Nor a Secret
After sorting out the general principles of distillation, we have a better foundation to expand on various discussions about distillation.
· Distillation is not a silver bullet, nor is it the most important factor determining model performance.
As mentioned repeatedly earlier, the distillation that makes large language models stronger mainly occurs in the post-training phase (and also the mid-training phase). The overall effectiveness of a model comes from the complete training process from pre-training to post-training. It is generally believed that pre-training is more important.
Therefore, distillation is not a silver bullet, nor is it the most important factor determining a model's performance.
In the DeepSeek-R1 technical report, using the same 800,000 R1 data items for distillation, the Qwen2.5-32B base model scored 72.6% on AIME 2024 after distillation, higher than the Llama-3.3-70B-Instruct's 70.0% after distillation, even though the latter has more than twice the parameters.
After K3 was released, Ai2 researcher Nathan Lambert, in a reply to a tweet about K3 topping the Frontend Code Arena leaderboard (evaluating frontend code capability), said: "At this point, the rhetoric about 'distillation' should stop. People should admit that China is also very good at building models."
A certain level of pre-training is the foundational condition for Chinese open-source models like K3, GLM-5.2, and DeepSeek-V4 to achieve their current performance.
· Distillation requires considerable operational experience, expertise, and engineering know-how.
Distillation is often compared to a shortcut, which implies ease and effort-saving. But in reality, conducting large-scale distillation today is a fairly complex systems engineering task.
Based on descriptions from multiple practitioners, large-scale distillation has several difficult aspects:
First, being able to stably, frequently, and massively call leading models and manage user operations.
One mentioned approach is: building a large number of intermediary stations, attracting specific users with discounts or other methods—users who have real usage behavior. They might be senior programmers or science/engineering students and researchers who need to handle a large number of scientific problems. In their daily use, they naturally generate high-quality multi-turn questions in real scenarios and real tasks, and then receive model answers. These question-and-answer data, after being filtered and processed in a certain way, can serve as the source for fermenting and amplifying more data.
This tests both the team's operational ability—whether they know where the high-quality users are and how to reach them—and the engineering ability to build such a system, such as ensuring the system is stable enough. It also requires some ecosystem capabilities, and the entire process may require cooperation with third-party companies or institutions.
Second, the team's own ability to construct high-quality questions and tasks. This requires a deep understanding of tasks, data, and the performance boundaries of current leading models. It overlaps with some of the capabilities needed for large model training itself.
Third, how to make good use of the data. This requires establishing a data pipeline—whether the task distribution is reasonable, how to sample, filter, clean, deduplicate, amplify, correct errors, and determine the format and ratio. This pipeline has some measurable optimization indicators, such as how much of the raw data obtained can ultimately be used for post-training, and the efficiency and quality of amplification. The quality of the data pipeline affects not only the results but also efficiency and cost.
Continuous distillation is also very expensive. There are some circulating figures about how much money companies in the industry have spent or budgeted for distillation this year, ranging from $100 million to $1 billion.
An AI investor said: "Distillation is not a simple button. You don't just click it and the model performance skyrockets. There are a lot of implementation issues. Distillation also requires calculating the return on investment."
· Can distillation become a barrier for a model team?
Then, can large-scale distillation, which is relatively complex to implement, become a barrier for a model development team?
Most of the practitioners we contacted, whether from companies rumored to have done distillation or from companies that do not, have a relatively consistent view: for first-tier companies, distillation does not constitute a long-term barrier.
Like many technologies in large models, the ideas and practices of distillation will gradually spread. Personnel turnover, open-source sharing, conference exchanges, and third-party service providers looking for more clients... People and information in the AI circle are constantly flowing. We have repeatedly heard similar sighs from researchers: "There are no real secrets in the field of large models."
The competitive advantage that a technical method itself can bring is mostly first-mover advantage. Those who do it first will have more experience, but it is not a strong barrier like network effects that are insurmountable and winner-takes-all.
In the field of large models, a phenomenon that is truly considered to have a strong competitive barrier is the "data flywheel": if a certain company's model is strong enough to reach a large number of users who use it for difficult tasks, it will obtain more high-quality data feedback. Moreover, this data is unique, non-public, and not available to others. After certain processing, this data can be used to help train stronger models, attracting more users to handle even harder tasks.
In this flywheel logic, applications that directly touch users have great value. For example, according to user agreements and permission settings, applications like Cursor, Devin, and Manus may obtain more complete data and user behavior than the models they call.
Yesterday (August 12), after Grok 4.6 was released, Musk replied to a tweet about Devin integrating Grok 4.6, saying: "Grok 4.7 will surpass all current models." "SpaceX's training corpus is so great, so unique." The $60 billion acquisition of Cursor seems well worth it.
However, the data flywheel also has its own controversies: model and application companies can obtain data, but can they use this data for training? Moreover, for harder tasks and higher-value scenarios, will customers and users tend to keep this data to themselves? At the same time, as the number of model users continues to expand, user types and scenarios will become more diversified—is it still worth panning for gold?
Some practitioners believe: in some life-assistant and entertainment AI applications, most of the data generated by users is garbage when it comes to training stronger models.
The Inner Line of Judgment: Black Box or White Box
Regarding distillation, the most interesting phenomenon is: no one is willing to talk about it publicly, but most of the practitioners we contacted do not genuinely believe it is a very shameful practice that violates their technical beliefs.
This inner line of judgment lies between black box and white box.
Currently, distillation against closed-source models is all black-box distillation. The data obtained is data generated during the use of these models after their release—the product of the model as a product.
So why can't other companies, as users, ask the model questions and get answers? Besides, they have paid real money for these questions and answers. (Of course, in practice, various methods are used to "fleece" and reduce costs.)
The more controversial part lies in the reasoning traces. Most models hide the complete reasoning traces, and the distilling party needs to use some technical means to infer and recover them. But the reasoning traces are still products of the model's usage phase.
Anthropic, Google, and OpenAI would say: "My user agreement clearly stipulates that other competitors cannot use my model to train and improve their own models."
But who was sued by The New York Times for copying and using the newspaper's archive of over 170 years accumulated by generations of journalists, critics, and authors without permission? It was OpenAI.
Who downloaded massive amounts of books from pirate platforms, refused to pay for any of them, and was sued by several US authors in a class action, just reaching a $1.5 billion settlement? It was Anthropic.
Even some US AI practitioners believe that Anthropic and other companies are very hypocritical. This is somewhat like the situation in Nolan's new film "The Odyssey": you sent the Trojan horse into Troy, and now your own homeland is being invaded by those coming from the sea.
Large-scale distillation is a new issue that has emerged after the development of new technologies. It is difficult to make the majority of people accept that it is an unacceptable, immoral, or even shameful and evil act just because a few companies say it violates their user agreements.
Moreover, violating a user agreement does not necessarily constitute legal infringement. This also involves other legal provisions and issues of jurisdiction.
The most broad sense of distillation—using data to improve models—has become ubiquitous.
A widely circulated method of verifying distillation is actually invalid: when you ask a model, "Who are you?" If model A says it is model B, this is not definitive proof that A distilled B. During the pre-training phase, everyone uses a large amount of public internet data, and much of it already contains data generated by various models themselves.
In July of this year, NVIDIA founder Jensen Huang, when asked about distillation in an interview with Axios, said: "(Broadly speaking) distillation—learning from AI, learning from other sources of knowledge—is the fundamental principle of intelligence."
For most model companies, including some US model companies, the accelerator on distillation has already been pressed down. Few people will voluntarily give up distillation in a short time.
Why Zhang Yiming Chose Not to Distill
Most companies are reluctant to talk about distillation publicly. ByteDance is an exception.
About half a month ago (end of July) at the most recent Seed all-hands meeting, ByteDance founder Zhang Yiming clearly stated that he opposes distillation.
ByteDance has gone through adjustments regarding distillation. At the end of 2023, ByteDance was the first major Chinese large model developer to be pointed out by OpenAI as possibly using GPT model outputs to improve its own models in an irregular manner. At that time, there was no large-scale distillation. OpenAI itself also said that ByteDance's usage of its API was minimal.
ByteDance later responded that GPT-generated data had been used for model annotation and evaluation, but the relevant data had been deleted from the training set in mid-2023.
According to "LatePost," in the more than two years since then, including the entire year of 2025 when distillation scale expanded rapidly, ByteDance Seed did not distill leading closed-source models. Instead, it obtained data through other means, such as hiring top winners of math, computer science, and other STEM competitions at high salaries to construct and annotate data.
During the same period, the controversy over TikTok's US business had not yet been resolved. It was not until the end of January 2026 that the restructuring transaction of TikTok's US business was officially completed.
Around the 2026 Spring Festival, Seed experienced a period of wavering. At that time, OpenClaw was popular, Anthropic's revenue surged, the usage of Chinese open-source models like GLM-5 increased dramatically, and the quantitative improvement in coding ability had caused a qualitative change, while ByteDance's models were relatively lagging in coding ability.
After a period of decision-making, based on Zhang Yiming's remarks at the Seed all-hands meeting, ByteDance has made a choice: not to distill external leading models.
According to an exclusive report by "LatePost," Zhang Yiming's views at the all-hands meeting were:
This leads to another discussion about distillation: can distillation really not surpass the teacher model?
We also asked several practitioners this question. The answers were relatively similar: technically, it is not impossible, but there are potential organizational pitfalls.
Distillation has some inherent technical issues, such as the student model may learn some of the teacher model's errors, biases, refusal habits, and expression patterns.
But distillation is only one part of model training. The complete model training process has many other areas for improvement: pre-training data, architecture, algorithms, infrastructure... Could the accumulation of multiple optimizations allow a student model to be better than a certain teacher model?
Some studies have already shown that on specific tasks, student models can surpass their teacher models. For example, in December 2024, Microsoft released Phi-4 with 14 billion parameters. A large amount of its training data was synthesized by teacher models such as GPT-4o. Phi-4 outperformed GPT-4o on two benchmarks:
Notably, as a small-parameter model, Phi-4 directly used the data generated by the teacher model in pre-training. However, when training truly massive models with trillions of parameters, the pre-training phase, which requires enormous data, can hardly use data obtained through distillation, because calling the teacher model to generate data piece by piece is slow and expensive compared to directly processing various web pages, code, and books.
Nevertheless, this also raises a speculation: when model inference speed greatly improves and prices drop significantly, can distillation—or data constructed with the help of stronger models—enter the pre-training phase more extensively?
At the same time, can ideas like multi-teacher distillation be used to make models stronger? In theory, a student model could learn from different strongest models such as Claude and GPT simultaneously. This might bring some new technical problems, such as distilling from different base models potentially causing training instability and interference between different capabilities.
More radical imaginations include: can the most leading companies achieve "left foot stepping on right foot" self-improvement by distilling their own models?
Purely from a research perspective, whether "a model using distillation can surpass its teacher model" is at least a topic to be verified and explored.
Zhang Yiming and ByteDance have given their answer through actions: he believes it cannot surpass.
This may be related to the organizational pitfalls mentioned by several practitioners. Distillation is a relatively economical and fast-acting method. An athlete can certainly both take stimulants and train hard. But in reality, it is often difficult to have both, because it breeds luck and laziness.
When a team's attention and resource allocation are heavily weighted toward distillation for a period, those projects and individuals exploring more uncertain, longer-term directions may not receive sufficient resources and recognition.
ByteDance is betting: mastering the ability to obtain data without relying on external opponents can build a more robust and long-term model technology advantage.
Since June of this year, according to "Intelligent Emergence," ByteDance began restructuring its data team, establishing a first-level AI department "AI Data and Security" parallel to Seed and Flow.
As mentioned earlier, most people do not believe that distillation methods constitute a long-term barrier. What can truly form a strong barrier in the large model field is the data flywheel.
Data has also become a high-value independent segment. For example, the US company Mercor mainly helps large model companies find scientists, PhDs in various disciplines, and other professionals to perform data construction, annotation, and model evaluation tasks. According to reports, in July of this year, Mercor was seeking a new round of financing at a $20 billion valuation.
In China, a company that started with data construction has also reached a valuation of $3 billion.
Under our exclusive report about "ByteDance not distilling," there was a comment: "He (Zhang Yiming) may not be the most technically savvy, but he is probably the most understanding of human nature."
No One Can Easily Hit the Brakes
When others accuse you of wrongdoing, retorting "Didn't you do the same?" does not resolve the conflict.
At this moment, open-source models are increasingly approaching the most advanced closed-source models in performance. Moreover, the objective fact is that the strongest open-source models come from China, and the strongest closed-source models come from the United States.
Previously, Zhipu was added to the US Department of Commerce's Entity List in January 2025. The "Intelligence Authorization Act for Fiscal Year 2026," which took effect at the end of the same year, required the removal of DeepSeek from US intelligence systems, national security systems, and related suppliers.
By April of this year, the US House Homeland Security Committee and the US-China Strategic Competition Special Committee began investigating US companies' use of Chinese models, asking why Cursor used Kimi K2.5 as the base for Composer, and why Airbnb used Alibaba's Qwen in its customer service operations.
After Kimi K3 was released on July 16, it was reported that the US government is considering restricting or even banning some Chinese open-source models. On July 24, 77 companies and institutions, including Microsoft, NVIDIA, Meta, and Fireworks AI, successively signed an open letter, "Open Weights and US AI Leadership," opposing hasty restrictions on open-source models.
Jensen Huang's first tweet on Twitter was sharing this open letter.
New restrictive measures considered in recent months are still under discussion and have not yet been implemented.
Anthropic, OpenAI, and Google are also adopting stricter technical measures to identify and block accounts suspected of being used for distillation. For example, Anthropic stated that it has established classifiers and behavioral fingerprint systems to identify distillation traffic, enabling detection of cross-account coordination, repeated questioning, and attempts to extract chains of thought. At the same time, it will strengthen identity verification for educational, research, and startup accounts.
The methods for recovering reasoning traces in the "Stealing Reasoning Traces" study from the University of Tübingen mentioned earlier have been reported to the relevant closed-source model companies by the research team. By the time they uploaded the article, some of those methods had already become ineffective.
All parties involved are making their own choices and preparations. The coming changes will affect the entire industrial chain from computing power, cloud services, and infrastructure to models, applications, and customer deployment.
The storm continues. The next eye of the storm may not necessarily be distillation; it is itself just one of many methods for optimizing models.
Why are so many companies investing so many resources, scrambling to participate in this intelligence competition? In the first half of this year, it was the explosive growth of coding and agents that reversed market expectations. Looking further ahead, if the scale and speed of large model application diffusion cannot keep up, how will the future of the model development race fluctuate? This is a question some people are already concerned about.
Text by Cheng Manqi
Edited by Song Wei