The Medium post goes over various flavors of distillation, including response-based distillation, feature-based distillation and relation-based distillation. It also covers two fundamentally different modes of distillation – off-line and online distillation.
Since Chinese artificial intelligence (AI) start-up DeepSeek rattled Silicon Valley and Wall Street with its cost-effective models, the company has been accused of data theft through a practice that is common across the industry.
David Sacks says OpenAI has evidence that Chinese company DeepSeek used a technique called "distillation" to build a rival model.
White House AI czar David Sacks alleged Tuesday that DeepSeek had used OpenAI’s data outputs to train its latest models through a process called distillation.
DeepSeek’s success learning from bigger AI models raises questions about the billions being spent on the most advanced technology.
What I can say is that it's a little rich for OpenAI to suddenly be so very publicly concerned about the sanctity of proprietary data. Collectively, the contributions from copyrighted sources are significant enough that OpenAI has said it would be "impossible" to build its large-language models without them.
DeepSeek faces allegations of using OpenAI's outputs to train its AI. Explore the legal, ethical and competitive implications of this dispute
OpenAI CEO Sam Altman downplayed the significance of a new artificial intelligence (AI) model released by Chinese startup DeepSeek on Thursday, saying it did a “couple of nice things” but has been
"I don't think OpenAI is very happy about this," said the White House's AI czar, who suggested that DeepSeek used a technique called distillation.
After DeepSeek AI shocked the world and tanked the market, OpenAI says it has evidence that ChatGPT distillation was used to train the model.
Chinese artificial intelligence company DeepSeek disrupted Silicon Valley with the release of cheaply developed AI models that compete with flagship offerings from OpenAI — but the ChatGPT maker suspects they were built upon OpenAI data.