Did DeepSeek copy OpenAI’s AI technology?

It is in this context that OpenAI has said that DeepSeek may have used a technique called “distillation,” which allows its model to learn from a pretrained model, in this case ChatGPT. While DeepSeek has been accused of intellectual property theft ever since it gained mainstream attention, some industry experts have dismissed these claims saying they stem from an inadequate understanding of how models such as DeepSeek are trained.

OpenAI’s suspicion about DeepSeek

OpenAI prohibits the practice of training a new AI model by repeatedly querying a larger, pre-trained model, a technique commonly referred to as distillation, according to their terms of use. And the company suspects DeepSeek may have tried something similar, which could be a breach of its terms.

“We know that groups in the P.R.C. (China) are actively working to use methods, including what’s known as distillation, to replicate advanced US AI models,” a spokesperson for OpenAI said in a statement. “We are aware of and reviewing indications that DeepSeek may have inappropriately distilled our models, and will share information as we know more.”

David Sacks, Trump’s AI adviser, told Fox News, “There’s substantial evidence that what DeepSeek did here is they distilled the knowledge out of OpenAI’s models… And I don’t think OpenAI is very happy about this.”

Industry players counter OpenAI’s assertions

Some, however, disagree with assertions that DeepSeek copied technology from OpenAI and the likes.

“There’s a lot of misconception that China ‘just cloned’ the outputs of OpenAI. This is far from true and reflects incomplete understanding of how these models are trained in the first place…”  Aravind Srinivas, CEO of Perplexity said in a post on X.

“DeepSeek R1 has figured out RL (reinforcement learning) finetuning. They wrote a whole paper on this topic called DeepSeek R1 Zero, where no SFT (supervised fine tuning) was used. And then combined it with some SFT to add domain knowledge with good rejection sampling (aka filtering). The main reason it’s so good is it learned reasoning from scratch rather than imitating other humans or models,” he added.

The idea of using reinforcement learning (RL) became a focus point for AI companies in 2024. “This new paradigm involves starting with the ordinary type of pretrained models, and then as a second stage using RL to add the reasoning skills,” explained Dario Amodei, CEO of Anthropic, in a blog post.

Supervised Fine-Tuning (SFT), is a process in machine learning where a pre-trained model is further trained (fine-tuned) on a labeled dataset specific to a particular task. This approach leverages the general knowledge the model has already acquired during its initial pre-training phase and adapts it to perform well on a more specialized task.

As per an attached summary with DeepSeek’s model on its Github page, the company said it applied reinforcement learning to the base model without relying on supervised fine-tuning as a preliminary step.

“This approach allows the model to explore chain-of-thought (CoT) for solving complex problems, resulting in the development of DeepSeek-R1-Zero. DeepSeek-R1-Zero demonstrates capabilities such as self-verification, reflection, and generating long CoTs, marking a significant milestone for the research community. Notably, it is the first open research to validate that reasoning capabilities of LLMs can be incentivized purely through RL, without the need for SFT. This breakthrough paves the way for future advancements in this area.,” the summary said.

OpenAI’s own copyright troubles

Around the world, and specifically in countries like the USA and India, there is growing scepticism of news publishers over concerns of copyrighted material, such as news reports, being used by companies like OpenAI for training their foundational models, without permission or payment.

Last November, news agency ANI had sued OpenAI in the Delhi High Court, accusing the company of unlawfully using Indian copyrighted material to train its AI models. Earlier this week, a number of digital news publishers, including The Indian Express, have filed an intervention in the case.

The contention is that companies like OpenAI have developed large language models (LLMs) by “training” on vast quantities of text, including, without a licence or permission, copyright-protected works. This “unlawful utilisation of copyrighted materials exclusively benefits OpenAI and its investors, to the detriment of the creative works across the entire industry in India,” said the Digital News Publishers Association (DNPA) said in a statement.

OpenAI is facing a number of similar lawsuits in other jurisdictions as well. In December 2023, The New York Times sued the company and Microsoft, citing “unlawful” use of copyrighted content. The publication has alleged that OpenAI and Microsoft’s large language models, which power ChatGPT and Copilot, “can generate output that recites Times content verbatim, closely summarises it, and mimics its expressive style.” This “undermine[s] and damage[s]” the Times’ relationship with readers, while also depriving it of “subscription, licensing, advertising, and affiliate revenue.”