DeepSeek claims its reasoning model beats OpenAI’s o1 on certain benchmarks

Chinese AI lab Dipsik has released an open source version of Dipsik-R1, its so-called logic model, which it claims works alongside OpenAI. o1 In certain AI benchmarks.

R1 is available from AI dev platform Hugging Face under an MIT license, meaning it can be used commercially without restrictions. According to DeepSeek, the R1 beats the o1 in AIME, MATH-500 and SWE-Bench verified benchmarks. AIME employs other models to evaluate the performance of one model, while MATH-500 is a collection of word problems. SWE-Bench Certified, meanwhile, focuses on programming tasks.

Being a logic model, R1 effectively fact-checks itself, which This helps avoid some of the problems that commonly trip up models. Reasoning models take a bit longer — typically seconds to minutes — to reach a solution than a simple irrational model. The upside is that they tend to be more reliable in domains like physics, science, and math.

R1 has 671 billion parameters, Dipsik revealed Technical report. Parameters roughly correspond to a model’s problem-solving ability, and models with more parameters generally perform better than models with fewer parameters.

671 billion parameters is huge, but Dipsic has also released “distilled” versions of R1 ranging in size from 1.5 billion parameters to 70 billion parameters. The smallest can run on a laptop. As with the full R1, it requires beefier hardware, but it is is Available through DeepSeek’s API at 90%-95% lower cost than OpenAI’s o1.

R1 has a downside. Being a Chinese model, it is subject to Benchmarking Confirmation by China’s Internet regulator that his responses “embody core socialist values.” R1 won’t answer questions about Tiananmen Square, for example, or Taiwan’s autonomy.

a lot Chinese AI systemincluding Other logic models, decrease To respond to matters that may raise the ire of the country’s regulators, such as speculation about Xi Jinping rule

R1 comes days after the outgoing Biden administration Recommended strict AI Technology Export Rules and Restrictions for Chinese Enterprises. Chinese companies were already barred from buying advanced AI chips, but if the new rules go into effect as written, companies will face strict caps on both the semiconductor technology and models needed to bootstrap sophisticated AI systems.

A Policy document Last week, OpenAI urged the US government to help develop US AI, lest Chinese models match or surpass them in capabilities. in a the interview With The Information, OpenAI’s VP of policy Chris Lehane identified Dipsic’s corporate parent High Flyer Capital Management as a firm of particular concern.

So far, at least three Chinese labs – Dipsik, Alibaba and likewhich is owned by the Chinese Unicorn Moonshot AI — have created models that they claim rival o1. (Note, Dipsic was first — it announcement A preview of R1 in late November.) a post Dean Ball, an AI researcher at George Mason University, told X that the trend suggests Chinese AI labs will continue to be “fast followers.”

“Impressive performance of DeepSeek’s distilled model […] This means that very capable logicians will continue to expand and be able to run on local hardware,” Ball writes, “out of sight of any top-down control system.”