New to this? Read it in simple words
- DeepSeek shared DeepInfer, the software that runs its own AI service, for free.
- It works with one model, DeepSeek V4.1 Flash, on Nvidia and Huawei chips.
- Some parts of DeepSeek’s own system are still not included.
- DeepSeek says one cluster serves 155 trillion tokens a day. Nobody outside has checked it.
- Inference
- Running a trained AI model so that it answers questions.
- Apache 2.0
- A free licence that lets anyone use, change and share the code.
- Token
- A small piece of text, often part of a word, that AI models read and write.
An inference framework is the software that answers users once an AI model is trained. It splits the work across chips, manages requests and produces the replies. DeepSeek, a Chinese AI company, published its own on Tuesday, October 20.
What DeepSeek released
DeepInfer is on GitHub under the Apache 2.0 licence, which lets anyone use and change the code. DeepSeek says it is the framework that “powers DeepSeek’s large-scale production services”.
It works with one model so far, DeepSeek V4.1 Flash, in the version DeepSeek released on October 17. It runs on Nvidia’s SM100 GPUs, the Blackwell data-centre chips, and on Huawei’s Ascend 950 chips.
DeepSeek also released deepseek-recipe, a small library that prepares requests and replies the way its official API does. The New Stack notes that this library first appeared quietly in September.
From DeepSeek’s GitHub repository and its deployment notes.
What is still missing
This release covers offline inference, DeepSeek’s documents say. The parts that manage the memory of long chats and spread requests across many servers are “not yet fully open-sourced”, because they depend on DeepSeek’s own systems.
Two features that make answers repeatable work only on Nvidia chips. Older Nvidia chips such as Hopper are not on the list, The New Stack points out.
In April 2025, DeepSeek had decided not to release its whole engine, because it was too tied to its own servers. It shared single parts instead. This release goes further, but it still leaves some of the system out.
DeepSeek’s numbers, and the Huawei angle
DeepSeek says a single cluster now serves 155 trillion tokens a day. It puts its revenue at $38 per GPU-hour at its off-peak prices. Nobody outside has checked these figures.
Its speed charts were measured on Nvidia chips: 4 GPUs for the first step of each answer and 64 for writing it. 36Kr and The New Stack both stress that Huawei’s chip is supported from the first day.
US export rules limit China’s access to Nvidia’s best chips, so working Huawei support matters to Chinese companies. The New Stack recalls that DeepSeek and Huawei are already working on tools for Ascend chips.
Sources
Every fact in this story comes from the sources below. Open them to check our work.
- 1Primary source · October 20, 2026deepseek-ai/DeepInfer: High-performance, large-scale inference framework for DeepSeek models DeepSeek (GitHub)
- 2
- 3
- 4Research · October 20, 2026DeepSeek open-sources the inference engine behind its API. And Huawei gets equal billing with Nvidia. The New Stack
- 5Research · October 20, 2026DeepSeek Fully Open-Sources Inference Framework: NVIDIA & Huawei Ascend Compatible 36Kr
- 6
What the code includes and which chips it supports comes from DeepSeek’s own repository. The traffic, revenue and speed figures are DeepSeek’s claims, and no outside group has tested them yet.
DeepSeek freely shares the software behind its own AI service, running on Nvidia and Huawei chips, though some parts stay closed. How we rate the mood