DeepSeek released DeepInfer, the inference framework behind its large-scale model services, on October 20. The framework supports Nvidia GPUs and Huawei Ascend processors and provides capabilities including distributed inference, speculative decoding and structured output. DeepSeek said the release is intended to help developers and businesses improve throughput and latency while reducing computing costs and energy consumption.
The company also open-sourced deepseek-recipe, a component that converts API requests and handles model input and output processing in line with DeepSeek’s own service. DeepInfer currently supports DeepSeek-V4.1-Flash-1017, and the company plans to continue releasing inference code and optimizations for future models through the project. [DeepSeek]