DeepSeek has open-sourced the inference engine it uses to serve its models in production, a step it said last year wasn’t yet practical. Dubbed DeepInfer , the framework is designed to let developers run DeepSeek’s V4.1 Flash model on their own infrastructure, with many of the same optimizations the company uses behind its API. Support for future DeepSeek models is also planned. “Running a model efficiently is a different matter altogether.” DeepSeek has, of course, long made its model weights available for developers to download and run using third-party software such as vLLM . But running a...

Source: The New Stack