JetBrains just released Mellum2, a 12-billion-parameter Mixture-of-Experts model trained from scratch on natural language and code. The headline number for engineers is the active parameter count: only 2.5B parameters fire per token. That’s the difference between a model that fits comfortably on a single GPU and one that requires a cluster.

Architecturally, Mellum2 is a sparse MoE. You get the representational capacity of a 12B dense model, but with inference cost closer to a 2.5B model. JetBrains reports more than 2x faster inference versus similarly sized open models. For high-throughput pipelines—think RAG post-processing, agent routing, or code completion in an IDE—that latency delta directly reduces serving cost and tail latency.

The model is Apache 2.0 licensed and available on Hugging Face. It targets text-and-code workloads exclusively; no multimodal bloat. That specialization keeps the model compact and easier to self-host on private infrastructure, which matters for teams dealing with proprietary codebases.

JetBrains positions Mellum2 as a “focal” model in multi-model systems: fast enough for routing, summarization, and tool selection, but not meant to replace larger reasoning models. The practical trade-off is clear: you trade benchmark parity (it’s competitive, not SOTA) for a much cheaper inference budget. For production systems that chain multiple model calls, that trade-off often wins.

The full technical report covers architecture, training setup, and evaluation methodology. Worth a read if you’re designing agent workflows or latency-sensitive code pipelines.

Source: Hugging Face Blog