Introducing Humming, a quantized inference kernel library from Ant Group and the vLLM community, with broad GPU and quantization support, dense and MoE performance results, and vLLM integration.
Announcing Humming: Fast, Flexible Kernels for Quantized Inference
Source:vLLM Blog ↗