A vLLM Deep Dive: Distributed Inference, KV Cache Evolution, and Model Compression

vLLM Community Night — Tokyo brings together core vLLM contributors and production engineers for a technical evening focused on high-throughput, low-latency LLM inference. Hosted in Roppongi-Itchome, the event features five talks covering the current state of vLLM, the evolution of the KV cache as a distributed component, practical model compression via Fujitsu's OneComp framework, distributed inference on AWS (SageMaker HyperPod, ParallelCluster, EFA/SRD networking, prefill-decode disaggregation), and an in-the-trenches production post-mortem from Shisa.AI. Speakers include Tun Jian Tan (vLLM committer, Embedded LLM), Yuma Ichikawa (Fujitsu), Toshinobu Akazawa (AWS), Tony Valderrama (Momento), and Leonard Lin (Shisa.AI). The agenda pairs low-level optimization topics with system-level scaling and real-world benchmarks, evals, and hardware-specific tuning tricks. Light dinner and networking are included. The event is aimed at AI/ML engineers, LLM researchers, infrastructure and platform builders, AI product folks, and open-source contributors interested in the future of LLM serving.