vLLM-Omni: Fully Disaggregated Serving for Any-to-Any Multimodal Models

How do you serve a model that works across text, images, video and audio without creating bottlenecks between its components?How do you serve a model that works across text, images, video and audio without creating bottlenecks between its components?

Any-to-any multimodal models combine components with different compute requirements, including autoregressive language models and diffusion transformers. Coordinating them efficiently is a challenge for serving systems designed around a single type of model.

Join the Nebius Science Paper Club with Han Gao, a co-founder of vLLM-Omni and co-author of the paper, to explore how vLLM-Omni breaks these complex pipelines into stages that can run independently. He’ll discuss the system’s design and share broader insights into optimizing multimodal LLM inference.

Read the paper before the session →