[Model] Add Dory support
The Issue & Context
Before this PR, vLLM did not support the Dory1.0-4B model, a latent-recurrent LLM with a unique looped architecture and hybrid sliding window attention, limiting users from running inference on this NVIDIA model.
Root Cause Analysis
The lack of a dedicated model implementation and configuration for Dory1.0-4B meant vLLM's model executor and transformers utilities could not correctly load or run this model, which requires handling loop counts for repeated application of a shared block and managing its interleaved attention layers.
Solution Strategy & Architectural Pattern
This PR introduces a new model class in vllm/model_executor/models/dory.py that implements Dory's normalized-GPT architecture with looped layers and hybrid sliding window attention. It adds corresponding configuration files in vllm/transformers_utils/configs/dory.py and integrates the model into the registry. Extensive tests were added to validate correct loading, configuration, and inference behavior, ensuring minimal impact on existing core components by reusing existing infrastructure where possible.
Key Takeaway for Contributors
When integrating models with non-standard architectures like looped layers and hybrid attention, encapsulate their unique execution logic in dedicated classes and configurations to maintain modularity and reuse existing infrastructure, enabling scalable support for diverse model designs.