Skip to content

vllm.models.mistral.common.layer_spec

Classes:

  • MistralLayerSpec –

    Architecture choices of one decoder layer, resolved from the config.

MistralLayerSpec dataclass

Architecture choices of one decoder layer, resolved from the config.

Source code in vllm/models/mistral/common/layer_spec.py
@dataclass(frozen=True)
class MistralLayerSpec:
    """Architecture choices of one decoder layer, resolved from the config."""

    layer_idx: int
    use_rope: bool
    # Layer whose latent KV cache this layer attends to (itself if it owns one).
    kv_source_layer: int
    # Sparse (DSA) attention: whether this layer runs its own indexer. Layers
    # without one reuse the top-k indices of the closest preceding indexer.
    use_sparse: bool
    has_indexer: bool
    indexer_use_rope: bool
    is_moe: bool

    @property
    def shares_kv(self) -> bool:
        return self.kv_source_layer != self.layer_idx