Skip to content

vllm.model_executor.kernels.attention.dsa.dcp_indexer_cutedsl

Functions:

_dcp_merge_topk_and_interleave(vllm_config)

(topk, cp_interleave) the DCP merge runs with.

Kpool indexers (GLM-5.x) merge pool ids: topk and the interleave are both in units of index_kpool tokens.

Source code in vllm/model_executor/kernels/attention/dsa/dcp_indexer_cutedsl.py
def _dcp_merge_topk_and_interleave(vllm_config: Any) -> tuple[int, int]:
    """(topk, cp_interleave) the DCP merge runs with.

    Kpool indexers (GLM-5.x) merge pool ids: topk and the interleave are both
    in units of ``index_kpool`` tokens.
    """
    hf_text_config = vllm_config.model_config.hf_text_config
    topk = hf_text_config.index_topk
    cp_interleave = vllm_config.parallel_config.cp_kv_cache_interleave_size
    kpool = getattr(hf_text_config, "index_kpool", None) or 1
    if kpool > 1:
        topk //= kpool
        cp_interleave //= kpool
    return topk, cp_interleave

dcp_topk_id_bits(max_model_len)

Bits that hold every candidate token id (a position below max_model_len).

Source code in vllm/model_executor/kernels/attention/dsa/dcp_indexer_cutedsl.py
def dcp_topk_id_bits(max_model_len: int) -> int:
    """Bits that hold every candidate token id (a position below max_model_len)."""
    return max(1, min(32, (max_model_len - 1).bit_length()))

stable_topk_from_gathered_candidates_cutedsl(gathered, topk, out=None, score_bits=32, id_bits=32)

score_bits: high score bits that rank (16 for upcast BF16 logits). id_bits: bits that hold every token id.

Source code in vllm/model_executor/kernels/attention/dsa/dcp_indexer_cutedsl.py
def stable_topk_from_gathered_candidates_cutedsl(
    gathered: torch.Tensor,
    topk: int,
    out: torch.Tensor | None = None,
    score_bits: int = 32,
    id_bits: int = 32,
) -> torch.Tensor:
    """`score_bits`: high score bits that rank (16 for upcast BF16 logits).
    `id_bits`: bits that hold every token id."""
    if out is None:
        out = torch.empty(
            (gathered.shape[0], topk),
            dtype=torch.int32,
            device=gathered.device,
        )
    _STABLE_TOPK_FROM_GATHERED_CANDIDATES_KERNEL(
        gathered, out, topk=topk, score_bits=score_bits, id_bits=id_bits
    )
    return out