vllm.v1.attention.ops.triton_decode_attention
¶
Memory-efficient attention for decoding. It supports page size >= 1.
Functions:
-
decode_attention_fwd–sliding_windowlimits each row to its lastsliding_windowkeys.
decode_attention_fwd(q, k_buffer, v_buffer, o, lse, req_to_token, b_seq_len, attn_logits, num_kv_splits, sm_scale, page_size=1, logit_cap=0.0, k_scale=None, v_scale=None, is_mla=False, sliding_window=None)
¶
sliding_window limits each row to its last sliding_window keys.