ggml 0.28.0 adds fused sparse-attention mask op
ggml v0.28.0 introduces GGML_OP_TOP_K_MASK, which fuses top-k selection with sparse-attention masking into a single kernel across CPU, CUDA and Metal backends. The release also adds precision introspection APIs ggml_prec_get_acc() and ggml_prec_get_src(), plus a -1 sentinel skip index for MUL_MAT_ID and ADD_ID.
- GGML_OP_TOP_K_MASK fuses top-k and masking in one kernel on CPU, CUDA, Metal
- A -1 index in MUL_MAT_ID and ADD_ID explicitly skips a compute slot
- ggml_prec_get_acc() and ggml_prec_get_src() expose node precision
- Scale and sum ops now accept non-contiguous tensors
Read next
AI