b12535
The Vulkan backend in llama.cpp has been updated to implement stream-k MUL_MAT, which splits data chunks of 256 K elements evenly across all SMs. Workgroups that finish a tile write to the destination, while others write to scratch, with a resolve shader similar to split-k. The feature is implemented for scalar+cm1+cm2 but currently enabled only for cm2. A simple heuristic is used for enabling on coopmat2 due to large tile size, though tuning for other tile sizes is not planned for NVIDIA devices.


