<details open> vulkan: Implement stream-k MUL_MAT (#28528) * vulkan: Implement stream-k MUL_MAT Stream-k tries to split chunks of 256 K elements evenly across all SMs. Workgroups that finish a tile write to dst, others write to scratch, and there's a resolve shader similar to split-k. The shader code is implemented for scalar+cm1+cm2, but only currently enabled for cm2. The heuristic for enabling on coopmat2 is pretty simple due to the large tile size (one workgroup per SM), but would be more complex for the tile sizes being used for cm1, and it doesn't make sense for me to tune that on NVIDIA devices. Assisted-by: Codex * address review feedback </details> **Website:** - <https://llama.app> **Attestations:** - <https://github.com/ggml-org/llama.cpp/attestations/69624611> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-arm64.tar.gz) - [Ubuntu 24.04 s390x (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-s390x.tar.gz) - [Ubuntu 26.04 s390x (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-26.04-s390x.tar.gz) - [Ubuntu 26.04 s390x (zDNN)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-zdnn-s390x.tar.gz) - [Ubuntu x64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-vulkan-x64.tar.gz) - [Ubuntu arm64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-vulkan-arm64.tar.gz) - [Ubuntu x64 (CUDA 12)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-cuda-12.8-x64.tar.gz) - [CUDA 12.8 libraries](https://github.com/ggml-org/llama.cpp/releases/download/b12535/cudart-llama-b12535-bin-ubuntu-cuda-12.8-x64.tar.gz) - [Ubuntu x64 (CUDA 13)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-cuda-13.5-x64.tar.gz) - [CUDA 13.5 libraries](https://github.com/ggml-org/llama.cpp/releases/download/b12535/cudart-llama-b12535-bin-ubuntu-cuda-13.5-x64.tar.gz) - [Ubuntu arm64 (CUDA 13)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-cuda-13.5-arm64.tar.gz) - [CUDA 13.5 libraries](https://github.com/ggml-org/llama.cpp/releases/download/b12535/cudart-llama-b12535-bin-ubuntu-cuda-13.5-arm64.tar.gz) - [Ubuntu x64 (ROCm 10.0)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-rocm-10.0-x64.tar.gz) - [Ubuntu x64 (OpenVINO)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-openvino-2026.4.1-x64.tar.gz) - [Ubuntu x64 (SYCL FP32)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-sycl-fp32-x64.tar.gz) - [Ubuntu x64 (SYCL FP16)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-ubuntu-sycl-fp16-x64.tar.gz) - [Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-linux-arm64-snapdragon.tar.gz) - [setup guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/linux.md) **Android:** - [Android arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-android-arm64.tar.gz) - [Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-android-arm64-snapdragon.tar.gz) - [setup guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/README.md) **Windows:** - [Windows x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-win-cpu-x64.zip) - [Windows arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-win-cpu-arm64.zip) - [Windows arm64 (OpenCL Adreno)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-win-opencl-adreno-arm64.zip) - [Windows x64 (CUDA 12)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-win-cuda-12.4-x64.zip) - [CUDA 12.4 DLLs](https://github.com/ggml-org/llama.cpp/releases/download/b12535/cudart-llama-bin-win-cuda-12.4-x64.zip) - [Windows x64 (CUDA 13)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-win-cuda-13.5-x64.zip) - [CUDA 13.5 DLLs](https://github.com/ggml-org/llama.cpp/releases/download/b12535/cudart-llama-bin-win-cuda-13.5-x64.zip) - [Windows arm64 (CUDA 13)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-win-cuda-13.5-arm64.zip) - [CUDA 13.5 DLLs](https://github.com/ggml-org/llama.cpp/releases/download/b12535/cudart-llama-bin-win-cuda-13.5-arm64.zip) - [Windows x64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-win-vulkan-x64.zip) - [Windows arm64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-win-vulkan-arm64.zip) - [Windows x64 (OpenVINO)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-win-openvino-2026.4.1-x64.zip) - [Windows x64 (SYCL)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-win-sycl-x64.zip) - [Windows x64 (ROCm 10.0)](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-bin-win-rocm-10.0-x64.zip) **openEuler:** - [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23705) - openEuler x86 (310p) - openEuler x86 (910b, ACL Graph) - openEuler aarch64 (310p) - openEuler aarch64 (910b, ACL Graph) **UI:** - [UI](https://github.com/ggml-org/llama.cpp/releases/download/b12535/llama-b12535-ui.tar.gz)
AI NEWS JOURNAL
llama.cpp releasesAI NEWS / JOURNAL
b12535
The Vulkan backend in llama.cpp has been updated to implement stream-k MUL_MAT, which splits data chunks of 256 K elements evenly across all SMs. Workgroups that finish a tile write to the destination, while others write to scratch, with a resolve shader similar to split-k. The feature is implemented for scalar+cm1+cm2 but currently enabled only for cm2. A simple heuristic is used for enabling on coopmat2 due to large tile size, though tuning for other tile sizes is not planned for NVIDIA devices.
翻译服务暂未运行,当前显示原文。