b12563
The update addresses a precision issue in CUDA's per-expert fallback path where the MUL_MAT_ID operation was not properly preserving precision due to an unset operation type. The fix sets the operation to GGML_OP_MUL_MAT and reverts an earlier change that forced fp32 output, ensuring consistent precision handling across quantized matrix multiplication paths. This resolves issues with overflow to infinity in fp16 when results exceed 65504, particularly affecting models using IQ1_M weights with large batches.



