cuBLASLt

cuBLASLt is NVIDIA's lightweight, high-performance library for highly-optimized General Matrix-to-matrix Multiply (GEMM) operations.

cuBLASLt (cuBLAS Light) delivers maximum throughput for critical deep learning and HPC workloads by focusing exclusively on advanced GEMM operations (Level 3 BLAS). It provides a flexible, multi-stage API that enables developers to programmatically select optimal algorithms and heuristics for specific GPU architectures, such as leveraging Tensor Cores on NVIDIA A100 or H100 GPUs. The library is engineered for low-latency kernel launches and supports mixed-precision compute (FP16, BF16, TF32, INT8), often employing kernel fusion to combine multiple operations and minimize overhead. This targeted optimization makes it the go-to tool for accelerating large-scale AI training and inference models.

https://docs.nvidia.com/cuda/cublas/index.html