INT8 Tensor Cores

Accelerate deep learning inference: INT8 Tensor Cores deliver high-throughput, low-precision matrix math on NVIDIA GPUs.

INT8 Tensor Cores are specialized hardware units on NVIDIA GPUs (Volta architecture and later) engineered to maximize deep learning inference performance. By utilizing 8-bit integer precision, they drastically reduce memory footprint and computational time compared to FP32 or FP16. This low-precision capability, combined with high-speed matrix multiplication, is critical for deploying large AI models at low latency and high throughput. Modern architectures, including Hopper, continue to advance this technology, ensuring massive speedups for production AI workloads across the data center and the edge.

https://www.nvidia.com/en-us/data-center/tensor-cores/