Cross-validation

A model validation technique that partitions a dataset into multiple subsets (folds) for iterative training and testing, providing a robust, low-variance estimate of performance on unseen data.

Cross-validation is the essential procedure for assessing a model’s generalization ability and preventing overfitting (memorization of training data). The standard approach, $k$-fold cross-validation, splits the dataset into $k$ equal-sized partitions, or folds. The process runs $k$ times: in each iteration, the model trains on $k-1$ folds and is validated on the single remaining fold. For example, a 5-fold CV trains five distinct models and generates five performance scores (e.g., accuracy or F1-score). The final, reported metric is the average of these $k$ scores, offering a significantly more stable and reliable performance measure than a single train/test split.

https://scikit-learn.org/stable/modules/cross_validation.html