K-Nearest Neighbors

A non-parametric supervised learning algorithm that predicts labels or values by identifying the K most similar data points in a feature space.

KNN operates on a core principle: similar data points exist in close proximity. It calculates the distance (typically Euclidean or Manhattan) between a query point and every sample in the dataset. For classification, the algorithm takes a majority vote of the K neighbors; for regression, it averages their output values. It is a lazy learner (it skips an explicit training phase) that often utilizes KD-trees or Ball Trees to accelerate search queries. Engineers deploy it for recommendation systems and pattern recognition tasks like the MNIST digit dataset. Performance relies on proper feature scaling and selecting an optimal K value (usually an odd integer) to avoid ties.

https://scikit-learn.org/stable/modules/neighbors.html