python - Why testing error rate increases at high values of K in KNN algorithm?

Question

I am getting the error rates like this up to 20 values what might be the reason for this ? k_values: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20] Error [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0020000000000000018, 0.0020000000000000018, 0.0020000000000000018, 0.0020000000000000018,0.0020000000000000018, 0.0020000000000000018, 0.006000000000000005, 0.0040000000000000036, 0.008000000000000007,0.006000000000000005, 0.010000000000000009, 0.008000000000000007, 0.014000000000000012, 0.01200000000000001] these are my testing error rates

I want to know the reason why the error rate increases with increase in k values?

Answer 1

The parameter K in KNN controls the complexity of the model. You don't give details of your specific problem, but what you likely seeing is the the bias/variance trade-off. This post is a good read about it.

Usually you try different values of the hyper parameters from the model (the value of K in the KNN) in a validation set and keep the best one. Notice that this validation set is not the same as the test set.