python - Execution time of AdaBoost with SVM base classifier

Question

I just made a Adaboost Classifier with these parameters,

1.n_estimators = 50

2.base_estimator = svc (support vector classifier)

3.learning_rate = 1

here is my code:

from sklearn.ensemble import AdaBoostClassifier
from sklearn.svm import SVC

svc = SVC(kernel = 'linear',probability = True)

ABC = AdaBoostClassifier(n_estimators = 50, base_estimator = svc, learning_rate = 1)

ABC.fit(X,Y)

Dataset has 18 independent variables and 1 categorical dependent variable dataset has 10480 datapoints

whenever i run this it will take so much time but no any result.

Is there any way to check execution time? Or any better way to do this?

Answer 1

In practice, we never use SVMs as base classifiers for Adaboost.

Adaboost (and similar ensemble methods) were conceived using decision trees as base classifiers (more specifically, decision stumps, i.e. DTs with a depth of only 1); there is good reason why still today, if you don't specify explicitly the base_classifier argument, it assumes a value of DecisionTreeClassifier(max_depth=1). DTs are suitable for such ensembling because they are essentially unstable classifiers, which is not the case with SVMs, hence the latter are not expected to offer much when used as base classifiers.

On top of this, SVMs are computationally much more expensive than decision trees (let alone decision stumps), which is the reason for the long processing times you have observed.

Unless you have a very good reason to stick to SVMs as base classifiers (and I highly doubt that you do), remove the base_estimator = svc in order to revert to the default setting, and most probably you will be fine.