python - SKLearn how to get decision probabilities for LinearSVC classifier

Question

I am using scikit-learn's linearSVC classifier for text mining. I have the y value as a label 0/1 and the X value as the TfidfVectorizer of the text document.

I use a pipeline like below

 pipeline = Pipeline([
    ('count_vectorizer',   TfidfVectorizer(ngram_range=(1, 2))),
    ('classifier',         LinearSVC())
  ])

For a prediction, I would like to get the confidence score or probability of a data point being classified as 1 in the range (0,1)

I currently use the decision function feature

pipeline.decision_function(test_X)

However it returns positive and negative values that seem to indicate confidence. I am not too sure about what they mean either.

However, is there a way to get the values in range 0-1?

For example here is the output of the decision function for some of the data points

    -0.40671879072078421, 
    -0.40671879072078421, 
    -0.64549376401063352, 
    -0.40610652684648957, 
    -0.40610652684648957, 
    -0.64549376401063352, 
    -0.64549376401063352, 
    -0.5468745098794594, 
    -0.33976011539714374, 
    0.36781572474117097, 
    -0.094943829974515004, 
    0.37728641897721765, 
    0.2856211778200019, 
    0.11775493140003235, 
    0.19387473663623439, 
    -0.062620918785563556, 
    -0.17080866610522819, 
    0.61791016307670399, 
    0.33631340372946961, 
    0.87081276844501176, 
    1.026991628346146, 
    0.092097790098391641, 
    -0.3266704728249083, 
    0.050368652422013376, 
    -0.046834129250376291, 

Answer 1

You can't. However you can use sklearn.svm.SVC with kernel='linear' and probability=True

It may run longer, but you can get probabilities from this classifier by using predict_proba method.

clf=sklearn.svm.SVC(kernel='linear',probability=True)
clf.fit(X,y)
clf.predict_proba(X_test)