python - nltk StanfordNERTagger : NoClassDefFoundError: org/slf4j/LoggerFactory (In Windows)

Question

NOTE: I am using Python 2.7 as part of Anaconda distribution. I hope this is not a problem for nltk 3.1.

I am trying to use nltk for NER as

import nltk
from nltk.tag.stanford import StanfordNERTagger 
#st = StanfordNERTagger('stanford-ner/all.3class.distsim.crf.ser.gz', 'stanford-ner/stanford-ner.jar')
st = StanfordNERTagger('english.all.3class.distsim.crf.ser.gz') 
print st.tag(str)

but i get

Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
    at edu.stanford.nlp.io.IOUtils.<clinit>(IOUtils.java:41)
    at edu.stanford.nlp.ie.AbstractSequenceClassifier.classifyAndWriteAnswers(AbstractSequenceClassifier.java:1117)
    at edu.stanford.nlp.ie.AbstractSequenceClassifier.classifyAndWriteAnswers(AbstractSequenceClassifier.java:1076)
    at edu.stanford.nlp.ie.AbstractSequenceClassifier.classifyAndWriteAnswers(AbstractSequenceClassifier.java:1057)
    at edu.stanford.nlp.ie.crf.CRFClassifier.main(CRFClassifier.java:3088)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 5 more

Traceback (most recent call last):
  File "X:jnk.py", line 47, in <module>
    print st.tag(str)
  File "X:Anaconda2libsite-packages
ltkagstanford.py", line 66, in tag
    return sum(self.tag_sents([tokens]), []) 
  File "X:Anaconda2libsite-packages
ltkagstanford.py", line 89, in tag_sents
    stdout=PIPE, stderr=PIPE)
  File "X:Anaconda2libsite-packages
ltkinternals.py", line 134, in java
    raise OSError('Java command failed : ' + str(cmd))
OSError: Java command failed : ['X:\PROGRA~1\Java\JDK18~1.0_6\bin\java.exe', '-mx1000m', '-cp', 'X:\stanford\stanford-ner.jar', 'edu.stanford.nlp.ie.crf.CRFClassifier', '-loadClassifier', 'X:\stanford\classifiers\english.all.3class.distsim.crf.ser.gz', '-textFile', 'x:\appdata\local\temp\tmpqjsoma', '-outputFormat', 'slashTags', '-tokenizerFactory', 'edu.stanford.nlp.process.WhitespaceTokenizer', '-tokenizerOptions', '"tokenizeNLs=false"', '-encoding', 'utf8']

but i can see that the slf4j jar is there in my lib folder. do i need to update an environment variable?

Edit

Thanks everyone for their help, but i still get the same error. Here is what i tried recently

import nltk
from nltk.tag import StanfordNERTagger 
print(nltk.__version__)
stanford_ner_dir = 'X:\stanford\'
eng_model_filename= stanford_ner_dir + 'classifiers\english.all.3class.distsim.crf.ser.gz'
my_path_to_jar= stanford_ner_dir + 'stanford-ner.jar'
st = StanfordNERTagger(model_filename=eng_model_filename, path_to_jar=my_path_to_jar) 
print st._stanford_model
print st._stanford_jar

st.tag('Rami Eid is studying at Stony Brook University in NY'.split())

and also

import nltk
from nltk.tag import StanfordNERTagger 
print(nltk.__version__)
st = StanfordNERTagger('english.all.3class.distsim.crf.ser.gz') 
print st._stanford_model
print st._stanford_jar
st.tag('Rami Eid is studying at Stony Brook University in NY'.split())

i get

3.1
X:stanfordclassifiersenglish.all.3class.distsim.crf.ser.gz
X:stanfordstanford-ner.jar

after that it goes on to print the same stacktrace as before. java.lang.ClassNotFoundException: org.slf4j.LoggerFactory

any idea why this might be happening? I updated my CLASSPATH as well. I even added all the relevant folders to my PATH environment variable.for example the folder where i unzipped the stanford jars, the place where i unzipped slf4j and even the lib folder inside the stanford folder. i have no idea why this is happening :(

Could it be windows? i have had problems with windows paths before

Update

1. The Stanford NER version i have is 3.6.0. The zip file says stanford-ner-2015-12-09.zip

2. I also tried using the stanford-ner-3.6.0.jar instead of stanford-ner.jar but still get the same error

3. When i right click on the stanford-ner-3.6.0.jar, i notice

i see this for all the files that i have extracted, even the slf4j files.could this be causing the problem?

4. Finally, why does the error message say

java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory

i do not see any folder named org anywhere

Update: Env variables

Here are my env variables

CLASSPATH
.;
X:jre1.8.0_60lib
t.jar;
X:stanfordstanford-ner-3.6.0.jar;
X:stanfordstanford-ner.jar;
X:stanfordlibslf4j-simple.jar;
X:stanfordlibslf4j-api.jar;
X:slf4jslf4j-1.7.13slf4j-1.7.13slf4j-log4j12-1.7.13.jar

STANFORD_MODELS
X:stanfordclassifiers

JAVA_HOME
X:PROGRA~1JavaJDK18~1.0_6

PATH
X:PROGRA~1JavaJDK18~1.0_6in;
X:stanford;
X:stanfordlib;
X:slf4jslf4j-1.7.13slf4j-1.7.13

anything wrong here?

Answer 1

EDITED

Note: The following answer will only work on:

• NLTK version 3.1
• Stanford Tools compiled since 2015-04-20

As both tools changes rather quickly and the API might look very different 3-6 months later. Please treat the following answer as temporal and not an eternal fix.

Always refer to https://github.com/nltk/nltk/wiki/Installing-Third-Party-Software for the latest instruction on how to interface Stanford NLP tools using NLTK!!

---

Step 1

First update your NLTK to the version 3.1 using

pip install -U nltk

or (for Windows) download the latest NLTK using http://pypi.python.org/pypi/nltk

Then check that you have version 3.1 using:

python3 -c "import nltk; print(nltk.__version__)"

Step 2

Then download the zip file from http://nlp.stanford.edu/software/stanford-ner-2015-04-20.zip and unzip the file and save to C:somepathostanford-ner (In windows)

Step 3

Then set the environment variable for CLASSPATH to C:somepathostanford-nerstanford-ner.jar

and the environment variable for STANFORD_MODELS to C:somepathostanford-nerclassifiers

Or in command line (ONLY for Windows):

set CLASSPATH=%CLASSPATH%;C:somepathostanford-nerstanford-ner.jar
set STANFORD_MODELS=%STANFORD_MODELS%;C:somepathostanford-nerclassifiers

(See https://stackoverflow.com/a/17176423/610569 for click-click GUI instructions for setting environment variables in Windows)

(See Stanford Parser and NLTK for details on setting environment variables in Linux)

Step 4

Then in python:

>>> from nltk.tag import StanfordNERTagger
>>> st = StanfordNERTagger('english.all.3class.distsim.crf.ser.gz') 
>>> st.tag('Rami Eid is studying at Stony Brook University in NY'.split())
[(u'Rami', u'PERSON'), (u'Eid', u'PERSON'), (u'is', u'O'), (u'studying', u'O'), (u'at', u'O'), (u'Stony', u'ORGANIZATION'), (u'Brook', u'ORGANIZATION'), (u'University', u'ORGANIZATION'), (u'in', u'O'), (u'NY', u'O')]

Without setting the environment variables, you can try:

from nltk.tag import StanfordNERTagger

stanford_ner_dir = 'C:\somepathostanford-ner'
eng_model_filename= stanford_ner_dir + 'classifiersenglish.all.3class.distsim.crf.ser.gz'
my_path_to_jar= stanford_ner_dir + 'stanford-ner.jar'

st = StanfordNERTagger(model_filename=eng_model_filename, path_to_jar=my_path_to_jar) 
st.tag('Rami Eid is studying at Stony Brook University in NY'.split())

See more detailed instructions on Stanford Parser and NLTK