NLTK vs Stanford NLP

Question

I have recently started to use NLTK toolkit for creating few solutions using Python.

I hear a lot of community activity regarding using Stanford NLP. Can anyone tell me the difference between NLTK and Stanford NLP? Are they two different libraries? I know that NLTK has an interface to Stanford NLP but can anyone throw some light on few basic differences or even more in detail.

Can Stanford NLP be used using Python?

Solution 1

Can anyone tell me what is the difference between NLTK and Stanford NLP? Are they 2 different libraries ? I know that NLTK has an interface to Stanford NLP but can anyone throw some light on few basic differences or even more in detail.

(I'm assuming you mean "Stanford CoreNLP".)

They are two different libraries.

• Stanford CoreNLP is written in Java
• NLTK is a Python library

The main functional difference is that NLTK has multiple versions or interfaces to other versions of NLP tools, while Stanford CoreNLP only has their version. NLTK also supports installing third-party Java projects, and even includes instructions for installing some Stanford NLP packages on the wiki.

Both have good support for English, but if you are dealing with other languages:

• Stanford CoreNLP comes with models for English, Chinese, French, German, Spanish, and Arabic.
• NLTK comes with corpora in additional languages like Portugese, Russian, and Polish. Individual tools may support even more languages (e.g. no Danish corpora, but has a DanishStemmer).

That said, which one is "best" will depend on your specific application and required performance (what features you are using, language, vocabulary, desired speed, etc.).

Can Stanford NLP be used using Python?

Yes, there are a number of interfaces and packages for using Stanford CoreNLP in Python (independent of NLTK).

Solution 2

The choice will depend upon your use case. NLTK is great for pre-processing and tokenizing text. It also includes a good POS tagger. Standford Core NLP for only tokenizing/POS tagging is a bit of overkill, because Standford NLP requires more resources.
But one fundamental difference is, you can't parse syntactic dependencies out of the box with NLTK. You need to specify a Grammar for that which can be very tedious if the text domain is not restricted. Whereas Standford NLP provides a probabilistic parser for general text as a down-loadable model, which is quite accurate. It also has built in NER (Named Entity Recognition) and more. Also I will recomend to take a look at Spacy, which is written in python, easy to use and much faster than CoreNLP.

Solution 3

It appears that you are new to NLP.

I have recently started to use NLTK toolkit

If indeed you are new to NLP, then the best thing would be to start simple. So ideally you would start off with nltk. I am relatively new to natural language processing (a few months old). I can confirm that for beginners, nltk is better, since it has a great and free online book which helps the beginner learn quickly.

Once you are comfortable and actually have a problem to solve, look at Stanford Core NLP to see if it will be better at solving your problem.

If you want to stick to NLTK, you can also access the Stanford CoreNLP API in NLTK.

Now for the similarities and differences:

Can anyone tell me what is the difference between NLTK and Stanford NLP ? Are they 2 different libraries?

Both offer natural language processing. Some of the most useful parts of Stanford Core NLP include the part-of-speech tagger, the named entity recognizer, sentiment analysis, and pattern learning.

The named entity recognizer is better in the Stanford Core NLP. Stanford Core NLP is better at grammatical functions for instance picking up subject, object, predictae (that is partially why I switched from nltk to Stanford Core NLP). As @user812786 said, NLTK has multiple interfaces to other versions of NLP tools. NLTK is also better for learning NLP. If you need to use multiple corpora, use NLTK, as you can easily access a wide multitude of text corpora and lexical resources. Both have POS tagging and sentiment analysis.

Can stanford NLP be used using Python ?

Yes absolutely. You can use StanfordNLP which is a Python natural language analysis package that is able to call the CoreNLP Java package. There are also multiple Python packages using the Stanford CoreNLP server

Solution 4

In 2020, Stanford released STANZA, Python library based on Stanford NLP. You can find it here https://stanfordnlp.github.io/stanza/

If you familiar with Spacy NLP, it quite similar :

>>> import stanza
>>> stanza.download('en') # download English model
>>> nlp = stanza.Pipeline('en') # initialize English neural pipeline
>>> doc = nlp("Barack Obama was born in Hawaii.") # run annotation over a sentence

Solution 5

NLTK can be used for the learning phase to and perform natural language process from scratch and basic level. Standford NLP gives you high-level flexibility to done task very fast and easiest way.

If you want fast and production use, can go for Standford NLP.

View more solutions