NLTK Lookup Error

Question

While running a Python script using NLTK I got this:

Traceback (most recent call last):
  File "cpicklesave.py", line 56, in <module>
    pos = nltk.pos_tag(words)
  File "/usr/lib/python2.7/site-packages/nltk/tag/__init__.py", line 110, in pos_tag
    tagger = PerceptronTagger()
  File "/usr/lib/python2.7/site-packages/nltk/tag/perceptron.py", line 140, in __init__
    AP_MODEL_LOC = str(find('taggers/averaged_perceptron_tagger/'+PICKLE))
  File "/usr/lib/python2.7/site-packages/nltk/data.py", line 641, in find
    raise LookupError(resource_not_found)
LookupError:
**********************************************************************
  Resource u'taggers/averaged_perceptron_tagger/averaged_perceptro
  n_tagger.pickle' not found.  Please use the NLTK Downloader to
  obtain the resource:  >>> nltk.download()
  Searched in:
    - '/root/nltk_data'
    - '/usr/share/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/lib/nltk_data'
    - '/usr/local/lib/nltk_data'
**********************************************************************

Can anyone explain the problem?

Solution 1

Use

>>> nltk.download()

to install the missing module (the Perceptron Tagger).

(check also the answers to Failed loading english.pickle with nltk.data.load)

Solution 2

First answer said the missing module is 'the Perceptron Tagger', actually its name in nltk.download is 'averaged_perceptron_tagger'

You can use this to fix the error

nltk.download('averaged_perceptron_tagger')

Solution 3

TL;DR

import nltk
nltk.download('averaged_perceptron_tagger')

Or to download all packages + data + docs:

import nltk
nltk.download('all')

See How do I download NLTK data?

Solution 4

Install all nltk resources in one line:

python3 -c "import nltk; nltk.download('all')"

the data will be saved at ~/nltk_data

---

You can also substitute "all" for "averaged_perceptron_tagger" to install only this module.

Solution 5

Problem: Lookup error when extracting count vectorizer from scikit learn. Below is code snippet.

from sklearn.feature_extraction.text import CountVectorizer
bow_transformer = CountVectorizer(analyzer=text_process).fit(X)

Solution: Try to run the below code and then try to install the stopwords from corpora natural language processing toolkit!!

import nltk
nltk.download()

View more solutions

Comments

[Papples]

it is python -m nltk.downloader averaged_perceptron_tagger if you want to download it from the command line

[Pyd]

Hi, may I know where this content will be saved after downloading all the nltk data by using nltk.download("all ")

[alvas]

See stackoverflow.com/questions/22211525/… and more specifically stackoverflow.com/a/36383314/610569

[Shiv Shankar]

nltk.download(), this is working fine. already marked as correct answer.

[Lucas Azevedo]

@ShivShankar you need to do python3 then import nltk, nltk.download() which will put you on a download prompt asking the package name that you want to download. It is not a wrong solution, it's just not as simple as the one I'm proposing.

[Andy]

I don't believe the punkt tokenizer is the missing piece here. See, for example, Lucas' answer.

[Golden Lion]

nltk.download starts a large download of data: all, all-corpora, all-nltk, book, popular,test, and thirdparty

[Partha Paul]

Hi, Its just downloads all the packages and stops...rest codes don't execute