from scrapy.selector import selector error
Solution 1
Try importing HtmlXPathSelector instead.
from scrapy.selector import HtmlXPathSelector
And then use the .select() method to parse out your html. For example,
sel = HtmlXPathSelector(response)
site_names = sel.select('//ul/li')
If you are following the tutorial on the Scrapy site (http://doc.scrapy.org/en/latest/intro/tutorial.html), the updated example would look like this:
from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector
class DmozSpider(BaseSpider):
name = "dmoz"
allowed_domains = ["dmoz.org"]
start_urls = [
"http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
"http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
]
def parse(self, response):
sel = HtmlXPathSelector(response)
sites = sel.select('//ul/li')
for site in sites:
title = site.select('a/text()').extract()
link = site.select('a/@href').extract()
desc = site.select('text()').extract()
print title, link, desc
Hope this helps!
Solution 2
I encounter the same problem. I think there is something wrong with your scrapy version.
You could type scrapy version -v
into cmd to check the version. As far as I know, the newest version is 0.24.4 (2014.10.23). You could visit http://scrapy.org/ to find the newest.
SMPLGRP
Updated on June 12, 2022Comments
-
SMPLGRP almost 2 years
I am unable to do the following:
from scrapy.selector import Selector
The error is:
File "/Desktop/KSL/KSL/spiders/spider.py", line 1, in from scrapy.selector import Selector ImportError: cannot import name Selector
It is as if LXML is not installed on my machine, but it is. Also, I thought this was a default module built into scrapy. Maybe not?
Thoughts?
-
chrishiestand over 10 yearsThis works, but you're missing a colon at the definition of parse(). I'd edit it, but stackoverflow has that weird 6 character minimum for editing....