Python Scrapy's ImagesPipeline not working

403
December 15, 2016, at 12:44 PM

I have this in settings.py

ITEM_PIPELINES = {
 'scrapy.pipelines.images.ImagesPipeline': 1,
 'craigslist_tickets.pipelines.CraigslistTicketsPipeline': 2,
}
IMAGES_STORE = os.path.abspath(os.path.join(os.path.dirname( __file__ ), '..', 'images'))
DOWNLOADER_MIDDLEWARES = {
        'craigslist_tickets.crawlera_proxy_middleware.CrawleraProxyMiddleware': 200,
        'craigslist_tickets.retrymiddleware.Retry': 300
    }

And in items I also have

images = scrapy.Field()
image_urls = scrapy.Field()

This images folder also exists, I have confirmed. But I dont see images being downloaded and the images Item field is empty.

BTW, I can clearly see that image_urls have image links in it.

But I am getting error when Scrapy tries to download images.

Error downloading image from <GET https://iwebsite/00N0N_5x7aEwmjsQS_600x450.jpg> referred in <None>: 'X-Crawlera-Session'

I am using Crawlera proxies too.

Another hint: there is no folder named full inside images as I think it should be there.

READ ALSO
Preserve DOCTYPE of an lxml.etree parsed xml with namespace

Preserve DOCTYPE of an lxml.etree parsed xml with namespace

This question is almost the same as Preserving original doctype and declaration of an lxmletree parsed xml

489
Cocotb using generics/parameters in gate level simulation

Cocotb using generics/parameters in gate level simulation

I have successfully setup a Cocotb verification environment for my design, and I am happy the way it works for RTL (VHDL in my case)

390
How to add whitespace after string.punctuation in Python?

How to add whitespace after string.punctuation in Python?

I want to clean my reviews dataHere's my code :

497