画像フォルダを作成し保存したい

Question

実現したいこと

スクレイピングで抽出した画像URLをbook_imagesフォルダへ格納するという処理を講座で習ったのですが、その通りにやっても画像URLは取得できるのですが、book_imagesフォルダが作成されません。どこかが間違っているのだとは思うのですが、見つけられませんでした。教えて頂きたいです。よろしくお願いします。

BOT_NAME = 'kinokuniya'

SPIDER_MODULES = ['kinokuniya.spiders']
NEWSPIDER_MODULE = 'kinokuniya.spiders'

ROBOTSTXT_OBEY = True


DOWNLOAD_DELAY = 3

ITEM_PIPELINES = {
   'kinokuniya.pipelines.CheckItemPipeline': 100,
    'scrapy.pipelines.images.ImagesPipeline': 400,
}

IMAGE_STORE = r'/Users/name/Desktop/books_toscrape/kinokuniya/book_images'
IMAGE_URLS_FIELD = 'image_urls'
middleware-settings
HTTPCACHE_ENABLED = True
HTTPCACHE_EXPIRATION_SECS = 0
HTTPCACHE_DIR = 'httpcache'
#HTTPCACHE_IGNORE_HTTP_CODES = []
#HTTPCACHE_STORAGE = 'scrapy.extensions.httpcache.FilesystemCacheStorage'

FEED_EXPORT_ENCODING='utf-8'

#処理する量を一つに限定する
CONCURRENT_REQUESTS = 1

#優先順を階層の浅いところから処理する
DEPTH_PRIORITY = 1

#リクエストを受けた最後から処理する⇨最初から処理するへ変更
SCHEDULER_DISK_QUEUE = 'scrapy.squeues.PickleFifoDiskQueue'
SCHEDULER_MEMORY_QUEUE = 'scrapy.squeues.FifoMemoryQueue'

import scrapy
from scrapy.linkextractors import LinkExtractor
from scrapy.spiders import CrawlSpider, Rule
import logging
from kinokuniya.items import BookItem
from scrapy.loader import ItemLoader


class ComputerBooksSpider(CrawlSpider):
    name = 'computer_books'
    allowed_domains = ['www.kinokuniya.co.jp']
    start_urls = ['https://www.kinokuniya.co.jp/f/dsd-101001037028005-06-']

    rules = (
        #書籍２０冊分のURLを辿る
        Rule(LinkExtractor(restrict_css='h3.heightLine-2 > a'), callback='parse_item', follow=False),
        #「次へ」へ行く
        # Rule(LinkExtractor(restrict_css='a:contains("次へ")')),
    )

    def parse_item(self, response):
        logging.info(response.url)

        loader = ItemLoader(item=BookItem(),response=response)
        loader.add_css('title','h3[itemprop="name"]::text'),
        loader.add_css('author','div.infobox.ml10.mt10 > ul > li > a::text'),
        loader.add_css('price','span.sale_price::text'),
        loader.add_css('publisher','a[href*=publisher-key]::text'),
        loader.add_css('size','div.infbox.dotted.ml10.mt05.pt05 > ul > li:first-of-type::text'),
        loader.add_css('page','div.infbox.dotted.ml10.mt05.pt05 > ul > li:first-of-type::text'),
        loader.add_css('isbn','li[itemprop="identifier"]::text'),
        #画像URL
        loader.add_value('image_urls',[response.urljoin(response.css('img[itemprop="image"]::attr(src)').get())])
    
        yield loader.load_item()

import scrapy
from itemloaders.processors import TakeFirst, MapCompose, Join

def strip_yen(element):
    if element:
        return element.replace('¥','')
    return element

def strip_comma(element):
    if element:
        return element.replace(',','')
    return element


def convert_integer(element):
    if element:
        return int(element)
    return 0

def get_size(element):
    if element:
        return element.split('／')[0].strip().replace('サイズ ','').replace('判','')
    return element

def get_page(element):
    if element:
        return element.split('／')[1].replace('ページ数 ','').replace('p','')
    return element

def strip_isbn(element):
    if element:
        return element.replace('商品コード ','')
    return element

class BookItem(scrapy.Item):
    title = scrapy.Field(
        input_processor = MapCompose(str.lstrip),
        output_processor = Join(' ') 
    )
    author = scrapy.Field(
        output_processor = TakeFirst()
    )
    price = scrapy.Field(
        input_processor = MapCompose(strip_yen, strip_comma, convert_integer),
        output_processor = TakeFirst()
    )
    publisher = scrapy.Field(
        output_processor = TakeFirst()
    )
    size = scrapy.Field(
        input_processor = MapCompose(get_size),
        output_processor = TakeFirst()
    )
    page = scrapy.Field(
        input_processor = MapCompose(get_page, convert_integer),
        output_processor = TakeFirst()
    )
    
    isbn = scrapy.Field(
        input_processor = MapCompose(strip_isbn),
        output_processor = TakeFirst()
    )

    image_urls = scrapy.Field()

(scrapy_env) name@name kinokuniya % scrapy crawl computer_books
2022-03-03 22:31:46 [scrapy.utils.log] INFO: Scrapy 2.4.1 started (bot: kinokuniya)
2022-03-03 22:31:46 [scrapy.utils.log] INFO: Versions: lxml 4.6.4.0, libxml2 2.9.10, cssselect 1.1.0, parsel 1.6.0, w3lib 1.22.0, Twisted 21.7.0, Python 3.8.12 (default, Oct 12 2021, 06:23:56) - [Clang 10.0.0 ], pyOpenSSL 21.0.0 (OpenSSL 1.1.1l  24 Aug 2021), cryptography 36.0.0, Platform macOS-10.16-x86_64-i386-64bit
2022-03-03 22:31:46 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.selectreactor.SelectReactor
2022-03-03 22:31:46 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'kinokuniya',
 'CONCURRENT_REQUESTS': 1,
 'DEPTH_PRIORITY': 1,
 'DOWNLOAD_DELAY': 3,
 'FEED_EXPORT_ENCODING': 'utf-8',
 'HTTPCACHE_ENABLED': True,
 'NEWSPIDER_MODULE': 'kinokuniya.spiders',
 'ROBOTSTXT_OBEY': True,
 'SCHEDULER_DISK_QUEUE': 'scrapy.squeues.PickleFifoDiskQueue',
 'SCHEDULER_MEMORY_QUEUE': 'scrapy.squeues.FifoMemoryQueue',
 'SPIDER_MODULES': ['kinokuniya.spiders']}
2022-03-03 22:31:46 [scrapy.extensions.telnet] INFO: Telnet Password: aaec40ad6141d1f7
2022-03-03 22:31:46 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.logstats.LogStats']
2022-03-03 22:31:46 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats',
 'scrapy.downloadermiddlewares.httpcache.HttpCacheMiddleware']
2022-03-03 22:31:46 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2022-03-03 22:31:46 [scrapy.middleware] INFO: Enabled item pipelines:
['kinokuniya.pipelines.CheckItemPipeline']
2022-03-03 22:31:46 [scrapy.core.engine] INFO: Spider opened
2022-03-03 22:31:46 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2022-03-03 22:31:46 [scrapy.extensions.httpcache] DEBUG: Using filesystem cache storage in /Users/name/Desktop/books_toscrape/kinokuniya/.scrapy/httpcache
2022-03-03 22:31:46 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2022-03-03 22:31:46 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.kinokuniya.co.jp/robots.txt> (referer: None) ['cached']

〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜省略〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜


2022-03-03 22:31:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.kinokuniya.co.jp/f/dsg-01-9784839969301>
{'author': '小川 卓【監修】',
 'image_urls': ['https://www.kinokuniya.co.jp/images/goods/ar2/web/imgdata2/48399/4839969302.jpg'],
 'isbn': '9784839969301',
 'page': 224,
 'price': 2728,
 'publisher': 'マイナビ出版',
 'size': 'B5',
 'title': 'コンバージョンを上げるＷｅｂデザイン改善集'}
2022-03-03 22:31:47 [root] INFO: https://www.kinokuniya.co.jp/f/dsg-01-9784873119236
2022-03-03 22:31:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.kinokuniya.co.jp/f/dsg-01-9784873119236>


〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜省略〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜


2022-03-03 22:31:47 [root] INFO: https://www.kinokuniya.co.jp/f/dsg-01-9784774191300
2022-03-03 22:31:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.kinokuniya.co.jp/f/dsg-01-9784774191300>
{'author': '山田 祥寛【著】',
 'image_urls': ['https://www.kinokuniya.co.jp/images/goods/ar2/web/imgdata2/47741/4774191302.jpg'],
 'isbn': '9784774191300',
 'page': 497,
 'price': 4070,
 'publisher': '技術評論社',
 'size': 'B5',
 'title': 'Ａｎｇｕｌａｒアプリケーションプログラミング'}
2022-03-03 22:31:47 [scrapy.core.engine] INFO: Closing spider (finished)
2022-03-03 22:31:47 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 11117,
 'downloader/request_count': 22,
 'downloader/request_method_count/GET': 22,
 'downloader/response_bytes': 1929878,
 'downloader/response_count': 22,
 'downloader/response_status_count/200': 22,
 'elapsed_time_seconds': 0.479828,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2022, 3, 3, 13, 31, 47, 266327),
 'httpcache/hit': 22,
 'item_scraped_count': 20,
 'log_count/DEBUG': 43,
 'log_count/INFO': 30,
 'memusage/max': 62787584,
 'memusage/startup': 62787584,
 'request_depth_max': 1,
 'response_received_count': 22,
 'robotstxt/request_count': 1,
 'robotstxt/response_count': 1,
 'robotstxt/response_status_count/200': 1,
 'scheduler/dequeued': 21,
 'scheduler/dequeued/memory': 21,
 'scheduler/enqueued': 21,
 'scheduler/enqueued/memory': 21,
 'start_time': datetime.datetime(2022, 3, 3, 13, 31, 46, 786499)}
2022-03-03 22:31:47 [scrapy.core.engine] INFO: Spider closed (finished)

Answer 1

それってpipelines.pyにどの様に書くかの問題では？