doc for creating custom cache storage backend.

This commit is contained in:
Anubhav Patel 2019-03-17 16:54:28 +05:30
parent d346b8cb0f
commit 044318920a
1 changed files with 53 additions and 0 deletions

View File

@ -496,6 +496,59 @@ In order to use this storage backend:
.. _LevelDB: https://github.com/google/leveldb
.. _leveldb python bindings: https://pypi.python.org/pypi/leveldb
.. _httpcache-storage-custom:
Implementing custom cache storage backend
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can implement custom cache storage backend by creating a Python class that
defines the methods described below.
.. module:: scrapy.extensions.httpcache
.. class:: CacheStorage
.. method:: open_spider(spider)
This method gets called after a spider has been opened for crawling.
:param spider: the spider which has been opened
:type spider: :class:`~scrapy.spiders.Spider` object
.. method:: close_spider(spider)
This method gets called after a spider has been closed.
:param spider: the spider which has been closed
:type spider: :class:`~scrapy.spiders.Spider` object
.. method:: retrieve_response(spider, request)
Returns response if present in cache, or ``None`` otherwise.
:param spider: the spider which generated the request
:type spider: :class:`~scrapy.spiders.Spider` object
:param request: the request to find cached reponse for
:type request: :class:`~scrapy.http.Request` object
.. method:: store_response(spider, request, response)
Stores the given response in the cache.
:param spider: the spider for which the response is intended
:type spider: :class:`~scrapy.spiders.Spider` object
:param request: corresponding request the spider generated
:type request: :class:`~scrapy.http.Request` object
:param response: the response to store in the cache
:type response: :class:`~scrapy.http.Response` object
In order to use your storage backend, set:
* :setting:`HTTPCACHE_STORAGE` to path of your custom storage class.
HTTPCache middleware settings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~