RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

add leveldb cache backend docs

This commit is contained in:
Daniel Graña 2014-06-23 14:28:55 -03:00
parent 752787e634
commit a90fd81c97
1 changed files with 21 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
docs/topics/downloader-middleware.rst
View File

@ -422,6 +422,27 @@ In order to use this storage backend, set:
* :setting:`HTTPCACHE_STORAGE` to ``scrapy.contrib.httpcache.DbmCacheStorage``
.. _httpcache-storage-leveldb:
LevelDB storage backend
~~~~~~~~~~~~~~~~~~~~~~~
.. versionadded:: 0.23
A LevelDB_ storage backend is also available for the HTTP cache middleware.
This backend is not recommended for development because only one process can
access LevelDB databases at the same time, so you can't run a crawl and open
the scrapy shell in parallel for the same spider.
In order to use this storage backend:
* set :setting:`HTTPCACHE_STORAGE` to ``scrapy.contrib.httpcache.LeveldbCacheStorage``
* install `LevelDB python bindings`_ like ``pip install leveldb``
.. _LevelDB: http://code.google.com/p/leveldb/
.. _leveldb python bindings: http://pypi.python.org/pypi/leveldb
HTTPCache middleware settings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~