From a90fd81c97e32694164b71a4df18e446bd2e3ee2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20Gra=C3=B1a?= Date: Mon, 23 Jun 2014 14:28:55 -0300 Subject: [PATCH] add leveldb cache backend docs --- docs/topics/downloader-middleware.rst | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/topics/downloader-middleware.rst b/docs/topics/downloader-middleware.rst index da5a85655..e201260f9 100644 --- a/docs/topics/downloader-middleware.rst +++ b/docs/topics/downloader-middleware.rst @@ -422,6 +422,27 @@ In order to use this storage backend, set: * :setting:`HTTPCACHE_STORAGE` to ``scrapy.contrib.httpcache.DbmCacheStorage`` +.. _httpcache-storage-leveldb: + +LevelDB storage backend +~~~~~~~~~~~~~~~~~~~~~~~ + +.. versionadded:: 0.23 + +A LevelDB_ storage backend is also available for the HTTP cache middleware. + +This backend is not recommended for development because only one process can +access LevelDB databases at the same time, so you can't run a crawl and open +the scrapy shell in parallel for the same spider. + +In order to use this storage backend: + +* set :setting:`HTTPCACHE_STORAGE` to ``scrapy.contrib.httpcache.LeveldbCacheStorage`` +* install `LevelDB python bindings`_ like ``pip install leveldb`` + +.. _LevelDB: http://code.google.com/p/leveldb/ +.. _leveldb python bindings: http://pypi.python.org/pypi/leveldb + HTTPCache middleware settings ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~