This commit is contained in:
Lazar-T 2014-11-09 00:04:22 +01:00
parent 5cb0cfb0cb
commit b0ae19993b
1 changed files with 1 additions and 1 deletions

View File

@ -11,7 +11,7 @@ Design goals
1. be nicer to sites instead of using default download delay of zero
2. automatically adjust scrapy to the optimum crawling speed, so the user
doesn't have to tune the download delays and concurrent requests to find the
optimum one. the user only needs to specify the maximum concurrent requests
optimum one. The user only needs to specify the maximum concurrent requests
it allows, and the extension does the rest.
How it works