mirror of https://github.com/scrapy/scrapy.git
Fix typos in documentation (#7076)
This commit is contained in:
parent
c577771838
commit
a02abdcf63
|
|
@ -138,7 +138,7 @@ To disable cookies use:
|
|||
Disable retries
|
||||
===============
|
||||
|
||||
Retrying failed HTTP requests can slow down the crawls substantially, specially
|
||||
Retrying failed HTTP requests can slow down the crawls substantially, especially
|
||||
when sites causes are very slow (or fail) to respond, thus causing a timeout
|
||||
error which gets retried many times, unnecessarily, preventing crawler capacity
|
||||
to be reused for other domains.
|
||||
|
|
|
|||
|
|
@ -267,7 +267,7 @@ CLOSESPIDER_TIMEOUT
|
|||
Default: ``0``
|
||||
|
||||
An integer which specifies a number of seconds. If the spider remains open for
|
||||
more than that number of second, it will be automatically closed with the
|
||||
more than that number of seconds, it will be automatically closed with the
|
||||
reason ``closespider_timeout``. If zero (or non set), spiders won't be closed by
|
||||
timeout.
|
||||
|
||||
|
|
|
|||
|
|
@ -281,7 +281,7 @@ finishes before starting the next one:
|
|||
Distributed crawls
|
||||
==================
|
||||
|
||||
Scrapy doesn't provide any built-in facility for running crawls in a distribute
|
||||
Scrapy doesn't provide any built-in facility for running crawls in a distributed
|
||||
(multi-server) manner. However, there are some ways to distribute crawls, which
|
||||
vary depending on how you plan to distribute them.
|
||||
|
||||
|
|
@ -289,10 +289,10 @@ If you have many spiders, the obvious way to distribute the load is to setup
|
|||
many Scrapyd instances and distribute spider runs among those.
|
||||
|
||||
If you instead want to run a single (big) spider through many machines, what
|
||||
you usually do is partition the urls to crawl and send them to each separate
|
||||
you usually do is partition the URLs to crawl and send them to each separate
|
||||
spider. Here is a concrete example:
|
||||
|
||||
First, you prepare the list of urls to crawl and put them into separate
|
||||
First, you prepare the list of URLs to crawl and put them into separate
|
||||
files/urls::
|
||||
|
||||
http://somedomain.com/urls-to-crawl/spider1/part1.list
|
||||
|
|
@ -319,7 +319,7 @@ consider contacting `commercial support`_ if in doubt.
|
|||
|
||||
Here are some tips to keep in mind when dealing with these kinds of sites:
|
||||
|
||||
* rotate your user agent from a pool of well-known ones from browsers (google
|
||||
* rotate your user agent from a pool of well-known ones from browsers (Google
|
||||
around to get a list of them)
|
||||
* disable cookies (see :setting:`COOKIES_ENABLED`) as some sites may use
|
||||
cookies to spot bot behaviour
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ If you have `IPython`_ installed, the Scrapy shell will use it (instead of the
|
|||
standard Python console). The `IPython`_ console is much more powerful and
|
||||
provides smart auto-completion and colorized output, among other things.
|
||||
|
||||
We highly recommend you install `IPython`_, specially if you're working on
|
||||
We highly recommend you install `IPython`_, especially if you're working on
|
||||
Unix systems (where `IPython`_ excels). See the `IPython installation guide`_
|
||||
for more info.
|
||||
|
||||
|
|
|
|||
|
|
@ -628,7 +628,7 @@ XMLFeedSpider
|
|||
This method is called for the nodes matching the provided tag name
|
||||
(``itertag``). Receives the response and an
|
||||
:class:`~scrapy.Selector` for each node. Overriding this
|
||||
method is mandatory. Otherwise, you spider won't work. This method
|
||||
method is mandatory. Otherwise, your spider won't work. This method
|
||||
must return an :ref:`item object <topics-items>`, a
|
||||
:class:`~scrapy.Request` object, or an iterable containing any of
|
||||
them.
|
||||
|
|
|
|||
Loading…
Reference in New Issue