diff --git a/docs/faq.rst b/docs/faq.rst
index f81ec3601..cef3e69f3 100644
--- a/docs/faq.rst
+++ b/docs/faq.rst
@@ -410,7 +410,7 @@ How can I make a blank request?
-------------------------------
.. code-block:: python
-
+
from scrapy import Request
diff --git a/docs/intro/install.rst b/docs/intro/install.rst
index 82a0e18c5..488a66f36 100644
--- a/docs/intro/install.rst
+++ b/docs/intro/install.rst
@@ -111,7 +111,7 @@ Once you've installed `Anaconda`_ or `Miniconda`_, install Scrapy with::
To install Scrapy on Windows using ``pip``:
.. warning::
- This installation method requires “Microsoft Visual C++” for installing some
+ This installation method requires “Microsoft Visual C++” for installing some
Scrapy dependencies, which demands significantly more disk space than Anaconda.
#. Download and execute `Microsoft C++ Build Tools`_ to install the Visual Studio Installer.
@@ -123,7 +123,7 @@ To install Scrapy on Windows using ``pip``:
#. Check the installation details and make sure following packages are selected as optional components:
* **MSVC** (e.g MSVC v142 - VS 2019 C++ x64/x86 build tools (v14.23) )
-
+
* **Windows SDK** (e.g Windows 10 SDK (10.0.18362.0))
#. Install the Visual Studio Build Tools.
diff --git a/docs/intro/tutorial.rst b/docs/intro/tutorial.rst
index 6e6caebf1..5041b49ea 100644
--- a/docs/intro/tutorial.rst
+++ b/docs/intro/tutorial.rst
@@ -292,7 +292,7 @@ As an alternative, you could've written:
>>> response.css("title::text")[0].get()
'Quotes to Scrape'
-Accessing an index on a :class:`~scrapy.selector.SelectorList` instance will
+Accessing an index on a :class:`~scrapy.selector.SelectorList` instance will
raise an :exc:`IndexError` exception if there are no results:
.. code-block:: pycon
@@ -302,8 +302,8 @@ raise an :exc:`IndexError` exception if there are no results:
...
IndexError: list index out of range
-You might want to use ``.get()`` directly on the
-:class:`~scrapy.selector.SelectorList` instance instead, which returns ``None``
+You might want to use ``.get()`` directly on the
+:class:`~scrapy.selector.SelectorList` instance instead, which returns ``None``
if there are no results:
.. code-block:: pycon
diff --git a/docs/news.rst b/docs/news.rst
index 8230c3aef..9a68f8852 100644
--- a/docs/news.rst
+++ b/docs/news.rst
@@ -934,10 +934,10 @@ Modified requirements
Backward-incompatible changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- The value of the :setting:`FEED_STORE_EMPTY` setting is now ``True``
- instead of ``False``. In earlier Scrapy versions empty files were created
- even when this setting was ``False`` (which was a bug that is now fixed),
- so the new default should keep the old behavior. (:issue:`872`,
+- The value of the :setting:`FEED_STORE_EMPTY` setting is now ``True``
+ instead of ``False``. In earlier Scrapy versions empty files were created
+ even when this setting was ``False`` (which was a bug that is now fixed),
+ so the new default should keep the old behavior. (:issue:`872`,
:issue:`5847`)
Deprecation removals
diff --git a/docs/topics/api.rst b/docs/topics/api.rst
index f7cffb61b..edc625be8 100644
--- a/docs/topics/api.rst
+++ b/docs/topics/api.rst
@@ -88,7 +88,7 @@ how you :ref:`configure the downloader middlewares
The execution engine, which coordinates the core crawling logic
between the scheduler, downloader and spiders.
- Some extension may want to access the Scrapy engine, to inspect or
+ Some extension may want to access the Scrapy engine, to inspect or
modify the downloader and scheduler behaviour, although this is an
advanced use and this API is not yet stable.
diff --git a/docs/topics/architecture.rst b/docs/topics/architecture.rst
index 0370dc538..4e53b6e3d 100644
--- a/docs/topics/architecture.rst
+++ b/docs/topics/architecture.rst
@@ -87,8 +87,8 @@ of the system, and triggering events when certain actions occur. See the
Scheduler
---------
-The :ref:`scheduler ` receives requests from the engine and
-enqueues them for feeding them later (also to the engine) when the engine
+The :ref:`scheduler ` receives requests from the engine and
+enqueues them for feeding them later (also to the engine) when the engine
requests them.
.. _component-downloader:
diff --git a/docs/topics/exporters.rst b/docs/topics/exporters.rst
index 7a85c099b..5c078568b 100644
--- a/docs/topics/exporters.rst
+++ b/docs/topics/exporters.rst
@@ -224,7 +224,7 @@ BaseItemExporter
.. [1] Not all exporters respect the specified field order.
.. [2] When using :ref:`item objects ` that do not expose
all their possible fields, exporters that do not support exporting
- a different subset of fields per item will only export the fields
+ a different subset of fields per item will only export the fields
found in the first item exported.
.. attribute:: export_empty_fields
diff --git a/docs/topics/extensions.rst b/docs/topics/extensions.rst
index c47a3226a..23bbcfcb5 100644
--- a/docs/topics/extensions.rst
+++ b/docs/topics/extensions.rst
@@ -256,14 +256,14 @@ Spider state extension
Manages spider state data by loading it before a crawl and saving it after.
Give a value to the :setting:`JOBDIR` setting to enable this extension.
-When enabled, this extension manages the :attr:`~scrapy.Spider.state`
+When enabled, this extension manages the :attr:`~scrapy.Spider.state`
attribute of your :class:`~scrapy.Spider` instance:
-
-- When your spider closes (:signal:`spider_closed`), the contents of its
- :attr:`~scrapy.Spider.state` attribute are serialized into a file named
+
+- When your spider closes (:signal:`spider_closed`), the contents of its
+ :attr:`~scrapy.Spider.state` attribute are serialized into a file named
``spider.state`` in the :setting:`JOBDIR` folder.
-- When your spider opens (:signal:`spider_opened`), if a previously-generated
- ``spider.state`` file exists in the :setting:`JOBDIR` folder, it is loaded
+- When your spider opens (:signal:`spider_opened`), if a previously-generated
+ ``spider.state`` file exists in the :setting:`JOBDIR` folder, it is loaded
into the :attr:`~scrapy.Spider.state` attribute.
@@ -291,8 +291,8 @@ settings:
.. note::
- When a certain closing condition is met, requests which are
- currently in the downloader queue (up to :setting:`CONCURRENT_REQUESTS`
+ When a certain closing condition is met, requests which are
+ currently in the downloader queue (up to :setting:`CONCURRENT_REQUESTS`
requests) are still processed.
.. setting:: CLOSESPIDER_TIMEOUT
diff --git a/docs/topics/feed-exports.rst b/docs/topics/feed-exports.rst
index 07a3f3678..7f401f0c7 100644
--- a/docs/topics/feed-exports.rst
+++ b/docs/topics/feed-exports.rst
@@ -180,7 +180,7 @@ FTP supports two different connection modes: `active or passive
mode by default. To use the active connection mode instead, set the
:setting:`FEED_STORAGE_FTP_ACTIVE` setting to ``True``.
-The default value for the ``overwrite`` key in the :setting:`FEEDS` for this
+The default value for the ``overwrite`` key in the :setting:`FEEDS` for this
storage backend is: ``True``.
.. caution:: The value ``True`` in ``overwrite`` will cause you to lose the
@@ -222,7 +222,7 @@ feeds using these settings:
- :setting:`AWS_ENDPOINT_URL`
- :setting:`AWS_REGION_NAME`
-The default value for the ``overwrite`` key in the :setting:`FEEDS` for this
+The default value for the ``overwrite`` key in the :setting:`FEEDS` for this
storage backend is: ``True``.
.. caution:: The value ``True`` in ``overwrite`` will cause you to lose the
@@ -255,7 +255,7 @@ You can set a *Project ID* and *Access Control List (ACL)* through the following
- :setting:`FEED_STORAGE_GCS_ACL`
- :setting:`GCS_PROJECT_ID`
-The default value for the ``overwrite`` key in the :setting:`FEEDS` for this
+The default value for the ``overwrite`` key in the :setting:`FEEDS` for this
storage backend is: ``True``.
.. caution:: The value ``True`` in ``overwrite`` will cause you to lose the
@@ -587,8 +587,8 @@ FEED_STORE_EMPTY
Default: ``True``
Whether to export empty feeds (i.e. feeds with no items).
-If ``False``, and there are no items to export, no new files are created and
-existing files are not modified, even if the :ref:`overwrite feed option
+If ``False``, and there are no items to export, no new files are created and
+existing files are not modified, even if the :ref:`overwrite feed option
` is enabled.
.. setting:: FEED_STORAGES
diff --git a/docs/topics/logging.rst b/docs/topics/logging.rst
index fe1c4d162..a398d6c83 100644
--- a/docs/topics/logging.rst
+++ b/docs/topics/logging.rst
@@ -266,9 +266,9 @@ e.g. in the spider's ``__init__`` method:
If you run this spider again then INFO messages from
``scrapy.spidermiddlewares.httperror`` logger will be gone.
-You can also filter log records by :class:`~logging.LogRecord` data. For
+You can also filter log records by :class:`~logging.LogRecord` data. For
example, you can filter log records by message content using a substring or
-a regular expression. Create a :class:`logging.Filter` subclass
+a regular expression. Create a :class:`logging.Filter` subclass
and equip it with a regular expression pattern to
filter out unwanted messages:
@@ -284,8 +284,8 @@ filter out unwanted messages:
if match:
return False
-A project-level filter may be attached to the root
-handler created by Scrapy, this is a wieldy way to
+A project-level filter may be attached to the root
+handler created by Scrapy, this is a wieldy way to
filter all loggers in different parts of the project
(middlewares, spider, etc.):
@@ -301,7 +301,7 @@ filter all loggers in different parts of the project
for handler in logging.root.handlers:
handler.addFilter(ContentFilter())
-Alternatively, you may choose a specific logger
+Alternatively, you may choose a specific logger
and hide it without affecting other loggers:
.. code-block:: python
diff --git a/docs/topics/media-pipeline.rst b/docs/topics/media-pipeline.rst
index f086a943e..cc1fe8703 100644
--- a/docs/topics/media-pipeline.rst
+++ b/docs/topics/media-pipeline.rst
@@ -414,7 +414,7 @@ class name. E.g. given pipeline class called MyPipeline you can set setting key:
and pipeline class MyPipeline will have expiration time set to 180.
-The last modified time from the file is used to determine the age of the file in days,
+The last modified time from the file is used to determine the age of the file in days,
which is then compared to the set expiration time to determine if the file is expired.
.. _topics-images-thumbnails:
@@ -519,7 +519,7 @@ See here the methods that you can override in your custom Files Pipeline:
In addition to ``response``, this method receives the original
:class:`request `,
- :class:`info ` and
+ :class:`info ` and
:class:`item `
You can override this method to customize the download path of each file.
@@ -541,9 +541,9 @@ See here the methods that you can override in your custom Files Pipeline:
def file_path(self, request, response=None, info=None, *, item=None):
return "files/" + PurePosixPath(urlparse_cached(request).path).name
- Similarly, you can use the ``item`` to determine the file path based on some item
+ Similarly, you can use the ``item`` to determine the file path based on some item
property.
-
+
By default the :meth:`file_path` method returns
``full/.``.
@@ -677,7 +677,7 @@ See here the methods that you can override in your custom Images Pipeline:
In addition to ``response``, this method receives the original
:class:`request `,
- :class:`info ` and
+ :class:`info ` and
:class:`item `
You can override this method to customize the download path of each file.
@@ -699,9 +699,9 @@ See here the methods that you can override in your custom Images Pipeline:
def file_path(self, request, response=None, info=None, *, item=None):
return "files/" + PurePosixPath(urlparse_cached(request).path).name
- Similarly, you can use the ``item`` to determine the file path based on some item
+ Similarly, you can use the ``item`` to determine the file path based on some item
property.
-
+
By default the :meth:`file_path` method returns
``full/.``.
diff --git a/docs/topics/practices.rst b/docs/topics/practices.rst
index 5f6798601..db91cd073 100644
--- a/docs/topics/practices.rst
+++ b/docs/topics/practices.rst
@@ -309,7 +309,7 @@ Here are some tips to keep in mind when dealing with these kinds of sites:
services like `ProxyMesh`_. An open source alternative is `scrapoxy`_, a
super proxy that you can attach your own proxies to.
* use a ban avoidance service, such as `Zyte API`_, which provides a `Scrapy
- plugin `__ and additional
+ plugin `__ and additional
features, like `AI web scraping `__
If you are still unable to prevent your bot getting banned, consider contacting
diff --git a/docs/topics/request-response.rst b/docs/topics/request-response.rst
index b187f3aaf..55e8518a5 100644
--- a/docs/topics/request-response.rst
+++ b/docs/topics/request-response.rst
@@ -1309,7 +1309,7 @@ JsonResponse objects
.. class:: JsonResponse(url[, ...])
- The :class:`JsonResponse` class is a subclass of :class:`TextResponse`
- that is used when the response has a `JSON MIME type
- `_ in its `Content-Type`
+ The :class:`JsonResponse` class is a subclass of :class:`TextResponse`
+ that is used when the response has a `JSON MIME type
+ `_ in its `Content-Type`
header.
diff --git a/docs/topics/selectors.rst b/docs/topics/selectors.rst
index b95e6eab3..dbef07b73 100644
--- a/docs/topics/selectors.rst
+++ b/docs/topics/selectors.rst
@@ -559,7 +559,7 @@ For example, suppose you want to extract all ``
`` elements inside ``
``
elements. First, you would get all ``
`` elements:
.. code-block:: pycon
-
+
>>> divs = response.xpath("//div")
At first, you may be tempted to use the following approach, which is wrong, as
@@ -610,7 +610,7 @@ As it turns out, Scrapy selectors allow you to chain selectors, so most of the t
you can just select by class using CSS and then switch to XPath when needed:
.. code-block:: pycon
-
+
>>> from scrapy import Selector
>>> sel = Selector(
... text=''
@@ -1032,7 +1032,7 @@ whereas the CSS lookup is translated into XPath and thus runs more efficiently,
so performance-wise its uses are limited to situations that are not easily
described with CSS selectors.
-Parsel also simplifies adding your own XPath extensions with
+Parsel also simplifies adding your own XPath extensions with
:func:`~parsel.xpathfuncs.set_xpathfunc`.
.. _topics-selectors-ref:
diff --git a/docs/topics/spiders.rst b/docs/topics/spiders.rst
index e1b1c5ad6..0a67240d6 100644
--- a/docs/topics/spiders.rst
+++ b/docs/topics/spiders.rst
@@ -379,8 +379,8 @@ The above example can also be written as follows:
def start_requests(self):
yield scrapy.Request(f"http://www.example.com/categories/{self.category}")
-If you are :ref:`running Scrapy from a script `, you can
-specify spider arguments when calling
+If you are :ref:`running Scrapy from a script `, you can
+specify spider arguments when calling
:class:`CrawlerProcess.crawl ` or
:class:`CrawlerRunner.crawl `:
diff --git a/docs/topics/stats.rst b/docs/topics/stats.rst
index be8ecb7a5..9572a3785 100644
--- a/docs/topics/stats.rst
+++ b/docs/topics/stats.rst
@@ -86,7 +86,7 @@ Available Stats Collectors
Besides the basic :class:`StatsCollector` there are other Stats Collectors
available in Scrapy which extend the basic Stats Collector. You can select
which Stats Collector to use through the :setting:`STATS_CLASS` setting. The
-default Stats Collector used is the :class:`MemoryStatsCollector`.
+default Stats Collector used is the :class:`MemoryStatsCollector`.
.. currentmodule:: scrapy.statscollectors
diff --git a/sep/sep-004.rst b/sep/sep-004.rst
index b1cef2600..7a4ebe886 100644
--- a/sep/sep-004.rst
+++ b/sep/sep-004.rst
@@ -11,7 +11,7 @@ SEP-004: Library API
====================
.. note:: the library API has been implemented, but slightly different from
proposed in this SEP. You can run a Scrapy crawler inside a Twisted
- reactor, but not outside it.
+ reactor, but not outside it.
Introduction
============
diff --git a/sep/sep-007.rst b/sep/sep-007.rst
index 0ca2036ce..73ce0d338 100644
--- a/sep/sep-007.rst
+++ b/sep/sep-007.rst
@@ -96,7 +96,7 @@ specified, else utf-8 is used) and returns a new unicode object. E.g:
``clean_spaces``
----------------
-
+
Converts multispaces into single spaces for the given string. E.g:
::
diff --git a/sep/sep-008.rst b/sep/sep-008.rst
index be5987e39..1c38b1c40 100644
--- a/sep/sep-008.rst
+++ b/sep/sep-008.rst
@@ -73,8 +73,8 @@ Alternative Public API Proposal
- ``ItemLoader.get_stored_values()`` or ``ItemLoader.get_values()`` *(returns the ``ItemLoader values)*
- ``ItemLoader.get_output_value()``
-- ``ItemLoader.get_input_processor()`` or ``ItemLoader.get_in_processor()`` *(short version)*
-- ``ItemLoader.get_output_processor()`` or ``ItemLoader.get_out_processor()`` *(short version)*
+- ``ItemLoader.get_input_processor()`` or ``ItemLoader.get_in_processor()`` *(short version)*
+- ``ItemLoader.get_output_processor()`` or ``ItemLoader.get_out_processor()`` *(short version)*
- ``ItemLoader.context``
diff --git a/sep/sep-014.rst b/sep/sep-014.rst
index e03a2b0f6..0a2e6b51e 100644
--- a/sep/sep-014.rst
+++ b/sep/sep-014.rst
@@ -21,7 +21,7 @@ Current flaws and inconsistencies
2. Link extractors are inflexible and hard to maintain, link
processing/filtering is tightly coupled. (e.g. canonicalize)
3. Isn't possible to crawl an url directly from command line because the Spider
- does not know which callback use.
+ does not know which callback use.
These flaws will be corrected by the changes proposed in this SEP.
@@ -55,7 +55,7 @@ Request Extractors
Request Extractors takes response object and determines which requests follow.
This is an enhancement to ``LinkExtractors`` which returns urls (links),
-Request Extractors return Request objects.
+Request Extractors return Request objects.
Request Processors
------------------
diff --git a/sep/sep-018.rst b/sep/sep-018.rst
index 13ab501ed..e6d601fe1 100644
--- a/sep/sep-018.rst
+++ b/sep/sep-018.rst
@@ -200,7 +200,7 @@ the same spider:
# extract item from response
return item
-The Spider Middleware that implements spider code
+The Spider Middleware that implements spider code
=================================================
There's gonna be one middleware that will take care of calling the proper
@@ -625,7 +625,7 @@ Resolved:
not the original one (think of redirections), but it does carry the ``meta``
of the original one. The original one may not be available anymore (in
memory) if we're using a persistent scheduler., but in that case it would be
- the deserialized request from the persistent scheduler queue.
+ the deserialized request from the persistent scheduler queue.
- No - this would make implementation more complex and we're not sure it's
really needed