mirror of https://github.com/scrapy/scrapy.git
Add sphinx-lint. (#6920)
This commit is contained in:
parent
f454465b14
commit
95172659af
|
|
@ -15,3 +15,7 @@ repos:
|
||||||
rev: v5.0.0
|
rev: v5.0.0
|
||||||
hooks:
|
hooks:
|
||||||
- id: trailing-whitespace
|
- id: trailing-whitespace
|
||||||
|
- repo: https://github.com/sphinx-contrib/sphinx-lint
|
||||||
|
rev: v1.0.0
|
||||||
|
hooks:
|
||||||
|
- id: sphinx-lint
|
||||||
|
|
|
||||||
|
|
@ -340,7 +340,7 @@ Deprecations
|
||||||
(:issue:`6708`, :issue:`6714`)
|
(:issue:`6708`, :issue:`6714`)
|
||||||
|
|
||||||
- ``scrapy.utils.versions.scrapy_components_versions()`` is deprecated, use
|
- ``scrapy.utils.versions.scrapy_components_versions()`` is deprecated, use
|
||||||
:func:`scrapy.utils.versions.get_versions()` instead.
|
:func:`scrapy.utils.versions.get_versions` instead.
|
||||||
(:issue:`6582`)
|
(:issue:`6582`)
|
||||||
|
|
||||||
- ``BaseDupeFilter.log()`` is deprecated. It does nothing and shouldn't be
|
- ``BaseDupeFilter.log()`` is deprecated. It does nothing and shouldn't be
|
||||||
|
|
@ -1786,7 +1786,7 @@ Bug fixes
|
||||||
(:issue:`5914`, :issue:`5917`)
|
(:issue:`5914`, :issue:`5917`)
|
||||||
|
|
||||||
- Fixed an error breaking user handling of send failures in
|
- Fixed an error breaking user handling of send failures in
|
||||||
:meth:`scrapy.mail.MailSender.send()`. (:issue:`1611`, :issue:`5880`)
|
:meth:`scrapy.mail.MailSender.send`. (:issue:`1611`, :issue:`5880`)
|
||||||
|
|
||||||
Documentation
|
Documentation
|
||||||
~~~~~~~~~~~~~
|
~~~~~~~~~~~~~
|
||||||
|
|
@ -5336,7 +5336,7 @@ Docs
|
||||||
- Added missing bullet point for the ``AUTOTHROTTLE_TARGET_CONCURRENCY``
|
- Added missing bullet point for the ``AUTOTHROTTLE_TARGET_CONCURRENCY``
|
||||||
setting. (:issue:`2756`)
|
setting. (:issue:`2756`)
|
||||||
- Update Contributing docs, document new support channels
|
- Update Contributing docs, document new support channels
|
||||||
(:issue:`2762`, issue:`3038`)
|
(:issue:`2762`, :issue:`3038`)
|
||||||
- Include references to Scrapy subreddit in the docs
|
- Include references to Scrapy subreddit in the docs
|
||||||
- Fix broken links; use ``https://`` for external links
|
- Fix broken links; use ``https://`` for external links
|
||||||
(:issue:`2978`, :issue:`2982`, :issue:`2958`)
|
(:issue:`2978`, :issue:`2982`, :issue:`2958`)
|
||||||
|
|
|
||||||
|
|
@ -83,4 +83,4 @@ and how well it's written.
|
||||||
|
|
||||||
Use scrapy-bench_ for more complex benchmarking.
|
Use scrapy-bench_ for more complex benchmarking.
|
||||||
|
|
||||||
.. _scrapy-bench: https://github.com/scrapy/scrapy-bench
|
.. _scrapy-bench: https://github.com/scrapy/scrapy-bench
|
||||||
|
|
|
||||||
|
|
@ -280,7 +280,7 @@ In more complex websites, it could be difficult to easily reproduce the
|
||||||
requests, as we could need to add ``headers`` or ``cookies`` to make it work.
|
requests, as we could need to add ``headers`` or ``cookies`` to make it work.
|
||||||
In those cases you can export the requests in `cURL <https://curl.se/>`_
|
In those cases you can export the requests in `cURL <https://curl.se/>`_
|
||||||
format, by right-clicking on each of them in the network tool and using the
|
format, by right-clicking on each of them in the network tool and using the
|
||||||
:meth:`~scrapy.Request.from_curl()` method to generate an equivalent
|
:meth:`~scrapy.Request.from_curl` method to generate an equivalent
|
||||||
request:
|
request:
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
|
||||||
|
|
@ -86,7 +86,7 @@ method and URL. However, you may also need to reproduce the body, headers and
|
||||||
form parameters (see :class:`~scrapy.FormRequest`) of that request.
|
form parameters (see :class:`~scrapy.FormRequest`) of that request.
|
||||||
|
|
||||||
As all major browsers allow to export the requests in curl_ format, Scrapy
|
As all major browsers allow to export the requests in curl_ format, Scrapy
|
||||||
incorporates the method :meth:`~scrapy.Request.from_curl()` to generate an equivalent
|
incorporates the method :meth:`~scrapy.Request.from_curl` to generate an equivalent
|
||||||
:class:`~scrapy.Request` from a cURL command. To get more information
|
:class:`~scrapy.Request` from a cURL command. To get more information
|
||||||
visit :ref:`request from curl <requests-from-curl>` inside the network
|
visit :ref:`request from curl <requests-from-curl>` inside the network
|
||||||
tool section.
|
tool section.
|
||||||
|
|
|
||||||
|
|
@ -116,10 +116,10 @@ Example:
|
||||||
2. Overriding the serialize_field() method
|
2. Overriding the serialize_field() method
|
||||||
------------------------------------------
|
------------------------------------------
|
||||||
|
|
||||||
You can also override the :meth:`~BaseItemExporter.serialize_field()` method to
|
You can also override the :meth:`~BaseItemExporter.serialize_field` method to
|
||||||
customize how your field value will be exported.
|
customize how your field value will be exported.
|
||||||
|
|
||||||
Make sure you call the base class :meth:`~BaseItemExporter.serialize_field()` method
|
Make sure you call the base class :meth:`~BaseItemExporter.serialize_field` method
|
||||||
after your custom code.
|
after your custom code.
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
|
|
|
||||||
|
|
@ -162,7 +162,7 @@ Too many spiders?
|
||||||
-----------------
|
-----------------
|
||||||
|
|
||||||
If your project has too many spiders executed in parallel,
|
If your project has too many spiders executed in parallel,
|
||||||
the output of :func:`prefs()` can be difficult to read.
|
the output of :func:`prefs` can be difficult to read.
|
||||||
For this reason, that function has a ``ignore`` argument which can be used to
|
For this reason, that function has a ``ignore`` argument which can be used to
|
||||||
ignore a particular class (and all its subclasses). For
|
ignore a particular class (and all its subclasses). For
|
||||||
example, this won't show any live references to spiders:
|
example, this won't show any live references to spiders:
|
||||||
|
|
|
||||||
|
|
@ -873,7 +873,7 @@ It is usual for web sites to provide pre-populated form fields through ``<input
|
||||||
type="hidden">`` elements, such as session related data or authentication
|
type="hidden">`` elements, such as session related data or authentication
|
||||||
tokens (for login pages). When scraping, you'll want these fields to be
|
tokens (for login pages). When scraping, you'll want these fields to be
|
||||||
automatically pre-populated and only override a couple of them, such as the
|
automatically pre-populated and only override a couple of them, such as the
|
||||||
user name and password. You can use the :meth:`.FormRequest.from_response()`
|
user name and password. You can use the :meth:`.FormRequest.from_response`
|
||||||
method for this job. Here's an example spider which uses it:
|
method for this job. Here's an example spider which uses it:
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
|
||||||
|
|
@ -37,8 +37,8 @@ Modules and classes
|
||||||
|
|
||||||
- ``scrapy.contrib.itemparser.ItemParser``
|
- ``scrapy.contrib.itemparser.ItemParser``
|
||||||
- ``scrapy.contrib.itemparser.XPathItemParser``
|
- ``scrapy.contrib.itemparser.XPathItemParser``
|
||||||
- ``scrapy.contrib.itemparser.parsers.``MapConcat`` *(ex. ``TreeExpander``)*
|
- ``scrapy.contrib.itemparser.parsers.MapConcat`` *(ex. ``TreeExpander``)*
|
||||||
- ``scrapy.contrib.itemparser.parsers.``TakeFirst``
|
- ``scrapy.contrib.itemparser.parsers.TakeFirst``
|
||||||
- ``scrapy.contrib.itemparser.parsers.Join``
|
- ``scrapy.contrib.itemparser.parsers.Join``
|
||||||
- ``scrapy.contrib.itemparser.parsers.Identity``
|
- ``scrapy.contrib.itemparser.parsers.Identity``
|
||||||
|
|
||||||
|
|
@ -70,7 +70,7 @@ Alternative Public API Proposal
|
||||||
- ``ItemLoader.replace_value()``
|
- ``ItemLoader.replace_value()``
|
||||||
- ``ItemLoader.load_item()`` *(returns loaded item)*
|
- ``ItemLoader.load_item()`` *(returns loaded item)*
|
||||||
|
|
||||||
- ``ItemLoader.get_stored_values()`` or ``ItemLoader.get_values()`` *(returns the ``ItemLoader values)*
|
- ``ItemLoader.get_stored_values()`` or ``ItemLoader.get_values()`` *(returns the ``ItemLoader`` values)*
|
||||||
- ``ItemLoader.get_output_value()``
|
- ``ItemLoader.get_output_value()``
|
||||||
|
|
||||||
- ``ItemLoader.get_input_processor()`` or ``ItemLoader.get_in_processor()`` *(short version)*
|
- ``ItemLoader.get_input_processor()`` or ``ItemLoader.get_in_processor()`` *(short version)*
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue