mirror of https://github.com/scrapy/scrapy.git
Add sphinx-lint. (#6920)
This commit is contained in:
parent
f454465b14
commit
95172659af
|
|
@ -15,3 +15,7 @@ repos:
|
|||
rev: v5.0.0
|
||||
hooks:
|
||||
- id: trailing-whitespace
|
||||
- repo: https://github.com/sphinx-contrib/sphinx-lint
|
||||
rev: v1.0.0
|
||||
hooks:
|
||||
- id: sphinx-lint
|
||||
|
|
|
|||
|
|
@ -340,7 +340,7 @@ Deprecations
|
|||
(:issue:`6708`, :issue:`6714`)
|
||||
|
||||
- ``scrapy.utils.versions.scrapy_components_versions()`` is deprecated, use
|
||||
:func:`scrapy.utils.versions.get_versions()` instead.
|
||||
:func:`scrapy.utils.versions.get_versions` instead.
|
||||
(:issue:`6582`)
|
||||
|
||||
- ``BaseDupeFilter.log()`` is deprecated. It does nothing and shouldn't be
|
||||
|
|
@ -1786,7 +1786,7 @@ Bug fixes
|
|||
(:issue:`5914`, :issue:`5917`)
|
||||
|
||||
- Fixed an error breaking user handling of send failures in
|
||||
:meth:`scrapy.mail.MailSender.send()`. (:issue:`1611`, :issue:`5880`)
|
||||
:meth:`scrapy.mail.MailSender.send`. (:issue:`1611`, :issue:`5880`)
|
||||
|
||||
Documentation
|
||||
~~~~~~~~~~~~~
|
||||
|
|
@ -5336,7 +5336,7 @@ Docs
|
|||
- Added missing bullet point for the ``AUTOTHROTTLE_TARGET_CONCURRENCY``
|
||||
setting. (:issue:`2756`)
|
||||
- Update Contributing docs, document new support channels
|
||||
(:issue:`2762`, issue:`3038`)
|
||||
(:issue:`2762`, :issue:`3038`)
|
||||
- Include references to Scrapy subreddit in the docs
|
||||
- Fix broken links; use ``https://`` for external links
|
||||
(:issue:`2978`, :issue:`2982`, :issue:`2958`)
|
||||
|
|
|
|||
|
|
@ -83,4 +83,4 @@ and how well it's written.
|
|||
|
||||
Use scrapy-bench_ for more complex benchmarking.
|
||||
|
||||
.. _scrapy-bench: https://github.com/scrapy/scrapy-bench
|
||||
.. _scrapy-bench: https://github.com/scrapy/scrapy-bench
|
||||
|
|
|
|||
|
|
@ -280,7 +280,7 @@ In more complex websites, it could be difficult to easily reproduce the
|
|||
requests, as we could need to add ``headers`` or ``cookies`` to make it work.
|
||||
In those cases you can export the requests in `cURL <https://curl.se/>`_
|
||||
format, by right-clicking on each of them in the network tool and using the
|
||||
:meth:`~scrapy.Request.from_curl()` method to generate an equivalent
|
||||
:meth:`~scrapy.Request.from_curl` method to generate an equivalent
|
||||
request:
|
||||
|
||||
.. code-block:: python
|
||||
|
|
|
|||
|
|
@ -86,7 +86,7 @@ method and URL. However, you may also need to reproduce the body, headers and
|
|||
form parameters (see :class:`~scrapy.FormRequest`) of that request.
|
||||
|
||||
As all major browsers allow to export the requests in curl_ format, Scrapy
|
||||
incorporates the method :meth:`~scrapy.Request.from_curl()` to generate an equivalent
|
||||
incorporates the method :meth:`~scrapy.Request.from_curl` to generate an equivalent
|
||||
:class:`~scrapy.Request` from a cURL command. To get more information
|
||||
visit :ref:`request from curl <requests-from-curl>` inside the network
|
||||
tool section.
|
||||
|
|
|
|||
|
|
@ -116,10 +116,10 @@ Example:
|
|||
2. Overriding the serialize_field() method
|
||||
------------------------------------------
|
||||
|
||||
You can also override the :meth:`~BaseItemExporter.serialize_field()` method to
|
||||
You can also override the :meth:`~BaseItemExporter.serialize_field` method to
|
||||
customize how your field value will be exported.
|
||||
|
||||
Make sure you call the base class :meth:`~BaseItemExporter.serialize_field()` method
|
||||
Make sure you call the base class :meth:`~BaseItemExporter.serialize_field` method
|
||||
after your custom code.
|
||||
|
||||
Example:
|
||||
|
|
|
|||
|
|
@ -162,7 +162,7 @@ Too many spiders?
|
|||
-----------------
|
||||
|
||||
If your project has too many spiders executed in parallel,
|
||||
the output of :func:`prefs()` can be difficult to read.
|
||||
the output of :func:`prefs` can be difficult to read.
|
||||
For this reason, that function has a ``ignore`` argument which can be used to
|
||||
ignore a particular class (and all its subclasses). For
|
||||
example, this won't show any live references to spiders:
|
||||
|
|
|
|||
|
|
@ -873,7 +873,7 @@ It is usual for web sites to provide pre-populated form fields through ``<input
|
|||
type="hidden">`` elements, such as session related data or authentication
|
||||
tokens (for login pages). When scraping, you'll want these fields to be
|
||||
automatically pre-populated and only override a couple of them, such as the
|
||||
user name and password. You can use the :meth:`.FormRequest.from_response()`
|
||||
user name and password. You can use the :meth:`.FormRequest.from_response`
|
||||
method for this job. Here's an example spider which uses it:
|
||||
|
||||
.. code-block:: python
|
||||
|
|
|
|||
|
|
@ -37,8 +37,8 @@ Modules and classes
|
|||
|
||||
- ``scrapy.contrib.itemparser.ItemParser``
|
||||
- ``scrapy.contrib.itemparser.XPathItemParser``
|
||||
- ``scrapy.contrib.itemparser.parsers.``MapConcat`` *(ex. ``TreeExpander``)*
|
||||
- ``scrapy.contrib.itemparser.parsers.``TakeFirst``
|
||||
- ``scrapy.contrib.itemparser.parsers.MapConcat`` *(ex. ``TreeExpander``)*
|
||||
- ``scrapy.contrib.itemparser.parsers.TakeFirst``
|
||||
- ``scrapy.contrib.itemparser.parsers.Join``
|
||||
- ``scrapy.contrib.itemparser.parsers.Identity``
|
||||
|
||||
|
|
@ -70,7 +70,7 @@ Alternative Public API Proposal
|
|||
- ``ItemLoader.replace_value()``
|
||||
- ``ItemLoader.load_item()`` *(returns loaded item)*
|
||||
|
||||
- ``ItemLoader.get_stored_values()`` or ``ItemLoader.get_values()`` *(returns the ``ItemLoader values)*
|
||||
- ``ItemLoader.get_stored_values()`` or ``ItemLoader.get_values()`` *(returns the ``ItemLoader`` values)*
|
||||
- ``ItemLoader.get_output_value()``
|
||||
|
||||
- ``ItemLoader.get_input_processor()`` or ``ItemLoader.get_in_processor()`` *(short version)*
|
||||
|
|
|
|||
Loading…
Reference in New Issue