RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

docs updated

This commit is contained in:
Georgiy Zatserklianyi 2024-12-13 13:48:03 +01:00
parent dd652e425d
commit 85c2d530ed
1 changed files with 38 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

38
docs/topics/downloader-middleware.rst
View File

@ -332,6 +332,11 @@ cookiejar.
p.crawl(Quotes)
p.start()
.. note::
If :reqmeta:`cookiejar` didn't specified in request meta, middleware will read this as ``None``, as result of this middleware will create cookiejar object under dict key ``None``.
Log output::
2024-02-23 10:51:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://quotes.toscrape.com/login> (referer: None)
@ -341,6 +346,39 @@ Log output::
2024-02-23 10:51:27 [scrapy.core.engine] INFO: Closing spider (finished)
If cookie middleware is enabled, ``get_cookiejar(response_or_request)`` method added to spider:
.. class:: scrapy.Spider
...
.. method:: get_cookiejar(spider, response_or_request)
Returns ``scrapy.http.cookies.CookieJar`` cookiejar object based on :reqmeta:`cookiejar` value of ``response_or_request`` argument
:param response_or_request: request or response object where target session/cookiejar object used or planned to use
:type response_or_request: :class:`~scrapy.Request` or :class:`~scrapy.Response` object
Technically this is short cut to ``request.meta.get('cookiejar')`` or ``response.request.meta.get('cookiejar')`` if ``response_or_request`` is :class:`~scrapy.Response`
It allows a bit easier access to cookiejar object:
.. code-block:: python
...
self.logger.info(self.get_cookiejar(response)) # scrapy.http.cookies.CookieJar object
self.logger.info(self.get_cookiejar(response).jar) # http.cookiejar object
locale_cookie = (
self.get_cookiejar(response).jar._cookies["quotes.toscrape.com"]["/"].get("session")
)
Known use cases where direct access to cookiejar objects can be useful:
1. Modifying specific cookie value (as provided on example code sample above).
2. Delete cookiejar. On some cases this can re-initialise session.
.. warning:: ``http.cookiejar`` object from python standart lib https://docs.python.org/3/library/http.cookies.html doesn't provide option to individually access/modify specific cookie values. It is possible to change it only using it's private attributes as provided on code sample above.
DefaultHeadersMiddleware
------------------------