mirror of https://github.com/scrapy/scrapy.git
DOC how to override log level for a specific Scrapy component
This commit is contained in:
parent
0fc73a9d55
commit
da19f0b7b7
|
|
@ -10,7 +10,7 @@ Logging
|
|||
about the new logging system.
|
||||
|
||||
Scrapy uses `Python's builtin logging system
|
||||
<https://docs.python.org/2/library/logging.html>`_ for event logging. We'll
|
||||
<https://docs.python.org/3/library/logging.html>`_ for event logging. We'll
|
||||
provide some simple examples to get you started, but for more advanced
|
||||
use-cases it's strongly suggested to read thoroughly its documentation.
|
||||
|
||||
|
|
@ -193,6 +193,43 @@ to override some of the Scrapy settings regarding logging.
|
|||
Module `logging.handlers <https://docs.python.org/2/library/logging.handlers.html>`_
|
||||
Further documentation on available handlers
|
||||
|
||||
Advanced customization
|
||||
----------------------
|
||||
|
||||
Because Scrapy uses stdlib logging module, you can customize logging using
|
||||
all features of stdlib logging.
|
||||
|
||||
For example, let's say you're scraping a website which returns many
|
||||
HTTP 404 and 500 responses, and you want to hide all messages like this::
|
||||
|
||||
2016-12-16 22:00:06 [scrapy.spidermiddlewares.httperror] INFO: Ignoring
|
||||
response <500 http://quotes.toscrape.com/page/1-34/>: HTTP status code
|
||||
is not handled or not allowed
|
||||
|
||||
The first thing to note is a logger name - it is in brackets:
|
||||
``[scrapy.spidermiddlewares.httperror]``. If you get just ``[scrapy]`` then
|
||||
:setting:`LOG_SHORT_NAMES` is likely set to True; set it to False and re-run
|
||||
the crawl.
|
||||
|
||||
Next, we can see that the message has INFO level. To hide it
|
||||
we should set logging level for ``scrapy.spidermiddlewares.httperror``
|
||||
higher than INFO; next level after INFO is WARNING. It could be done
|
||||
e.g. in the spider's ``__init__`` method::
|
||||
|
||||
import logging
|
||||
import scrapy
|
||||
|
||||
|
||||
class MySpider(scrapy.Spider):
|
||||
# ...
|
||||
def __init__(self, *args, **kwargs):
|
||||
logger = logging.getLogger('scrapy.spidermiddlewares.httperror')
|
||||
logger.setLevel(logging.WARNING)
|
||||
super().__init__(*args, **kwargs)
|
||||
|
||||
If you run this spider again then INFO messages from
|
||||
``scrapy.spidermiddlewares.httperror`` logger will be gone.
|
||||
|
||||
scrapy.utils.log module
|
||||
=======================
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue