3.7 KiB

Raw Blame History

Logging

Scrapy provides a logging facility which can be used through the :mod:`scrapy.log` module. The current underling implementation uses Twisted logging but this may change in the future.

System Message: ERROR/3 (<stdin>, line 7); backlink

Unknown interpreted text role "mod".

Logging service must be explicitly started through the :func:`scrapy.log.start` function.

System Message: ERROR/3 (<stdin>, line 13); backlink

Unknown interpreted text role "func".

Log levels

Scrapy provides 5 logging levels:

:data:`~scrapy.log.CRITICAL` - for critical errors

System Message: ERROR/3 (<stdin>, line 22); backlink

Unknown interpreted text role "data".
:data:`~scrapy.log.ERROR` - for regular errors

System Message: ERROR/3 (<stdin>, line 23); backlink

Unknown interpreted text role "data".
:data:`~scrapy.log.WARNING` - for warning messages

System Message: ERROR/3 (<stdin>, line 24); backlink

Unknown interpreted text role "data".
:data:`~scrapy.log.INFO` - for informational messages

System Message: ERROR/3 (<stdin>, line 25); backlink

Unknown interpreted text role "data".
:data:`~scrapy.log.DEBUG` - for debugging messages

System Message: ERROR/3 (<stdin>, line 26); backlink

Unknown interpreted text role "data".

How to set the log level

You can set the log level using the --loglevel/-L command line option, or using the :setting:`LOG_LEVEL` setting.

System Message: ERROR/3 (<stdin>, line 31); backlink

Unknown interpreted text role "setting".

How to log messages

Here's a quick example of how to log a message using the WARNING level:

from scrapy import log
log.msg("This is a warning", level=log.WARNING)

Logging from Spiders

The recommended way to log from spiders is by using the Spider :meth:`~scrapy.spider.BaseSpider.log` method, which already populates the domain argument of the :func:`scrapy.log.msg` function. The other arguments are passed directly to the :func:`~scrapy.log.msg` function.

System Message: ERROR/3 (<stdin>, line 45); backlink

Unknown interpreted text role "meth".

System Message: ERROR/3 (<stdin>, line 45); backlink

Unknown interpreted text role "func".

System Message: ERROR/3 (<stdin>, line 45); backlink

Unknown interpreted text role "func".

scrapy.log module

System Message: ERROR/3 (<stdin>, line 53)

Unknown directive type "module".

.. module:: scrapy.log
   :synopsis: Logging facility

System Message: ERROR/3 (<stdin>, line 56)

Unknown directive type "attribute".

.. attribute:: log_level

   The current log level being used

System Message: ERROR/3 (<stdin>, line 60)

Unknown directive type "attribute".

.. attribute:: started

   A boolean which is ``True`` is logging has been started or ``False`` otherwise.

System Message: ERROR/3 (<stdin>, line 64)

Unknown directive type "function".

.. function:: start(logfile=None, loglevel=None, logstdout=None)

    Start the logging facility. This must be called before actually logging any
    messages. Otherwise, messages logged before this call will get lost.

    ``logfile`` is a string with the file path to use for logging output. If
    omitted, :setting:`LOG_FILE` setting will be used. If both are ``None``, log
    will be sent to standard output (if ``logstdout`` or :setting:`LOG_STDOUT` is
    ``True``) or standard error (if ``log_stderr`` or :setting:`LOG_STDOUT` is
    ``False``).

    ``loglevel`` is the logging level. Availables ones are: :data:`CRITICAL`,
    :data:`ERROR`, :data:`WARNING`, :data:`INFO` and :data:`DEBUG`.

    ``log_stdout`` is a boolean which specifies if log should be sent to standard
    output (if ``True``) or standard error (if ``False``)

System Message: ERROR/3 (<stdin>, line 81)

Unknown directive type "function".

.. function:: msg(message, level=INFO, component=BOT_NAME, domain=None)

    Log a message

    :param message: the message to log
    :type message: str

    :param level: the log level for this message. See
        :ref:`topics-logging-levels`.

    :param component: the component to use for logging, it defaults to
        :setting:`BOT_NAME`
    :type component: str

    :param domain: the spider domain to use for logging this message. This
        parameter should always be used when logging things related to a
        particular spider.
    :type domain: str

System Message: ERROR/3 (<stdin>, line 100)

Unknown directive type "function".

.. function:: exc(message, level=ERROR, component=BOT_NAME, domain=None)

    Log an exception. Similar to ``msg()`` but it also appends a stack trace
    report using `traceback.format_exc`.

    .. _traceback.format_exc: http://docs.python.org/library/traceback.html#traceback.format_exc

    It accepts the same parameters as the :func:`msg` function.

System Message: ERROR/3 (<stdin>, line 109)