3.4 KiB

Raw Blame History

Logging

Scrapy provides a logging facility which can be used through the :mod:`scrapy.log` module. The current underlying implementation uses Twisted logging but this may change in the future.

System Message: ERROR/3 (<stdin>, line 7); backlink

Unknown interpreted text role "mod".

The logging service must be explicitly started through the :func:`scrapy.log.start` function.

System Message: ERROR/3 (<stdin>, line 13); backlink

Unknown interpreted text role "func".

Log levels

Scrapy provides 5 logging levels:

:data:`~scrapy.log.CRITICAL` - for critical errors

System Message: ERROR/3 (<stdin>, line 22); backlink

Unknown interpreted text role "data".
:data:`~scrapy.log.ERROR` - for regular errors

System Message: ERROR/3 (<stdin>, line 23); backlink

Unknown interpreted text role "data".
:data:`~scrapy.log.WARNING` - for warning messages

System Message: ERROR/3 (<stdin>, line 24); backlink

Unknown interpreted text role "data".
:data:`~scrapy.log.INFO` - for informational messages

System Message: ERROR/3 (<stdin>, line 25); backlink

Unknown interpreted text role "data".
:data:`~scrapy.log.DEBUG` - for debugging messages

System Message: ERROR/3 (<stdin>, line 26); backlink

Unknown interpreted text role "data".

How to set the log level

You can set the log level using the --loglevel/-L command line option, or using the :setting:`LOG_LEVEL` setting.

System Message: ERROR/3 (<stdin>, line 31); backlink

Unknown interpreted text role "setting".

How to log messages

Here's a quick example of how to log a message using the WARNING level:

from scrapy import log
log.msg("This is a warning", level=log.WARNING)

Logging from Spiders

The recommended way to log from spiders is by using the Spider :meth:`~scrapy.spider.Spider.log` method, which already populates the spider argument of the :func:`scrapy.log.msg` function. The other arguments are passed directly to the :func:`~scrapy.log.msg` function.

System Message: ERROR/3 (<stdin>, line 45); backlink

Unknown interpreted text role "meth".

System Message: ERROR/3 (<stdin>, line 45); backlink

Unknown interpreted text role "func".

System Message: ERROR/3 (<stdin>, line 45); backlink

Unknown interpreted text role "func".

scrapy.log module

System Message: ERROR/3 (<stdin>, line 53)

Unknown directive type "module".

.. module:: scrapy.log
   :synopsis: Logging facility

System Message: ERROR/3 (<stdin>, line 56)

Unknown directive type "function".

.. function:: start(logfile=None, loglevel=None, logstdout=None)

    Start the logging facility. This must be called before actually logging any
    messages. Otherwise, messages logged before this call will get lost.

    :param logfile: the file path to use for logging output. If omitted, the
        :setting:`LOG_FILE` setting will be used. If both are ``None``, the log
        will be sent to standard error.
    :type logfile: str

    :param loglevel: the minimum logging level to log. Available values are:
        :data:`CRITICAL`, :data:`ERROR`, :data:`WARNING`, :data:`INFO` and
        :data:`DEBUG`.

    :param logstdout: if ``True``, all standard output (and error) of your
        application will be logged instead. For example if you "print 'hello'"
        it will appear in the Scrapy log. If omitted, the :setting:`LOG_STDOUT`
        setting will be used.
    :type logstdout: boolean

System Message: ERROR/3 (<stdin>, line 76)

Unknown directive type "function".

.. function:: msg(message, level=INFO, spider=None)

    Log a message

    :param message: the message to log
    :type message: str

    :param level: the log level for this message. See
        :ref:`topics-logging-levels`.

    :param spider: the spider to use for logging this message. This parameter
        should always be used when logging things related to a particular
        spider.
    :type spider: :class:`~scrapy.spider.Spider` object

System Message: ERROR/3 (<stdin>, line 91)