Logging¶
Scrapy provides a logging facility which can be used through the
scrapy.log
module. The current underlying implementation uses Twisted
logging but this may change in the future.
The logging service must be explicitly started through the scrapy.log.start()
function.
Log levels¶
Scrapy provides 5 logging levels:
How to set the log level¶
You can set the log level using the –loglevel/-L command line option, or
using the LOG_LEVEL
setting.
How to log messages¶
Here’s a quick example of how to log a message using the WARNING
level:
from scrapy import log
log.msg("This is a warning", level=log.WARNING)
Logging from Spiders¶
The recommended way to log from spiders is by using the Spider
log()
method, which already populates the
spider
argument of the scrapy.log.msg()
function. The other arguments
are passed directly to the msg()
function.
scrapy.log module¶
-
scrapy.log.
start
(logfile=None, loglevel=None, logstdout=None)¶ Start the logging facility. This must be called before actually logging any messages. Otherwise, messages logged before this call will get lost.
Parameters: - logfile (str) – the file path to use for logging output. If omitted, the
LOG_FILE
setting will be used. If both areNone
, the log will be sent to standard error. - loglevel – the minimum logging level to log. Available values are:
CRITICAL
,ERROR
,WARNING
,INFO
andDEBUG
. - logstdout (boolean) – if
True
, all standard output (and error) of your application will be logged instead. For example if you “print ‘hello’” it will appear in the Scrapy log. If omitted, theLOG_STDOUT
setting will be used.
- logfile (str) – the file path to use for logging output. If omitted, the
-
scrapy.log.
msg
(message, level=INFO, spider=None)¶ Log a message
Parameters: - message (str) – the message to log
- level – the log level for this message. See Log levels.
- spider (
Spider
object) – the spider to use for logging this message. This parameter should always be used when logging things related to a particular spider.
-
scrapy.log.
CRITICAL
¶ Log level for critical errors
-
scrapy.log.
ERROR
¶ Log level for errors
-
scrapy.log.
WARNING
¶ Log level for warnings
-
scrapy.log.
INFO
¶ Log level for informational messages (recommended level for production deployments)
-
scrapy.log.
DEBUG
¶ Log level for debugging messages (recommended level for development)
Logging settings¶
These settings can be used to configure the logging: