RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Improvements.

This commit is contained in:
Andrey Rakhmatullin 2026-07-17 17:58:03 +05:00
parent bf33614e62
commit 028777d617
2 changed files with 30 additions and 23 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
docs/topics/commands.rst
View File

@ -662,23 +662,37 @@ and overriding specific methods. Here's what you need to know:
**Attributes you can set:**
* ``requires_project`` (bool): If True, the command requires a Scrapy project to be present (default: False)
* ``requires_crawler_process`` (bool): If True, the command requires a crawler process to be available (default: True)
* ``default_settings`` (dict): Default settings to use for this command instead of global defaults (default: {})
* ``exitcode`` (int): Exit code to return when command completes (default: 0)
* :attr:`~scrapy.commands.ScrapyCommand.requires_project` (bool): If ``True``,
the command only runs inside a Scrapy project (default: ``False``).
* :attr:`~scrapy.commands.ScrapyCommand.requires_crawler_process` (bool): If
``True``, a :class:`~scrapy.crawler.AsyncCrawlerProcess` or
:class:`~scrapy.crawler.CrawlerProcess` instance will be created by Scrapy
when the command runs and made available in the
:attr:`~scrapy.commands.ScrapyCommand.crawler_process` attribute (default:
``True``).
* :attr:`~scrapy.commands.ScrapyCommand.default_settings` (dict): Settings that
will override the default ones when running this command (default: ``{}``).
* :attr:`~scrapy.commands.ScrapyCommand.exitcode` (int): Process exit code to
set when the command completes (default: ``0``).
**Methods you must override:**
* :meth:`~scrapy.commands.ScrapyCommand.syntax`: Return command syntax (preferably one-line, without command name)
* :meth:`~scrapy.commands.ScrapyCommand.short_desc`: Return a short description of the command
* :meth:`~scrapy.commands.ScrapyCommand.run`: Main entry point for command execution (must implement)
* :meth:`~scrapy.commands.ScrapyCommand.short_desc`: Return a short description
of the command.
* :meth:`~scrapy.commands.ScrapyCommand.run`: Main entry point for the command
execution.
**Methods you can override:**
* :meth:`~scrapy.commands.ScrapyCommand.long_desc`: Return a detailed description (can contain newlines)
* :meth:`~scrapy.commands.ScrapyCommand.help`: Return extensive help text (can contain newlines)
* :meth:`~scrapy.commands.ScrapyCommand.add_options`: Add command-specific options to argument parser
* :meth:`~scrapy.commands.ScrapyCommand.process_options`: Process parsed command-line options
* :meth:`~scrapy.commands.ScrapyCommand.syntax`: Return command syntax
(preferably one-line, without command name).
* :meth:`~scrapy.commands.ScrapyCommand.long_desc`: Return a detailed command
description.
* :meth:`~scrapy.commands.ScrapyCommand.add_options`: Add command-specific
options to the argument parser.
* :meth:`~scrapy.commands.ScrapyCommand.process_options`: Process parsed
command-line options and set settings before
:attr:`~scrapy.commands.ScrapyCommand.crawler_process` is instantiated.
**Example custom command:**

17
scrapy/commands/__init__.py
View File

@ -60,16 +60,12 @@ class ScrapyCommand(ABC):
self._crawler: Crawler = crawler
def syntax(self) -> str:
"""
Command syntax (preferably one-line). Do not include command name.
"""
"""Command syntax (preferably one-line). Do not include command name."""
return ""
@abstractmethod
def short_desc(self) -> str:
"""
A short description of the command
"""
"""A short description of the command."""
return ""
def long_desc(self) -> str:
@ -88,9 +84,7 @@ class ScrapyCommand(ABC):
return self.long_desc()
def add_options(self, parser: argparse.ArgumentParser) -> None:
"""
Populate option parse with options available for this command
"""
"""Populate the option parser with the options available for this command."""
assert self.settings is not None
group = parser.add_argument_group(title="Global Options")
group.add_argument(
@ -124,6 +118,7 @@ class ScrapyCommand(ABC):
group.add_argument("--pdb", action="store_true", help="enable pdb on failure")
def process_options(self, args: list[str], opts: argparse.Namespace) -> None:
"""Set settings based on the command line options."""
assert self.settings is not None
try:
self.settings.setdict(arglist_to_dict(opts.set), priority="cmdline")
@ -153,9 +148,7 @@ class ScrapyCommand(ABC):
@abstractmethod
def run(self, args: list[str], opts: argparse.Namespace) -> None:
"""
Entry point for running commands
"""
"""Entry point for running commands."""
raise NotImplementedError