diff --git a/docs/topics/commands.rst b/docs/topics/commands.rst index c14d0d488..418648c8e 100644 --- a/docs/topics/commands.rst +++ b/docs/topics/commands.rst @@ -153,8 +153,7 @@ Project-only commands: * :command:`list` * :command:`parse` * :command:`genspider` -* :command:`runserver` -* :command:`queue` +* :command:`server` * :command:`deploy` .. command:: startproject @@ -235,24 +234,23 @@ Usage examples:: $ scrapy crawl http://example.com/some/page.html [ ... spider that handles example.com starts crawling from that url ... ] -.. command:: runserver +.. command:: server -runserver ---------- +server +------ -* Syntax: ``scrapy runserver`` +* Syntax: ``scrapy server`` * Requires project: *yes* -Start Scrapy in server mode, which can be controlled by the :command:`queue` -command. +Start Scrapyd server for this project, which can be referred from the JSON API +with the project name ``default``. For more info see: :ref:`topics-scrapyd`. Usage example:: - $ scrapy runserver - [ ... scrapy starts and stays idle waiting for spiders to get scheduled ... ] + $ scrapy server + [ ... scrapyd starts and stays idle waiting for spiders to get scheduled ... ] -You can now schedule spiders to run using the :command:`queue` command. If -there were spiders already enqueued, it will start crawling them. +To schedule spiders, use the Scrapyd JSON API. .. command:: list @@ -410,35 +408,6 @@ Example usage:: $ scrapy runspider myspider.py [ ... spider starts crawling ... ] -.. command:: queue - -queue ------ - -* Syntax: ``scrapy queue `` -* Requires project: *yes* - -Manage the execution queue of a Scrapy project. - -This command is meant to be used to control a Scrapy server started with the -:command:`runserver` command. - -Example usage:: - - $ scrapy queue add example.com - -If there is a Scrapy server running (see :command:`runserver` command), it will -start crawling the ``example.com`` spider. Otherwise, it will only get -enqueued,, and it will start crawling once the Scrapy server is started. - -You can also view the spiders enqueued but not yet started:: - - $ scrapy queue list - -And clear the queue:: - - $ scrapy queue clear - .. command:: version version diff --git a/docs/topics/scrapyd.rst b/docs/topics/scrapyd.rst index 331b41738..e8542f803 100644 --- a/docs/topics/scrapyd.rst +++ b/docs/topics/scrapyd.rst @@ -31,7 +31,7 @@ for spiders that need to run. When a spider needs to run, a process is started to crawl the spider:: - scrapy crawl my spider + scrapy crawl myspider Scrapyd also runs multiple processes in parallel, allocating them in a fixed number of slots given by the `max_proc`_ and `max_proc_per_cpu`_ options, @@ -58,6 +58,11 @@ distribution, like this:: That should get your Scrapyd started. +Or, if you want to start Scrapyd from inside a Scrapy project you can use the +:command:`server` command, like this:: + + scrapy server + Installing Scrapyd ================== diff --git a/scrapy/commands/queue.py b/scrapy/commands/queue.py index f4a6d126e..dbede1414 100644 --- a/scrapy/commands/queue.py +++ b/scrapy/commands/queue.py @@ -14,7 +14,7 @@ class Command(runserver.Command): return "[options] " def short_desc(self): - return "Control the spider queue" + return "Deprecated command. See Scrapyd documentation." def add_options(self, parser): ScrapyCommand.add_options(self, parser) diff --git a/scrapy/commands/runserver.py b/scrapy/commands/runserver.py index 7f4dc970b..c40133418 100644 --- a/scrapy/commands/runserver.py +++ b/scrapy/commands/runserver.py @@ -7,7 +7,7 @@ class Command(ScrapyCommand): default_settings = {'KEEP_ALIVE': True} def short_desc(self): - return "Start Scrapy in server mode" + return "Deprecated command. Use 'server' command instead" def run(self, args, opts): self.crawler.start() diff --git a/scrapy/commands/scrapyd.py b/scrapy/commands/server.py similarity index 100% rename from scrapy/commands/scrapyd.py rename to scrapy/commands/server.py