RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
scrapy/docs/index.rst

5.2 KiB
Raw Blame History

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> </head>

Scrapy |version| documentation

This documentation contains everything you need to know about Scrapy.

Getting help

Having trouble? We'd like to help!

First steps

System Message: ERROR/3 (<stdin>, line 30)

Unknown directive type "toctree".

.. toctree::
   :hidden:

   intro/overview
   intro/install
   intro/tutorial
   intro/examples
:doc:`intro/overview`

System Message: ERROR/3 (<stdin>, line 39); backlink

Unknown interpreted text role "doc".

Understand what Scrapy is and how it can help you.

:doc:`intro/install`

System Message: ERROR/3 (<stdin>, line 42); backlink

Unknown interpreted text role "doc".

Get Scrapy installed on your computer.

:doc:`intro/tutorial`

System Message: ERROR/3 (<stdin>, line 45); backlink

Unknown interpreted text role "doc".

Write your first Scrapy project.

:doc:`intro/examples`

System Message: ERROR/3 (<stdin>, line 48); backlink

Unknown interpreted text role "doc".

Learn more by playing with a pre-made Scrapy project.

Basic concepts

System Message: ERROR/3 (<stdin>, line 55)

Unknown directive type "toctree".

.. toctree::
   :hidden:

   topics/commands
   topics/items
   topics/spiders
   topics/link-extractors
   topics/selectors
   topics/loaders
   topics/shell
   topics/item-pipeline
   topics/feed-exports
   topics/link-extractors
:doc:`topics/commands`

System Message: ERROR/3 (<stdin>, line 70); backlink

Unknown interpreted text role "doc".

Learn about the command-line tool used to manage your Scrapy project.

:doc:`topics/items`

System Message: ERROR/3 (<stdin>, line 73); backlink

Unknown interpreted text role "doc".

Define the data you want to scrape.

:doc:`topics/spiders`

System Message: ERROR/3 (<stdin>, line 76); backlink

Unknown interpreted text role "doc".

Write the rules to crawl your websites.

:doc:`topics/selectors`

System Message: ERROR/3 (<stdin>, line 79); backlink

Unknown interpreted text role "doc".

Extract the data from web pages.

:doc:`topics/shell`

System Message: ERROR/3 (<stdin>, line 82); backlink

Unknown interpreted text role "doc".

Test your extraction code in an interactive environment.

:doc:`topics/loaders`

System Message: ERROR/3 (<stdin>, line 85); backlink

Unknown interpreted text role "doc".

Populate your items with the extracted data.

:doc:`topics/item-pipeline`

System Message: ERROR/3 (<stdin>, line 88); backlink

Unknown interpreted text role "doc".

Post-process and store your scraped data.

:doc:`topics/feed-exports`

System Message: ERROR/3 (<stdin>, line 91); backlink

Unknown interpreted text role "doc".

Output your scraped data using different formats and storages.

:doc:`topics/link-extractors`

System Message: ERROR/3 (<stdin>, line 94); backlink

Unknown interpreted text role "doc".

Convenient classes to extract links to follow from pages.

Built-in services

System Message: ERROR/3 (<stdin>, line 99)

Unknown directive type "toctree".

.. toctree::
   :hidden:

   topics/logging
   topics/stats
   topics/email
   topics/telnetconsole
   topics/webservice
:doc:`topics/logging`

System Message: ERROR/3 (<stdin>, line 109); backlink

Unknown interpreted text role "doc".

Understand the simple logging facility provided by Scrapy.

:doc:`topics/stats`

System Message: ERROR/3 (<stdin>, line 112); backlink

Unknown interpreted text role "doc".

Collect statistics about your scraping crawler.

:doc:`topics/email`

System Message: ERROR/3 (<stdin>, line 115); backlink

Unknown interpreted text role "doc".

Send email notifications when certain events occur.

:doc:`topics/telnetconsole`

System Message: ERROR/3 (<stdin>, line 118); backlink

Unknown interpreted text role "doc".

Inspect a running crawler using a built-in Python console.

:doc:`topics/webservice`

System Message: ERROR/3 (<stdin>, line 122); backlink

Unknown interpreted text role "doc".

Monitor and control a crawler using a web service.

Solving specific problems

System Message: ERROR/3 (<stdin>, line 127)

Unknown directive type "toctree".

.. toctree::
   :hidden:

   faq
   topics/firefox
   topics/firebug
   topics/leaks
   topics/images
   topics/ubuntu
   topics/scrapyd
   topics/jobs
:doc:`faq`

System Message: ERROR/3 (<stdin>, line 140); backlink

Unknown interpreted text role "doc".

Get answers to most frequently asked questions.

:doc:`topics/firefox`

System Message: ERROR/3 (<stdin>, line 143); backlink

Unknown interpreted text role "doc".

Learn how to scrape with Firefox and some useful add-ons.

:doc:`topics/firebug`

System Message: ERROR/3 (<stdin>, line 146); backlink

Unknown interpreted text role "doc".

Learn how to scrape efficiently using Firebug.

:doc:`topics/leaks`

System Message: ERROR/3 (<stdin>, line 149); backlink

Unknown interpreted text role "doc".

Learn how to find and get rid of memory leaks in your crawler.

:doc:`topics/images`

System Message: ERROR/3 (<stdin>, line 152); backlink

Unknown interpreted text role "doc".

Download static images associated with your scraped items.

:doc:`topics/ubuntu`

System Message: ERROR/3 (<stdin>, line 155); backlink

Unknown interpreted text role "doc".

Install latest Scrapy packages easily on Ubuntu

:doc:`topics/scrapyd`

System Message: ERROR/3 (<stdin>, line 158); backlink

Unknown interpreted text role "doc".

Deploying your Scrapy project in production.

:doc:`topics/jobs`

System Message: ERROR/3 (<stdin>, line 161); backlink

Unknown interpreted text role "doc".

Learn how to pause and resume crawls for large spiders.

Extending Scrapy

System Message: ERROR/3 (<stdin>, line 168)

Unknown directive type "toctree".

.. toctree::
   :hidden:

   topics/architecture
   topics/downloader-middleware
   topics/spider-middleware
   topics/extensions
:doc:`topics/architecture`

System Message: ERROR/3 (<stdin>, line 177); backlink

Unknown interpreted text role "doc".

Understand the Scrapy architecture.

:doc:`topics/downloader-middleware`

System Message: ERROR/3 (<stdin>, line 180); backlink

Unknown interpreted text role "doc".

Customize how pages get requested and downloaded.

:doc:`topics/spider-middleware`

System Message: ERROR/3 (<stdin>, line 183); backlink

Unknown interpreted text role "doc".

Customize the input and output of your spiders.

:doc:`topics/extensions`

System Message: ERROR/3 (<stdin>, line 188); backlink

Unknown interpreted text role "doc".

Add any custom functionality using :doc:`signals <topics/signals>` and the Scrapy API

System Message: ERROR/3 (<stdin>, line 186); backlink

Unknown interpreted text role "doc".

Reference

System Message: ERROR/3 (<stdin>, line 193)

Unknown directive type "toctree".

.. toctree::
   :hidden:

   topics/request-response
   topics/settings
   topics/signals
   topics/exceptions
   topics/exporters
:doc:`topics/commands`

System Message: ERROR/3 (<stdin>, line 203); backlink

Unknown interpreted text role "doc".

Learn about the command-line tool and see all :ref:`available commands <topics-commands-ref>`.

System Message: ERROR/3 (<stdin>, line 203); backlink

Unknown interpreted text role "ref".
:doc:`topics/request-response`

System Message: ERROR/3 (<stdin>, line 206); backlink

Unknown interpreted text role "doc".

Understand the classes used to represent HTTP requests and responses.

:doc:`topics/settings`

System Message: ERROR/3 (<stdin>, line 209); backlink

Unknown interpreted text role "doc".

Learn how to configure Scrapy and see all :ref:`available settings <topics-settings-ref>`.

System Message: ERROR/3 (<stdin>, line 209); backlink

Unknown interpreted text role "ref".
:doc:`topics/signals`

System Message: ERROR/3 (<stdin>, line 212); backlink

Unknown interpreted text role "doc".

See all available signals and how to work with them.

:doc:`topics/exceptions`

System Message: ERROR/3 (<stdin>, line 215); backlink

Unknown interpreted text role "doc".

See all available exceptions and their meaning.

:doc:`topics/exporters`

System Message: ERROR/3 (<stdin>, line 219); backlink

Unknown interpreted text role "doc".

Quickly export your scraped items to a file (XML, CSV, etc).

All the rest

System Message: ERROR/3 (<stdin>, line 224)

Unknown directive type "toctree".

.. toctree::
   :hidden:

   news
   contributing
   versioning
   experimental/index
:doc:`news`

System Message: ERROR/3 (<stdin>, line 233); backlink

Unknown interpreted text role "doc".

See what has changed in recent Scrapy versions.

:doc:`contributing`

System Message: ERROR/3 (<stdin>, line 236); backlink

Unknown interpreted text role "doc".

Learn how to contribute to the Scrapy project.

:doc:`versioning`

System Message: ERROR/3 (<stdin>, line 239); backlink

Unknown interpreted text role "doc".

Understand Scrapy versioning and API stability.

:doc:`experimental/index`

System Message: ERROR/3 (<stdin>, line 241); backlink

Unknown interpreted text role "doc".

Learn about bleeding-edge features.

Docutils System Messages

System Message: ERROR/3 (<stdin>, line 5); backlink

Undefined substitution referenced: "version".
</html>