diff --git a/Related-projects.md b/Related-projects.md new file mode 100644 index 0000000..b93e36b --- /dev/null +++ b/Related-projects.md @@ -0,0 +1,70 @@ +Here you'll find a list of projects that add extend Scrapy functionality or add +unexisting features. + + +## Projects maintained by Scrapy devs + +These following projects are maintained by Scrapy developers. + +### Slybot + +Slybot is a Python web crawler for doing web scraping. It's implemented on top +of the Scrapy web crawling framework and the `Scrapely`_ extraction library. + +Project page: [slybot](https://github.com/scrapy/slybot) + + + +## Projects not maintained by Scrapy devs + +The projects below are **not** maintained by Scrapy developers. So, don't +expect official support. Existing features may also stop working on newer +Scrapy versions. Use at your own risk. Also, the descriptions below were copied +from the project's own description, but always read each project's +documentation for further details. + +### scrapy-sentry + +Logs Scrapy exceptions into Sentry. A python library that glues Sentry with +Scrapy. + +Project page: [scrapy-sentry](https://pypi.python.org/pypi/scrapy-sentry) + +### ScrapyGraphite + +Output scrapy statistics to carbon/graphite. + +Project page: [ScrapyGraphite](https://pypi.python.org/pypi/ScrapyGraphite) + +### scrapy-mongo + +It's a pipeline which allow you to store scrapy items in MongoDB database. + +Project page: [scrapy-mongo](https://pypi.python.org/pypi/scrapy-mongodb) + + +### scrapy-boilerplate + +scrapy-boilerplate is a small set of utilities for Scrapy to simplify writing +low-complexity spiders that are very common in small and one-off projects. + +Project page: [scrapy-boilerplate](https://pypi.python.org/pypi/scrapy-boilerplate) + + +### scrapy-inline-requests + +This module provides a decorator to allow to write Scrapy's spider callbacks +which performs multiple requests without the need to write multiple callbacks +for each request. The code is experimental, might not work in all cases and +even might be hard to debug + +Project page: [scrapy-inline-requests](https://pypi.python.org/pypi/scrapy-inline-requests) + + +### scrapy-redis + +A initial work on Scrapy-Redis integration, not production-tested. Use it at +your own risk! Features distributed crawling/scraping and distributed +post-processing. + +Project page: [scrapy-redis](https://pypi.python.org/pypi/scrapy-redis)