Scrapy, a fast high-level web crawling & scraping framework for Python.
Go to file
Pablo Hoffman 635ac1ca64 Simplified domain prioritizers, so that they don't receive domains in the
constructor (domain prioritizers will be refactored later anyway) and
simplified Scrapy Manager code thanks to this.

Added make_request_from_url method to BaseSpider, splitting funtionality to
create requests from URLs which was previously done all in start_requests.
2009-06-10 14:21:36 -03:00
bin sorted out running of unittests: 2009-05-16 20:11:23 -03:00
docs Simplified domain prioritizers, so that they don't receive domains in the 2009-06-10 14:21:36 -03:00
examples Some changes to HTTP cache middleware: 2009-05-24 19:13:06 -03:00
extras/sql mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
profiling/priorityqueue mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
scrapy Simplified domain prioritizers, so that they don't receive domains in the 2009-06-10 14:21:36 -03:00
scripts mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
.hgignore adding .svn to hgignore to help with hg2svn autocommits 2009-05-07 17:29:30 -03:00
AUTHORS mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
INSTALL mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
LICENSE mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
README mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
setup.cfg mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
setup.py mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00

README

This is Scrapy, an opensource screen scraping framework written in Python.

For more visit the project home page at http://scrapy.org