Scrapy, a fast high-level web crawling & scraping framework for Python.

crawlercrawlingframeworkhacktoberfestpythonscrapingweb-scrapingweb-scraping-python

olveyra 338a485bc1 - load only one domain per node (and load the following when the run ... callback is executed). This way, we avoid to load lot of domains that will bounce. Also, we mix up better the domains between available nodes. --HG-- extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4068		2008-07-16 11:20:53 +00:00
scrapy/trunk	- load only one domain per node (and load the following when the run	2008-07-16 11:20:53 +00:00
sites	css styling changes.	2008-06-30 15:07:27 +00:00
.hgignore	dummy file to shift a revision	2009-05-04 18:16:20 +00:00