Scrapy, a fast high-level web crawling & scraping framework for Python.

crawlercrawlingframeworkhacktoberfestpythonscrapingweb-scrapingweb-scraping-python

olveyra 29cd1bc3cb Added pb-capable crawler. The idea is to improve cluster performance ... adding communication between crawler and master. At the momento, a remote stop method to the crawler was added to sustitute the previous stop based on kernel signal. Further will add monitoring functionality, because the processes are very silent, mainly when unavailable report is not issued, and offen happens lots of thing that nobody realize on if some fortuite events wouldn't happent --HG-- extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40153		2008-08-08 16:57:33 +00:00
scrapy/trunk	Added pb-capable crawler. The idea is to improve cluster performance	2008-08-08 16:57:33 +00:00
sites	Adding missing templates	2008-07-30 23:47:28 +00:00
.hgignore	dummy file to shift a revision	2009-05-04 18:16:20 +00:00