RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
Scrapy, a fast high-level web crawling & scraping framework for Python.
2,214 Commits 49 Branches 132 Tags 145 MiB
Python 99.8%
HTML 0.1%
Go to file
Pablo Hoffman 8d24175c81 Added CrawlerProcess class, isolating all (Twisted) reactor-controlling code 
into this class and leaving the Crawler class free of any reactor control.

This allows embedding the Scrapy crawler in other Twisted applications, or any
Twisted asynchronous code.

The "scrapy" command will use the new CrawlerProcess class, which resembles the
behaviour of the old (all-in-one) Crawler class.

Closes #214.

Also moved log.start() call from Crawler class to scrapy.cmdline module.
 		2010-08-25 05:24:02 -03:00
bin Removed scrapy-sqs script, as it has been superseded by the new scrapy 'queue' command 2010-08-23 22:04:49 -03:00
debian Removed scrapy-sqs script, as it has been superseded by the new scrapy 'queue' command 2010-08-23 22:04:49 -03:00
docs Added documentation for Ubuntu packages. Refs #211 2010-08-23 21:28:32 -03:00
examples example projects: added scrapy.cfg and removed scrapy-ctl.py 2010-08-21 04:56:19 -03:00
extras Minor improvement to bash autocompletion 2010-08-22 20:10:11 -03:00
profiling/priorityqueue mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
scrapy Added CrawlerProcess class, isolating all (Twisted) reactor-controlling code 2010-08-25 05:24:02 -03:00
scripts removed python2.5 from rpm-install.sh script 2009-06-16 13:14:40 -03:00
.hgignore ignore docs/build 2009-07-25 15:21:22 -03:00
.hgtags Added tag 0.9 for changeset 5caf3dc10a92 2010-06-28 00:54:47 -03:00
AUTHORS Applied documentation patch provided by Lucian Ursu (closes #207) 2010-08-21 01:26:35 -03:00
INSTALL mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
LICENSE mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
MANIFEST.in added some missing file to MANIFEST.in 2009-09-28 23:55:00 -03:00
README added scrapy-sqs.py to deployed scripts 2010-06-13 19:17:17 -03:00
setup.cfg added bitmap for windows installer 2009-09-17 02:01:40 -03:00
setup.py Removed scrapy-sqs script, as it has been superseded by the new scrapy 'queue' command 2010-08-23 22:04:49 -03:00

README 

This is Scrapy, an opensource screen scraping framework written in Python.

For more visit the project home page at http://scrapy.org