RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
Scrapy, a fast high-level web crawling & scraping framework for Python.
2,926 Commits 49 Branches 132 Tags 143 MiB
Python 99.8%
HTML 0.1%
Go to file
Daniel Graña cc6e297062 Merge pull request #98 from kalessin/start_requests 
This will break any spider that extends `start_requests` and expect a `list` as return value.

In the other side:

* [Docs](http://doc.scrapy.org/en/latest/topics/spiders.html#scrapy.spider.BaseSpider.start_requests) says that return value is **iterable** not list: 
* Scrapy core already support consuming start_requests generator on demand so we can avoid problems like #47
* it allows extensions to change starting requests on `spider_opened` signal
 		2012-03-05 08:51:22 -08:00
bin slightly improved version of scrapyd script 2011-06-20 11:04:38 -03:00
debian make scrapyd debian package dependent on the same (or higher) version of scrapy package 2012-01-13 10:55:20 -02:00
docs Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25 2012-03-01 08:18:12 -02:00
extras Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25 2012-03-01 08:18:12 -02:00
scrapy replace list by generator also in start_requests method of Sitemap 2012-03-05 14:25:12 -02:00
scrapyd Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25 2012-03-01 08:18:12 -02:00
.coveragerc Added rules to Makefile.buildbot for generating coverage reports 2010-12-15 11:13:45 -02:00
.gitignore Merge branch '0.12' 2011-09-18 01:08:04 -03:00
.hgtags Added tag 0.10.3 for changeset 803efdb19e0b 2010-09-29 13:34:56 -03:00
AUTHORS added Darian Moody to AUTHORS 2011-06-12 01:42:30 -03:00
INSTALL mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
LICENSE mv scrapy/trunk to root as part of svn2hg migration 2009-05-06 15:55:17 -03:00
MANIFEST.in moved module scrapy.core.downloader.responsetypes to scrapy.responsetypes 2011-08-07 02:49:57 -03:00
Makefile.buildbot moved coverage report script to extras/ 2010-12-15 12:08:59 -02:00
README fixed typo 2011-04-07 02:04:42 -03:00
setup.cfg moved rpm-install.sh to extras/ 2011-09-18 06:08:28 -03:00
setup.py Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25 2012-03-01 08:18:12 -02:00

README 

This is Scrapy, an opensource screen scraping framework written in Python.

For more info visit the project home page at http://scrapy.org