| .. |
|
downloadermiddleware
|
update docs for defaultheaders middleware and change spider attribute to match global setting name
|
2010-07-16 16:17:08 -03:00 |
|
exporter
|
Added scrapy.utils.py26.json to use python2.6 json module when available, otherwise failback to simplejson module or scrapy.xlib.simplejson. This way we can always assume json and avoid conditional code.
|
2010-04-12 10:44:07 -03:00 |
|
ibl
|
removed nltk dependency from IBL code
|
2010-06-22 13:38:32 -03:00 |
|
linkextractors
|
LinkExtractor: split _process_links from _extract_links
|
2010-04-27 14:58:11 +08:00 |
|
loader
|
{add,replace}_xpath: add processors, kw args and allow field_name to be None
|
2010-04-23 01:34:55 +08:00 |
|
pipeline
|
mediapipeline: bugfix error raised when media requests has not callbacks, remove item_media_{downloaded,failed} hooks in favour or request.{errback,calback}, and add tests
|
2010-06-14 12:34:52 -03:00 |
|
queue
|
Added SQS Execution Queue, and example script to add spiders to the queue
|
2010-06-11 17:22:14 -03:00 |
|
schedulermiddleware
|
Changed (unstable) scheduler middleware API to receive spider (instead of domain) in enqueue_request method
|
2009-09-13 20:51:43 -03:00 |
|
spidermiddleware
|
Removed Scrapy engine singleton from scrapy.core.engine.scrapyengine. Now
|
2010-05-26 10:29:32 -03:00 |
|
spiders
|
Added support for Requests without callbacks (#166) - the Spider.parse() method
|
2010-06-08 18:18:02 -03:00 |
|
webconsole
|
* Added Scrapy Web Service with documentation and tests.
|
2010-06-09 13:46:22 -03:00 |
|
webservice
|
* Added Scrapy Web Service with documentation and tests.
|
2010-06-09 13:46:22 -03:00 |
|
__init__.py
|
mv scrapy/trunk to root as part of svn2hg migration
|
2009-05-06 15:55:17 -03:00 |
|
closespider.py
|
Removed Scrapy engine singleton from scrapy.core.engine.scrapyengine. Now
|
2010-05-26 10:29:32 -03:00 |
|
corestats.py
|
StatsCollector: ported methods to receive spider instances (closes #113), removed list_domains() method, added iter_spider_stats() method
|
2009-11-14 20:28:59 -02:00 |
|
debug.py
|
added Debugger extension, removed StackTraceDump from extensions available by default
|
2009-09-08 22:32:17 -03:00 |
|
dupefilter.py
|
Changed (unstable) scheduler middleware API to receive spider (instead of domain) in enqueue_request method
|
2009-09-13 20:51:43 -03:00 |
|
itemsampler.py
|
Removed Scrapy engine singleton from scrapy.core.engine.scrapyengine. Now
|
2010-05-26 10:29:32 -03:00 |
|
logformatter.py
|
some improvements to 'Crawled' log lines, delegating the formatting to a pluggable function
|
2009-10-06 22:47:17 -02:00 |
|
memdebug.py
|
updated some documentation references in source code
|
2009-08-19 22:41:26 -03:00 |
|
memusage.py
|
Fix memusage report concatenation
|
2010-07-14 18:47:09 -03:00 |
|
resolver.py
|
Fixed memory leak in CachingResolver
|
2009-11-25 16:54:36 -02:00 |
|
spiderclosedelay.py
|
Removed Scrapy engine singleton from scrapy.core.engine.scrapyengine. Now
|
2010-05-26 10:29:32 -03:00 |
|
spidermanager.py
|
Added ExecutionQueue class for feeding spiders and requests to scrape. This
|
2010-05-26 10:29:32 -03:00 |
|
statsmailer.py
|
SEP12 implementation
|
2010-04-01 18:27:22 -03:00 |
3e013f564b