| .. |
|
__init__.py
|
mv scrapy/trunk to root as part of svn2hg migration
|
2009-05-06 15:55:17 -03:00 |
|
conf.py
|
Reversed scrapy.cfg lookup order so that the one in the current project has more precedence. Also added alternative system-wide location for windows.
|
2010-10-18 03:18:54 -02:00 |
|
console.py
|
scrapy.utils.python: fixed bug introduced when adding support for new IPython 0.11. refs #335
|
2011-08-09 01:38:31 -03:00 |
|
datatypes.py
|
Initial support for a persistent scheduler, to support pausing and resuming
|
2011-08-02 11:57:55 -03:00 |
|
decorator.py
|
replaced DeprecationWarning by a new ScrapyDeprecationWarning category, since the default DeprecationWarning is silenced on Python 2.7+
|
2011-08-08 10:39:53 -03:00 |
|
defer.py
|
Improve spider errors logging which were previously logged as confusing "Unhandled errors" - closes #196
|
2010-08-13 01:45:47 -03:00 |
|
deprecate.py
|
replaced DeprecationWarning by a new ScrapyDeprecationWarning category, since the default DeprecationWarning is silenced on Python 2.7+
|
2011-08-08 10:39:53 -03:00 |
|
display.py
|
"parse" command refactoring. This fixes #173 and renders #106 invalid.
|
2010-08-22 05:04:17 -03:00 |
|
engine.py
|
Another step towards singleton removal: deprecated crawler singleton import (from scrapy.project import crawler) by a new class method that extensions can implement to receive the crawler
|
2011-08-08 11:42:44 -03:00 |
|
ftp.py
|
Added new Feed exports extension with documentation and storage tests. Closes #197.
|
2010-08-17 14:27:48 -03:00 |
|
gz.py
|
fixed struct.error on http compression middleware. closes #87
|
2012-02-11 20:28:09 -02:00 |
|
http.py
|
added new downloader middleware: ChunkedTransferMiddleware
|
2011-08-09 03:03:25 -03:00 |
|
httpobj.py
|
remove Url class and use str instead for Request and Response urls. Also added urlparse_cached function for achieving the same caching functionality provided by old Url class
|
2009-08-17 21:16:55 -03:00 |
|
iterators.py
|
format log lines lazily in case they are dropped by loglevels
|
2012-08-31 17:14:35 -03:00 |
|
job.py
|
Initial support for a persistent scheduler, to support pausing and resuming
|
2011-08-02 11:57:55 -03:00 |
|
jsonrpc.py
|
Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25
|
2012-03-01 08:18:12 -02:00 |
|
markup.py
|
Moved several functions from scrapy.utils.{http,markup,multipart,response,url} (and their tests) to a new library called 'w3lib'. Scrapy will now depend on w3lib.
|
2011-04-18 22:37:19 -03:00 |
|
misc.py
|
make selector re() method use re.UNICODE flag to compile regexes
|
2012-04-01 00:41:03 -03:00 |
|
multipart.py
|
Moved several functions from scrapy.utils.{http,markup,multipart,response,url} (and their tests) to a new library called 'w3lib'. Scrapy will now depend on w3lib.
|
2011-04-18 22:37:19 -03:00 |
|
ossignal.py
|
Some enhancements to Scrapy core:
|
2009-09-03 08:27:48 -03:00 |
|
pqueue.py
|
fixed subtle bug in disk-based priority queues caused by serialization errors, and added tests
|
2011-09-02 09:40:52 -03:00 |
|
project.py
|
if you put in settings.py something like import local_settings and local_settings doesn't exist or contains errors you get:
|
2012-07-09 08:39:14 +04:00 |
|
py26.py
|
Restored scrapy.utils.py26 module for backwards compatibility, with a deprecation message. This is needed because the module was used a lot by users and the change causes too much trouble
|
2012-03-05 17:15:49 -02:00 |
|
py27.py
|
Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25
|
2012-03-01 08:18:12 -02:00 |
|
python.py
|
added tests for get_func_args() and support for more cases
|
2012-09-04 18:07:44 -03:00 |
|
queue.py
|
Small improvement on *MemoryQueue performance (mainly on push time)
|
2012-08-10 10:38:20 -03:00 |
|
reactor.py
|
more core refactoring including moving engine next request call logic to a separate class
|
2011-07-25 10:46:00 -03:00 |
|
reqser.py
|
Initial support for a persistent scheduler, to support pausing and resuming
|
2011-08-02 11:57:55 -03:00 |
|
request.py
|
Scheduler refactoring which introduces the following changes:
|
2011-06-06 03:16:56 -03:00 |
|
response.py
|
support TextResponse in open_in_browser util
|
2012-04-25 16:04:37 -03:00 |
|
serialize.py
|
use ScrapyJSONEncoder JsonItemExporter & JsonLinesItemExporter, to support nested items properly
|
2012-08-22 13:46:50 -03:00 |
|
signal.py
|
format log lines lazily in case they are dropped by loglevels
|
2012-08-31 17:14:35 -03:00 |
|
sitemap.py
|
scrapy.utils.sitemap: added one more case of parsing invalid sitemaps
|
2011-08-07 03:24:32 -03:00 |
|
spider.py
|
format log lines lazily in case they are dropped by loglevels
|
2012-08-31 17:14:35 -03:00 |
|
template.py
|
Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25
|
2012-03-01 08:18:12 -02:00 |
|
test.py
|
fixed tests to work on windows
|
2012-08-30 11:24:29 -03:00 |
|
testproc.py
|
utils.testproc: make spawned process use the original CWD, instead of the temporary one created by twisted trial
|
2010-11-29 09:56:14 -02:00 |
|
testsite.py
|
* Added tests for shell/fetch/version commands (closes #255)
|
2010-11-28 18:14:45 -02:00 |
|
trackref.py
|
added trackref stats to memory debugger report. closes #272
|
2010-10-27 21:18:58 -02:00 |
|
txweb.py
|
Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25
|
2012-03-01 08:18:12 -02:00 |
|
url.py
|
handle missing paths in urls as /
|
2012-05-03 13:56:45 -03:00 |
caa64908d2