RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,344 Commits 49 Branches 132 Tags 180 MiB
Commit Graph

2344 Commits

Author SHA1 Message Date
Pablo Hoffman 49ffe528a3 Fixed listen_tcp function when receiving None or 0 in portrange argument. Closes #252 2010-09-27 12:27:32 -03:00
Pablo Hoffman 9206806770 setup.py: added support for generating version from hg revision 2010-09-27 08:19:32 -03:00
Pablo Hoffman 52d198afc9 Removed forked cookielib tests, because Python cookielib has been suffering several changes and maintaining a fork of the tests has become a pain. Instead, we've added specific tests for the urllib2 request/response wrappers 2010-09-27 07:55:27 -03:00
Pablo Hoffman 3db8d367c4 Another fix to tests, broken by recent Python changes 2010-09-26 21:09:14 -03:00
Pablo Hoffman 5d4e0655d0 Fixed test_utils_url, broken on Windows after recent Python urllib change. Closes #251 2010-09-26 16:57:42 -03:00
Pablo Hoffman 1ef2cd400c Fixed issue with unicode keyword arguments bug in earlier Python versions (see http://bugs.python.org/issue2646). Closes #250 2010-09-25 21:17:36 -03:00
Pablo Hoffman b11e11037f Make custom project commands discovery work with eggified projects. Fixes #247 2010-09-22 15:42:31 -03:00
Pablo Hoffman 2ebfa7e68d Removed unneeded code (since autodoc is not used in Sphinx doc) 2010-09-22 10:52:02 -03:00
Pablo Hoffman f3769651af Refactored Scrapyd code to fix a couple of bugs that ocurred when running projects without eggs 2010-09-22 01:04:15 -03:00
Pablo Hoffman 0ffc8d29b6 Don't silence warnings after log is started. Closes #246 2010-09-22 01:00:34 -03:00
Pablo Hoffman 4c61df7abb get_spider_list_from_eggfile(): fixed bug when SCRAPY_SETTINGS_DISABLED is set 2010-09-20 08:47:55 -03:00
Pablo Hoffman 8329491296 Added Shuaib Khan to AUTHORS 2010-09-20 08:34:22 -03:00
Shuaib 9288f622f9 Added formname parameter for FormRequest.from_response 2010-09-20 08:33:24 -03:00
Pablo Hoffman 400c4134af Make scrapyd.eggutils compatible with Python 2.5 and added tests for get_spider_list_from_eggfile() function (closes #242) 2010-09-19 21:08:27 -03:00
Ping Yin aa86f180b9 BaseSgmlLinkExtractor: Fix extract '>>' as '>' 
The anchor text is extracted as '>' in
<a href="/">&gt;&gt;</a>

Signed-off-by: Ping Yin <pkufranky@gmail.com>
 2010-09-19 19:28:35 -03:00
Ping Yin f41fd3214e test_engine.py: fix typo brwoser => browser 
Signed-off-by: Ping Yin <pkufranky@gmail.com>
 2010-09-19 19:28:13 -03:00
Pablo Hoffman 08952562e9 Make Ubuntu package conflict with standard Scrapy Debian/Ubuntu package 2010-09-17 16:25:02 -03:00
Daniel Grana cad077574b Fix black shapes appearing round transparent images 2010-09-16 14:19:32 -03:00
Pablo Hoffman 4cecbcdc5b Fixed bug in Scrapyd launcher when running projects without eggs. Refs #238 2010-09-15 21:03:43 -03:00
Pablo Hoffman a5908f052f Bumped version to 0.10.3 2010-09-15 21:03:21 -03:00
Pablo Hoffman b33c8e49e2 Added tag 0.10.2 for changeset 68d08e757571 2010-09-15 14:48:47 -03:00
Pablo Hoffman c01b8651ff Bumped version to 0.10.2 2010-09-15 14:04:33 -03:00
Pablo Hoffman 5fe9a49e51 Force recalculating the encoding on HTTPCompression middleware until we are sure the responsetypes guessing is reliable. Refs #239, #240 2010-09-15 13:57:17 -03:00
Pablo Hoffman 28f6bdf7bb Added tag 0.10.1 for changeset bb80107f959d 2010-09-15 02:44:41 -03:00
Pablo Hoffman beee7e4fbc Merge with trunk 2010-09-15 01:50:42 -03:00
Pablo Hoffman 39499a2437 Fixed bug in HTTP Compression middleware which was failing to properly discover the encoding when the encoding was declared inside the response body. Closes #239. Also changed responsetypes to return Response class (instead of HtmlResponse) when the response has a Content-Encoding header 2010-09-14 20:22:25 -03:00
Pablo Hoffman 9acc99e723 Added 'inthread' decorator 2010-09-14 14:36:35 -03:00
Pablo Hoffman c559b06a85 Removed unused import 2010-09-14 01:53:05 -03:00
Pablo Hoffman 833baa6041 Support running projects without eggs in Scrapyd. Closes #238 2010-09-14 01:44:25 -03:00
Pablo Hoffman b76cd42690 Added tests for Scrapyd components. Closes #237 2010-09-14 01:44:10 -03:00
Pablo Hoffman d0c7a946bd Bumped version to 0.10.1 2010-09-13 19:02:21 -03:00
Pablo Hoffman 1fa96dbcd3 Bumped version to 0.11 2010-09-13 19:02:15 -03:00
Pablo Hoffman e2a3ab5141 Added tag 0.10 for changeset 3b32720c96a3 2010-09-13 18:49:33 -03:00
Pablo Hoffman a3fc60cfcd Added missing mime.types files to MANIFEST.in 2010-09-11 04:30:08 -03:00
Pablo Hoffman bf467fc37a Check 'dont_merge_cookies' membership in request.meta, instead of getting its value 2010-09-10 15:29:15 -03:00
Pablo Hoffman e7a958a035 Fixed exception thrown when in FreeBSD when /proc exists but it's not mounted. Closes #235 2010-09-10 15:17:36 -03:00
Pablo Hoffman b16eff227b Bumped version to 0.10 2010-09-10 01:48:45 -03:00
Pablo Hoffman 9b9b3a2b64 Removed unused import 2010-09-09 23:35:51 -03:00
Pablo Hoffman 7d14a52234 Reference dont_merge_cookies in list of special Request.meta keys 2010-09-09 21:54:26 -03:00
Pablo Hoffman 7f21a6384f Documented handle_httpstatus_list request.meta key 2010-09-09 21:50:40 -03:00
Pablo Hoffman f1c943543a Added dont_retry request.meta key to make RetryMiddleware ignore requests. Closes #234 2010-09-09 21:43:44 -03:00
Pablo Hoffman 9f01e3e79e Added dont_redirect request.meta key to make RedirectMiddleware ignore requests. Closes #233 2010-09-09 21:37:35 -03:00
Pablo Hoffman ad2b979e0f Changed TextResponse.replace() behaviour by keeping previous encoding when not specified 2010-09-09 18:01:00 -03:00
Pablo Hoffman b9ea9dcf1a Restored wrongly removed import 2010-09-09 17:42:54 -03:00
Pablo Hoffman c041328b45 Fixed bug with thread unsafety calls from Scrapy shell 2010-09-09 15:26:46 -03:00
Pablo Hoffman 24c0088bde Added tag 0.10-rc1 for changeset 131f8e906f75 2010-09-09 00:41:18 -03:00
Pablo Hoffman 38bbec8dca Bumped version to 0.10-rc1 2010-09-09 00:40:58 -03:00
Pablo Hoffman fe77794f76 Added SpiderContext extension. Closes #203 2010-09-08 15:52:35 -03:00
Pablo Hoffman 1d5a7df125 Fixed encoding error with some item dropped messages. Closes #232 2010-09-08 15:34:13 -03:00
Pablo Hoffman cb7cc7c8bb Fixed bugs with SQLiteDict and added tests 2010-09-08 14:32:28 -03:00