RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,785 Commits 49 Branches 132 Tags 132 MiB
Commit Graph

1785 Commits

Author SHA1 Message Date
Pablo Hoffman d03bf1f27f Automated merge with http://hg.scrapy.org/scrapy-stable 2009-10-07 20:38:46 -02:00
Pablo Hoffman 37d9e015bb minor fix to tutorial 2009-10-07 20:15:49 -02:00
Pablo Hoffman 1450af257a some improvements to 'Crawled' log lines, delegating the formatting to a pluggable function 2009-10-06 22:47:17 -02:00
Daniel Grana bc64ca3e13 Add support to set http proxies per request, and obey enviroment variables http_proxy and no_proxy by default. 2009-10-05 04:10:22 -02:00
Daniel Grana 8aa7d153ae rewrote of downloader handlers 
* add REQUEST_HANDLERS setting with defaults for file, http and https schemes
* add documentation of new setting
* add unittests for all the builtin handlers
* remove unused getPage function
 2009-10-05 04:10:22 -02:00
Pablo Hoffman 40dc867ac3 Automated merge with http://hg.scrapy.org/scrapy-stable 2009-10-03 14:09:21 -03:00
Pablo Hoffman aacb7965b4 improved check for missing callback in Requests 2009-10-03 14:08:40 -03:00
Pablo Hoffman 9710f2635a Automated merge with http://hg.scrapy.org/scrapy-stable 2009-10-02 09:21:21 -03:00
Pablo Hoffman 0f1bb87198 fixed bug in parse command with requests which don't have __dict__ attribute 2009-10-02 09:20:22 -03:00
Ismael Carnales 64ad96ba1e added LxmlItemLoader 2009-09-29 16:17:30 -03:00
Pablo Hoffman ea9f2360eb merge with scrapy-stable 2009-09-29 16:16:35 -03:00
Ismael Carnales bb4871b7a7 added xpathitemloader tests 2009-09-29 16:05:34 -03:00
Ismael Carnales 16ada1d5dc fixed ItemLoader replace_value 2009-09-29 16:05:32 -03:00
Pablo Hoffman 64ce9542d2 Automated merge with http://hg.scrapy.org/scrapy-stable 
--HG--
rename : extras/make_release.sh => extras/build_release.sh
 2009-09-29 14:22:10 -03:00
Pablo Hoffman a0eec7eaf6 some typos fixes and updates to install doc 2009-09-29 09:44:02 -03:00
Pablo Hoffman 4a371a6cd3 removed unmaintained spidermonkey bindings 2009-09-29 09:33:20 -03:00
Ismael Carnales 1646482bef reformatted installation guide 2009-09-29 08:41:34 -03:00
Pablo Hoffman f565127eb2 added some missing file to MANIFEST.in 2009-09-28 23:55:00 -03:00
Pablo Hoffman 4a9d9282bc removed obsolete scrapy architecture dia diagram 2009-09-28 23:54:01 -03:00
Pablo Hoffman 7e12d3d41b removed old adaptors tests sample data 2009-09-28 23:16:17 -03:00
Pablo Hoffman 7428e4cebf splitted make_release.sh in: build_release.sh and sign_release.sh 
--HG--
rename : extras/make_release.sh => extras/build_release.sh
 2009-09-28 23:07:00 -03:00
Pablo Hoffman 1dcb9ed72f Merge with stable repo 2009-09-28 22:53:05 -03:00
Pablo Hoffman 504293b8c4 renamed tag: 0.7.0-rc1 to 0.7-rc1 2009-09-28 22:29:23 -03:00
Pablo Hoffman 2be739964d bumped version to 0.7 final 2009-09-28 22:28:49 -03:00
Daniel Grana e0ccb2cceb ignore request if spider is not opened in downloader when called from engine.download 2009-09-28 12:44:05 -03:00
Daniel Grana 749b136595 Automated merge with ssh://hg.scrapy.org/scrapy-stable 2009-09-28 12:40:16 -03:00
Daniel Grana 1f9e0f10eb don't log ignored requests errors at images pipeline 2009-09-28 12:38:49 -03:00
Daniel Grana 706cb439c5 Automated merge with ssh://hg.scrapy.org/scrapy-stable 2009-09-23 14:54:33 -03:00
Daniel Grana c795771282 remove unused line from project settings template 2009-09-23 14:53:40 -03:00
Daniel Grana bd0dc21e8d Automated merge with ssh://hg.scrapy.org/scrapy-stable 2009-09-23 14:28:41 -03:00
Daniel Grana dfd35c5280 Add common javascript mimetypes to be detected as TextResponses, also improve mimetype detection to avoid querying twice classes dict 2009-09-23 14:26:58 -03:00
Daniel Grana bae726a172 Automated merge with ssh://hg.scrapy.org/scrapy-stable 2009-09-22 11:33:15 -03:00
Ismael Carnales ef0e5d368f updated spider templates to reflect the new spider callback return policy 2009-09-22 11:25:42 -03:00
Ismael Carnales 5862ba7db7 modified doc to reflect the new spider callback return policy (lists not needed) 2009-09-22 11:25:40 -03:00
Ismael Carnales 802f918b69 removed obsolete doc static file 2009-09-22 11:25:38 -03:00
Daniel Grana 337253f967 Automated merge with file:///home/dan/src/hg/scrapy-stable 2009-09-21 13:59:46 -03:00
Daniel Grana 6467c7055d check iterable output of crawl spider and add tests for common function used to iterate spider output 2009-09-21 13:57:16 -03:00
Pablo Hoffman 201ce525e5 make_release.sh: disabled bdist_wininst command as it doesn't package data files properly on Linux - refs #109 2009-09-21 13:07:52 -03:00
Pablo Hoffman 12ebcfd6c4 bumped version to 0.8 2009-09-18 15:31:54 -03:00
Pablo Hoffman b6155801f0 Added tag 0.7.0-rc1 for changeset f4a14daa04e6 2009-09-18 00:52:41 -03:00
Pablo Hoffman bc778a0e5b make_release.sh: added warning about broken bdist_wininst 2009-09-17 15:28:36 -03:00
Daniel Grana 1b28858bf9 improve XmlRpcRequest and update tests to cover more cases 2009-09-17 14:26:42 -03:00
Pablo Hoffman 48c3fb3ace made XmlRpcRequestTest inherit from RequestTest and removed redundant test 2009-09-17 12:21:43 -03:00
Pablo Hoffman 6845505e7a added make_release.sh script 2009-09-17 11:46:47 -03:00
Pablo Hoffman 6e10914454 added pypi classifiers to setup.py 2009-09-17 11:22:48 -03:00
Pablo Hoffman 6e93872955 updated installation guide for using releases 2009-09-17 11:06:55 -03:00
Pablo Hoffman 653db1d673 added bitmap for windows installer 2009-09-17 02:01:40 -03:00
Pablo Hoffman 132557dd14 some deployment changes in preparation for the 0.7.0 release candidate 2009-09-16 22:40:36 -03:00
Ismael Carnales fd41f06056 added doc on how to enable an Item Pipeline component 2009-09-16 14:19:16 -03:00
Ismael Carnales 404e7e09d7 changed spider doc references in BaseSpider class 2009-09-16 14:10:11 -03:00