RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,710 Commits 49 Branches 132 Tags 158 MiB
Commit Graph

10710 Commits

Author SHA1 Message Date
Daniel Grana 3e013f564b update docs for defaultheaders middleware and change spider attribute to match global setting name 2010-07-16 16:17:08 -03:00
Daniel Grana 6883a99c1e Automated merge with ssh://hg.scrapy.org/scrapy-0.9 2010-07-16 14:56:00 -03:00
Daniel Grana b799e5ee37 Support default headers per spider. closes #181 
--HG--
extra : rebase_source : 60162dffa4fbab525501e46b479dc272b8998942
 2010-07-16 14:51:14 -03:00
Pablo Hoffman b91d40ba78 Fixed grammar error in doc (patch by stav) - closes #176 2010-07-16 11:34:18 -03:00
Pablo Hoffman b8aa74ee9e bugfix in request_httprepr() function 2010-07-15 12:04:55 -03:00
Martin Olveyra ec850b9fd1 Fix memusage report concatenation 2010-07-14 18:47:09 -03:00
Pablo Hoffman 90a04f0530 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-07-13 19:47:55 -03:00
Pablo Hoffman cc32f6ec66 Applied patch to ClientForm to fix bug with wrong entities. Also added tests and left patch in repo in case we upgrade ClientForm in the future and need to re-apply it 2010-07-13 19:46:53 -03:00
Pablo Hoffman 9e37ec4230 fixed documentation typo (closes #151) 2010-07-13 19:03:02 -03:00
Ping Yin b3a65d3313 HTTPCACHE: Don't cache response with codes in HTTPCACHE_IGNORE_HTTP_CODES 2010-07-09 13:14:25 -03:00
Pablo Hoffman 2067bcd8d0 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-07-08 14:03:58 -03:00
Juan Picca 2ddbbc8152 allow passing custom headers in FormRequest.from_response() 2010-07-08 14:02:28 -03:00
Pablo Hoffman a6a86d9b4a Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-07-01 11:48:36 -03:00
Martin Olveyra b258fc3305 Fixed bug with float values in meta refresh 2010-07-01 11:46:06 -03:00
Pablo Hoffman 3e976e5005 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-06-28 00:55:35 -03:00
Pablo Hoffman b23af5ccf6 Added tag 0.9 for changeset 5caf3dc10a92 2010-06-28 00:54:47 -03:00
Pablo Hoffman 6525d3fe8c bumped version to 0.9 final 2010-06-28 00:54:43 -03:00
Pablo Hoffman 66053291d6 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-06-27 19:57:54 -03:00
Pablo Hoffman 861c04fb7e made encoding explicit in test_get_meta_refresh, to avoid depending on 
unreliable UnicodeDammit criteria.
 2010-06-27 19:55:39 -03:00
Pablo Hoffman 696bb63ac5 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-06-27 19:32:47 -03:00
Pablo Hoffman 22555df56a response_httprepr: fixed error with unknown response codes (closes #169) 2010-06-27 19:32:26 -03:00
Pablo Hoffman 5408202697 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-06-27 19:23:41 -03:00
Ismael Carnales 2571e1b7aa docs: Some DjangoItem docs improvements, closes #134. Thanks tn! 2010-06-27 09:09:54 -03:00
Daniel Grana 7b80cce148 Automated merge with ssh://hg.scrapy.org/scrapy 2010-06-25 13:06:14 -03:00
Daniel Grana 9148321765 do not redirect when there is a commented meta refresh header. closes #170 2010-06-25 12:57:15 -03:00
Pablo Hoffman a61df93a22 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-06-22 14:00:51 -03:00
Pablo Hoffman 3bbad36998 Raise when trying to set an item field value using setattr api, and added tests. 2010-06-22 14:00:31 -03:00
Pablo Hoffman 504f90c6f7 Automated merge with http://hg.scrapy.org/scrapy-0.9 2010-06-22 13:39:29 -03:00
Pablo Hoffman baa523055f removed nltk dependency from IBL code 2010-06-22 13:38:32 -03:00
Pablo Hoffman e665e5abb7 bumped version to 0.10-dev 2010-06-14 22:00:54 -03:00
Pablo Hoffman 8815de94ff Added tag 0.9-rc1 for changeset 8b9c31e18c08 2010-06-14 18:37:55 -03:00
Pablo Hoffman 61e374eb56 bumped version to 0.9-rc1 2010-06-14 18:37:52 -03:00
Pablo Hoffman 115e9f2162 Added FAQ entry about running Scrapy deployment. 2010-06-14 18:21:12 -03:00
Daniel Grana f3d2ee41cc mediapipeline: bugfix error raised when media requests has not callbacks, remove item_media_{downloaded,failed} hooks in favour or request.{errback,calback}, and add tests 
--HG--
extra : rebase_source : 72172406ab4ffc748e1648b46fe976e403b87c29
 2010-06-14 12:34:52 -03:00
Pablo Hoffman 0731c42e24 added "hg purge" to make tarball 2010-06-14 09:58:17 -03:00
Pablo Hoffman 4262a0af8c moved sign_release.sh code to Makefile 2010-06-14 09:49:07 -03:00
Pablo Hoffman d8ac4857a5 a couple of fixes to make tests pass on win32 2010-06-14 09:11:35 -03:00
Pablo Hoffman 2d3d5b6aba use mercurial revision to construct version, when building a non-final version 2010-06-14 08:59:20 -03:00
Pablo Hoffman 68e8e6ac11 removed unused code 2010-06-14 08:28:48 -03:00
Pablo Hoffman ede1df4b4f updated copyright year, and indentation space 2010-06-14 07:16:51 -03:00
Pablo Hoffman 247fc26598 moved scrapy.tac to extras/ 
--HG--
rename : bin/scrapy.tac => extras/scrapy.tac
 2010-06-13 23:09:08 -03:00
Pablo Hoffman 09182efaff added scrapy-sqs.py to deployed scripts 2010-06-13 19:17:17 -03:00
Pablo Hoffman 37f71a9957 upstart script: exec twistd and use pidfile 2010-06-13 18:59:52 -03:00
Pablo Hoffman 91e1e0aff3 fixed bug and updated old code in googledir example project 2010-06-13 17:31:33 -03:00
Pablo Hoffman bd16d1cd48 Added SMTP-AUTH support to scrapy.mail (closes #149) 2010-06-13 17:14:46 -03:00
Pablo Hoffman 495f23dea2 utils.serialize: added support for encoding Deferreds, and to refer spiders by name using 'spider::name' 2010-06-11 18:16:09 -03:00
Pablo Hoffman 1b083911e6 scrapy-ws.py: added stop command 2010-06-11 18:14:01 -03:00
Pablo Hoffman ed5d7561f9 Added SQS Execution Queue, and example script to add spiders to the queue 2010-06-11 17:22:14 -03:00
olveyra efe9811d92 Populate annotation metadata with data not used by IBL extractor. 2010-06-11 13:09:56 -03:00
Pablo Hoffman ea8b5ddfd5 debian package: fix dh_auto_build confusing with Makefile, added scrapy-ws.py to deployed scripts 2010-06-11 12:48:35 -03:00