RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,468 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

3468 Commits

Author SHA1 Message Date
nramirezuy bef8ade956 removed request_received and added request_scheduled 2013-06-26 16:45:46 -03:00
Pablo Hoffman 819b2776dd Merge pull request #326 from berendiwema/master 
Include example of how to stop the reactor from script
 2013-06-25 13:30:07 -07:00
nramirezuy 83b2774354 remove wrong default httpcache 2013-06-25 17:01:29 -03:00
Daniel Graña e06836ff48 fix import of deprecated http handler for S3 handler 2013-06-25 14:34:43 -03:00
Daniel Graña 22da89b748 callLater must happens before timeout callback is added to avoid AttributeError 2013-06-25 12:45:48 -03:00
Berend Iwema aec314db09 added a bit more documentation on how to close the reactor when running scrapy from a script 2013-06-25 16:08:22 +02:00
Pablo Hoffman bbde1d0e0b Merge pull request #275 from stav/doc 
doc: Response.replace() cannot take meta argument
 2013-06-24 11:09:28 -07:00
Daniel Graña 7af94be3a9 fix helpers used to sign S3 requests with boto connections 2013-06-20 10:56:48 -03:00
Daniel Graña 7a4c012db4 show a meaningful skip message for image tests 2013-06-20 10:51:59 -03:00
Daniel Graña de57e27d81 do not skip image tests and set a fixed pyOpenSSL version 2013-06-20 10:44:20 -03:00
Daniel Graña 26809d2254 fix download_timeout for servers that returns response headers but hangs sending its body 2013-06-20 10:15:23 -03:00
Daniel Graña 133091af3a rename http11 handler methods and split latency and timeout check 2013-06-19 16:23:51 -03:00
Daniel Graña 1e1b161671 Merge pull request #321 from barraponto/document_crawlspider_parse_start_urls 
Document CrawlSpider.parse_start_urls method
 2013-06-10 06:20:16 -07:00
Capi Etheriel 50fa46d183 Document CrawlSpider.parse_start_urls method 2013-06-09 04:03:20 -03:00
Pablo Hoffman 5a097cc192 log optional features available on startup 2013-06-04 19:25:26 -03:00
Daniel Graña 8ea041528f Merge pull request #318 from scrapy/http11 
Add HTTP 1.1 download handler
 2013-06-04 14:34:54 -07:00
Daniel Graña e7acc2171d show deprecation warning for HttpDownloadHandler 2013-06-04 17:28:43 -03:00
Daniel Graña cf10474b8b improve twisted version tests and enable HTTP11 handler by default if supported 2013-06-04 16:59:35 -03:00
Daniel Graña bb806fa0f8 restore http10 as default handler and skip tests if http1.1 is not supported 2013-06-04 16:36:21 -03:00
Daniel Graña 8b0c9466d6 add required files to support twisted 11.1 (precise) 2013-06-04 15:35:31 -03:00
Daniel Graña 7729f939e9 implement download timeouts based on deferred cancellation 2013-06-03 19:18:45 -03:00
Pablo Hoffman 4ac0eea55d Merge pull request #316 from nramirezuy/closespider/errorcount 
standarizes close_spider to use signals
 2013-05-31 12:20:58 -07:00
nramirezuy b61a4e719c standarizes close_spider to use signals 2013-05-31 15:54:17 -03:00
Daniel Graña b4fca90bba merge 0.16.5 release notes 2013-05-30 18:49:00 -03:00
Daniel Graña 58dc16f48d obey request method when scrapy deploy is redirected to a new endpoint 2013-05-30 18:34:59 -03:00
Pablo Hoffman 0effe139e5 Merge pull request #311 from nramirezuy/logstats-6017 
removed multispider support from logstats
 2013-05-30 08:47:33 -07:00
nramirezuy 14e382ae48 test_closespider fixed for python 2.6 2013-05-30 10:57:06 -03:00
Pablo Hoffman 461eace4af Merge pull request #313 from nramirezuy/closespider-6017 
drop multi-spider support from CloseSpider extension
 2013-05-30 06:34:52 -07:00
nramirezuy 971ef5c78e removed multispider support, test added 2013-05-28 17:39:37 -03:00
Pablo Hoffman 2bb2ba66f6 simplify scrapy.tests.mockserver 2013-05-28 15:12:38 -03:00
Pablo Hoffman f35266e6a5 use os.pathsep 2013-05-28 15:10:05 -03:00
nramirezuy 96b8fb69be removed multispider support from logstats 2013-05-28 14:45:02 -03:00
Pablo Hoffman 98aa3efea7 Merge pull request #312 from nramirezuy/cmd/bech/test 
added test for bench command
 2013-05-28 08:46:31 -07:00
nramirezuy 1ed5643eb2 added test for bench command 2013-05-28 12:42:50 -03:00
Pablo Hoffman a4b5bfbb5e Merge pull request #309 from amferraz/patch-1 
Add FAQ entry referencing Request.meta usage
 2013-05-27 09:07:48 -07:00
cacovsky 8007762890 Add FAQ entry referencing Request.meta usage 2013-05-27 13:02:17 -03:00
Nicolás Alejandro Ramírez Quiros 7874eef7af Merge pull request #307 from DeaconDesperado/raise-format 
Raise a usage error when an invalid or unrecognized output format is entered on Command line
 2013-05-23 12:50:17 -07:00
Mark Grey 485a954571 raise on unrecognized format 2013-05-23 15:11:45 -04:00
Pablo Hoffman 845c64b89d add benchmarking to 0.18 release notes 2013-05-17 10:38:42 -03:00
Pablo Hoffman ca12886acb update copyright notes 2013-05-16 15:05:52 -03:00
Pablo Hoffman 8e49fed918 minor improvements to benchmarking doc 2013-05-16 13:23:13 -03:00
Pablo Hoffman 76087e336a add scrapy bench command for benchmarking, with documentation 2013-05-16 13:15:25 -03:00
Pablo Hoffman 5c40741d65 added context manager for mock server, moved test spiders into a separate module (scrapy.tests.spiders) 2013-05-16 13:01:02 -03:00
Pablo Hoffman 214bcdf3be Merge pull request #301 from tpeng/fix_telnet_doc 
change manager in telnet variable to crawler
 2013-05-08 07:00:52 -07:00
tpeng edbe525d19 change manager in telnet variable to crawler 2013-05-08 11:58:15 +08:00
Pablo Hoffman 99f8d5733a added mock tests for retries (more to come) 2013-05-06 20:04:15 -03:00
Pablo Hoffman a7b7c89216 improved logging of downloader errors, when they contain tracebacks 2013-05-06 20:03:54 -03:00
Pablo Hoffman db195ee4df added more methods to mockserver 2013-05-06 20:03:17 -03:00
Pablo Hoffman c2561ca160 use get_testenv() shortcut instead of get_pythonpath() 2013-05-06 17:35:09 -03:00
Pablo Hoffman 14e7121674 make sure the mockserver forked from tests uses the current installation of scrapy, not the system one 2013-05-06 17:27:59 -03:00