RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,578 Commits 49 Branches 132 Tags 141 MiB
Commit Graph

3578 Commits

Author SHA1 Message Date
cacovsky 71b320914a Update request-response.rst 
Fix small doc typo (too many backticks)
 2013-09-18 11:45:25 -03:00
Pablo Hoffman ee8464b471 Merge pull request #390 from kmike/mock-in-tox 
mock is required for running tests
 2013-09-17 15:51:54 -07:00
Mikhail Korobov 7ffc786fb6 mock is required for running tests since 45ff6ec28a 
This fixes tox test running. Mock is available at Travis, but there is no harm in listing it explicitly.
 2013-09-18 00:02:23 +06:00
Pablo Hoffman 2a89265b96 Merge pull request #387 from ktharmal/patch-1 
Fixed directory location for dmoz_spider.py file
 2013-09-16 11:36:13 -07:00
Kumara Tharmalingam bbb0603091 Fixed directory location for dmoz_spider.py file 
It should be under 'tutorial/spiders' not 'dmoz/spiders'
 2013-09-15 21:55:52 -07:00
Daniel Graña 0400b18efa docs: list lxml as installation prerequisite 2013-09-09 12:44:26 -03:00
Daniel Graña b326b87154 forms: do not submit reset inputs 2013-09-07 21:53:27 -03:00
Daniel Graña 65135db70c increase unittest timeouts to decrease travis false positive failures 2013-09-05 21:53:50 -03:00
Daniel Graña 801564c2c9 exporters linting 2013-09-05 17:00:16 -03:00
Daniel Graña 2be3f7132a Do not fail on when json exporter are instanciated with extra keywords. closes #379 2013-09-05 16:57:27 -03:00
Daniel Graña 92d14d4a03 Fix permission and set umask before generating sdist tarball 2013-09-03 23:22:17 -03:00
Daniel Graña e6b3ca0180 Add 0.18.2 release notes 2013-09-03 14:29:40 -03:00
Daniel Graña 9b821df482 Merge pull request #330 from alexcepoi/contracts_fix 2013-09-03 11:41:04 -03:00
Daniel Graña f54d5c5896 Merge pull request #370 from loucash/master 
FilesPipeline extracted and separated from ImagesPipeline
 2013-09-03 07:29:30 -07:00
Alex Cepoi 50b653e5ae don't start reactor if no crawler is scheduled 2013-09-02 18:43:25 +02:00
Lukasz Biedrycki 0a8bf2c9e4 after additional tests with actual s3: k.set_contents_from_string is working 2013-08-28 18:00:54 +02:00
Daniel Graña 0f00b1602a merge 0.18 release notes 2013-08-27 18:47:46 -03:00
Daniel Graña ac218ed418 no need to test latest versions of dependencies on python 2.6 2013-08-27 17:38:28 -03:00
Daniel Graña c5222b4b8a remove phanton build row from travis matrix due to travis-ci/travis-core#1027 2013-08-27 17:19:45 -03:00
Daniel Graña ad140a2763 another try to limit travis build matrix 2013-08-27 16:53:54 -03:00
Daniel Graña 7394969236 test pypy with travis 2013-08-27 15:31:08 -03:00
Daniel Graña 401e888ced test lucid and precise python packages under their respective python versions 2013-08-27 15:30:10 -03:00
Daniel Graña 588a262b73 fix crawling tests under twisted pre 11.0.0 2013-08-27 14:05:37 -03:00
Daniel Graña 92826586d5 py26 can not format zero length fields {} 2013-08-27 12:12:50 -03:00
Daniel Graña caa0f90263 test PotentiaDataLoss errors on unbound responses 2013-08-27 11:53:18 -03:00
Daniel Graña 6720acc1af Treat responses without content-length or Transfer-Encoding as good responses 
There is not a way to determine if responses without Content-Length or Transfer-Encoding are complete, this change treat them as good responses but flags them as "partial".
This is backout change only for this functionality of 3c64a989
 2013-08-27 11:10:20 -03:00
Pablo Hoffman da1f6d3144 remove unused imports and some assorted pylint-ing 2013-08-23 13:03:34 -03:00
Pablo Hoffman 19ff9ac4f9 url/body attributes of Request/Response objects are now immutable 2013-08-23 12:43:22 -03:00
Pablo Hoffman 86230c0ab8 added quantal & raring to support ubuntu releases 2013-08-22 21:49:55 -03:00
Lukasz Biedrycki 83bd151c57 backwards compatibility: image_key & image_downloaded 2013-08-22 15:13:59 +02:00
Lukasz Biedrycki 2b11e7da93 backwards compatibility: image_key 2013-08-22 14:44:04 +02:00
Lukasz Biedrycki 02d08722e7 typo 2013-08-22 14:33:28 +02:00
Lukasz Biedrycki 4c89d8b298 remarks 2013-08-22 12:26:56 +02:00
Daniel Graña efb4f32d80 do no include ResponseFailed if http11 handler is not enabled 2013-08-21 19:04:12 -03:00
Daniel Graña 3c64a989ca New HTTP client wraps connection losts in ResponseFailed exception. fix #373 2013-08-21 17:28:26 -03:00
Pablo Hoffman 071172cbd6 fix retry middleware which didn't retry certain connection errors after the upgrade to http1 client, closes GH-373 2013-08-21 14:32:33 -03:00
Pablo Hoffman 6e4ed8b004 Merge pull request #372 from kmike/xml-exporter-fix-for-2.7.4 
fix XmlItemExporter in Python 2.7.4 and 2.7.5
 2013-08-21 09:55:51 -07:00
Mikhail Korobov a6e6ca06b9 fix XmlItemExporter in Python 2.7.4 and 2.7.5 2013-08-21 06:05:40 +06:00
Alex Cepoi 9c092536c5 remove redundant kwarg 2013-08-21 01:52:36 +02:00
Alex Cepoi 7bdeabc0ed fix log intermixing in scrapy shell command 2013-08-21 01:42:03 +02:00
Alex Cepoi 902208ca58 fix some missing logs with new crawlerprocess 2013-08-20 14:02:55 +02:00
Lukasz Biedrycki 96077ccf48 typo 2013-08-20 11:29:55 +02:00
Alex Cepoi a812def59e port all scrapy commands to new CrawlerProcess 2013-08-19 23:40:23 +02:00
Alex Cepoi a9a911fb09 default to multi crawler commands 
compatibility for old single crawler process
deprecated command's `crawler` property
logs about settings only shown once
 2013-08-19 22:43:29 +02:00
Mikhail Korobov 034ffae60f Recommend Pillow instead of PIL. Closes GH-317. 2013-08-18 00:44:01 +06:00
Lukasz Biedrycki 45ff6ec28a Test reorganization and new tests for Files and Images Pipelines, PEP8 changes in MediaPipeline 2013-08-16 18:47:56 +02:00
Lukasz Biedrycki 76ce8c5238 FilesPipeline which enalbes to download any files. 
It has been extracted from ImagesPipelines.
ImagesPipeline is built on top of FilesPipeline and consist only with convert image and thumbnail generation logic.
 2013-08-16 17:02:31 +02:00
Pablo Hoffman f95b164d06 Merge pull request #367 from berendiwema/master 
#327 - Support STARTTLS / SSL option in email sender
 2013-08-14 07:17:30 -07:00
Berend Iwema 32b6364bcd #327 - Support STARTTLS / SSL option in email sender 2013-08-14 12:59:01 +02:00
Pablo Hoffman 49952a45ae add docstring to PythonItemExporter 2013-08-14 02:49:14 -03:00