RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,809 Commits 49 Branches 132 Tags 133 MiB
Commit Graph

6809 Commits

Author SHA1 Message Date
Konstantin Lopuhin c3d17659b3 Fix queue serialization test on PyPy 
It is not affected by Twisted bug #7989 and is more permissive
with pickling (especially with protocol=2).
 2017-06-15 13:05:45 +03:00
Konstantin Lopuhin 6014856df5 Fix test_output_processor_error undere PyPy 
For float(u'$10') PyPy includes "u'" in the error message,
and it's more fair to check error message on input we are really
passing.
 2017-06-15 12:57:32 +03:00
Konstantin Lopuhin 5a08cf3b96 Fix test_start_requests_errors for PyPy 
Twisted prints errors in DebugInfo.__del__, but PyPy does not run
gc.collect() on exit:
http://doc.pypy.org/en/latest/cpython_differences.html?highlight=gc.collect#differences-related-to-garbage-collection-strategies
 2017-06-15 12:02:47 +03:00
Pengyu CHEN f712513ed7 Added doc for `scrapy.exceptions.DontCloseSpider`. Also fixes inaccurate doc for `scrapy.signals.spider_idle`. 2017-06-15 11:09:17 +08:00
Pengyu CHEN b33e0d5a54 Added: Now supporting <link> tags in Response.follow 2017-06-14 17:09:42 +08:00
Mikhail Korobov 5aebdac45d Merge pull request #2781 from crasker/patch-1 
use suggest method instead of DEPRECATED one
 2017-06-14 03:29:45 +05:00
Mikhail Korobov 4e84424f26 Merge pull request #2777 from chuanjin/patch-1 
Update extensions.rst
 2017-06-14 03:28:31 +05:00
Casker ae679f6499 Create item-pipeline.rst 2017-06-09 16:12:20 +08:00
Paul Tremberth 39ad0d0bdd Fix setting name reference 2017-06-06 10:48:30 +02:00
Paul Tremberth e7061f7a41 Reformat a bit 2017-06-06 10:47:43 +02:00
Chuan Jin 3f8542eb56 Update extensions.rst 
#2759
 2017-06-05 23:31:37 +02:00
Valdir Stumm Junior 4b6f68b9ee make reqser tests create Request with proper callback/errback 2017-06-05 17:26:52 -03:00
Valdir Stumm Junior 0f6f486769 fix parse command issue with callback as a string 2017-06-05 16:19:00 -03:00
Paul Tremberth fad6b70d92 Use https:// for readthedocs links 2017-06-01 16:41:52 +02:00
Paul Tremberth 6b092c6680 Handle Twisted versions before 15.0 2017-06-01 12:29:01 +02:00
Paul Tremberth e162c1ff40 Pass proxy URI to ProxyAgent as bytes 2017-06-01 11:58:17 +02:00
Valdir Stumm Junior 60727dedf6 verify if Request callback is callable 2017-05-31 15:00:38 -03:00
Paul Tremberth c8dc158697 Use HTTP pool and proper endpoint key for ProxyAgent 2017-05-30 17:22:23 +02:00
Mikhail Korobov 5e1f7a9ead DOC change "releases" section content 2017-05-30 00:50:32 +05:00
Mikhail Korobov 083880888b DOC fixed rst syntax in DOWNLOAD_FAIL_ON_DATALOSS docs 2017-05-30 00:44:11 +05:00
Paul Tremberth d847e65aae Merge pull request #2762 from stummjr/add-subreddit-to-docs 
Include references to Scrapy subreddit in the docs
 2017-05-29 20:35:40 +02:00
Valdir Stumm Junior 80b160d0d7 include references to scrapy subreddit in the docs 2017-05-29 14:56:49 -03:00
Kurt Peek af2963d0eb Update autothrottle.rst 
Added missing bullet point for the AUTOTHROTTLE_TARGET_CONCURRENCY setting.
 2017-05-24 19:59:36 +05:00
Paul Tremberth 5f69ec98f7 Bump version: 1.3.2 → 1.4.0 2017-05-18 23:01:05 +02:00
Paul Tremberth fc2846d637 Set release date for v1.4.0 2017-05-18 22:59:46 +02:00
Paul Tremberth 79b0d8605c Merge pull request #2630 from scrapy/release-notes-1.4.0 
Release notes for 1.4.0
 2017-05-18 22:57:51 +02:00
Mikhail Korobov 76e5b0f65c DOC 1.4 deprecations and backwards incompatible changes, add recent commits to news. 2017-05-18 22:37:15 +02:00
Mikhail Korobov edcde7a2cf DOC tweak release notes: promote response.follow, mention logging/stats changes 2017-05-18 22:37:15 +02:00
Paul Tremberth a3d3cd4cb7 Update with latest merges 2017-05-18 22:37:15 +02:00
Paul Tremberth 432668acf7 Mention implementation of #667 2017-05-18 22:37:15 +02:00
Paul Tremberth 896c30a8eb Reference items pretty-printing issue number 2017-05-18 22:37:15 +02:00
Paul Tremberth 55d1082360 Reference recent fixes and commits 2017-05-18 22:37:15 +02:00
Paul Tremberth 7d72394794 Reword mention of new response.follow() shortcut 2017-05-18 22:37:15 +02:00
Paul Tremberth c6464cc4f5 Add verbose introduction to new features 2017-05-18 22:37:14 +02:00
Paul Tremberth e139d990fc Fix sphinx-build warning on deprecated latex_paper_size 2017-05-18 22:37:14 +02:00
Paul Tremberth cba55cd190 Rephrase other sections 2017-05-18 22:37:14 +02:00
Paul Tremberth 8729a91f7a Rephrase "New features" section 2017-05-18 22:37:14 +02:00
Paul Tremberth 851adcedf2 List merged pull requests since 1.3.3 2017-05-18 22:37:14 +02:00
Paul Tremberth 0c237c6cac Merge pull request #2750 from scrapy/codecov 
tweak coverage reports
 2017-05-18 21:38:54 +02:00
Daniel Graña 25f609e2a3 Merge pull request #2675 from simongartz/png-p-to-jpg-conversion-fix 
[MRG+1] Fixes conversion of transparent PNG with palette images to jpg #2452
 2017-05-18 16:34:02 -03:00
Paul Tremberth 8aa2e4f997 Merge pull request #1829 from nyov/nyov/editor 
[MRG+1] Remove dependency on os.environ from default settings
 2017-05-18 21:32:51 +02:00
Daniel Graña b9d3b447a5 Merge pull request #2670 from qhuang872/master 
[MRG+1] Update spiders.rst
 2017-05-18 16:31:01 -03:00
Paul Tremberth 143810410f Merge pull request #2740 from luzfcb/luzfcb-run-scrapy-from-module 
[MRG+1] Add support for executing scrapy using -m option of python
 2017-05-18 21:30:19 +02:00
Mikhail Korobov 9ce03d096d codecov config: disable project check, tweak PR comments 2017-05-19 00:01:27 +05:00
Bernardas 1a452c038c increase ptpython priority since it can use other shells as backend 2017-05-18 16:57:13 +00:00
Mikhail Korobov 532400f993 Merge pull request #2643 from harshasrinivas/set-retry-times-per-request 
[MRG+1] Add feature to set RETRY_TIMES per request (#2642)
 2017-05-17 15:46:33 +05:00
Mikhail Korobov 1acacab93b Merge pull request #2741 from eliat123/2576_cleanup_MEMUSAGE_REPORT 
[FIX #2576] cleanup: removed unused MEMUSAGE_REPORT
 2017-05-17 13:48:00 +05:00
Daniel Graña 73668ce407 Merge pull request #2721 from HarrisonGregg/feature-drop-from-response-field 
[MRG+1] Allow dropping field in from_response formdata
 2017-05-16 09:47:45 -03:00
Eli Atzaba b74b98fa3e cleanup: removed unused MEMUSAGE_REPORT 
Signed-off-by: Eli Atzaba <eliat123@gmail.com>
 2017-05-16 13:59:58 +03:00
Fábio C. Barrionuevo da Luz df7a5c4aa4 Add support for executing scrapy using -m option of python 
python -m scrapy
 2017-05-15 22:52:23 -03:00