RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,095 Commits 49 Branches 132 Tags 136 MiB
Commit Graph

6095 Commits

Author SHA1 Message Date
Mikhail Korobov f0c4e5dfd5 TST fixed ProjectUtilsTest on OS X 
Temp folder can be a symlink on OS X.
 2017-02-16 16:32:23 +05:00
Mikhail Korobov afac3fd2c2 Merge pull request #2530 from elacuesta/proxy_credentials 
[MRG+1] Use credentials from request.meta['proxy']
 2017-02-14 20:58:39 +06:00
Eugenio Lacuesta ae0ea31abd Add HTTPPROXY_ENABLED setting (default True) 2017-02-14 11:33:01 -03:00
Daniel Graña 7dd7646e65 Bump version: 1.3.1 → 1.3.2 2017-02-13 14:57:55 -03:00
Daniel Graña 9315e944a2 Release notes for 1.3.2 2017-02-13 14:56:29 -03:00
Daniel Graña 45f19021a5 Merge pull request #2558 from scrapy/twisted-17-fix 
Fixed compatibility with twisted 17+
 2017-02-13 14:48:12 -03:00
Mikhail Korobov 1bc4d8b6b6 fixed tls in Twisted 17+ 2017-02-13 20:03:53 +05:00
Mikhail Korobov de65ad3fb1 TST replace Ubuntu 12.04 tox environment with 14.04 2017-02-13 18:44:39 +05:00
Mikhail Korobov 5ee595d695 Merge pull request #2551 from redapple/tutorial-author-selector 
Use consistent selectors for author field in tutorial
 2017-02-09 18:16:49 +05:00
Paul Tremberth 29e60213db Use consistent selectors for author field in tutorial 2017-02-09 10:41:21 +01:00
Paul Tremberth f32a229e3d Merge pull request #2510 from elacuesta/reqser_request_class 
[MRG+1] Preserve request class when converting to/from dicts (utils.reqser)
 2017-02-08 18:30:07 +01:00
Eugenio Lacuesta 9c0aae724e Use credentials from request.meta['proxy'] if present 2017-02-08 13:22:30 -03:00
Paul Tremberth af55a8713e Bump version: 1.3.0 → 1.3.1 2017-02-08 17:08:19 +01:00
Paul Tremberth ff8a564b1a Set date for 1.3.1 release 2017-02-08 17:05:06 +01:00
Mikhail Korobov 094937bc27 Merge pull request #2536 from scrapy/release-notes-1.3.1 
[MRG] Update changelog for upcoming 1.3.1 release
 2017-02-08 20:47:10 +05:00
Paul Tremberth 48c8c679de Update changelog for upcoming 1.3.1 release 2017-02-08 15:53:44 +01:00
Paul Tremberth d205206aaa Merge pull request #2345 from gustavodeandrade/master 
[MRG+1] Fix documentation about HTML entities decoding with selector extraction
 2017-02-08 13:21:15 +01:00
Paul Tremberth 7d0b89042f Merge pull request #2533 from djrobust/patch-1 
[MRG+1] Use yield with nested parsing of responses
 2017-02-08 13:02:50 +01:00
Paul Tremberth a8893190a1 Merge pull request #2464 from elacuesta/component_order_integer 
[MRG+2] Validate numeric values (components order)
 2017-02-08 13:01:02 +01:00
Paul Tremberth 3becb861af Merge pull request #2544 from scrapy/twisted-version-cleanup 
[MRG+1] Remove code required to support ancient twisted versions.
 2017-02-08 12:59:45 +01:00
Paul Tremberth e87f18447c Merge pull request #2538 from scrapy/linkextractor-fail-fast 
LinkExtractor: don’t check all regexes if one of them matches
 2017-02-08 12:58:41 +01:00
Paul Tremberth 5ad64204ac Merge pull request #2509 from rolando-contrib/py36-classcell-compat 
[MRG+1] BUG: Fix __classcell__ propagation required in Python 3.6.
 2017-02-08 12:15:15 +01:00
Rolando Espinoza 4e765acaed BUG: Fix __classcell__ propagation. 
Python 3.6 added simpler customization of class creation but this
requires to propagate correctly the __classcell__ attribute in custom
__new__ methods.

See https://docs.python.org/3.6/whatsnew/3.6.html#pep-487-simpler-
customization-of-class-creation
 2017-02-08 02:12:52 -04:00
Mikhail Korobov f2f9350c47 Merge pull request #2542 from scrapy/redirect-cleanup 
[MRG+1] Cleanup MetaRefreshMiddleware: remove redundant check
 2017-02-08 00:57:21 +05:00
Mikhail Korobov 04b2f79e7a Remove code required to support ancient twisted versions. See GH-1887. 2017-02-07 22:30:58 +05:00
Mikhail Korobov eaf62ab69c cleanup MetaRefreshMiddleware: remove redundant check 2017-02-07 17:56:43 +05:00
Eugenio Lacuesta 24e82bfe75 Validate values for components order 2017-02-07 09:13:57 -03:00
Mikhail Korobov f73eb715ac LinkExtractor: don’t check all regexes if one of them matches 2017-02-07 02:16:06 +05:00
Mikhail Korobov 3b8e6d4d82 Merge pull request #2531 from Lukas0907/patch-1 
Fix typo in downloader-middleware.rst.
 2017-02-06 13:09:35 +05:00
Mikhail Korobov 2743786d1d Merge pull request #2534 from shio-phys/fix/typo_of_doc 
fix typo
 2017-02-06 13:05:46 +05:00
Takehiro Shiozaki fcb3daf4fa fix typo 2017-02-06 14:03:41 +09:00
djrobust 3021084f37 Use 'yield' when parsing multiple responses 
Use 'yield' consistently across examples of parse functions.
 2017-02-04 20:07:05 -08:00
Lukas Anzinger 09643796b4 Fix typo in downloader-middleware.rst. 2017-02-03 20:05:17 +01:00
Mikhail Korobov 814ce37dff Merge pull request #2528 from redapple/dns-retry-test 
Make DNS retry test compatible with Twisted 17+
 2017-02-03 02:43:46 +05:00
Mikhail Korobov 7b8bbc9205 Merge pull request #2215 from redapple/travis-pypy 
Enable PyPy tests on Travis (allowed to fail)
 2017-02-03 02:38:49 +05:00
Paul Tremberth 02e1d2b1fd Add trailing dot 2017-02-02 22:28:37 +01:00
Paul Tremberth e604c0f3ab Remove unused imports 2017-02-02 18:26:41 +01:00
Mikhail Korobov ae2a5292d8 Merge pull request #2457 from redapple/parsel-selector-kwargs 
[MRG] Support kwargs for response.xpath()
 2017-02-02 22:17:26 +05:00
Paul Tremberth 3358254c5c Make DNS retry test compatible with Twisted 17+ 2017-02-02 17:53:28 +01:00
Paul Tremberth 1295c17a26 Bump parsel requirement to at least parsel v1.1 2017-02-02 17:47:22 +01:00
Paul Tremberth 1c0b805357 DOC Mention XPath variables in Selectors section 2017-02-02 17:44:57 +01:00
Paul Tremberth 803d8c4b57 Add tests for passing kwargs on response .xpath() shortcut 2017-02-02 17:44:57 +01:00
Paul Tremberth 0cf6344cc2 Support kwargs for response.xpath() 2017-02-02 17:44:57 +01:00
Mikhail Korobov c305c46254 Merge pull request #2503 from redapple/view-no-redirect 
[MRG] Fix view command against new --no-redirect option
 2017-02-02 21:24:56 +05:00
Mikhail Korobov d4a8f43816 Merge pull request #2519 from redapple/settings-getbool-new 
[MRG+1] Support 'True' and 'False' strings as boolean settings values
 2017-02-02 21:15:24 +05:00
Mikhail Korobov 245287fd5a Merge pull request #2496 from redapple/dns-resolver-timeout 
[MRG] Enforce DNS resolution timeout
 2017-02-02 21:13:59 +05:00
Paul Tremberth 73f985cbde Merge pull request #2525 from scrapy/cache-docs 
DOC mention LevelDB cache storage backend
 2017-02-02 13:22:23 +01:00
Mikhail Korobov 9c75403176 Merge pull request #2466 from eLRuLL/settings-pipeline-template 
[MRG+1] settings: fixing name of the pipeline template
 2017-02-02 17:16:57 +05:00
Mikhail Korobov 55742c0392 DOC mention LevelDB cache storage backend 2017-02-01 22:43:28 +05:00
Paul Tremberth 70f260d353 Don't run coverage stats when on PyPy 2017-02-01 15:19:17 +01:00