RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,103 Commits 49 Branches 132 Tags 152 MiB
Commit Graph

6103 Commits

Author SHA1 Message Date
Daniel Graña 58a18e3007 Merge pull request #2535 from pawelmhm/brotli 
[MRG+1] [httpcompression] add support for br - brotli content encoding
 2017-02-20 11:04:55 -03:00
Mikhail Korobov 7ad5156287 Merge pull request #2570 from scrapy/tests-cleanup-resources 
[MRG+1] TST remove temp files and folders
 2017-02-20 20:00:32 +06:00
Mikhail Korobov 78cb46a079 TST fix a weird OS X testing issue 
pytest intercepts temp file creation to provide readable file/folder
names; path is built from method name; in case of conflicts
pytests uses increasing numbers, but it seems it doesn’t account
for case-insensitive (but case preserving) OS X filesystem. There
are methods named test_encoding, pytest thinks test_Encoding is
different and fails to create a test folder
 2017-02-16 18:22:04 +05:00
Mikhail Korobov b9928558b2 TST remove temporary files and folders 2017-02-16 17:50:38 +05:00
Mikhail Korobov afac3fd2c2 Merge pull request #2530 from elacuesta/proxy_credentials 
[MRG+1] Use credentials from request.meta['proxy']
 2017-02-14 20:58:39 +06:00
Eugenio Lacuesta ae0ea31abd Add HTTPPROXY_ENABLED setting (default True) 2017-02-14 11:33:01 -03:00
Daniel Graña 7dd7646e65 Bump version: 1.3.1 → 1.3.2 2017-02-13 14:57:55 -03:00
Daniel Graña 9315e944a2 Release notes for 1.3.2 2017-02-13 14:56:29 -03:00
Daniel Graña 45f19021a5 Merge pull request #2558 from scrapy/twisted-17-fix 
Fixed compatibility with twisted 17+
 2017-02-13 14:48:12 -03:00
Mikhail Korobov 1bc4d8b6b6 fixed tls in Twisted 17+ 2017-02-13 20:03:53 +05:00
Mikhail Korobov de65ad3fb1 TST replace Ubuntu 12.04 tox environment with 14.04 2017-02-13 18:44:39 +05:00
Mikhail Korobov 5ee595d695 Merge pull request #2551 from redapple/tutorial-author-selector 
Use consistent selectors for author field in tutorial
 2017-02-09 18:16:49 +05:00
Paul Tremberth 29e60213db Use consistent selectors for author field in tutorial 2017-02-09 10:41:21 +01:00
Paul Tremberth f32a229e3d Merge pull request #2510 from elacuesta/reqser_request_class 
[MRG+1] Preserve request class when converting to/from dicts (utils.reqser)
 2017-02-08 18:30:07 +01:00
Eugenio Lacuesta 9c0aae724e Use credentials from request.meta['proxy'] if present 2017-02-08 13:22:30 -03:00
Paul Tremberth af55a8713e Bump version: 1.3.0 → 1.3.1 2017-02-08 17:08:19 +01:00
Paul Tremberth ff8a564b1a Set date for 1.3.1 release 2017-02-08 17:05:06 +01:00
Mikhail Korobov 094937bc27 Merge pull request #2536 from scrapy/release-notes-1.3.1 
[MRG] Update changelog for upcoming 1.3.1 release
 2017-02-08 20:47:10 +05:00
Paul Tremberth 48c8c679de Update changelog for upcoming 1.3.1 release 2017-02-08 15:53:44 +01:00
Paul Tremberth d205206aaa Merge pull request #2345 from gustavodeandrade/master 
[MRG+1] Fix documentation about HTML entities decoding with selector extraction
 2017-02-08 13:21:15 +01:00
Paul Tremberth 7d0b89042f Merge pull request #2533 from djrobust/patch-1 
[MRG+1] Use yield with nested parsing of responses
 2017-02-08 13:02:50 +01:00
Paul Tremberth a8893190a1 Merge pull request #2464 from elacuesta/component_order_integer 
[MRG+2] Validate numeric values (components order)
 2017-02-08 13:01:02 +01:00
Paul Tremberth 3becb861af Merge pull request #2544 from scrapy/twisted-version-cleanup 
[MRG+1] Remove code required to support ancient twisted versions.
 2017-02-08 12:59:45 +01:00
Paul Tremberth e87f18447c Merge pull request #2538 from scrapy/linkextractor-fail-fast 
LinkExtractor: don’t check all regexes if one of them matches
 2017-02-08 12:58:41 +01:00
Paul Tremberth 5ad64204ac Merge pull request #2509 from rolando-contrib/py36-classcell-compat 
[MRG+1] BUG: Fix __classcell__ propagation required in Python 3.6.
 2017-02-08 12:15:15 +01:00
Rolando Espinoza 4e765acaed BUG: Fix __classcell__ propagation. 
Python 3.6 added simpler customization of class creation but this
requires to propagate correctly the __classcell__ attribute in custom
__new__ methods.

See https://docs.python.org/3.6/whatsnew/3.6.html#pep-487-simpler-
customization-of-class-creation
 2017-02-08 02:12:52 -04:00
Mikhail Korobov f2f9350c47 Merge pull request #2542 from scrapy/redirect-cleanup 
[MRG+1] Cleanup MetaRefreshMiddleware: remove redundant check
 2017-02-08 00:57:21 +05:00
Mikhail Korobov 04b2f79e7a Remove code required to support ancient twisted versions. See GH-1887. 2017-02-07 22:30:58 +05:00
Mikhail Korobov eaf62ab69c cleanup MetaRefreshMiddleware: remove redundant check 2017-02-07 17:56:43 +05:00
Eugenio Lacuesta 24e82bfe75 Validate values for components order 2017-02-07 09:13:57 -03:00
Pawel Miech fb4ef21a1d [httpcompression] minor style edits 2017-02-07 10:22:42 +01:00
Mikhail Korobov f73eb715ac LinkExtractor: don’t check all regexes if one of them matches 2017-02-07 02:16:06 +05:00
Pawel Miech af802bad14 [httpcompression] add brotlipy for python 3 2017-02-06 15:45:21 +01:00
Pawel Miech 3daf473686 [httpcompression] skip test if no brotli 2017-02-06 13:49:41 +01:00
Pawel Miech f8f8bbe080 [httpcompression] import brotli when available 2017-02-06 12:19:00 +01:00
Pawel Miech 16c4b4e184 [httpcompression] add support for br - brotli content encoding 2017-02-06 11:41:08 +01:00
Mikhail Korobov 3b8e6d4d82 Merge pull request #2531 from Lukas0907/patch-1 
Fix typo in downloader-middleware.rst.
 2017-02-06 13:09:35 +05:00
Mikhail Korobov 2743786d1d Merge pull request #2534 from shio-phys/fix/typo_of_doc 
fix typo
 2017-02-06 13:05:46 +05:00
Takehiro Shiozaki fcb3daf4fa fix typo 2017-02-06 14:03:41 +09:00
djrobust 3021084f37 Use 'yield' when parsing multiple responses 
Use 'yield' consistently across examples of parse functions.
 2017-02-04 20:07:05 -08:00
Lukas Anzinger 09643796b4 Fix typo in downloader-middleware.rst. 2017-02-03 20:05:17 +01:00
Mikhail Korobov 814ce37dff Merge pull request #2528 from redapple/dns-retry-test 
Make DNS retry test compatible with Twisted 17+
 2017-02-03 02:43:46 +05:00
Mikhail Korobov 7b8bbc9205 Merge pull request #2215 from redapple/travis-pypy 
Enable PyPy tests on Travis (allowed to fail)
 2017-02-03 02:38:49 +05:00
Paul Tremberth 02e1d2b1fd Add trailing dot 2017-02-02 22:28:37 +01:00
Paul Tremberth e604c0f3ab Remove unused imports 2017-02-02 18:26:41 +01:00
Mikhail Korobov ae2a5292d8 Merge pull request #2457 from redapple/parsel-selector-kwargs 
[MRG] Support kwargs for response.xpath()
 2017-02-02 22:17:26 +05:00
Paul Tremberth 3358254c5c Make DNS retry test compatible with Twisted 17+ 2017-02-02 17:53:28 +01:00
Paul Tremberth 1295c17a26 Bump parsel requirement to at least parsel v1.1 2017-02-02 17:47:22 +01:00
Paul Tremberth 1c0b805357 DOC Mention XPath variables in Selectors section 2017-02-02 17:44:57 +01:00
Paul Tremberth 803d8c4b57 Add tests for passing kwargs on response .xpath() shortcut 2017-02-02 17:44:57 +01:00