RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,542 Commits 49 Branches 132 Tags 143 MiB
Commit Graph

5542 Commits

Author SHA1 Message Date
Mikhail Korobov 16512a32ea Merge pull request #1825 from redapple/relnotes-s3policy 
[MRG+1] Update release notes about change of default S3 ACL policy to "private"
 2016-02-29 17:02:18 +05:00
Paul Tremberth 90c64a6d13 Update release notes about change of default S3 ACL policy to "private" 2016-02-29 12:27:25 +01:00
Paul Tremberth 74158611c8 Merge pull request #1818 from lagenar/master 
[MRG+1] Refactored SpiderLoader class constructor for easier subclassing
 2016-02-26 23:23:34 +01:00
Paul Tremberth 241ae9f22f Merge pull request #1820 from redapple/http-tls-settings 
[MRG+1] Document DOWNLOADER_* settings for HTTP/1.0 and TLS
 2016-02-26 23:20:33 +01:00
Paul Tremberth 709b4fa884 Update release notes about HTTPS downloader 2016-02-26 18:35:29 +01:00
Paul Tremberth 174f526709 Document DOWNLOADER_* settings for HTTP/1.0 and TLS 2016-02-26 18:23:57 +01:00
Lucas Moauro 1ce4c86c2b Refactored SpiderLoader class constructor for easier subclassing 2016-02-25 21:08:50 -03:00
Paul Tremberth 84dea19455 Update release notes 2016-02-25 00:59:15 +01:00
Mikhail Korobov f7a48b0c6b Merge pull request #1794 from redapple/twisted-tls 
[MRG+1] Use best practices for TLS connections when using Twisted>=14.0
 2016-02-25 02:23:33 +05:00
Paul Tremberth 4e93501304 Merge pull request #1796 from lopuhin/s3-acl-private 
[MRG+1] Change default S3 ACL to "private" and allow customization via settings
 2016-02-24 22:18:43 +01:00
Paul Tremberth 0336c2504a Use context factory class name in warning message 2016-02-24 16:42:25 +01:00
Paul Tremberth c29a1b98b3 Make warning message a bit nicer in logs 2016-02-24 16:01:18 +01:00
Konstantin Lopuhin 6137dd96d9 Fix documentation for S3_STORE_ACL (now settings.FILES_STORE_S3_ACL) settings: it has nothing to do with feed exporters. 2016-02-24 10:16:41 +03:00
Konstantin Lopuhin 164f300762 See #1778 - change default S3 ACL to "private" and allow customization via settings 2016-02-24 10:16:41 +03:00
Paul Tremberth c9890d5f76 Add warning for context factories not accepting `method` param 2016-02-24 01:26:04 +01:00
Paul Tremberth 095495e99c Backward-compatibility for common Scrapy context factory patterns 2016-02-24 01:24:58 +01:00
Daniel Graña 513ba7a1fb Merge pull request #1800 from redapple/http11-post-content-length 
[MRG+1] Add "Content-Length: 0" for body-less HTTP/1.1 POST requests
 2016-02-23 15:00:33 -03:00
Elias Dorneles 329a77a3d0 Merge pull request #1808 from scrapy/revert-1701-cleanup-http11-tunneling 
Revert "Cleanup http11 tunneling connection after #1678"
 2016-02-23 13:59:15 -03:00
Elias Dorneles 10bcdb49b0 Merge pull request #1787 from scrapy/improve-errors 
[MRG+1] Better tracebacks
 2016-02-23 13:16:47 -03:00
Paul Tremberth 35fb630c4e Revert "[MRG+1] Cleanup http11 tunneling connection after #1678" 2016-02-23 16:28:48 +01:00
Mikhail Korobov 62a517163b Merge pull request #1801 from redapple/botocore-notconfigured-exception 
Fix SkipTest() message for botocore import test
 2016-02-21 12:33:48 +05:00
Paul Tremberth ad4c1169c4 Pass exception directly to SkipTest() 
It prints the same as passing `str(e)`
 2016-02-21 01:16:46 +01:00
Paul Tremberth e9bd328936 Fix SkipTest() message for botocore import test 2016-02-21 01:08:50 +01:00
Paul Tremberth ac8f97c33b Fix typo in comment 2016-02-21 00:49:41 +01:00
Paul Tremberth 6174192564 Add "Content-Length: 0" for body-less HTTP/1.1 POST requests 
GH-823 was fixed only for HTTP/1.0 (in GH-1089)
 2016-02-20 23:27:04 +01:00
Paul Tremberth 3f946b5278 Fix super() call 2016-02-20 02:07:45 +01:00
Paul Tremberth 49c757f208 Fix import for Ubuntu 12.04 precise (Twisted 11.1.0) 
Revert test server certificate change
 2016-02-20 01:58:59 +01:00
Paul Tremberth 57990fba7a Backward compatibility for HTTP/10 context factory 
New DOWNLOADER_CLIENT_TLS_METHOD setting to configure TLS method
 2016-02-20 01:32:21 +01:00
Paul Tremberth 2969e34e82 Merge pull request #1795 from AmbientLighter/patch-1 
Fix typos in docstrings
 2016-02-19 16:44:20 +01:00
Victor Mireyev 523e98dabe Fix typo in docstring 2016-02-19 17:32:01 +03:00
Victor Mireyev 5f5374209f Fix typo in docstring 2016-02-19 17:20:34 +03:00
Mikhail Korobov 18a381816d Remove unneeded `raise` (thanks @lopuhin). 2016-02-19 18:31:49 +05:00
Paul Tremberth 406b9a06fe Add missing import for implementer 
Also remove TLSv1.1 and TLSv1.2 method: these are available only
from pyOpenSSL 0.14
https://github.com/pyca/pyopenssl/releases/tag/v0.14a1
 2016-02-19 02:56:26 +01:00
Paul Tremberth 45f972cb3f Implement IPolicyForHTTPS 2016-02-19 02:31:57 +01:00
Paul Tremberth 30a27effa9 Use best practices for TLS connections when using Twisted>=14.0 2016-02-19 02:09:42 +01:00
Paul Tremberth da36b7d386 Merge pull request #1761 from lopuhin/py3-s3-botocore 
[MRG+1] Py3 S3 botocore
 2016-02-18 15:11:03 +01:00
Paul Tremberth 104027d78d Minor change on quotes 
Trying to force Travis CI to build
 2016-02-18 11:45:03 +01:00
Konstantin Lopuhin d61fbcc8b5 Support headers in S3FilesStore.persist_file for botocore 2016-02-18 10:57:02 +03:00
Konstantin Lopuhin 617631f264 Fix method name: this always returns unicode keys and values 2016-02-18 10:10:16 +03:00
Mikhail Korobov f766dd0ba8 Preserve tracebacks better. Fixes GH-1760. 2016-02-17 23:07:03 +05:00
Mikhail Korobov 06da7af9e2 TST clean up RunSpiderCommandTest 2016-02-17 23:03:12 +05:00
Mikhail Korobov 4e2b7207ec Merge pull request #1786 from redapple/header-encoding 
More liberal Content-Disposition header parsing
 2016-02-17 21:33:43 +05:00
Paul Tremberth cabed6f183 More liberal Content-Disposition header parsing 
Fixes #1782
 2016-02-17 16:55:28 +01:00
Konstantin Lopuhin 49313a6988 use absolute_import to import external boto package 2016-02-15 20:16:40 +03:00
Konstantin Lopuhin d1ecb8cd38 Fix S3TestCase for precise env: we reraise TypeError as NotConfigured in this case 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin e7c4806c5e Update feedstorage docs: add botocore, mention that boto is supported only on Python 2 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 77ebb13684 fix assertRaises for precise env 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 08bc41cc68 py3: reviewed s3 downloader handlers 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 3cb7a567ea py3 fix for TestS3FilesStore: checksum is a native string 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin cfc567f48e botocore support for S3FilesStore 2016-02-15 19:59:48 +03:00