RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,222 Commits 42 Branches 132 Tags 162 MiB
Commit Graph

6222 Commits

Author SHA1 Message Date
Paul Tremberth ad4c1169c4 Pass exception directly to SkipTest() 
It prints the same as passing `str(e)`
 2016-02-21 01:16:46 +01:00
Paul Tremberth e9bd328936 Fix SkipTest() message for botocore import test 2016-02-21 01:08:50 +01:00
Paul Tremberth ac8f97c33b Fix typo in comment 2016-02-21 00:49:41 +01:00
Paul Tremberth 6174192564 Add "Content-Length: 0" for body-less HTTP/1.1 POST requests 
GH-823 was fixed only for HTTP/1.0 (in GH-1089)
 2016-02-20 23:27:04 +01:00
Paul Tremberth 3f946b5278 Fix super() call 2016-02-20 02:07:45 +01:00
Paul Tremberth 49c757f208 Fix import for Ubuntu 12.04 precise (Twisted 11.1.0) 
Revert test server certificate change
 2016-02-20 01:58:59 +01:00
Paul Tremberth 57990fba7a Backward compatibility for HTTP/10 context factory 
New DOWNLOADER_CLIENT_TLS_METHOD setting to configure TLS method
 2016-02-20 01:32:21 +01:00
Paul Tremberth 2969e34e82 Merge pull request #1795 from AmbientLighter/patch-1 
Fix typos in docstrings
 2016-02-19 16:44:20 +01:00
Victor Mireyev 523e98dabe Fix typo in docstring 2016-02-19 17:32:01 +03:00
Victor Mireyev 5f5374209f Fix typo in docstring 2016-02-19 17:20:34 +03:00
Mikhail Korobov 18a381816d Remove unneeded `raise` (thanks @lopuhin). 2016-02-19 18:31:49 +05:00
Paul Tremberth 406b9a06fe Add missing import for implementer 
Also remove TLSv1.1 and TLSv1.2 method: these are available only
from pyOpenSSL 0.14
https://github.com/pyca/pyopenssl/releases/tag/v0.14a1
 2016-02-19 02:56:26 +01:00
Paul Tremberth 45f972cb3f Implement IPolicyForHTTPS 2016-02-19 02:31:57 +01:00
Paul Tremberth 30a27effa9 Use best practices for TLS connections when using Twisted>=14.0 2016-02-19 02:09:42 +01:00
Paul Tremberth da36b7d386 Merge pull request #1761 from lopuhin/py3-s3-botocore 
[MRG+1] Py3 S3 botocore
 2016-02-18 15:11:03 +01:00
Paul Tremberth 104027d78d Minor change on quotes 
Trying to force Travis CI to build
 2016-02-18 11:45:03 +01:00
Konstantin Lopuhin d61fbcc8b5 Support headers in S3FilesStore.persist_file for botocore 2016-02-18 10:57:02 +03:00
Konstantin Lopuhin 617631f264 Fix method name: this always returns unicode keys and values 2016-02-18 10:10:16 +03:00
Mikhail Korobov f766dd0ba8 Preserve tracebacks better. Fixes GH-1760. 2016-02-17 23:07:03 +05:00
Mikhail Korobov 06da7af9e2 TST clean up RunSpiderCommandTest 2016-02-17 23:03:12 +05:00
Mikhail Korobov 4e2b7207ec Merge pull request #1786 from redapple/header-encoding 
More liberal Content-Disposition header parsing
 2016-02-17 21:33:43 +05:00
Paul Tremberth cabed6f183 More liberal Content-Disposition header parsing 
Fixes #1782
 2016-02-17 16:55:28 +01:00
Konstantin Lopuhin 49313a6988 use absolute_import to import external boto package 2016-02-15 20:16:40 +03:00
Konstantin Lopuhin d1ecb8cd38 Fix S3TestCase for precise env: we reraise TypeError as NotConfigured in this case 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin e7c4806c5e Update feedstorage docs: add botocore, mention that boto is supported only on Python 2 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 77ebb13684 fix assertRaises for precise env 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 08bc41cc68 py3: reviewed s3 downloader handlers 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 3cb7a567ea py3 fix for TestS3FilesStore: checksum is a native string 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin cfc567f48e botocore support for S3FilesStore 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 32cd8c9165 add direct test for S3FilesStore 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin d1470e85a2 S3FeedStorageTest: pass on py3, add some non-ascii content to be sure 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 3ada45a9bb S3FeedStorageTest: add botocore support, and organize boto/botocore checks 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin 5d2f067458 S3FeedStorageTest: delete key after test 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin 19b2910ad1 Fix assert_aws_environ: check for botocore with boto fallback on PY2 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin 408bc1580b Pass env variables required for running tests against real s3 via tox. 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin bcb92b50dc check that no extra kwargs are silently discarded 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin d6bea3bf2e botocore not only does not allow passing our own Date header, but does not handle x-amz-date according to the spec 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin 7748ee6bba mock date in s3 tests when using botocore 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin c3fec83e7e use botocore by default, boto is still used in "precise" env 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin 1b1092b7d0 add Headers.to_native_string_dict - useful when interfacing with other libraries 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin 467553cc29 fix anon test: in this case we do no signing, just change the url 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin eaf3a239e4 using botocore for s3 request signing: proof of concept 2016-02-15 19:59:46 +03:00
Paul Tremberth 41588397c0 Merge pull request #1765 from scrapy/add-deprecation-for-pydispatch 
[MRG+1] Add fallback and deprecation warning for pydispatch (fixes #1762)
 2016-02-11 19:29:53 +01:00
Elias Dorneles 164493df2e add deprecation for pydispatch (thanks for the help @redapple) 2016-02-11 16:15:28 -02:00
Elias Dorneles 150d1c370b Merge pull request #1769 from orangain/stdout-typeerror 
[MRG+1] PY3: Fix TypeError when outputting to stdout
 2016-02-10 14:30:11 -02:00
Paul Tremberth c083935806 Merge pull request #1771 from orangain/secure-cookies 
[MRG+1] PY3: Implement some attributes of WrappedRequest required in Python 3
 2016-02-08 11:52:00 +01:00
Mikhail Korobov 44bc4c06ff Merge pull request #1767 from orangain/sitemap-robotstxt 
[MRG+1] PY3: Fix SitemapSpider to extract sitemap urls from robots.txt properly
 2016-02-08 10:11:58 +05:00
orangain 1f743996ff PY3: Implement some attributes of WrappedRequest required in Python 3 
This will fix #1770.
 2016-02-07 14:19:27 +09:00
orangain 1cb841bfb3 PY3: Fix TypeError when outputting to stdout 
This will fix #1768.
 2016-02-07 10:06:42 +09:00
orangain 25c56159b8 Fix SitemapSpider to extract sitemap urls from robots.txt properly 
This will fix #1766.
 2016-02-06 23:54:07 +09:00