RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,522 Commits 49 Branches 132 Tags 132 MiB
Commit Graph

5522 Commits

Author SHA1 Message Date
Konstantin Lopuhin 6137dd96d9 Fix documentation for S3_STORE_ACL (now settings.FILES_STORE_S3_ACL) settings: it has nothing to do with feed exporters. 2016-02-24 10:16:41 +03:00
Konstantin Lopuhin 164f300762 See #1778 - change default S3 ACL to "private" and allow customization via settings 2016-02-24 10:16:41 +03:00
Daniel Graña 513ba7a1fb Merge pull request #1800 from redapple/http11-post-content-length 
[MRG+1] Add "Content-Length: 0" for body-less HTTP/1.1 POST requests
 2016-02-23 15:00:33 -03:00
Elias Dorneles 329a77a3d0 Merge pull request #1808 from scrapy/revert-1701-cleanup-http11-tunneling 
Revert "Cleanup http11 tunneling connection after #1678"
 2016-02-23 13:59:15 -03:00
Elias Dorneles 10bcdb49b0 Merge pull request #1787 from scrapy/improve-errors 
[MRG+1] Better tracebacks
 2016-02-23 13:16:47 -03:00
Paul Tremberth 35fb630c4e Revert "[MRG+1] Cleanup http11 tunneling connection after #1678" 2016-02-23 16:28:48 +01:00
Mikhail Korobov 62a517163b Merge pull request #1801 from redapple/botocore-notconfigured-exception 
Fix SkipTest() message for botocore import test
 2016-02-21 12:33:48 +05:00
Paul Tremberth ad4c1169c4 Pass exception directly to SkipTest() 
It prints the same as passing `str(e)`
 2016-02-21 01:16:46 +01:00
Paul Tremberth e9bd328936 Fix SkipTest() message for botocore import test 2016-02-21 01:08:50 +01:00
Paul Tremberth ac8f97c33b Fix typo in comment 2016-02-21 00:49:41 +01:00
Paul Tremberth 6174192564 Add "Content-Length: 0" for body-less HTTP/1.1 POST requests 
GH-823 was fixed only for HTTP/1.0 (in GH-1089)
 2016-02-20 23:27:04 +01:00
Paul Tremberth 2969e34e82 Merge pull request #1795 from AmbientLighter/patch-1 
Fix typos in docstrings
 2016-02-19 16:44:20 +01:00
Victor Mireyev 523e98dabe Fix typo in docstring 2016-02-19 17:32:01 +03:00
Victor Mireyev 5f5374209f Fix typo in docstring 2016-02-19 17:20:34 +03:00
Mikhail Korobov 18a381816d Remove unneeded `raise` (thanks @lopuhin). 2016-02-19 18:31:49 +05:00
Paul Tremberth da36b7d386 Merge pull request #1761 from lopuhin/py3-s3-botocore 
[MRG+1] Py3 S3 botocore
 2016-02-18 15:11:03 +01:00
Paul Tremberth 104027d78d Minor change on quotes 
Trying to force Travis CI to build
 2016-02-18 11:45:03 +01:00
Konstantin Lopuhin d61fbcc8b5 Support headers in S3FilesStore.persist_file for botocore 2016-02-18 10:57:02 +03:00
Konstantin Lopuhin 617631f264 Fix method name: this always returns unicode keys and values 2016-02-18 10:10:16 +03:00
Mikhail Korobov f766dd0ba8 Preserve tracebacks better. Fixes GH-1760. 2016-02-17 23:07:03 +05:00
Mikhail Korobov 06da7af9e2 TST clean up RunSpiderCommandTest 2016-02-17 23:03:12 +05:00
Mikhail Korobov 4e2b7207ec Merge pull request #1786 from redapple/header-encoding 
More liberal Content-Disposition header parsing
 2016-02-17 21:33:43 +05:00
Paul Tremberth cabed6f183 More liberal Content-Disposition header parsing 
Fixes #1782
 2016-02-17 16:55:28 +01:00
Konstantin Lopuhin 49313a6988 use absolute_import to import external boto package 2016-02-15 20:16:40 +03:00
Konstantin Lopuhin d1ecb8cd38 Fix S3TestCase for precise env: we reraise TypeError as NotConfigured in this case 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin e7c4806c5e Update feedstorage docs: add botocore, mention that boto is supported only on Python 2 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 77ebb13684 fix assertRaises for precise env 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 08bc41cc68 py3: reviewed s3 downloader handlers 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 3cb7a567ea py3 fix for TestS3FilesStore: checksum is a native string 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin cfc567f48e botocore support for S3FilesStore 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 32cd8c9165 add direct test for S3FilesStore 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin d1470e85a2 S3FeedStorageTest: pass on py3, add some non-ascii content to be sure 2016-02-15 19:59:48 +03:00
Konstantin Lopuhin 3ada45a9bb S3FeedStorageTest: add botocore support, and organize boto/botocore checks 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin 5d2f067458 S3FeedStorageTest: delete key after test 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin 19b2910ad1 Fix assert_aws_environ: check for botocore with boto fallback on PY2 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin 408bc1580b Pass env variables required for running tests against real s3 via tox. 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin bcb92b50dc check that no extra kwargs are silently discarded 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin d6bea3bf2e botocore not only does not allow passing our own Date header, but does not handle x-amz-date according to the spec 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin 7748ee6bba mock date in s3 tests when using botocore 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin c3fec83e7e use botocore by default, boto is still used in "precise" env 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin 1b1092b7d0 add Headers.to_native_string_dict - useful when interfacing with other libraries 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin 467553cc29 fix anon test: in this case we do no signing, just change the url 2016-02-15 19:59:47 +03:00
Konstantin Lopuhin eaf3a239e4 using botocore for s3 request signing: proof of concept 2016-02-15 19:59:46 +03:00
Paul Tremberth 41588397c0 Merge pull request #1765 from scrapy/add-deprecation-for-pydispatch 
[MRG+1] Add fallback and deprecation warning for pydispatch (fixes #1762)
 2016-02-11 19:29:53 +01:00
Elias Dorneles 164493df2e add deprecation for pydispatch (thanks for the help @redapple) 2016-02-11 16:15:28 -02:00
Elias Dorneles 150d1c370b Merge pull request #1769 from orangain/stdout-typeerror 
[MRG+1] PY3: Fix TypeError when outputting to stdout
 2016-02-10 14:30:11 -02:00
Paul Tremberth c083935806 Merge pull request #1771 from orangain/secure-cookies 
[MRG+1] PY3: Implement some attributes of WrappedRequest required in Python 3
 2016-02-08 11:52:00 +01:00
Mikhail Korobov 44bc4c06ff Merge pull request #1767 from orangain/sitemap-robotstxt 
[MRG+1] PY3: Fix SitemapSpider to extract sitemap urls from robots.txt properly
 2016-02-08 10:11:58 +05:00
orangain 1f743996ff PY3: Implement some attributes of WrappedRequest required in Python 3 
This will fix #1770.
 2016-02-07 14:19:27 +09:00
orangain 1cb841bfb3 PY3: Fix TypeError when outputting to stdout 
This will fix #1768.
 2016-02-07 10:06:42 +09:00