RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,505 Commits 49 Branches 132 Tags 133 MiB
Commit Graph

5505 Commits

Author SHA1 Message Date
Mikhail Korobov 9a992cdfd7 declare Python 3 support in setup.py 2016-03-01 14:38:49 +01:00
Mikhail Korobov a83ea5d50b Extract a function to build CONNECT request; add tests for it. See GH-1701 and GH-1808. 2016-03-01 14:38:39 +01:00
Paul Tremberth fe3a2fcce9 Build universal wheels 2016-03-01 14:18:14 +01:00
Paul Tremberth ab5e2accda Bump version: 1.1.0rc1 → 1.1.0rc2 2016-02-29 13:15:54 +01:00
Paul Tremberth a4cc8cd2a2 Update release notes about change of default S3 ACL policy to "private" 2016-02-29 13:15:50 +01:00
Paul Tremberth acfc915b7a Update release notes about HTTPS downloader 2016-02-27 15:13:30 +01:00
Paul Tremberth c1337d74ab Document DOWNLOADER_* settings for HTTP/1.0 and TLS 2016-02-27 15:12:56 +01:00
Paul Tremberth ed3ef17397 Merge pull request #1821 from redapple/backport-1.1-pr1818 
[backport][1.1] Refactored SpiderLoader class constructor for easier subclassing
 2016-02-27 14:50:36 +01:00
Lucas Moauro 9766695550 Refactored SpiderLoader class constructor for easier subclassing 2016-02-27 14:47:20 +01:00
Paul Tremberth 75743c95fd Merge pull request #1816 from redapple/backport-1.1-pr1769 
[backport][1.1] PY3: Fix TypeError when outputting to stdout
 2016-02-25 00:46:21 +01:00
orangain 838bc08098 PY3: Fix TypeError when outputting to stdout 
This will fix #1768.
 2016-02-25 00:35:41 +01:00
Victor Mireyev bc170a2495 Fix typo in docstring 2016-02-25 00:28:54 +01:00
Paul Tremberth 394f42440b Merge pull request #1815 from redapple/backport-1.1-pr1794 
[backport][1.1] Use best practices for TLS connections when using Twisted>=14.0
 2016-02-24 22:59:54 +01:00
Paul Tremberth bdc1356871 Use best practices for TLS connections when using Twisted>=14.0 
Implement IPolicyForHTTPS
Backward compatibility for HTTP/10 context factory
New DOWNLOADER_CLIENT_TLS_METHOD setting to configure TLS method
Backward-compatibility for common Scrapy context factory patterns
Add warning for context factories not accepting `method` param
Use context factory class name in warning message
 2016-02-24 22:47:44 +01:00
Paul Tremberth 300f162a31 Merge pull request #1813 from redapple/backport-1.1-pr1796 
[backport][1.1] Change default S3 ACL to "private" and allow customization via settings
 2016-02-24 22:39:21 +01:00
Konstantin Lopuhin 9083b68125 Fix documentation for S3_STORE_ACL (now settings.FILES_STORE_S3_ACL) settings: it has nothing to do with feed exporters. 2016-02-24 22:26:22 +01:00
Konstantin Lopuhin b9c6d67365 See #1778 - change default S3 ACL to "private" and allow customization via settings 2016-02-24 22:26:22 +01:00
Paul Tremberth 6d4c51b266 Merge pull request #1809 from scrapy/backport-1.1-1787 
[backport][1.1] Better tracebacks
 2016-02-24 17:54:56 +01:00
Paul Tremberth f0c07a3121 Merge pull request #1810 from scrapy/backport-1.1-pr1800 
[backport][1.1] Add "Content-Length: 0" for body-less HTTP/1.1 POST requests
 2016-02-24 17:54:26 +01:00
Paul Tremberth 6801af2419 Merge pull request #1811 from scrapy/backport-1.1-revert-pr1678 
[backport][1.1] Revert "Cleanup http11 tunneling connection after #1678"
 2016-02-24 17:53:56 +01:00
Paul Tremberth 5e57169394 Merge pull request #1812 from scrapy/backport-1.1-pr1801 
[backport][1.1] Fix SkipTest() message for botocore import test
 2016-02-24 17:52:20 +01:00
Paul Tremberth 9d51e1fc3e Fix SkipTest() message for botocore import test 2016-02-24 17:42:42 +01:00
Paul Tremberth 0b2bb3bc7a Revert "[MRG+1] Cleanup http11 tunneling connection after #1678" 2016-02-24 17:24:54 +01:00
Paul Tremberth 2bcaff97b4 Fix typo in comment 2016-02-24 17:21:08 +01:00
Paul Tremberth e25052e4ef Add "Content-Length: 0" for body-less HTTP/1.1 POST requests 
GH-823 was fixed only for HTTP/1.0 (in GH-1089)
 2016-02-24 17:21:08 +01:00
Mikhail Korobov 374d7b04fc Remove unneeded `raise` (thanks @lopuhin). 2016-02-24 17:14:48 +01:00
Paul Tremberth 50f7b416c5 Minor change on quotes 
Trying to force Travis CI to build
 2016-02-24 17:14:48 +01:00
Mikhail Korobov 0d596c27a9 Preserve tracebacks better. Fixes GH-1760. 2016-02-24 17:14:48 +01:00
Mikhail Korobov 4459ee5d53 TST clean up RunSpiderCommandTest 2016-02-24 17:14:48 +01:00
Paul Tremberth a6bc758db1 Merge pull request #1792 from scrapy/backport-1.1-pr1761 
[backport][1.1.x] Py3 S3 botocore
 2016-02-19 11:17:20 +01:00
Paul Tremberth eafd054092 Merge pull request #1793 from scrapy/backport-1.1-pr1786 
[backport][1.1.x] More liberal Content-Disposition header parsing
 2016-02-18 23:23:16 +01:00
Paul Tremberth 3331cea788 More liberal Content-Disposition header parsing 
Fixes #1782
 2016-02-18 22:56:17 +01:00
Konstantin Lopuhin 25e481fd14 [backport][1.1.x] Py3 S3 botocore 
Originally from https://github.com/scrapy/scrapy/pull/1761
 2016-02-18 22:48:37 +01:00
Mikhail Korobov 7ce32422c4 Merge pull request #1774 from scrapy/backport-1.1 
[MRG+1] [1.1.x][backport] #1766, #1770, #1750, #1662, #1765
 2016-02-15 16:05:31 +05:00
Elias Dorneles f928014d91 add deprecation for pydispatch (thanks for the help @redapple) 2016-02-15 11:40:08 +01:00
Paul Tremberth b1fac01258 Update release notes 2016-02-11 18:44:26 +01:00
Nicolas Pennequin 9cdbe177b7 Update the email doc for the charset argument to send(). 2016-02-09 19:52:06 +01:00
Nicolas Pennequin 72229f92de MailSender.send: allow passing a charset. 
Resolves Issue #348
 2016-02-09 19:52:06 +01:00
Mikhail Korobov 8aca66252a more efficient ExecutionEngine.spider_is_idle 2016-02-09 19:52:06 +01:00
orangain 281e9bd046 PY3: Implement some attributes of WrappedRequest required in Python 3 
This will fix #1770.
 2016-02-09 16:10:58 +01:00
orangain 522c71bb8a Fix SitemapSpider to extract sitemap urls from robots.txt properly 
This will fix #1766.
 2016-02-09 16:10:58 +01:00
Daniel Graña 56ad73fe99 Enable travis builds on tag patterns 2016-02-03 23:11:17 -03:00
Elias Dorneles a0b63a7707 Bump version: 1.1.0dev1 → 1.1.0rc1 2016-02-03 21:52:47 -02:00
Elias Dorneles 94b2c47da5 Merge pull request #1751 from scrapy/release-notes-1.1 
[MRG+1] Release notes 1.1
 2016-02-03 21:51:10 -02:00
Elias Dorneles ca08314654 makes more explicit source of problems in Python 3 limitations 2016-02-03 21:38:25 -02:00
Elias Dorneles 2167573b32 fix comment about disabling robots.txt 2016-02-03 21:31:27 -02:00
Elias Dorneles 3180abc800 applying more review comments 2016-02-03 21:28:05 -02:00
Elias Dorneles d40add7b27 add note about robots.txt waiting and make it explicit builtin extensions only are ported 2016-02-03 21:18:25 -02:00
Elias Dorneles 9cfefd5277 favor beta over basic to describe PY3 support 2016-02-03 20:59:56 -02:00
Elias Dorneles 2f697ce59f added note about how to revert to old behavior for robotstxt 2016-02-03 20:51:26 -02:00