RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,517 Commits 49 Branches 132 Tags 141 MiB
Commit Graph

5517 Commits

Author SHA1 Message Date
Aron Bordin 7b1dddceb3 small doc style fixes 2016-03-30 18:36:34 +02:00
nyov 718d386ebd Update documentation links 2016-03-30 18:36:34 +02:00
djunzu 3d6abacc55 Add pps extension to IGNORED_EXTENSIONS 
modified:   scrapy/linkextractors/__init__.py
 2016-03-30 18:33:58 +02:00
Paul Tremberth d381f8d930 Merge pull request #1892 from scrapy/backport-1.1-pr1822 
[Backport][1.1] Allow core Scheduler priority queue customization
 2016-03-30 18:17:22 +02:00
nyov 1adc47ce93 Allow core Scheduler priority queue customization 2016-03-30 17:51:37 +02:00
Paul Tremberth 844f040bc4 Add support for Sphinx 1.4 
See http://www.sphinx-doc.org/en/stable/changes.html#release-1-4-released-mar-28-2016

sphinx_rtd_theme has become optional, needs to be added to reqs

https://github.com/sphinx-doc/sphinx/pull/2320 changes node entries tuples
to 5 values instead of 4

`sh` syntax highlighting added very locally in selectors.rst
because of this warning/error with Sphinx 1.4:

```
Warning, treated as error:
/home/paul/src/scrapy/docs/topics/selectors.rst:743:
WARNING: Could not lex literal_block as "python". Highlighting skipped.
```
 2016-03-30 17:49:14 +02:00
nanolab e7a5e3717b Update httpcache.py 
It checks cache directory modification time, but have to check file modification time.
 2016-03-30 10:53:54 +02:00
Lele 15ffb076ca Changed sel. to response. for clarity 
Changed sel. to response. to comply with the rest of the examples in the same section, to avoid confusion.
 2016-03-30 10:53:54 +02:00
Paul Tremberth e025f374e8 Bump version: 1.1.0rc2 → 1.1.0rc3 2016-03-01 23:40:30 +01:00
Paul Tremberth 9a4ead8e5d Merge pull request #1831 from redapple/backport-1.1-pr1804 
[Backport][1.1] Re-enable HTTPS tests for download timeouts
 2016-03-01 15:22:31 +01:00
Paul Tremberth 4547a7e759 Explicitly call Twisted transport stopProducing() on HTTP/1.0 timeouts 2016-03-01 15:00:58 +01:00
Paul Tremberth a0c5c61855 Explicitly call Twisted transport stopProducing() on HTTP/1.1 timeouts 2016-03-01 15:00:44 +01:00
Mikhail Korobov 9a992cdfd7 declare Python 3 support in setup.py 2016-03-01 14:38:49 +01:00
Mikhail Korobov a83ea5d50b Extract a function to build CONNECT request; add tests for it. See GH-1701 and GH-1808. 2016-03-01 14:38:39 +01:00
Paul Tremberth fe3a2fcce9 Build universal wheels 2016-03-01 14:18:14 +01:00
Paul Tremberth ab5e2accda Bump version: 1.1.0rc1 → 1.1.0rc2 2016-02-29 13:15:54 +01:00
Paul Tremberth a4cc8cd2a2 Update release notes about change of default S3 ACL policy to "private" 2016-02-29 13:15:50 +01:00
Paul Tremberth acfc915b7a Update release notes about HTTPS downloader 2016-02-27 15:13:30 +01:00
Paul Tremberth c1337d74ab Document DOWNLOADER_* settings for HTTP/1.0 and TLS 2016-02-27 15:12:56 +01:00
Paul Tremberth ed3ef17397 Merge pull request #1821 from redapple/backport-1.1-pr1818 
[backport][1.1] Refactored SpiderLoader class constructor for easier subclassing
 2016-02-27 14:50:36 +01:00
Lucas Moauro 9766695550 Refactored SpiderLoader class constructor for easier subclassing 2016-02-27 14:47:20 +01:00
Paul Tremberth 75743c95fd Merge pull request #1816 from redapple/backport-1.1-pr1769 
[backport][1.1] PY3: Fix TypeError when outputting to stdout
 2016-02-25 00:46:21 +01:00
orangain 838bc08098 PY3: Fix TypeError when outputting to stdout 
This will fix #1768.
 2016-02-25 00:35:41 +01:00
Victor Mireyev bc170a2495 Fix typo in docstring 2016-02-25 00:28:54 +01:00
Paul Tremberth 394f42440b Merge pull request #1815 from redapple/backport-1.1-pr1794 
[backport][1.1] Use best practices for TLS connections when using Twisted>=14.0
 2016-02-24 22:59:54 +01:00
Paul Tremberth bdc1356871 Use best practices for TLS connections when using Twisted>=14.0 
Implement IPolicyForHTTPS
Backward compatibility for HTTP/10 context factory
New DOWNLOADER_CLIENT_TLS_METHOD setting to configure TLS method
Backward-compatibility for common Scrapy context factory patterns
Add warning for context factories not accepting `method` param
Use context factory class name in warning message
 2016-02-24 22:47:44 +01:00
Paul Tremberth 300f162a31 Merge pull request #1813 from redapple/backport-1.1-pr1796 
[backport][1.1] Change default S3 ACL to "private" and allow customization via settings
 2016-02-24 22:39:21 +01:00
Konstantin Lopuhin 9083b68125 Fix documentation for S3_STORE_ACL (now settings.FILES_STORE_S3_ACL) settings: it has nothing to do with feed exporters. 2016-02-24 22:26:22 +01:00
Konstantin Lopuhin b9c6d67365 See #1778 - change default S3 ACL to "private" and allow customization via settings 2016-02-24 22:26:22 +01:00
Paul Tremberth 6d4c51b266 Merge pull request #1809 from scrapy/backport-1.1-1787 
[backport][1.1] Better tracebacks
 2016-02-24 17:54:56 +01:00
Paul Tremberth f0c07a3121 Merge pull request #1810 from scrapy/backport-1.1-pr1800 
[backport][1.1] Add "Content-Length: 0" for body-less HTTP/1.1 POST requests
 2016-02-24 17:54:26 +01:00
Paul Tremberth 6801af2419 Merge pull request #1811 from scrapy/backport-1.1-revert-pr1678 
[backport][1.1] Revert "Cleanup http11 tunneling connection after #1678"
 2016-02-24 17:53:56 +01:00
Paul Tremberth 5e57169394 Merge pull request #1812 from scrapy/backport-1.1-pr1801 
[backport][1.1] Fix SkipTest() message for botocore import test
 2016-02-24 17:52:20 +01:00
Paul Tremberth 9d51e1fc3e Fix SkipTest() message for botocore import test 2016-02-24 17:42:42 +01:00
Paul Tremberth 0b2bb3bc7a Revert "[MRG+1] Cleanup http11 tunneling connection after #1678" 2016-02-24 17:24:54 +01:00
Paul Tremberth 2bcaff97b4 Fix typo in comment 2016-02-24 17:21:08 +01:00
Paul Tremberth e25052e4ef Add "Content-Length: 0" for body-less HTTP/1.1 POST requests 
GH-823 was fixed only for HTTP/1.0 (in GH-1089)
 2016-02-24 17:21:08 +01:00
Mikhail Korobov 374d7b04fc Remove unneeded `raise` (thanks @lopuhin). 2016-02-24 17:14:48 +01:00
Paul Tremberth 50f7b416c5 Minor change on quotes 
Trying to force Travis CI to build
 2016-02-24 17:14:48 +01:00
Mikhail Korobov 0d596c27a9 Preserve tracebacks better. Fixes GH-1760. 2016-02-24 17:14:48 +01:00
Mikhail Korobov 4459ee5d53 TST clean up RunSpiderCommandTest 2016-02-24 17:14:48 +01:00
Paul Tremberth a6bc758db1 Merge pull request #1792 from scrapy/backport-1.1-pr1761 
[backport][1.1.x] Py3 S3 botocore
 2016-02-19 11:17:20 +01:00
Paul Tremberth eafd054092 Merge pull request #1793 from scrapy/backport-1.1-pr1786 
[backport][1.1.x] More liberal Content-Disposition header parsing
 2016-02-18 23:23:16 +01:00
Paul Tremberth 3331cea788 More liberal Content-Disposition header parsing 
Fixes #1782
 2016-02-18 22:56:17 +01:00
Konstantin Lopuhin 25e481fd14 [backport][1.1.x] Py3 S3 botocore 
Originally from https://github.com/scrapy/scrapy/pull/1761
 2016-02-18 22:48:37 +01:00
Mikhail Korobov 7ce32422c4 Merge pull request #1774 from scrapy/backport-1.1 
[MRG+1] [1.1.x][backport] #1766, #1770, #1750, #1662, #1765
 2016-02-15 16:05:31 +05:00
Elias Dorneles f928014d91 add deprecation for pydispatch (thanks for the help @redapple) 2016-02-15 11:40:08 +01:00
Paul Tremberth b1fac01258 Update release notes 2016-02-11 18:44:26 +01:00
Nicolas Pennequin 9cdbe177b7 Update the email doc for the charset argument to send(). 2016-02-09 19:52:06 +01:00
Nicolas Pennequin 72229f92de MailSender.send: allow passing a charset. 
Resolves Issue #348
 2016-02-09 19:52:06 +01:00