RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,575 Commits 49 Branches 132 Tags 134 MiB
Commit Graph

5575 Commits

Author SHA1 Message Date
Paul Tremberth 4ea2453ae0 DOC Place FEED_TEMPDIR setting at lexicographical position 2016-06-08 17:24:33 +02:00
Paul Tremberth e4af849edc Bump version: 1.1.0rc4 → 1.1.0 2016-05-11 18:15:47 +02:00
Paul Tremberth 2a9639377a Merge pull request #1984 from scrapy/release-notes-1.1.0 
[MRG+1] Update release notes and date for 1.1.0
 2016-05-11 18:13:11 +02:00
Paul Tremberth bcf902bde0 Update release notes and date for 1.1.0 2016-05-11 17:31:50 +02:00
Panayiotis Lipiridis 28efd8dd35 Github >> GitHub 2016-05-11 17:10:12 +02:00
Paul Tremberth 1e7b216f2f Merge pull request #1983 from redapple/backport-1.1-pr1979 
[backport][1.1] Prevent empty warnings in case of certificate verification failures (PR #1979)
 2016-05-11 16:46:05 +02:00
Paul Tremberth 933bc8b251 Prevent empty warnings in case of certificate verification failures 
Fixes #1976
 2016-05-11 16:29:53 +02:00
Paul Tremberth c9fa29cbd9 Merge 1.0.6 release notes 2016-05-11 14:51:15 +02:00
Paul Tremberth 60d94465e8 Merge pull request #1972 from mgachhui/1.1 
Added Python 3.3+ to readme
 2016-05-11 10:54:04 +02:00
mgachhui 8527ce0c7b Added Python 3.3+ to readme 2016-05-08 18:39:56 +05:30
Paul Tremberth 946fc678f2 Bump w3lib version dependency in setup.py 2016-05-03 13:31:18 +02:00
Paul Tremberth 0f45a70b88 Bump version: 1.1.0rc3 → 1.1.0rc4 2016-04-28 22:51:35 +02:00
Mikhail Korobov 730da2ac17 Merge pull request #1960 from scrapy/release-nots-1.1.0rc4 
Update release notes with recent features and bug fixes.
 2016-04-29 01:19:23 +06:00
Paul Tremberth 651c6f639e Add notes on botocore support and JsonItemExporter opening/closing brackets 2016-04-28 17:54:37 +02:00
Paul Tremberth 15ed4bfd8d Update release notes with recent features and bug fixes. 2016-04-28 15:39:19 +02:00
Paul Tremberth 0890aa6be9 Merge pull request #1958 from redapple/backport-1.1-pr1947 
[backport][1.1] Fix canonicalize_url() on Python 3 and re-enable tests (PR #1947)
 2016-04-27 14:08:08 +02:00
Paul Tremberth a19f3475be Add idempotence tests for canonicalize_url 2016-04-27 13:55:07 +02:00
Paul Tremberth 53f1b044bb Use six.PY2 also for conditional imports 2016-04-27 13:54:56 +02:00
Paul Tremberth 0e65eb33c3 Use six.PY2 instead of six.PY3 for Python version variations 
Also don't test passed encoding against 'utf8';
Just consider that if encoding failed, it must have been another encoding.
 2016-04-27 13:54:45 +02:00
Paul Tremberth 7f94b57b3f Fix canonicalize_url() on Python 3 and re-enable tests 2016-04-27 13:54:36 +02:00
Paul Tremberth 12e10e9862 Merge pull request #1951 from redapple/backport-1.1-pr1950 
[backport][1.1] Place brackets on own lines with JsonItemExporter (PR #1950)
 2016-04-25 15:39:53 +02:00
Patrick Connolly 9c29215bef Place brackets on own lines with JsonItemExporter. 
Placing the opening and closing brackets on their own lines makes it slightly easier to sort lines after the `spider_closed` signal is fired.
 2016-04-25 15:28:41 +02:00
Paul Tremberth 19b585361e Allow more pre-releases with bumpversion 2016-04-21 16:53:17 +02:00
Paul Tremberth 11b22a35dd Merge pull request #1945 from redapple/backport-1.1-pr1923 
[backport][1.1] Use newer w3lib.url.safe_url_string() (PR #1923)
 2016-04-20 17:55:55 +02:00
Paul Tremberth 04977aa8d5 Bump up w3lib requirement to v1.14.2 2016-04-20 17:37:16 +02:00
Paul Tremberth 08232a3f82 Use newer w3lib.url.safe_url_string() and re-enable HTTP request tests 2016-04-20 17:36:58 +02:00
Paul Tremberth d61a71b723 Merge pull request #1944 from redapple/backport-1.1-pr1933 
[backport][1.1] Ignore HTTPS certificate verification failures (PR #1933)
 2016-04-20 17:35:21 +02:00
Paul Tremberth 33a7caac90 Add HTTPS tests with non-hostname-maching server certificate 2016-04-20 17:25:25 +02:00
Paul Tremberth ebbec3cd39 Catch VerificationError but keep the rest of ClientTLSOptions 2016-04-20 17:25:07 +02:00
Paul Tremberth 2f54f9680d Ignore HTTPS certificate verification failures 
Fixes #1930
 2016-04-20 17:24:49 +02:00
Paul Tremberth 6e4742755e Merge pull request #1943 from redapple/backport-1.1-pr1912 
[backport][1.1] Fix HTTP Pool key for HTTPS proxy tunneled connections (PR #1912)
 2016-04-20 17:19:39 +02:00
Paul Tremberth 7d3a529dbc Fix HTTP Pool key for HTTPS proxy tunneled connections (CONNECT method) 2016-04-20 17:02:47 +02:00
Paul Tremberth f70d538d10 Merge pull request #1942 from scrapy/backport-1.1-pr1938 
[backport][1.1] Set SNI properly when using CONNECT (PR #1938)
 2016-04-20 17:01:34 +02:00
Paul Tremberth 352d0d36ee Fall back to no-SNi context factory is Twisted<14 is used 2016-04-20 16:48:32 +02:00
Paul Tremberth 5cb071b466 Set SNI properly when using CONNECT 2016-04-20 16:48:20 +02:00
Paul Tremberth bd6b3c0608 Reference StackOverflow's "minimal, complete, and verifiable example" guide 2016-04-12 16:13:05 +02:00
Konstantin Lopuhin db61d54663 Fix FAQ entry about python versions support 2016-04-12 16:12:46 +02:00
Paul Tremberth f1020effc5 Merge pull request #1920 from scrapy/backport-1.1-pr1891 
[backport][1.1] Change Files/ImagesPipelines class attributes to instance attributes (PR #1891)
 2016-04-12 16:07:07 +02:00
Paul Tremberth 9f1d215b4a Merge pull request #1919 from scrapy/backport-1.1-pr1881 
[backport][1.1] Remove duplicate code now handled by newer w3lib (PR #1881)
 2016-04-12 16:06:35 +02:00
djunzu fd98e3c031 Update docs. 
modified:   docs/topics/media-pipeline.rst
 2016-04-11 18:06:03 +02:00
djunzu 6d56faf11f Change ImagesPipeline class attributes to instance attributes. 
modified:   scrapy/pipelines/images.py
 2016-04-11 18:06:03 +02:00
djunzu bb68cc120b Change FilesPipeline class attributes to instance attributes. 
modified:   scrapy/pipelines/files.py
	modified:   tests/test_pipeline_files.py
 2016-04-11 18:06:03 +02:00
djunzu 6d8184953a Move default settings to settings/default_settings.py. 
modified:   scrapy/pipelines/files.py
	modified:   scrapy/pipelines/images.py
	modified:   scrapy/settings/default_settings.py
 2016-04-11 18:06:02 +02:00
djunzu 6627e05e92 Add tests. 
modified:   tests/test_pipeline_files.py
	modified:   tests/test_pipeline_images.py
 2016-04-11 18:06:02 +02:00
nyov 1a62d87e35 Remove duplicate code now handled by newer w3lib 
see f3029a6a10
 2016-04-11 17:39:17 +02:00
nblock 00792557ec Fix spelling mistake 2016-04-11 17:36:31 +02:00
Paul Tremberth d30dfada11 Merge pull request #1921 from scrapy/backport-1.1-pr1913 
[backport][1.1] Fix link extractor tests for non-ASCII characters from latin1 document (PR #1913)
 2016-04-11 17:34:47 +02:00
Paul Tremberth a1a799249b Add link extractor test for non-ASCII characters in query part of URL 2016-04-11 16:55:56 +02:00
Paul Tremberth 6672883df8 Fix link extractor tests for non-ASCII characters from latin1 document 
URL path component should use UTF-8 before percent-encoding (that's what
browsers do when you open scrapy/tests/sample_data/link_extractor/linkextractor_latin1.html
and follow the links)
This matches current w3lib v1.14.1
 2016-04-11 16:55:42 +02:00
Pengyu CHEN 160bcbacfc Added: Making it case-insensitive when extracting sitemap URLs from a robots.txt 2016-04-04 15:25:49 +02:00