RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,635 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

5635 Commits

Author SHA1 Message Date
omer c8ec79d959 fix issue with '' in python 3 2016-05-25 06:56:06 +03:00
omer ffa77e1ac6 Do not decode of head response (2) 2016-05-24 14:24:29 +03:00
omer ce48bae533 Do not decode of head response 2016-05-23 19:52:38 +03:00
Elias Dorneles d3ced85ec3 Merge pull request #1995 from scrapy/docs-errback 
DOC Add info and example on errbacks
 2016-05-18 13:18:47 -03:00
Paul Tremberth b3367c7acd DOC Add info and example on errbacks 2016-05-18 18:00:09 +02:00
Paul Tremberth c2c8036a97 DOC Update copyright notice 2016-05-18 16:52:09 +02:00
Elias Dorneles ed4b9afbc3 Merge pull request #1994 from scrapy/overview-code 
[DOC][Overview] Use idiomatic .extract_first()
 2016-05-17 13:41:53 -03:00
Paul Tremberth 149c4cd41e [DOC][Overview] Use idiomatic .extract_first() 2016-05-17 17:53:40 +02:00
Paul Tremberth 9b3c72cb37 DOC Place FEED_TEMPDIR setting at lexicographical position 2016-05-17 16:25:57 +02:00
Paul Tremberth 0700e437f2 Update release notes and date for 1.1.0 2016-05-11 20:39:35 +02:00
mgachhui aecc23d24a Added Python 3.3+ to readme 2016-05-11 20:39:23 +02:00
Elias Dorneles 0e2b49c8c2 [MRG+1] Prevent empty warnings in case of certificate verification failures 
[MRG+1] Prevent empty warnings in case of certificate verification failures
 2016-05-11 10:51:45 -03:00
Paul Tremberth bd3e71a1ea Merge pull request #1980 from redapple/release-notes-1.0.6-merge 
Merge 1.0.6 release notes
 2016-05-11 14:50:08 +02:00
Paul Tremberth d62654dfda Merge 1.0.6 release notes 2016-05-11 14:39:55 +02:00
Paul Tremberth 137197b98b Prevent empty warnings in case of certificate verification failures 
Fixes #1976
 2016-05-11 13:12:35 +02:00
Paul Tremberth 6aa85aee2a Merge pull request #1975 from lipis/github 
Github >> GitHub
 2016-05-09 16:43:40 +02:00
Panayiotis Lipiridis 3f8be374c3 Github >> GitHub 2016-05-09 16:15:18 +02:00
Mikhail Korobov 064f05349f Merge pull request #1962 from redapple/w3lib-dep 
Bump w3lib version dependency in setup.py
 2016-04-29 16:55:10 +06:00
Paul Tremberth c8bff23751 Bump w3lib version dependency in setup.py 2016-04-29 10:29:37 +02:00
Paul Tremberth cb38f3896b Add notes on botocore support and JsonItemExporter opening/closing brackets 2016-04-28 22:34:17 +02:00
Paul Tremberth dc9de7524f Update release notes with recent features and bug fixes. 2016-04-28 22:34:07 +02:00
Mikhail Korobov dbef7e2b45 Merge pull request #1947 from scrapy/canonicalize-url 
[MRG+1] Fix canonicalize_url() on Python 3 and re-enable tests
 2016-04-27 01:32:42 +06:00
Paul Tremberth 0e11b3e6f0 Add idempotence tests for canonicalize_url 2016-04-26 20:03:17 +02:00
Paul Tremberth efbe75ea5e Use six.PY2 also for conditional imports 2016-04-26 16:14:15 +02:00
Paul Tremberth 25401fd322 Use six.PY2 instead of six.PY3 for Python version variations 
Also don't test passed encoding against 'utf8';
Just consider that if encoding failed, it must have been another encoding.
 2016-04-26 15:12:00 +02:00
Paul Tremberth cf716ea200 Merge pull request #1950 from patcon/patch-1 
[MRG+1] Place brackets on own lines with JsonItemExporter
 2016-04-25 15:21:21 +02:00
Patrick Connolly 19a4a0adf9 Place brackets on own lines with JsonItemExporter. 
Placing the opening and closing brackets on their own lines makes it slightly easier to sort lines after the `spider_closed` signal is fired.
 2016-04-24 14:55:08 -04:00
Paul Tremberth 8efa98794d Allow more pre-releases with bumpversion 2016-04-21 16:51:17 +02:00
Paul Tremberth 68dedf54cb Fix canonicalize_url() on Python 3 and re-enable tests 2016-04-21 14:50:59 +02:00
Mikhail Korobov 73a5571c60 Merge pull request #1923 from scrapy/request-new-safe_url_string 
[MRG+1] Use newer w3lib.url.safe_url_string() and re-enable HTTP request tests
 2016-04-20 19:33:45 +06:00
Paul Tremberth 417279cceb Bump up w3lib requirement to v1.14.2 2016-04-20 15:12:19 +02:00
Mikhail Korobov 679a680638 Merge pull request #1933 from scrapy/cert-verif-ignore 
Ignore HTTPS certificate verification failures
 2016-04-20 18:57:33 +06:00
Paul Tremberth cd979ace40 Add HTTPS tests with non-hostname-maching server certificate 2016-04-20 14:42:03 +02:00
Mikhail Korobov 3735eb8e5f Merge pull request #1912 from redapple/https-proxy-pool-key 
[MRG+1] Fix HTTP Pool key for HTTPS proxy tunneled connections (CONNECT method)
 2016-04-20 17:32:04 +06:00
Mikhail Korobov a8b49472b5 Merge pull request #1938 from redapple/https-proxy-connect-sni 
Set SNI properly when using CONNECT
 2016-04-20 17:30:13 +06:00
Paul Tremberth dcea11a70c Fall back to no-SNi context factory is Twisted<14 is used 2016-04-19 10:41:13 +02:00
Paul Tremberth d6760dbaac Set SNI properly when using CONNECT 2016-04-18 18:30:01 +02:00
Paul Tremberth 25ee023561 Catch VerificationError but keep the rest of ClientTLSOptions 2016-04-14 17:19:55 +02:00
Paul Tremberth a087d2593a Ignore HTTPS certificate verification failures 
Fixes #1930
 2016-04-14 15:59:41 +02:00
Mikhail Korobov ba6dbad1e0 Merge pull request #1926 from redapple/faq-mcve 
Reference StackOverflow's "minimal, complete, and verifiable example" guide
 2016-04-12 18:35:34 +06:00
Paul Tremberth 2849ebf4c6 Reference StackOverflow's "minimal, complete, and verifiable example" guide 2016-04-12 14:07:33 +02:00
Paul Tremberth 47bfac1669 Merge pull request #1924 from lopuhin/faq-fix-py3 
[MRG+1] Fix FAQ entry about python versions support (add Python 3.3+)
 2016-04-12 12:15:17 +02:00
Konstantin Lopuhin 1ec49c2ada Fix FAQ entry about python versions support 2016-04-12 11:48:57 +03:00
Paul Tremberth d42a98d3b5 Use newer w3lib.url.safe_url_string() and re-enable HTTP request tests 2016-04-12 00:33:25 +02:00
Paul Tremberth 86e4442d17 Fix HTTP Pool key for HTTPS proxy tunneled connections (CONNECT method) 2016-04-11 18:28:38 +02:00
Paul Tremberth 10d03ee419 Merge pull request #1916 from nblock/patch-1 
Fix spelling mistake
 2016-04-11 15:12:24 +02:00
nblock a3557dd34d Fix spelling mistake 2016-04-11 14:06:57 +02:00
Mikhail Korobov ff80e1c381 Merge pull request #1913 from redapple/link-extractor-new-w3lib 
Fix link extractor tests for non-ASCII characters from latin1 document
 2016-04-09 22:05:59 +06:00
Paul Tremberth 7b5243a263 Add link extractor test for non-ASCII characters in query part of URL 2016-04-09 15:15:01 +02:00
Paul Tremberth 1656fbcffa Fix link extractor tests for non-ASCII characters from latin1 document 
URL path component should use UTF-8 before percent-encoding (that's what
browsers do when you open scrapy/tests/sample_data/link_extractor/linkextractor_latin1.html
and follow the links)
This matches current w3lib v1.14.1
 2016-04-08 23:25:50 +02:00