RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,649 Commits 49 Branches 132 Tags 133 MiB
Commit Graph

5649 Commits

Author SHA1 Message Date
Luar Roji 5f010724af Updated pypi name 2016-06-17 01:11:06 -03:00
Luar Roji 8e03ef7720 Test 2016-06-17 01:08:16 -03:00
Luar Roji a63af8e756 Add travis configs for debian package building 2016-06-17 01:03:27 -03:00
Mikhail Korobov 80c296e091 Merge pull request #2048 from redapple/bs4-faq 
[MRG] Add FAQ entry on using BeautifulSoup in spider callbacks
 2016-06-14 23:45:04 +06:00
Paul Tremberth 1ff9a4828b Do not commit on any HTML parsing speed assumption with BS4+lxml 2016-06-14 19:33:56 +02:00
Paul Tremberth 6cbd92fac1 Add note on how to choose parser with BeautifulSoup 2016-06-14 15:33:34 +02:00
Elias Dorneles edca28326f Merge pull request #2054 from matt-oconnell/tutorial-typos 
Typo fixes
 2016-06-13 17:35:34 -03:00
Matt O'Connell 0bb1288969 Typo fixes 2016-06-13 16:21:02 -04:00
Paul Tremberth 7978237ed5 Add FAQ entry on using BeautifulSoup in spider callbacks 2016-06-10 17:42:18 +02:00
Paul Tremberth b7925e4220 Merge pull request #2008 from foromer4/master 
[MRG+1] Fix issue HttpCompressionMiddleware tries to decode HEAD responses #1899
 2016-06-06 16:14:27 +02:00
Mikhail Korobov 6ff605b488 Merge pull request #1974 from starrify/more-errors-to-retry 
[MRG+1] Added: Retrying scrapy.core.downloader.handlers.http11.TunnelError
 2016-06-06 20:13:24 +06:00
Paul Tremberth f929853cc9 Merge pull request #2015 from lopuhin/patch-1 
[MRG+1] Use "url" variable in the example
 2016-06-06 16:11:29 +02:00
Konstantin Lopuhin da8d0eadd6 Use "url" variable in the example 
Instead of hardcoded http://www.example.com: without it url variable is unused and only one request will make it past dupefilter.
 2016-05-27 10:47:57 +03:00
omer c8ec79d959 fix issue with '' in python 3 2016-05-25 06:56:06 +03:00
omer ffa77e1ac6 Do not decode of head response (2) 2016-05-24 14:24:29 +03:00
omer ce48bae533 Do not decode of head response 2016-05-23 19:52:38 +03:00
Elias Dorneles d3ced85ec3 Merge pull request #1995 from scrapy/docs-errback 
DOC Add info and example on errbacks
 2016-05-18 13:18:47 -03:00
Paul Tremberth b3367c7acd DOC Add info and example on errbacks 2016-05-18 18:00:09 +02:00
Paul Tremberth c2c8036a97 DOC Update copyright notice 2016-05-18 16:52:09 +02:00
Elias Dorneles ed4b9afbc3 Merge pull request #1994 from scrapy/overview-code 
[DOC][Overview] Use idiomatic .extract_first()
 2016-05-17 13:41:53 -03:00
Paul Tremberth 149c4cd41e [DOC][Overview] Use idiomatic .extract_first() 2016-05-17 17:53:40 +02:00
Paul Tremberth 9b3c72cb37 DOC Place FEED_TEMPDIR setting at lexicographical position 2016-05-17 16:25:57 +02:00
Paul Tremberth 0700e437f2 Update release notes and date for 1.1.0 2016-05-11 20:39:35 +02:00
mgachhui aecc23d24a Added Python 3.3+ to readme 2016-05-11 20:39:23 +02:00
Elias Dorneles 0e2b49c8c2 [MRG+1] Prevent empty warnings in case of certificate verification failures 
[MRG+1] Prevent empty warnings in case of certificate verification failures
 2016-05-11 10:51:45 -03:00
Paul Tremberth bd3e71a1ea Merge pull request #1980 from redapple/release-notes-1.0.6-merge 
Merge 1.0.6 release notes
 2016-05-11 14:50:08 +02:00
Paul Tremberth d62654dfda Merge 1.0.6 release notes 2016-05-11 14:39:55 +02:00
Paul Tremberth 137197b98b Prevent empty warnings in case of certificate verification failures 
Fixes #1976
 2016-05-11 13:12:35 +02:00
Paul Tremberth 6aa85aee2a Merge pull request #1975 from lipis/github 
Github >> GitHub
 2016-05-09 16:43:40 +02:00
Panayiotis Lipiridis 3f8be374c3 Github >> GitHub 2016-05-09 16:15:18 +02:00
Pengyu CHEN 334ee40e3d Added: Retrying scrapy.core.downloader.handlers.http11.TunnelError 2016-05-09 20:43:54 +08:00
Mikhail Korobov 064f05349f Merge pull request #1962 from redapple/w3lib-dep 
Bump w3lib version dependency in setup.py
 2016-04-29 16:55:10 +06:00
Paul Tremberth c8bff23751 Bump w3lib version dependency in setup.py 2016-04-29 10:29:37 +02:00
Paul Tremberth cb38f3896b Add notes on botocore support and JsonItemExporter opening/closing brackets 2016-04-28 22:34:17 +02:00
Paul Tremberth dc9de7524f Update release notes with recent features and bug fixes. 2016-04-28 22:34:07 +02:00
Mikhail Korobov dbef7e2b45 Merge pull request #1947 from scrapy/canonicalize-url 
[MRG+1] Fix canonicalize_url() on Python 3 and re-enable tests
 2016-04-27 01:32:42 +06:00
Paul Tremberth 0e11b3e6f0 Add idempotence tests for canonicalize_url 2016-04-26 20:03:17 +02:00
Paul Tremberth efbe75ea5e Use six.PY2 also for conditional imports 2016-04-26 16:14:15 +02:00
Paul Tremberth 25401fd322 Use six.PY2 instead of six.PY3 for Python version variations 
Also don't test passed encoding against 'utf8';
Just consider that if encoding failed, it must have been another encoding.
 2016-04-26 15:12:00 +02:00
Paul Tremberth cf716ea200 Merge pull request #1950 from patcon/patch-1 
[MRG+1] Place brackets on own lines with JsonItemExporter
 2016-04-25 15:21:21 +02:00
Patrick Connolly 19a4a0adf9 Place brackets on own lines with JsonItemExporter. 
Placing the opening and closing brackets on their own lines makes it slightly easier to sort lines after the `spider_closed` signal is fired.
 2016-04-24 14:55:08 -04:00
Paul Tremberth 8efa98794d Allow more pre-releases with bumpversion 2016-04-21 16:51:17 +02:00
Paul Tremberth 68dedf54cb Fix canonicalize_url() on Python 3 and re-enable tests 2016-04-21 14:50:59 +02:00
Mikhail Korobov 73a5571c60 Merge pull request #1923 from scrapy/request-new-safe_url_string 
[MRG+1] Use newer w3lib.url.safe_url_string() and re-enable HTTP request tests
 2016-04-20 19:33:45 +06:00
Paul Tremberth 417279cceb Bump up w3lib requirement to v1.14.2 2016-04-20 15:12:19 +02:00
Mikhail Korobov 679a680638 Merge pull request #1933 from scrapy/cert-verif-ignore 
Ignore HTTPS certificate verification failures
 2016-04-20 18:57:33 +06:00
Paul Tremberth cd979ace40 Add HTTPS tests with non-hostname-maching server certificate 2016-04-20 14:42:03 +02:00
Mikhail Korobov 3735eb8e5f Merge pull request #1912 from redapple/https-proxy-pool-key 
[MRG+1] Fix HTTP Pool key for HTTPS proxy tunneled connections (CONNECT method)
 2016-04-20 17:32:04 +06:00
Mikhail Korobov a8b49472b5 Merge pull request #1938 from redapple/https-proxy-connect-sni 
Set SNI properly when using CONNECT
 2016-04-20 17:30:13 +06:00
Paul Tremberth dcea11a70c Fall back to no-SNi context factory is Twisted<14 is used 2016-04-19 10:41:13 +02:00