RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,689 Commits 50 Branches 132 Tags 142 MiB
Commit Graph

5689 Commits

Author SHA1 Message Date
Mikhail Korobov 2dd1a9e3bc Merge pull request #2094 from redapple/dns-invalid-id 
Catch and ignore certification verification exception for IP-address hosts
 2016-07-13 10:48:08 +00:00
Paul Tremberth 005cf949b8 Change wording of warning + docstring for ScrapyClientTLSOptions 2016-07-13 11:01:30 +02:00
Elias Dorneles 3b5915568b Merge pull request #2120 from kas/kas-patch-1 
(docs) Fix small grammar issue in practices.rst (Supersedes #2119)
 2016-07-12 17:05:39 -03:00
Kenneth Schnall 2489f84d39 Update practices.rst 2016-07-12 15:34:50 -04:00
Paul Tremberth 859bcf4820 Rephrase warning 2016-07-12 17:53:19 +02:00
Paul Tremberth c3109daa72 Merge pull request #2034 from dracony/master 
[MRG+1] Added option to turn off ensure_ascii for JSON exporters
 2016-07-12 17:01:09 +02:00
Dracony 33a39b368f added FEED_EXPORT_ENCODING setting to allow encoding specification 2016-07-12 16:20:17 +02:00
Paul Tremberth 8a22a74e48 Merge pull request #2065 from redapple/octet-stream-no-decompress 
[MRG+2] [HttpCompressionMW] Do not decompress gzip binary/octet-stream responses
 2016-07-11 12:15:38 +02:00
Paul Tremberth 962eb11c73 Simplify regex conditions 2016-07-11 11:30:05 +02:00
Paul Tremberth 778f1cf84c Merge remote-tracking branch 'origin/master' into octet-stream-no-decompress 2016-07-08 18:13:20 +02:00
Elias Dorneles d43a35735a Merge pull request #2050 from Tethik/is_gzipped_fix 
[MRG+1] Is_gzipped for application/x-gzip;charset=utf-8
 2016-07-08 08:47:56 -03:00
Mikhail Korobov b7553d921a Merge pull request #2038 from redapple/canonicalize-idna-failures 
[MRG] Do not fail on canonicalizing URLs with wrong netlocs
 2016-07-08 10:47:54 +06:00
Mikhail Korobov 5f261cf295 Merge pull request #2026 from Digenis/contributing-full_subjects 
[MRG+1] Encourage complete titles in pull requests
 2016-07-08 10:45:22 +06:00
Mikhail Korobov 52a52e2388 Merge pull request #2001 from matveinazaruk/issue-2000 
[MRG+1] Fixed choosing of response class.
 2016-07-08 10:44:24 +06:00
Elias Dorneles 0ab7c1fbf9 Merge pull request #2052 from stummjr/genspider-standalone 
[MRG+1] Enable genspider command outside project folder
 2016-07-06 18:01:30 -03:00
Valdir Stumm Junior 081595a2e4 document new genspider behavior 2016-07-06 15:10:48 -03:00
Valdir Stumm Junior 8987b17730 remove references to Item classes in templates 2016-07-06 15:10:48 -03:00
Valdir Stumm Junior 1779f5feca enable genspider command outside projects 2016-07-06 15:10:48 -03:00
Mikhail Korobov 759a555d28 Merge pull request #2069 from redapple/https-connect-host 
[MRG] Add "Host" header in CONNECT requests to HTTPS proxies
 2016-07-06 21:43:41 +06:00
Paul Tremberth 1737c15826 Merge pull request #2091 from eLRuLL/default_headers_priority 
[MRG+1] prioritize default headers over user agent middlewares
 2016-07-06 17:29:39 +02:00
Paul Tremberth 15d0c89159 Cleanup unused argument 2016-07-06 17:15:21 +02:00
Mikhail Korobov 8ec6c22d60 Merge pull request #2095 from scrapy/testing-deps 
[MRG+1] TST pin pytest-cov to 2.2.1; upgrade pytest
 2016-07-06 20:07:19 +06:00
Mikhail Korobov 4273734744 TST pin pytest-cov to 2.2.1; upgrade pytest 2016-07-06 18:29:49 +05:00
Paul Tremberth 37efdde3e3 Catch and ignore TLS verification exception for IP-address hosts 
Fixes GH-2092
 2016-07-06 14:20:13 +02:00
Raul Gallegos 49ac7de231 prioritize default headers over user agent 2016-07-05 15:38:17 -05:00
Elias Dorneles 441df4c853 Merge pull request #2060 from pawelmhm/windows-python3 
[MRG+1] [docs] add note about windows + python3
 2016-07-05 10:26:33 -03:00
Νικόλαος-Διγενής Καραγιάννης 2a92ffb409 Encourage descriptive PR titles 2016-07-05 14:21:28 +03:00
Pawel Miech b67440dec0 docs on Scrapy on Windows + Python 3 2016-07-04 16:37:51 +02:00
Paul Tremberth 6539277f99 Fix CONNECT request timeout (with an ugly hack) 2016-06-21 17:14:41 +02:00
Paul Tremberth d9343463cb Add "Host" header in CONNECT requests to HTTPS proxies 2016-06-21 13:26:32 +02:00
Paul Tremberth 10a2c46e12 [HttpCompressionMiddleware] Do not decompress binary/octet-stream responses 2016-06-20 16:37:00 +02:00
Pawel Miech 07d1605586 [docs] warnings about windows + python 3 in faq and install 2016-06-17 13:28:51 +02:00
Pawel Miech 73cc066caa [docs] add note about windows + python3 2016-06-17 09:21:58 +02:00
Joakim Uddholm 23f99e98c4 is_gzipped: Separated tests again. 2016-06-14 21:33:51 +02:00
Mikhail Korobov 80c296e091 Merge pull request #2048 from redapple/bs4-faq 
[MRG] Add FAQ entry on using BeautifulSoup in spider callbacks
 2016-06-14 23:45:04 +06:00
Paul Tremberth 1ff9a4828b Do not commit on any HTML parsing speed assumption with BS4+lxml 2016-06-14 19:33:56 +02:00
Joakim Uddholm 36928d897c is_gzipped: improved readability 2016-06-14 15:40:20 +02:00
Paul Tremberth 6cbd92fac1 Add note on how to choose parser with BeautifulSoup 2016-06-14 15:33:34 +02:00
Joakim Uddholm 259426ec99 is_gzipped: Changed to regex to check the content-type header. Also suggested by @redapple. 2016-06-14 14:39:16 +02:00
Joakim Uddholm 124e218a3b Added new testcases suggested by @redapple. 2016-06-14 14:22:18 +02:00
Elias Dorneles edca28326f Merge pull request #2054 from matt-oconnell/tutorial-typos 
Typo fixes
 2016-06-13 17:35:34 -03:00
Matvei Nazaruk b76b374648 Added test for http11 choosing response type without content-type header. 2016-06-13 23:21:38 +03:00
Matt O'Connell 0bb1288969 Typo fixes 2016-06-13 16:21:02 -04:00
Joakim Uddholm 2c98a88a0e Separated tests based on case 2016-06-12 10:49:34 +02:00
Joakim Uddholm db729f5b30 Suggested fix for is_gzipped 2016-06-12 02:26:16 +02:00
Joakim Uddholm 989f6b8843 Test to show bug with is_gzipped and Content-Type: application/gzip;charset. 2016-06-12 01:38:27 +02:00
Paul Tremberth 7978237ed5 Add FAQ entry on using BeautifulSoup in spider callbacks 2016-06-10 17:42:18 +02:00
Paul Tremberth 1aec5200bc Do not fail on canonicalizing URLs with wrong netlocs 
Fixes #2010
 2016-06-08 16:49:33 +02:00
Paul Tremberth b7925e4220 Merge pull request #2008 from foromer4/master 
[MRG+1] Fix issue HttpCompressionMiddleware tries to decode HEAD responses #1899
 2016-06-06 16:14:27 +02:00
Mikhail Korobov 6ff605b488 Merge pull request #1974 from starrify/more-errors-to-retry 
[MRG+1] Added: Retrying scrapy.core.downloader.handlers.http11.TunnelError
 2016-06-06 20:13:24 +06:00