RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,603 Commits 49 Branches 132 Tags 158 MiB
Commit Graph

5603 Commits

Author SHA1 Message Date
Paul Tremberth 8c937ed000 Merge pull request #2103 from redapple/backport-1.1-pr2038 
[backport][1.1] Do not fail on canonicalizing URLs with wrong netlocs (PR #2038)
 2016-07-08 12:46:31 +02:00
Paul Tremberth 53be5dc3e0 Do not fail on canonicalizing URLs with wrong netlocs 
Fixes #2010
 2016-07-08 12:35:56 +02:00
Paul Tremberth 0ef490e9ce Merge pull request #2102 from redapple/backport-1.1-pr2026 
[backport][1.1] Encourage descriptive PR titles (PR #2026)
 2016-07-08 12:31:47 +02:00
Νικόλαος-Διγενής Καραγιάννης 8ffb30fbef Encourage descriptive PR titles 2016-07-08 12:30:54 +02:00
Paul Tremberth 060a94b398 Merge pull request #2101 from redapple/backport-1.1-pr2001 
[backport][1.1] Fixed choosing of response class (PR #2001)
 2016-07-08 12:29:33 +02:00
Matvei Nazaruk eb0c1a3921 Added test for http11 choosing response type without content-type header. 2016-07-08 12:20:22 +02:00
Matvei Nazaruk 5e60d957e9 Fixed choosing of response class based on body. 2016-07-08 12:20:21 +02:00
Paul Tremberth 1c34e491d6 Merge pull request #2100 from redapple/backport-1.1-pr2069 
[backport][1.1] Add "Host" header in CONNECT requests to HTTPS proxies (PR #2069)
 2016-07-08 12:14:14 +02:00
Paul Tremberth 556fc2c334 Cleanup unused argument 2016-07-08 11:59:57 +02:00
Paul Tremberth d89f9df52e Fix CONNECT request timeout (with an ugly hack) 2016-07-08 11:59:56 +02:00
Paul Tremberth 99a93a0967 Add "Host" header in CONNECT requests to HTTPS proxies 2016-07-08 11:59:56 +02:00
Paul Tremberth cedf46c9bf Merge pull request #2099 from redapple/backport-1.1-pr2095 
[backport][1.1] TST pin pytest-cov to 2.2.1; upgrade pytest (PR #2095)
 2016-07-08 11:48:56 +02:00
Paul Tremberth c5b78f7712 Merge pull request #2098 from redapple/backport-1.1-pr2060 
[backport][1.1] [docs] Add note about Windows + Python3 (PR #2060)
 2016-07-08 11:22:53 +02:00
Mikhail Korobov cb8eccf603 TST pin pytest-cov to 2.2.1; upgrade pytest 2016-07-08 11:21:24 +02:00
Pawel Miech dc8db8b35e docs on Scrapy on Windows + Python 3 2016-07-08 11:17:04 +02:00
Pawel Miech adb837d7b7 [docs] warnings about windows + python 3 in faq and install 2016-07-08 11:16:52 +02:00
Pawel Miech d191785194 [docs] add note about windows + python3 2016-07-08 11:16:42 +02:00
Paul Tremberth 7b7e492d28 Do not commit on any HTML parsing speed assumption with BS4+lxml 2016-06-16 22:27:17 +02:00
Paul Tremberth 9db63f588d Add note on how to choose parser with BeautifulSoup 2016-06-16 22:27:04 +02:00
Paul Tremberth 05ce6afa3f Add FAQ entry on using BeautifulSoup in spider callbacks 2016-06-16 22:26:54 +02:00
Matt O'Connell 04d4cca545 Typo fixes 2016-06-16 22:21:41 +02:00
Paul Tremberth a04dbca22d Merge pull request #2041 from scrapy/backport-1.1-pr2015 
[backport][1.1] Use "url" variable in the example (PR #2015)
 2016-06-08 17:39:15 +02:00
Konstantin Lopuhin e6a0baa8fe Use "url" variable in the example 
Instead of hardcoded http://www.example.com: without it url variable is unused and only one request will make it past dupefilter.
 2016-06-08 17:37:26 +02:00
Paul Tremberth 7415a41225 Merge pull request #2040 from scrapy/backport-1.1-pr1995 
[backport][1.1] Add info and example on errbacks (PR #1995)
 2016-06-08 17:34:16 +02:00
Paul Tremberth 1be1b454ef DOC Add info and example on errbacks 2016-06-08 17:32:56 +02:00
Paul Tremberth efaabbdb99 DOC Update copyright notice 2016-06-08 17:31:37 +02:00
Paul Tremberth e1277ff70b Merge pull request #2039 from scrapy/backport-1.1-pr1994 
[backport][1.1][DOC] Use idiomatic .extract_first() (PR #1994)
 2016-06-08 17:30:15 +02:00
Paul Tremberth 239254b282 [DOC][Overview] Use idiomatic .extract_first() 2016-06-08 17:27:55 +02:00
Paul Tremberth 4ea2453ae0 DOC Place FEED_TEMPDIR setting at lexicographical position 2016-06-08 17:24:33 +02:00
Paul Tremberth e4af849edc Bump version: 1.1.0rc4 → 1.1.0 2016-05-11 18:15:47 +02:00
Paul Tremberth 2a9639377a Merge pull request #1984 from scrapy/release-notes-1.1.0 
[MRG+1] Update release notes and date for 1.1.0
 2016-05-11 18:13:11 +02:00
Paul Tremberth bcf902bde0 Update release notes and date for 1.1.0 2016-05-11 17:31:50 +02:00
Panayiotis Lipiridis 28efd8dd35 Github >> GitHub 2016-05-11 17:10:12 +02:00
Paul Tremberth 1e7b216f2f Merge pull request #1983 from redapple/backport-1.1-pr1979 
[backport][1.1] Prevent empty warnings in case of certificate verification failures (PR #1979)
 2016-05-11 16:46:05 +02:00
Paul Tremberth 933bc8b251 Prevent empty warnings in case of certificate verification failures 
Fixes #1976
 2016-05-11 16:29:53 +02:00
Paul Tremberth c9fa29cbd9 Merge 1.0.6 release notes 2016-05-11 14:51:15 +02:00
Paul Tremberth 60d94465e8 Merge pull request #1972 from mgachhui/1.1 
Added Python 3.3+ to readme
 2016-05-11 10:54:04 +02:00
mgachhui 8527ce0c7b Added Python 3.3+ to readme 2016-05-08 18:39:56 +05:30
Paul Tremberth 946fc678f2 Bump w3lib version dependency in setup.py 2016-05-03 13:31:18 +02:00
Paul Tremberth 0f45a70b88 Bump version: 1.1.0rc3 → 1.1.0rc4 2016-04-28 22:51:35 +02:00
Mikhail Korobov 730da2ac17 Merge pull request #1960 from scrapy/release-nots-1.1.0rc4 
Update release notes with recent features and bug fixes.
 2016-04-29 01:19:23 +06:00
Paul Tremberth 651c6f639e Add notes on botocore support and JsonItemExporter opening/closing brackets 2016-04-28 17:54:37 +02:00
Paul Tremberth 15ed4bfd8d Update release notes with recent features and bug fixes. 2016-04-28 15:39:19 +02:00
Paul Tremberth 0890aa6be9 Merge pull request #1958 from redapple/backport-1.1-pr1947 
[backport][1.1] Fix canonicalize_url() on Python 3 and re-enable tests (PR #1947)
 2016-04-27 14:08:08 +02:00
Paul Tremberth a19f3475be Add idempotence tests for canonicalize_url 2016-04-27 13:55:07 +02:00
Paul Tremberth 53f1b044bb Use six.PY2 also for conditional imports 2016-04-27 13:54:56 +02:00
Paul Tremberth 0e65eb33c3 Use six.PY2 instead of six.PY3 for Python version variations 
Also don't test passed encoding against 'utf8';
Just consider that if encoding failed, it must have been another encoding.
 2016-04-27 13:54:45 +02:00
Paul Tremberth 7f94b57b3f Fix canonicalize_url() on Python 3 and re-enable tests 2016-04-27 13:54:36 +02:00
Paul Tremberth 12e10e9862 Merge pull request #1951 from redapple/backport-1.1-pr1950 
[backport][1.1] Place brackets on own lines with JsonItemExporter (PR #1950)
 2016-04-25 15:39:53 +02:00
Patrick Connolly 9c29215bef Place brackets on own lines with JsonItemExporter. 
Placing the opening and closing brackets on their own lines makes it slightly easier to sort lines after the `spider_closed` signal is fired.
 2016-04-25 15:28:41 +02:00