RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,589 Commits 49 Branches 132 Tags 137 MiB
Commit Graph

5589 Commits

Author SHA1 Message Date
Pawel Miech dc8db8b35e docs on Scrapy on Windows + Python 3 2016-07-08 11:17:04 +02:00
Pawel Miech adb837d7b7 [docs] warnings about windows + python 3 in faq and install 2016-07-08 11:16:52 +02:00
Pawel Miech d191785194 [docs] add note about windows + python3 2016-07-08 11:16:42 +02:00
Paul Tremberth 7b7e492d28 Do not commit on any HTML parsing speed assumption with BS4+lxml 2016-06-16 22:27:17 +02:00
Paul Tremberth 9db63f588d Add note on how to choose parser with BeautifulSoup 2016-06-16 22:27:04 +02:00
Paul Tremberth 05ce6afa3f Add FAQ entry on using BeautifulSoup in spider callbacks 2016-06-16 22:26:54 +02:00
Matt O'Connell 04d4cca545 Typo fixes 2016-06-16 22:21:41 +02:00
Paul Tremberth a04dbca22d Merge pull request #2041 from scrapy/backport-1.1-pr2015 
[backport][1.1] Use "url" variable in the example (PR #2015)
 2016-06-08 17:39:15 +02:00
Konstantin Lopuhin e6a0baa8fe Use "url" variable in the example 
Instead of hardcoded http://www.example.com: without it url variable is unused and only one request will make it past dupefilter.
 2016-06-08 17:37:26 +02:00
Paul Tremberth 7415a41225 Merge pull request #2040 from scrapy/backport-1.1-pr1995 
[backport][1.1] Add info and example on errbacks (PR #1995)
 2016-06-08 17:34:16 +02:00
Paul Tremberth 1be1b454ef DOC Add info and example on errbacks 2016-06-08 17:32:56 +02:00
Paul Tremberth efaabbdb99 DOC Update copyright notice 2016-06-08 17:31:37 +02:00
Paul Tremberth e1277ff70b Merge pull request #2039 from scrapy/backport-1.1-pr1994 
[backport][1.1][DOC] Use idiomatic .extract_first() (PR #1994)
 2016-06-08 17:30:15 +02:00
Paul Tremberth 239254b282 [DOC][Overview] Use idiomatic .extract_first() 2016-06-08 17:27:55 +02:00
Paul Tremberth 4ea2453ae0 DOC Place FEED_TEMPDIR setting at lexicographical position 2016-06-08 17:24:33 +02:00
Paul Tremberth e4af849edc Bump version: 1.1.0rc4 → 1.1.0 2016-05-11 18:15:47 +02:00
Paul Tremberth 2a9639377a Merge pull request #1984 from scrapy/release-notes-1.1.0 
[MRG+1] Update release notes and date for 1.1.0
 2016-05-11 18:13:11 +02:00
Paul Tremberth bcf902bde0 Update release notes and date for 1.1.0 2016-05-11 17:31:50 +02:00
Panayiotis Lipiridis 28efd8dd35 Github >> GitHub 2016-05-11 17:10:12 +02:00
Paul Tremberth 1e7b216f2f Merge pull request #1983 from redapple/backport-1.1-pr1979 
[backport][1.1] Prevent empty warnings in case of certificate verification failures (PR #1979)
 2016-05-11 16:46:05 +02:00
Paul Tremberth 933bc8b251 Prevent empty warnings in case of certificate verification failures 
Fixes #1976
 2016-05-11 16:29:53 +02:00
Paul Tremberth c9fa29cbd9 Merge 1.0.6 release notes 2016-05-11 14:51:15 +02:00
Paul Tremberth 60d94465e8 Merge pull request #1972 from mgachhui/1.1 
Added Python 3.3+ to readme
 2016-05-11 10:54:04 +02:00
mgachhui 8527ce0c7b Added Python 3.3+ to readme 2016-05-08 18:39:56 +05:30
Paul Tremberth 946fc678f2 Bump w3lib version dependency in setup.py 2016-05-03 13:31:18 +02:00
Paul Tremberth 0f45a70b88 Bump version: 1.1.0rc3 → 1.1.0rc4 2016-04-28 22:51:35 +02:00
Mikhail Korobov 730da2ac17 Merge pull request #1960 from scrapy/release-nots-1.1.0rc4 
Update release notes with recent features and bug fixes.
 2016-04-29 01:19:23 +06:00
Paul Tremberth 651c6f639e Add notes on botocore support and JsonItemExporter opening/closing brackets 2016-04-28 17:54:37 +02:00
Paul Tremberth 15ed4bfd8d Update release notes with recent features and bug fixes. 2016-04-28 15:39:19 +02:00
Paul Tremberth 0890aa6be9 Merge pull request #1958 from redapple/backport-1.1-pr1947 
[backport][1.1] Fix canonicalize_url() on Python 3 and re-enable tests (PR #1947)
 2016-04-27 14:08:08 +02:00
Paul Tremberth a19f3475be Add idempotence tests for canonicalize_url 2016-04-27 13:55:07 +02:00
Paul Tremberth 53f1b044bb Use six.PY2 also for conditional imports 2016-04-27 13:54:56 +02:00
Paul Tremberth 0e65eb33c3 Use six.PY2 instead of six.PY3 for Python version variations 
Also don't test passed encoding against 'utf8';
Just consider that if encoding failed, it must have been another encoding.
 2016-04-27 13:54:45 +02:00
Paul Tremberth 7f94b57b3f Fix canonicalize_url() on Python 3 and re-enable tests 2016-04-27 13:54:36 +02:00
Paul Tremberth 12e10e9862 Merge pull request #1951 from redapple/backport-1.1-pr1950 
[backport][1.1] Place brackets on own lines with JsonItemExporter (PR #1950)
 2016-04-25 15:39:53 +02:00
Patrick Connolly 9c29215bef Place brackets on own lines with JsonItemExporter. 
Placing the opening and closing brackets on their own lines makes it slightly easier to sort lines after the `spider_closed` signal is fired.
 2016-04-25 15:28:41 +02:00
Paul Tremberth 19b585361e Allow more pre-releases with bumpversion 2016-04-21 16:53:17 +02:00
Paul Tremberth 11b22a35dd Merge pull request #1945 from redapple/backport-1.1-pr1923 
[backport][1.1] Use newer w3lib.url.safe_url_string() (PR #1923)
 2016-04-20 17:55:55 +02:00
Paul Tremberth 04977aa8d5 Bump up w3lib requirement to v1.14.2 2016-04-20 17:37:16 +02:00
Paul Tremberth 08232a3f82 Use newer w3lib.url.safe_url_string() and re-enable HTTP request tests 2016-04-20 17:36:58 +02:00
Paul Tremberth d61a71b723 Merge pull request #1944 from redapple/backport-1.1-pr1933 
[backport][1.1] Ignore HTTPS certificate verification failures (PR #1933)
 2016-04-20 17:35:21 +02:00
Paul Tremberth 33a7caac90 Add HTTPS tests with non-hostname-maching server certificate 2016-04-20 17:25:25 +02:00
Paul Tremberth ebbec3cd39 Catch VerificationError but keep the rest of ClientTLSOptions 2016-04-20 17:25:07 +02:00
Paul Tremberth 2f54f9680d Ignore HTTPS certificate verification failures 
Fixes #1930
 2016-04-20 17:24:49 +02:00
Paul Tremberth 6e4742755e Merge pull request #1943 from redapple/backport-1.1-pr1912 
[backport][1.1] Fix HTTP Pool key for HTTPS proxy tunneled connections (PR #1912)
 2016-04-20 17:19:39 +02:00
Paul Tremberth 7d3a529dbc Fix HTTP Pool key for HTTPS proxy tunneled connections (CONNECT method) 2016-04-20 17:02:47 +02:00
Paul Tremberth f70d538d10 Merge pull request #1942 from scrapy/backport-1.1-pr1938 
[backport][1.1] Set SNI properly when using CONNECT (PR #1938)
 2016-04-20 17:01:34 +02:00
Paul Tremberth 352d0d36ee Fall back to no-SNi context factory is Twisted<14 is used 2016-04-20 16:48:32 +02:00
Paul Tremberth 5cb071b466 Set SNI properly when using CONNECT 2016-04-20 16:48:20 +02:00
Paul Tremberth bd6b3c0608 Reference StackOverflow's "minimal, complete, and verifiable example" guide 2016-04-12 16:13:05 +02:00