RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,564 Commits 49 Branches 132 Tags 136 MiB
Commit Graph

6564 Commits

Author SHA1 Message Date
Mikhail Korobov d4e5671d07 make release docs more readable, add highlights 2017-12-29 07:06:00 +05:00
Mikhail Korobov 45b0e1a0e4 DOC draft 1.5 release notes 2017-12-28 07:33:43 +05:00
Paul Tremberth 461f9daff5 Update release notes for upcoming 1.4.1 version 2017-12-28 05:50:18 +05:00
Raphael a1cc5a63d3 Add mention to dont_merge_cookies in CookiesMiddlewares docs (#2999) (#3030) 
Add mention to dont_merge_cookies in CookiesMiddlewares docs (#2999)
 2017-12-27 17:54:17 -03:00
Daniel Graña 57d04aa960
Merge pull request #2767 from redapple/http-proxy-endpoint-key 
[MRG+1] Use HTTP pool and proper endpoint key for ProxyAgent
 2017-12-26 14:12:47 -03:00
Mikhail Korobov 930f6ed800
Merge pull request #3050 from lopuhin/pypy3 
Add PyPy3 support
 2017-12-25 18:50:58 +05:00
Konstantin Lopuhin bb1f311891 Add PyPy3 support to faq and install doc 2017-12-25 15:46:05 +03:00
Konstantin Lopuhin 041308afe7 Fix get_func_args test for pypy3 
These built-in functions are exposed as methods in PyPy3.
For scrapy this does not matter as:
1) they do not work for CPython at all
2) get_func_args is checked for presense of an argument in scrapy,
   extra "self" does not matter.
But it still makes sense to leave these tests so that we know we
shouldn't use get_func_args for built-in functions/methods.
 2017-12-25 14:48:10 +03:00
Konstantin Lopuhin f71df6f9ad Run tests for PyPy3 2017-12-25 14:48:06 +03:00
Mikhail Korobov 632f1cc073
Merge pull request #3049 from scrapy/trove-classifiers 
[MRG+1] setup.py: mention that we support PyPy. See GH-2213.
 2017-12-25 15:59:56 +05:00
Mikhail Korobov 9f9edeadfc
Merge pull request #3048 from lopuhin/pypy-install-docs 
[MRG+1] Mention PyPy support, add PyPy to install docs
 2017-12-25 15:32:50 +05:00
Mikhail Korobov 1058169f0e setup.py: mention that we support PyPy. See GH-2213. 2017-12-25 15:31:07 +05:00
Konstantin Lopuhin ea41114cf0 Mention PyPy support, add PyPy to install docs 2017-12-25 12:35:04 +03:00
Mikhail Korobov bdc12f3994
Merge pull request #3045 from hugovk/rm-3.3 
[MRG+1] Drop support for EOL Python 3.3
 2017-12-20 21:08:13 +05:00
Hugo cbcf80b98f
Fix typo 
[CI skip]
 2017-12-20 17:34:13 +02:00
Hugo f11c21c6fc Test on Python 3.4 2017-12-20 17:05:56 +02:00
Hugo 44623687ab Drop support for EOL Python 3.3 2017-12-19 17:59:05 +02:00
Daniel Graña 86c322c3a8
Merge pull request #3038 from scrapy/update-contributing-docs 
DOC update "Contributing" docs
 2017-12-13 13:17:28 -03:00
Mikhail Korobov 9aa9dd8d45 DOC mention an easier way to track pull requests locally. 
Thanks @eliasdorneles!
 2017-12-12 19:17:00 +05:00
Mikhail Korobov f716843a66 DOC update "Contributing" docs: 
* suggest Stack Overflow for Scrapy usage questions;
* encourage users to submit test-only pull requests with reproducable examples;
* encourage users to pick up stalled pull requests;
* we don't use AUTHORS file as a main acknowledgement source;
* suggest using Sphinx autodocs extension
 2017-12-12 16:19:43 +05:00
Konstantin Lopuhin a21b800419
Merge pull request #3011 from Jane222/master 
[MRG+1] Issues a warning when user puts a URL into allowed_domains (#2250)
 2017-12-08 15:45:16 +03:00
Jana Cavojska 22c68baf99 url_pattern is now being compiled before entering the loop 2017-12-07 18:38:29 +01:00
Daniel Graña 3cf0332ec3
Merge pull request #2957 from ScrapingLab/add_meta_json_to_parse_command 
[MRG+1] Scrapy Command: add --meta/-m to the "parse" command to pass additional meta data into the request
 2017-11-29 16:26:48 -03:00
Jana Cavojska 454d5e5733 checking for subclass of URLWarning instead of checking error message text when URL in allowed_domains 2017-11-26 20:07:04 +01:00
Jana Cavojska 8ec3b476b0 triggering a warning when user puts URL in allowed_domains now covered by test 2017-11-26 16:36:15 +01:00
IAlwaysBeCoding 6af323d7c8
Fix spelling mistake on scrapy parse command docs 
Fixed spelling mistake from "will be pass" to "will be passed"
 2017-11-26 00:24:52 +01:00
Konstantin Lopuhin 97dc527191
Merge pull request #3020 from Jesse-Bakker/add-fromcrawler-middleware-docs 
[MRG+1] Added from_crawler to middleware docs
 2017-11-24 17:56:57 +03:00
Mikhail Korobov bd16951a05
Merge pull request #3013 from KosayJabre/patch-1 
Separated import statements
 2017-11-24 16:49:56 +05:00
Jesse Bakker 0b14cb44aa Added from_crawler to middleware docs 2017-11-23 15:25:43 +01:00
Jana Cavojska 91ff194d1e looping over allowed_domains directly instead of via index 2017-11-20 21:23:31 +01:00
KosayJabre 5441cc18e4
Separated import statements 
Just separated the import statements. Tiny change - testing GitHub!
 2017-11-19 18:09:38 -04:00
Jana Cavojska 62a6261028 Issues a warning when user puts a URL into allowed_domains (#2250) 2017-11-18 20:03:59 +01:00
IAlwaysBeCoding 846fd83512 removed commented out code, wrapped line to pep-8 and removed backlashes 2017-11-11 18:30:01 -05:00
Daniel Graña b8870ee8a1
Merge pull request #2989 from colinmorris/ItemExporterDocsExample 
[MRG+1] Revise/modernize item exporter example in docs
 2017-11-03 13:15:31 -02:00
Konstantin Lopuhin fd7334518f
Merge pull request #2983 from codeaditya/https-links 
[MRG+1] Use https link in default user agent
 2017-11-02 19:23:46 +03:00
Mikhail Korobov b7a88fadbc
Merge pull request #2991 from lopuhin/pypy-build-2 
Fix PyPy build
 2017-11-02 18:59:20 +05:00
Konstantin Lopuhin abb6d0a1c1 Use portable pypy directly 
They are provided by https://github.com/squeaky-pl/portable-pypy
 2017-11-01 17:52:32 +03:00
colinmorris 23e571e860 fix issues identified in review 2017-10-31 18:08:47 -04:00
colinmorris 8a7552370d revise/modernize item exporter example in docs 2017-10-31 17:14:53 -04:00
Aditya df7e0a4315 Use https link in default user agent 2017-10-28 23:37:44 +05:30
Mikhail Korobov 108f8c4fd2
Merge pull request #2982 from codeaditya/https-links 
Fix broken links and use https links wherever possible
 2017-10-28 21:45:05 +05:00
Aditya dae7b1cdd0 Migrate all subdomains on readthedocs.org to readthedocs.io 2017-10-28 16:53:32 +05:30
Aditya 97d047a055 Fix link for Tox 2017-10-28 16:48:41 +05:30
Aditya 23c7437e46 Fix link for 'XPath and XSLT with lxml' 2017-10-28 16:34:49 +05:30
Aditya 9d9d83a8c3 Use https links wherever possible 2017-10-28 16:24:40 +05:30
Mikhail Korobov 79df51aae1 Merge pull request #2978 from codeaditya/https-links 
Use https for external links wherever possible in docs
 2017-10-27 14:14:37 +05:00
Daniel Graña 1d9c8f5295 Merge pull request #2921 from revolter/hotfix/disable-logging 
[MRG+1] Add option to disable automatic log handler install
 2017-10-26 15:04:18 -03:00
Aditya 9dd680d5c9 Use https for external links wherever possible in docs 2017-10-26 23:33:45 +05:30
Daniel Graña dc4b36a89e Merge pull request #2952 from NoExitTV/Unhelpful-log-message-from-core.downloader.handlers.http11 
[MRG+1] Changed unhelpful log message from core.downloader.handlers.http11
 2017-10-26 11:38:43 -03:00
Daniel Graña 4f8b45ec83 Merge pull request #2929 from djunzu/add_m4v_to_ignored_extensions 
[MRG+1] Add m4v extension to IGNORED_EXTENSIONS in LinkExtractor.
 2017-10-26 11:35:32 -03:00