RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,606 Commits 49 Branches 132 Tags 141 MiB
Commit Graph

6606 Commits

Author SHA1 Message Date
Viral Mehta dd064413a4 corrected syntax error in XPath 2018-03-19 19:28:41 +05:30
Viral Mehta a5acc9373f Resolving Comments 2018-03-19 18:19:39 +05:30
Viral Mehta e25e2afe17 Removed unnecessary print statements 2018-03-17 18:20:14 +05:30
Viral Mehta ff5f717f7a Fixed formatting issues 2018-03-17 18:17:48 +05:30
Viral Mehta 65744c2199 Corrected Test 2018-03-03 20:07:50 +05:30
Viral Mehta d5b7ebcfdc Fixed bug FormRequest.from_response() clickdata ignores input[type=image] 2018-03-03 18:17:49 +05:30
NewUserHa acd2b8d43b [MRG+1] Fix part of issue #3128 - None should not be a valid type for 'url' in Response.follow (#3131) 
* fix one issue of issue#3128

because @kmike posted: 'If url is '', Scrapy should follow the same page, this is an intended behavior.'

*  fix one issue of issue#3128

because @kmike posted: 'If url is '', Scrapy should follow the same page, this is an intended behavior.'
 2018-02-22 03:37:26 +05:00
Cathal Garvey 426da0ed07
Merge pull request #3127 from Anjalizi/patch-2 
Updated contributing.rst
 2018-02-16 18:09:48 +00:00
Anjali Jain bbc2a3569f further edited 2018-02-16 23:33:10 +05:30
Anjali Jain 6954da1366
Updated contributing.rst 
Rectified grammatical errors
 2018-02-15 23:27:40 +05:30
Konstantin Lopuhin f8c688e6f4
Merge pull request #3123 from scrapy/fix-docs-build 
fix docs building on CI
 2018-02-14 18:50:26 +03:00
Mikhail Korobov dc0304fde1 fix docs building with recent sphinx: don't use deprecated sphinx options and imports 2018-02-13 19:47:41 +05:00
Daniel Graña c56f7b3c8d
Merge pull request #3113 from WenbinZhang/master 
[MRG+1] Update robotstxt.py
 2018-02-08 18:22:56 -03:00
Daniel Graña 68e45d32e0
Merge pull request #3115 from scrapy/telnet-log-level 
[MRG+1] use INFO log level to show telnet host/port
 2018-02-08 18:22:36 -03:00
Konstantin Lopuhin 936dbc7bf6
Merge branch 'master' into master 2018-02-08 23:43:29 +03:00
Konstantin Lopuhin 8d2240d066
Merge pull request #3082 from elacuesta/pickle-requests 
[MRG+1] Do not serialize unpickable objects (py3)
 2018-02-08 23:39:37 +03:00
Eugenio Lacuesta 6edd4114c4 Clarify comment about Pyhton versions 2018-02-08 15:47:20 -03:00
Eugenio Lacuesta 0d87e77afe Bump parsel dependency 2018-02-08 15:03:57 -03:00
Eugenio Lacuesta e4558cb27e Update test for unpickable objects 2018-02-08 15:03:57 -03:00
Eugenio Lacuesta a56540877c Do not serialize unpickable objects (py3) 2018-02-08 15:03:57 -03:00
Konstantin Lopuhin eff469245a
Merge pull request #3100 from scrapy/robots-stats 
[MRG+1] more stats for RobotsTxtMiddleware
 2018-02-08 12:01:36 +03:00
Mikhail Korobov 0c374c00fb use INFO log level to show telnet host/port 2018-02-08 05:09:02 +05:00
Wenbin Zhang 4d5e5378bd
Update robotstxt.py 
Add message to IgnoreRequest exception so that it can be detectedin the errbak method of a spider
 2018-02-07 10:59:32 -05:00
Konstantin Lopuhin fd02fa808c
Merge pull request #3096 from jesuslosada/fix-signal-names 
[MRG+1] Fix OS signal names
 2018-02-06 16:21:59 +03:00
Mikhail Korobov 6f264ab190 more stats for RobotsTxtMiddleware 2018-01-30 05:47:28 +05:00
Jesús Losada c1916626c1 Fix OS signal names 2018-01-27 21:24:15 +00:00
Daniel Graña 8be28fe4ca
Merge pull request #3092 from scrapy/fix-tests-for-w3lib-1.19 
Fix tests to account for changes in w3lib 1.19
 2018-01-26 13:52:21 -03:00
Mikhail Korobov ba15b63ed6 TST fix tests to account for changes in w3lib 1.19 2018-01-26 02:11:49 +05:00
Yash Sharma 1d1581266c Changed some documentations (#3089) 
DOC typo fix in defer_fail docstring
 2018-01-26 01:12:17 +05:00
Mikhail Korobov 7c9e32213d
Merge pull request #3059 from jesuslosada/fix-typo 
Fix typo in comment
 2018-01-11 01:34:11 +05:00
Mikhail Korobov 786144e0c7
Merge pull request #3058 from jesuslosada/fix-link 
Fix link in news.rst
 2018-01-11 01:33:38 +05:00
Daniel Graña b170e9fb96
Merge pull request #2609 from otobrglez/extending-s3-files-store 
S3FilesStore support for other S3 providers (botocore options)
 2018-01-07 19:16:11 -05:00
Jesús Losada 61c0b14782 Fix typo in comment 2018-01-01 16:03:55 +00:00
Jesús Losada a0836b8fd9 Fix link in news.rst 2018-01-01 15:59:38 +00:00
Daniel Graña 2dee191374
Merge branch 'master' into extending-s3-files-store 2017-12-31 16:44:38 -03:00
Mikhail Korobov aa83e159c9 Bump version: 1.4.0 → 1.5.0 2017-12-30 02:09:52 +05:00
Mikhail Korobov d07fe11981 set release date 2017-12-30 02:09:41 +05:00
Daniel Graña 9b4d6a40a6
Merge pull request #3053 from scrapy/release-notes-1.5 
Release notes for the upcoming 1.5.0 version
 2017-12-29 11:57:04 -03:00
Mikhail Korobov c107059ef8 DOC fix rst syntax 2017-12-29 07:07:43 +05:00
Mikhail Korobov d4e5671d07 make release docs more readable, add highlights 2017-12-29 07:06:00 +05:00
Mikhail Korobov 45b0e1a0e4 DOC draft 1.5 release notes 2017-12-28 07:33:43 +05:00
Paul Tremberth 461f9daff5 Update release notes for upcoming 1.4.1 version 2017-12-28 05:50:18 +05:00
Raphael a1cc5a63d3 Add mention to dont_merge_cookies in CookiesMiddlewares docs (#2999) (#3030) 
Add mention to dont_merge_cookies in CookiesMiddlewares docs (#2999)
 2017-12-27 17:54:17 -03:00
Daniel Graña 57d04aa960
Merge pull request #2767 from redapple/http-proxy-endpoint-key 
[MRG+1] Use HTTP pool and proper endpoint key for ProxyAgent
 2017-12-26 14:12:47 -03:00
Mikhail Korobov 930f6ed800
Merge pull request #3050 from lopuhin/pypy3 
Add PyPy3 support
 2017-12-25 18:50:58 +05:00
Konstantin Lopuhin bb1f311891 Add PyPy3 support to faq and install doc 2017-12-25 15:46:05 +03:00
Konstantin Lopuhin 041308afe7 Fix get_func_args test for pypy3 
These built-in functions are exposed as methods in PyPy3.
For scrapy this does not matter as:
1) they do not work for CPython at all
2) get_func_args is checked for presense of an argument in scrapy,
   extra "self" does not matter.
But it still makes sense to leave these tests so that we know we
shouldn't use get_func_args for built-in functions/methods.
 2017-12-25 14:48:10 +03:00
Konstantin Lopuhin f71df6f9ad Run tests for PyPy3 2017-12-25 14:48:06 +03:00
Mikhail Korobov 632f1cc073
Merge pull request #3049 from scrapy/trove-classifiers 
[MRG+1] setup.py: mention that we support PyPy. See GH-2213.
 2017-12-25 15:59:56 +05:00
Mikhail Korobov 9f9edeadfc
Merge pull request #3048 from lopuhin/pypy-install-docs 
[MRG+1] Mention PyPy support, add PyPy to install docs
 2017-12-25 15:32:50 +05:00