RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,291 Commits 49 Branches 132 Tags 155 MiB
Commit Graph

9291 Commits

Author SHA1 Message Date
Lucy Wang 57b0e6b695 improve document about functions as processors 2018-04-19 13:35:46 +08:00
rhoboro 6ef6585b5a update docs 2018-04-13 19:06:29 +09:00
rhoboro 560ee623fd set defalut value "" to FILES_STORE_GCS_ACL 2018-04-13 19:00:27 +09:00
rhoboro 464973489e Using bucket's default object ACL 2018-04-13 12:06:39 +09:00
Konstantin Lopuhin da1256a9c8
Merge pull request #3189 from stav/master 
[MRG+1] Doc: update wording for COOKIES_ENABLED
 2018-04-05 11:42:49 +03:00
grammy-jiang cb76b88331 fix a mistake in topic spider-middleware.rst 2018-04-04 05:56:05 -04:00
rhoboro 74a9c65290 update docs for support gcs acl 2018-04-03 18:20:37 +09:00
rhoboro 5254ac393b added test for gcs policy 2018-04-03 18:06:34 +09:00
rhoboro 8e8994c6b5 add acl support for gcs 2018-04-02 15:36:47 +09:00
Steven Almeroth c6d20bdd82 Doc: update wording for COOKIES_ENABLED 2018-03-27 16:21:07 -04:00
Daniel Graña 6c3970e672
Merge pull request #3153 from virmht/new_bug 
[MRG+1] Fixed bug FormRequest.from_response() clickdata ignores input[type=image]
 2018-03-21 16:32:12 -03:00
Viral Mehta dd064413a4 corrected syntax error in XPath 2018-03-19 19:28:41 +05:30
Viral Mehta a5acc9373f Resolving Comments 2018-03-19 18:19:39 +05:30
Viral Mehta e25e2afe17 Removed unnecessary print statements 2018-03-17 18:20:14 +05:30
Viral Mehta ff5f717f7a Fixed formatting issues 2018-03-17 18:17:48 +05:30
Daniel Graña 6cc6bbb5fc
Merge pull request #3166 from lucywang000/catch-tls-certificate-error 
catch CertificateError in tls verification
 2018-03-14 11:47:58 -03:00
Lucy Wang 2c58da19a6 update docstring of ScrapyClientTLSOptions 2018-03-14 09:27:59 +08:00
Lucy Wang 1a2f0193a3 fix tests on jessie 2018-03-13 19:14:52 +08:00
siulkilulki 6a7cdf9a6c [MRG+1] Add 'flv' to ignored video extensions. (#3165) 2018-03-13 10:35:27 +03:00
Lucy Wang d9e6c73fb3 revert wrong changes 2018-03-13 13:05:37 +08:00
Lucy Wang e487100987 add a test case 2018-03-13 08:59:03 +08:00
Lucy Wang 13a74d77e2 catch CertificateError in tls verification 2018-03-12 22:25:19 +08:00
Arvind Prasanna 412f852602 A few typo fixes and some grammatical enhancements 2018-03-06 23:58:27 -05:00
Patience Shyu f10a43d562 [WIP] Install Twisted from branch for py3.7 2018-03-05 11:43:39 +01:00
Patience Shyu 5d1f5245f2 [WIP] Install Twisted from branch to bypass syntax issue 2018-03-05 11:14:50 +01:00
Patience Shyu ca7d79c29a Install Twisted from branch to bypass syntax issue 2018-03-05 10:46:51 +01:00
Viral Mehta 65744c2199 Corrected Test 2018-03-03 20:07:50 +05:30
Viral Mehta d5b7ebcfdc Fixed bug FormRequest.from_response() clickdata ignores input[type=image] 2018-03-03 18:17:49 +05:30
nctl144 4c05441450 add ftp to the scheme list 2018-03-03 00:00:03 -05:00
Patience Shyu fab68ff626 Use 3.7-dev version for travis 2018-03-02 17:05:14 +01:00
Patience Shyu aca2655c12 [WIP] Run tests for Python 3.7 2018-03-02 14:57:39 +01:00
NewUserHa acd2b8d43b [MRG+1] Fix part of issue #3128 - None should not be a valid type for 'url' in Response.follow (#3131) 
* fix one issue of issue#3128

because @kmike posted: 'If url is '', Scrapy should follow the same page, this is an intended behavior.'

*  fix one issue of issue#3128

because @kmike posted: 'If url is '', Scrapy should follow the same page, this is an intended behavior.'
 2018-02-22 03:37:26 +05:00
Cathal Garvey 426da0ed07
Merge pull request #3127 from Anjalizi/patch-2 
Updated contributing.rst
 2018-02-16 18:09:48 +00:00
Anjali Jain bbc2a3569f further edited 2018-02-16 23:33:10 +05:30
Anjali Jain 6954da1366
Updated contributing.rst 
Rectified grammatical errors
 2018-02-15 23:27:40 +05:30
Konstantin Lopuhin f8c688e6f4
Merge pull request #3123 from scrapy/fix-docs-build 
fix docs building on CI
 2018-02-14 18:50:26 +03:00
Mikhail Korobov dc0304fde1 fix docs building with recent sphinx: don't use deprecated sphinx options and imports 2018-02-13 19:47:41 +05:00
Daniel Graña c56f7b3c8d
Merge pull request #3113 from WenbinZhang/master 
[MRG+1] Update robotstxt.py
 2018-02-08 18:22:56 -03:00
Daniel Graña 68e45d32e0
Merge pull request #3115 from scrapy/telnet-log-level 
[MRG+1] use INFO log level to show telnet host/port
 2018-02-08 18:22:36 -03:00
Konstantin Lopuhin 936dbc7bf6
Merge branch 'master' into master 2018-02-08 23:43:29 +03:00
Konstantin Lopuhin 8d2240d066
Merge pull request #3082 from elacuesta/pickle-requests 
[MRG+1] Do not serialize unpickable objects (py3)
 2018-02-08 23:39:37 +03:00
Eugenio Lacuesta 6edd4114c4 Clarify comment about Pyhton versions 2018-02-08 15:47:20 -03:00
Eugenio Lacuesta 0d87e77afe Bump parsel dependency 2018-02-08 15:03:57 -03:00
Eugenio Lacuesta e4558cb27e Update test for unpickable objects 2018-02-08 15:03:57 -03:00
Eugenio Lacuesta a56540877c Do not serialize unpickable objects (py3) 2018-02-08 15:03:57 -03:00
Konstantin Lopuhin eff469245a
Merge pull request #3100 from scrapy/robots-stats 
[MRG+1] more stats for RobotsTxtMiddleware
 2018-02-08 12:01:36 +03:00
Mikhail Korobov 0c374c00fb use INFO log level to show telnet host/port 2018-02-08 05:09:02 +05:00
Wenbin Zhang 4d5e5378bd
Update robotstxt.py 
Add message to IgnoreRequest exception so that it can be detectedin the errbak method of a spider
 2018-02-07 10:59:32 -05:00
Konstantin Lopuhin fd02fa808c
Merge pull request #3096 from jesuslosada/fix-signal-names 
[MRG+1] Fix OS signal names
 2018-02-06 16:21:59 +03:00
Mikhail Korobov 6f264ab190 more stats for RobotsTxtMiddleware 2018-01-30 05:47:28 +05:00