RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,629 Commits 49 Branches 132 Tags 136 MiB
Commit Graph

6629 Commits

Author SHA1 Message Date
Ryan P Kilby 2dfc5d128b Update DEPTH_STATS refs to DEPTH_STATS_VERBOSE 2018-05-17 11:00:21 -04:00
Konstantin Lopuhin bac1e2d47d
Merge pull request #3263 from whalebot-helmsman/no_twisted_18_4_0 
[MRG+1] Blacklist twisted version with regression
 2018-05-17 13:21:42 +03:00
Vostretsov Nikita c5ddfddb7e blacklist twisted version with regression in constraints file 2018-05-17 08:53:42 +00:00
Vostretsov Nikita 0d015e5c0f blacklist twisted version with regression 2018-05-16 09:36:07 +00:00
Konstantin Lopuhin c4f096d3a5
Merge pull request #3224 from lucywang000/better-processors-doc 
[MRG+1] Improve document about functions as processors
 2018-04-26 09:36:39 +03:00
Konstantin Lopuhin f36e1b53d4
Merge pull request #3199 from rhoboro/gcs_acl 
[MRG+1] FilesPipeline supports ACL for Google Cloud Storage
 2018-04-25 19:42:19 +03:00
Lucy Wang 57b0e6b695 improve document about functions as processors 2018-04-19 13:35:46 +08:00
rhoboro 6ef6585b5a update docs 2018-04-13 19:06:29 +09:00
rhoboro 560ee623fd set defalut value "" to FILES_STORE_GCS_ACL 2018-04-13 19:00:27 +09:00
rhoboro 464973489e Using bucket's default object ACL 2018-04-13 12:06:39 +09:00
Konstantin Lopuhin da1256a9c8
Merge pull request #3189 from stav/master 
[MRG+1] Doc: update wording for COOKIES_ENABLED
 2018-04-05 11:42:49 +03:00
rhoboro 74a9c65290 update docs for support gcs acl 2018-04-03 18:20:37 +09:00
rhoboro 5254ac393b added test for gcs policy 2018-04-03 18:06:34 +09:00
rhoboro 8e8994c6b5 add acl support for gcs 2018-04-02 15:36:47 +09:00
Steven Almeroth c6d20bdd82 Doc: update wording for COOKIES_ENABLED 2018-03-27 16:21:07 -04:00
Daniel Graña 6c3970e672
Merge pull request #3153 from virmht/new_bug 
[MRG+1] Fixed bug FormRequest.from_response() clickdata ignores input[type=image]
 2018-03-21 16:32:12 -03:00
Viral Mehta dd064413a4 corrected syntax error in XPath 2018-03-19 19:28:41 +05:30
Viral Mehta a5acc9373f Resolving Comments 2018-03-19 18:19:39 +05:30
Viral Mehta e25e2afe17 Removed unnecessary print statements 2018-03-17 18:20:14 +05:30
Viral Mehta ff5f717f7a Fixed formatting issues 2018-03-17 18:17:48 +05:30
Daniel Graña 6cc6bbb5fc
Merge pull request #3166 from lucywang000/catch-tls-certificate-error 
catch CertificateError in tls verification
 2018-03-14 11:47:58 -03:00
Lucy Wang 2c58da19a6 update docstring of ScrapyClientTLSOptions 2018-03-14 09:27:59 +08:00
Lucy Wang 1a2f0193a3 fix tests on jessie 2018-03-13 19:14:52 +08:00
siulkilulki 6a7cdf9a6c [MRG+1] Add 'flv' to ignored video extensions. (#3165) 2018-03-13 10:35:27 +03:00
Lucy Wang d9e6c73fb3 revert wrong changes 2018-03-13 13:05:37 +08:00
Lucy Wang e487100987 add a test case 2018-03-13 08:59:03 +08:00
Lucy Wang 13a74d77e2 catch CertificateError in tls verification 2018-03-12 22:25:19 +08:00
Viral Mehta 65744c2199 Corrected Test 2018-03-03 20:07:50 +05:30
Viral Mehta d5b7ebcfdc Fixed bug FormRequest.from_response() clickdata ignores input[type=image] 2018-03-03 18:17:49 +05:30
NewUserHa acd2b8d43b [MRG+1] Fix part of issue #3128 - None should not be a valid type for 'url' in Response.follow (#3131) 
* fix one issue of issue#3128

because @kmike posted: 'If url is '', Scrapy should follow the same page, this is an intended behavior.'

*  fix one issue of issue#3128

because @kmike posted: 'If url is '', Scrapy should follow the same page, this is an intended behavior.'
 2018-02-22 03:37:26 +05:00
Cathal Garvey 426da0ed07
Merge pull request #3127 from Anjalizi/patch-2 
Updated contributing.rst
 2018-02-16 18:09:48 +00:00
Anjali Jain bbc2a3569f further edited 2018-02-16 23:33:10 +05:30
Anjali Jain 6954da1366
Updated contributing.rst 
Rectified grammatical errors
 2018-02-15 23:27:40 +05:30
Konstantin Lopuhin f8c688e6f4
Merge pull request #3123 from scrapy/fix-docs-build 
fix docs building on CI
 2018-02-14 18:50:26 +03:00
Mikhail Korobov dc0304fde1 fix docs building with recent sphinx: don't use deprecated sphinx options and imports 2018-02-13 19:47:41 +05:00
Daniel Graña c56f7b3c8d
Merge pull request #3113 from WenbinZhang/master 
[MRG+1] Update robotstxt.py
 2018-02-08 18:22:56 -03:00
Daniel Graña 68e45d32e0
Merge pull request #3115 from scrapy/telnet-log-level 
[MRG+1] use INFO log level to show telnet host/port
 2018-02-08 18:22:36 -03:00
Konstantin Lopuhin 936dbc7bf6
Merge branch 'master' into master 2018-02-08 23:43:29 +03:00
Konstantin Lopuhin 8d2240d066
Merge pull request #3082 from elacuesta/pickle-requests 
[MRG+1] Do not serialize unpickable objects (py3)
 2018-02-08 23:39:37 +03:00
Eugenio Lacuesta 6edd4114c4 Clarify comment about Pyhton versions 2018-02-08 15:47:20 -03:00
Eugenio Lacuesta 0d87e77afe Bump parsel dependency 2018-02-08 15:03:57 -03:00
Eugenio Lacuesta e4558cb27e Update test for unpickable objects 2018-02-08 15:03:57 -03:00
Eugenio Lacuesta a56540877c Do not serialize unpickable objects (py3) 2018-02-08 15:03:57 -03:00
Konstantin Lopuhin eff469245a
Merge pull request #3100 from scrapy/robots-stats 
[MRG+1] more stats for RobotsTxtMiddleware
 2018-02-08 12:01:36 +03:00
Mikhail Korobov 0c374c00fb use INFO log level to show telnet host/port 2018-02-08 05:09:02 +05:00
Wenbin Zhang 4d5e5378bd
Update robotstxt.py 
Add message to IgnoreRequest exception so that it can be detectedin the errbak method of a spider
 2018-02-07 10:59:32 -05:00
Konstantin Lopuhin fd02fa808c
Merge pull request #3096 from jesuslosada/fix-signal-names 
[MRG+1] Fix OS signal names
 2018-02-06 16:21:59 +03:00
Mikhail Korobov 6f264ab190 more stats for RobotsTxtMiddleware 2018-01-30 05:47:28 +05:00
Jesús Losada c1916626c1 Fix OS signal names 2018-01-27 21:24:15 +00:00
Daniel Graña 8be28fe4ca
Merge pull request #3092 from scrapy/fix-tests-for-w3lib-1.19 
Fix tests to account for changes in w3lib 1.19
 2018-01-26 13:52:21 -03:00