RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,393 Commits 49 Branches 132 Tags 153 MiB
Commit Graph

10393 Commits

Author SHA1 Message Date
Anubhav Patel e9cd4ee03a fix list alignment and line width 2019-03-10 20:37:56 +05:30
Anubhav Patel 82049e9c41 make suggested changes. 2019-03-10 20:17:07 +05:30
Adrián Chaves 91aec8b3bb
Update developer-tools.rst 
Fixes #3674
 2019-03-08 18:19:30 +01:00
Adrián Chaves b1063d9b2c Use the description from README.rst on index.rst 2019-03-08 17:22:49 +01:00
Adrián Chaves e108e3adbf Clarify the documentation of DEPTH_PRIORITY further 2019-03-08 15:13:11 +01:00
Adrián Chaves 4ef38d925e Remove the unexisting retry_complete signal from the documentation 2019-03-08 14:21:00 +01:00
Adrián Chaves 120007c057 Add a FAQ entry on how to deal with long lists of allowed domains 2019-03-08 13:53:47 +01:00
Anubhav Patel 924b67437b move api docs to source code 2019-03-07 16:40:59 +05:30
Anubhav Patel 184def1060 fix a link inside docs 2019-03-07 00:09:10 +05:30
Maram Sumanth f7bf3abbd0
Modified code 2019-03-06 14:10:03 +05:30
Anubhav Patel 82d239f3b1 docs for scrapy.logformatter 2019-03-06 12:08:09 +05:30
Maram Sumanth 7da460b793
Update form.py 2019-03-04 17:25:15 +05:30
Maram Sumanth 8831fafabc
Update test_http_request.py 2019-03-04 15:42:48 +05:30
Maram Sumanth fdf03a6d0d
correcting tests 2019-03-04 15:12:44 +05:30
Maram Sumanth d75b61b96a
Update test_http_request.py 2019-03-04 15:07:12 +05:30
Maram Sumanth 6eca6f92c6
Update form.py 2019-03-04 14:59:34 +05:30
Adrián Chaves 75d6f56c8a Switch from ` to `` where inline code formatting is desired 2019-03-01 16:56:58 +01:00
Adrián Chaves 858f5be747 backwards → backward (adj.) 2019-03-01 16:10:23 +01:00
Matthieu Grandrie e3b15252c8 New constructor arg *restrict_text* for FilteringLinkExtractor. 
Same as allow and deny args, it holds a string, a regex or an iterable of. Links whose text don't match one of the regex are filtered out.
DOC restrict_text in LxmlLinkExtractor
 2019-02-28 17:21:17 +01:00
Júlio César Batista 0bb3d8ca93 Updating Google Cloud Storage scheme to gs instead of gcs 2019-02-27 18:41:01 -03:00
Daniel Graña c72ab1d4ba
Merge pull request #3625 from Ordepsousa/master 
[MRG+1] A different S3 Endpoint URL is now possible when uploading images
 2019-02-20 11:39:21 -03:00
John de la Garza b02d26fae8 rel_has_nofollow: remove redundant if statement 2019-02-15 16:54:33 -08:00
Victor Torres 9fed6fcb51 trigger tests 2019-02-14 16:59:51 -02:00
Victor Torres 9b8ba4c383 try to import botocore before runing some tests 2019-02-14 16:20:56 -02:00
Victor Torres ea8be627d1 botocore is not supported on debian jessie 2019-02-13 20:32:36 -02:00
Victor Torres dc0b643832 refactoring tests to avoid mocking private method 2019-02-13 19:44:50 -02:00
Victor Torres b4d132b9f0 setting botocore version as described in debian jessie website 
https://packages.debian.org/en/jessie/python-botocore
 2019-02-13 19:21:14 -02:00
Pedro Sousa 430e939248
Added missing AWS Settings for ImagesPipeline 2019-02-13 19:59:40 +00:00
Adrián Chaves 50bf4c60c4 Document that the main entry point of downloader and spider middlewares is from_crawler() 2019-02-13 17:39:20 +01:00
Pedro Sousa 04ccf79e38
A different S3 Endpoint URL is now possible when uploading images 2019-02-13 15:39:45 +00:00
Victor Torres 984e706fd2 using blank string instead of None as default value as proposed by @kmike 2019-02-12 12:26:57 -02:00
Victor Torres c2dede27bd reduce code with simple ternary operator 2019-02-12 12:22:05 -02:00
Victor Torres 7c9f0bd86c using named params with optional amazon s3 params 2019-02-12 12:19:30 -02:00
Adrián Chaves 03e61b9908 Check that spidercls arguments in scrapy.crawler classes are not spider objects 2019-02-12 14:46:27 +01:00
Victor Torres 1eac2a163c simplifying how we deal with threads.deferToThread calls 2019-02-08 16:50:39 -02:00
Victor Torres f824f5b2d1 testing public method store instead of private method _store_in_thread 
need to mock deferToThread function
 2019-02-08 15:51:23 -02:00
Victor Torres cfd183a9d1 no need to use get here since we're defining a default value in default_settings.py 2019-02-08 15:51:23 -02:00
Victor Torres ceae356e62 add FEED_STORAGE_S3_ACL to default_settings.py file 2019-02-08 15:51:23 -02:00
Victor Torres 079af889e7 also testing without botocore 2019-02-08 15:51:23 -02:00
Victor Torres dbeb088eea trying to fix jessie testenv by adding botocore to requirements and fixing its version 2019-02-08 15:51:23 -02:00
Victor Torres e25b9a2323 calling it feeds instead of objects 2019-02-08 15:51:23 -02:00
Victor Torres 7b83ed7c5e remove typo 2019-02-08 15:51:23 -02:00
Victor Torres e0f34be383 update docs 2019-02-08 15:51:23 -02:00
Victor Torres 126207fb7b PEP8: use short name for mock method 2019-02-08 15:51:23 -02:00
Victor Torres ad83ffdf1f refactoring 2019-02-08 15:51:23 -02:00
Victor Torres 013568097d add FEED_STORAGE_S3_ACL setting 2019-02-08 15:51:23 -02:00
Júlio César Batista cb5f800b0f Adding documentation about Google Cloud Storage Feed Export 2019-02-08 11:26:33 -02:00
Júlio César Batista 2bbbd02bda Adding an option to set ACL while uploading the blob to GCS 2019-02-08 09:45:10 -02:00
Júlio César Batista 4a53de165a Sorted schemas alphabetically 2019-02-08 09:09:56 -02:00
Júlio César Batista fc6809b024 Add gcs schema to FEED_STORAGES_BASE 2019-02-08 09:08:54 -02:00