RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,291 Commits 49 Branches 132 Tags 153 MiB
Commit Graph

9291 Commits

Author SHA1 Message Date
John de la Garza b02d26fae8 rel_has_nofollow: remove redundant if statement 2019-02-15 16:54:33 -08:00
Victor Torres 9fed6fcb51 trigger tests 2019-02-14 16:59:51 -02:00
Victor Torres 9b8ba4c383 try to import botocore before runing some tests 2019-02-14 16:20:56 -02:00
Victor Torres ea8be627d1 botocore is not supported on debian jessie 2019-02-13 20:32:36 -02:00
Victor Torres dc0b643832 refactoring tests to avoid mocking private method 2019-02-13 19:44:50 -02:00
Victor Torres b4d132b9f0 setting botocore version as described in debian jessie website 
https://packages.debian.org/en/jessie/python-botocore
 2019-02-13 19:21:14 -02:00
Pedro Sousa 430e939248
Added missing AWS Settings for ImagesPipeline 2019-02-13 19:59:40 +00:00
Adrián Chaves 50bf4c60c4 Document that the main entry point of downloader and spider middlewares is from_crawler() 2019-02-13 17:39:20 +01:00
Pedro Sousa 04ccf79e38
A different S3 Endpoint URL is now possible when uploading images 2019-02-13 15:39:45 +00:00
Victor Torres 984e706fd2 using blank string instead of None as default value as proposed by @kmike 2019-02-12 12:26:57 -02:00
Victor Torres c2dede27bd reduce code with simple ternary operator 2019-02-12 12:22:05 -02:00
Victor Torres 7c9f0bd86c using named params with optional amazon s3 params 2019-02-12 12:19:30 -02:00
Adrián Chaves 03e61b9908 Check that spidercls arguments in scrapy.crawler classes are not spider objects 2019-02-12 14:46:27 +01:00
Victor Torres 1eac2a163c simplifying how we deal with threads.deferToThread calls 2019-02-08 16:50:39 -02:00
Victor Torres f824f5b2d1 testing public method store instead of private method _store_in_thread 
need to mock deferToThread function
 2019-02-08 15:51:23 -02:00
Victor Torres cfd183a9d1 no need to use get here since we're defining a default value in default_settings.py 2019-02-08 15:51:23 -02:00
Victor Torres ceae356e62 add FEED_STORAGE_S3_ACL to default_settings.py file 2019-02-08 15:51:23 -02:00
Victor Torres 079af889e7 also testing without botocore 2019-02-08 15:51:23 -02:00
Victor Torres dbeb088eea trying to fix jessie testenv by adding botocore to requirements and fixing its version 2019-02-08 15:51:23 -02:00
Victor Torres e25b9a2323 calling it feeds instead of objects 2019-02-08 15:51:23 -02:00
Victor Torres 7b83ed7c5e remove typo 2019-02-08 15:51:23 -02:00
Victor Torres e0f34be383 update docs 2019-02-08 15:51:23 -02:00
Victor Torres 126207fb7b PEP8: use short name for mock method 2019-02-08 15:51:23 -02:00
Victor Torres ad83ffdf1f refactoring 2019-02-08 15:51:23 -02:00
Victor Torres 013568097d add FEED_STORAGE_S3_ACL setting 2019-02-08 15:51:23 -02:00
Júlio César Batista cb5f800b0f Adding documentation about Google Cloud Storage Feed Export 2019-02-08 11:26:33 -02:00
Júlio César Batista 2bbbd02bda Adding an option to set ACL while uploading the blob to GCS 2019-02-08 09:45:10 -02:00
Júlio César Batista 4a53de165a Sorted schemas alphabetically 2019-02-08 09:09:56 -02:00
Júlio César Batista fc6809b024 Add gcs schema to FEED_STORAGES_BASE 2019-02-08 09:08:54 -02:00
Júlio César Batista 1bb6c4154c Turning into instance attributes 2019-02-08 09:04:01 -02:00
Raul Gallegos b364bfb68b
Merge pull request #3615 from Gallaecio/form-request-example 
Indicate that users must implement their own authentication result check
 2019-02-06 23:11:51 -05:00
Adrián Chaves 38af090f4d Indicate that users must implement their own authentication result check 
The example of form-based login could lead some users to think its authentication result
check was final. See https://stackoverflow.com/a/54410966/939364

This change should make it more obvious that users are expected to implement their
own logic to check whether authentication worked or not.
 2019-02-04 11:17:58 +01:00
Júlio César Batista a4059851e7 Refactoring tests 2019-01-31 18:29:15 -02:00
Júlio César Batista 5a55c4269d Adding GCSFeedStorage 2019-01-31 17:20:29 -02:00
Daniel Graña 65d631329a Be consistent with domain used for links to documentation website 2019-01-31 01:28:53 -03:00
Daniel Graña 88326cd8be Set release date to 1.6.0 2019-01-31 01:16:45 -03:00
Daniel Graña b8594353d0 Bump version: 1.5.0 → 1.6.0 2019-01-30 18:00:40 -03:00
Daniel Graña 1312174607
Merge pull request #3549 from scrapy/release-notes-1.6 
1.6 release notes
 2019-01-30 15:03:49 -03:00
Mikhail Korobov 91791cd329 DOC final changelog cleanups 2019-01-30 12:49:36 -03:00
Mikhail Korobov 2c8c8b2dd8 DOC fix after bad merge - remove duplicate entries in changelog 2019-01-30 12:49:36 -03:00
Mikhail Korobov 0fc9d705c2 DOC mention that telnet security improvements happened in 1.5.2 2019-01-30 12:49:36 -03:00
Mikhail Korobov 4cf4dd1d3e DOC add recent changes to changelog 2019-01-30 12:49:36 -03:00
Mikhail Korobov 638469f9ef DOC extract_first/extract matches get/getall better 
Thanks @Gallaecio!
 2019-01-30 12:49:36 -03:00
Mikhail Korobov e479f5aa15 DOC update changelog 
* changes from recently merged pull requests
* more highlights
* re-organized headers
* Selector API changes
 2019-01-30 12:49:36 -03:00
Mikhail Korobov 706910790b [wip] draft 1.6 release notes 2019-01-30 12:49:36 -03:00
Eugenio Lacuesta e3e804cfb0 Styling nitpick :-) 2019-01-28 15:10:34 -03:00
Daniel Graña b5026b842c
Merge pull request #3544 from joaquingx/fix-item-pipeline-x 
Fix item-pipeline example
 2019-01-28 10:44:19 -03:00
Daniel Graña c4f57608d6
Merge pull request #3588 from ejulio/dupefilter-debug 
[MRG+1] Adding requests referer to RFPDupeFilter log messages
 2019-01-28 10:22:48 -03:00
Harry Moreno 8fca98616a fix grammar 2019-01-28 10:19:31 -03:00
Daniel Graña 0e5b6755ba
Merge pull request #3605 from morenoh149/patch-2 
fix grammar
 2019-01-28 10:18:45 -03:00