RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,888 Commits 49 Branches 132 Tags 144 MiB
Commit Graph

6888 Commits

Author SHA1 Message Date
Eugenio Lacuesta 174ba3cc56 Rule.process_request: update docs 2019-03-22 19:16:18 -03:00
Eugenio Lacuesta 56929e77d9 Rule.process_request: deprecate the use of functions taking only one argument 2019-03-22 18:34:55 -03:00
Eugenio Lacuesta bbf24b7a1c Rule.process_request: use scrapy.utils.python.get_func_args 2019-03-22 18:02:31 -03:00
Eugenio Lacuesta 92bbc5290d Rule.process_request - Renaming 2019-03-16 13:19:38 +00:00
Eugenio Lacuesta 01ed605d02 PEP8 changes to test_spider.py 2019-03-15 16:54:14 +00:00
Eugenio Lacuesta 83ec947fe7 Rule.process_request defaults to None in the docs 2019-03-13 11:23:51 +00:00
Eugenio Lacuesta b30ca379b6 Rule.process_request: docs 2019-03-13 11:02:51 +00:00
Eugenio Lacuesta 22fda61d62 Rule.process_request: tests 2019-03-13 10:54:38 +00:00
Eugenio Lacuesta 43fd622968 Rule.process_request: optionally take a Response object 2019-03-13 10:43:44 +00:00
Daniel Graña c72ab1d4ba
Merge pull request #3625 from Ordepsousa/master 
[MRG+1] A different S3 Endpoint URL is now possible when uploading images
 2019-02-20 11:39:21 -03:00
Pedro Sousa 430e939248
Added missing AWS Settings for ImagesPipeline 2019-02-13 19:59:40 +00:00
Pedro Sousa 04ccf79e38
A different S3 Endpoint URL is now possible when uploading images 2019-02-13 15:39:45 +00:00
Raul Gallegos b364bfb68b
Merge pull request #3615 from Gallaecio/form-request-example 
Indicate that users must implement their own authentication result check
 2019-02-06 23:11:51 -05:00
Adrián Chaves 38af090f4d Indicate that users must implement their own authentication result check 
The example of form-based login could lead some users to think its authentication result
check was final. See https://stackoverflow.com/a/54410966/939364

This change should make it more obvious that users are expected to implement their
own logic to check whether authentication worked or not.
 2019-02-04 11:17:58 +01:00
Daniel Graña 65d631329a Be consistent with domain used for links to documentation website 2019-01-31 01:28:53 -03:00
Daniel Graña 88326cd8be Set release date to 1.6.0 2019-01-31 01:16:45 -03:00
Daniel Graña b8594353d0 Bump version: 1.5.0 → 1.6.0 2019-01-30 18:00:40 -03:00
Daniel Graña 1312174607
Merge pull request #3549 from scrapy/release-notes-1.6 
1.6 release notes
 2019-01-30 15:03:49 -03:00
Mikhail Korobov 91791cd329 DOC final changelog cleanups 2019-01-30 12:49:36 -03:00
Mikhail Korobov 2c8c8b2dd8 DOC fix after bad merge - remove duplicate entries in changelog 2019-01-30 12:49:36 -03:00
Mikhail Korobov 0fc9d705c2 DOC mention that telnet security improvements happened in 1.5.2 2019-01-30 12:49:36 -03:00
Mikhail Korobov 4cf4dd1d3e DOC add recent changes to changelog 2019-01-30 12:49:36 -03:00
Mikhail Korobov 638469f9ef DOC extract_first/extract matches get/getall better 
Thanks @Gallaecio!
 2019-01-30 12:49:36 -03:00
Mikhail Korobov e479f5aa15 DOC update changelog 
* changes from recently merged pull requests
* more highlights
* re-organized headers
* Selector API changes
 2019-01-30 12:49:36 -03:00
Mikhail Korobov 706910790b [wip] draft 1.6 release notes 2019-01-30 12:49:36 -03:00
Daniel Graña b5026b842c
Merge pull request #3544 from joaquingx/fix-item-pipeline-x 
Fix item-pipeline example
 2019-01-28 10:44:19 -03:00
Daniel Graña c4f57608d6
Merge pull request #3588 from ejulio/dupefilter-debug 
[MRG+1] Adding requests referer to RFPDupeFilter log messages
 2019-01-28 10:22:48 -03:00
Harry Moreno 8fca98616a fix grammar 2019-01-28 10:19:31 -03:00
Daniel Graña 0e5b6755ba
Merge pull request #3605 from morenoh149/patch-2 
fix grammar
 2019-01-28 10:18:45 -03:00
Harry Moreno b828b5f8c8
fix grammar 2019-01-26 18:39:05 -05:00
Daniel Graña 71743a6546 Add release notes for v1.5.2 2019-01-22 18:56:47 -03:00
Júlio César Batista 8eade7d864 Testing stats and log messages from RFPDupeFilter 2019-01-18 11:39:35 -02:00
Júlio César Batista bdf12f7750 Logging the request referer when DUPEFILTER_DEBUG is active 2019-01-18 11:38:59 -02:00
Joaquin Garmendia Cabrera e1f8b55ba0
Improve syntax for readability 2018-12-28 16:53:12 -05:00
Mikhail Korobov 094dde6fdb
Merge pull request #3512 from victor-torres/sitemap_filter 
[MRG+1] Add sitemap_filter function to SitemapSpider class
 2018-12-28 20:11:46 +05:00
Victor Torres 5a824c906c using shorter import version and moving datetime import to the beginning of the code snippet 2018-12-27 18:34:41 -03:00
Victor Torres bfbcf52e9d fix SitemapSpider import 2018-12-27 18:12:31 -03:00
Victor Torres b68308779a improving docs 2018-12-27 17:37:59 -03:00
Victor Torres e1597f7c42 improve readability 2018-12-27 11:40:14 -03:00
Victor Torres fe283bcd05 add test case for sitemap filter with alternate links 2018-12-27 11:40:14 -03:00
Victor Torres 10f46bca54 documenting sitemap entries as suggested by @kmike 2018-12-27 11:40:14 -03:00
Victor Torres 5e7ecf9dc1 add tests for sitemapindex 2018-12-27 11:40:14 -03:00
Victor Torres 657f0663b3 rename param from urls to entries 2018-12-27 11:40:14 -03:00
Victor Torres d7d5917ff1 add tests for the sitemap_filter method in the SitemapSpider class 2018-12-27 11:40:14 -03:00
Victor Torres 672385a371 using a method definition instead of a None attribute 2018-12-27 11:40:14 -03:00
Victor Torres a5e1b7bb47 add sitemap_filter attribute to SitemapSpider class 
it makes it possible to filter sitemap urls by any available attribute

for example, you can filter urls with lastmod greater than a given datetime

it can be helpful when the url loc itself does not aggregate that information
 2018-12-27 11:40:14 -03:00
Daniel Graña 93cf38354d
Merge pull request #3394 from hcoura/dh-lazyloading-optional 
Make lazy loading Download Handlers optional
 2018-12-26 11:16:19 -03:00
Daniel Graña f6ce716372
Merge pull request #3476 from elacuesta/deque_appendleft 
[MRG+1] Use collections.deque instead of list to store MiddlewareManager's methods
 2018-12-26 11:13:52 -03:00
Daniel Graña 76433feb4f
Merge pull request #3548 from scrapy/fix-aws-settings-docs 
fix docs for AWS_... settings. A follow-up to GH-2609.
 2018-12-26 11:10:57 -03:00
Daniel Graña 5cf26c9985
Merge pull request #3547 from scrapy/setup.py-3.7 
declare Python 3.7 support in setup.py
 2018-12-26 11:09:07 -03:00