c3b690a5b5
Merge pull request #4290 from dekimsey/patch-1
...
FilesPipeline.file_path has optional arguments
2020-02-07 23:41:31 +05:00
afbaf9d430
Merge pull request #4303 from whalebot-helmsman/request_left_downloader_signal
...
request_left_downloader signal
2020-02-07 23:33:51 +05:00
153b78e53f
Update docs/topics/signals.rst
...
Co-Authored-By: elacuesta <elacuesta@users.noreply.github.com>
2020-02-07 11:08:55 +05:00
8817b9e8e9
Update docs/topics/signals.rst
...
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
2020-02-07 11:07:53 +05:00
2f83f3e2cb
Update docs/topics/signals.rst
...
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
2020-02-07 11:07:43 +05:00
84b55b7364
Update docs/topics/signals.rst
...
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
2020-02-07 11:07:35 +05:00
042e71e2b8
Merge pull request #4311 from Gallaecio/metarefresh-ignore-tags
...
Make METAREFRESH_IGNORE_TAGS an empty list by default
2020-02-06 23:40:45 +05:00
8d2705f23c
Merge pull request #4305 from Respawnz/patch-1
...
fix a typo in devloper-tools.rst
2020-02-06 23:17:28 +05:00
35dafef7f1
Specify Twisted reactor (TWISTED_REACTOR setting) ( #4294 )
...
* Add the ability to install a specific reactor
* Add docs for the TWISTED_REACTOR setting
* Add tests for the TWISTED_REACTOR setting
* Update asyncio reactor test
* Ignore W503 globally
W503 is not PEP8-compliant:
c59c4376ad
2020-02-06 22:42:34 +05:00
6733f4d976
Update docs/topics/signals.rst
...
Co-Authored-By: elacuesta <elacuesta@users.noreply.github.com>
2020-02-06 18:40:42 +05:00
576663e5a7
Make METAREFRESH_IGNORE_TAGS an empty list by default
2020-02-06 10:43:20 +01:00
c2cca36821
typo
2020-02-06 05:39:15 +08:00
aab39f6341
docummentation for new signal
2020-02-05 11:35:03 +00:00
752e8f7018
FilesPipeline.file_path has optional arguments
...
Documented signature doesn't match the actual interface in [files.py](https://github.com/scrapy/scrapy/blob/master/scrapy/pipelines/files.py#L520 ).
Specifically, it looks like it may be [called](https://github.com/scrapy/scrapy/blob/master/scrapy/pipelines/files.py#L422 ) without a response value.
I found this when I was implementing the pipeline with the signature `file_path(self, request, response, info)` and the following error was being return in my results :
[(False, <twisted.python.failure.Failure builtins.TypeError: file_path() missing 1 required positional argument: 'response'>)]
Scrapy==1.8.0
2020-01-28 15:21:15 -06:00
f72d4e93e6
[Docs] 2 typos + 1 clarification in docs
...
Fixing 2 small typos and adding 1 word as clarification in the downloader-middlewares.
Also, I was confused with the entries like `ref:Reppy <reppy-parser>` and similar entries.
Are these supposed to be links to other parts of the doc, or is this the intended way of showing these references ?
2020-01-26 10:48:28 +01:00
8b8df31961
Merge pull request #4227 from elacuesta/name-resolver
...
Name resolver with IPv6 support
2020-01-25 05:03:13 +05:00
fc98aa6b67
Merge branch 'master' into ftp_files#3928
2020-01-24 14:52:40 +02:00
6a98d660e5
Merge pull request #3551 from jpbalarini/change_scraper_slot
...
[MRG+1] Add ability to change max_active_size by setting
2020-01-23 23:40:05 +05:00
0f2d871d88
Use PEP 515 style for SCRAPER_SLOT_MAX_ACTIVE_SIZE documentation
2020-01-20 11:28:28 -03:00
eaa8ed02d0
Add ability to change max_active_size by settings
2020-01-20 11:27:58 -03:00
302d3f552b
[doc] DNS_RESOLVER setting
2020-01-18 01:46:24 -03:00
2e405d2d5c
Merge branch 'master' into response_follow_all
2020-01-05 00:33:19 -03:00
14d4428e70
Rephrasing documentation for image and file pipelines ( #4252 )
...
* scrapy#4034 Clarify documentation for image and file pipelines
* scrapy#4034 Clarify documentation for file pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Clarify documentation for image and file pipelines
* scrapy#4034 Clarify documentation for file pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Revert image, file pipeline docs. Enhance custom media pipeline docs.
* scrapy#4034 rebase master
* scrapy#4034 Clarify documentation for image and file pipelines
* scrapy#4034 Clarify documentation for file pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Clarify documentation for image and file pipelines
* scrapy#4034 Clarify documentation for file pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Revert image, file pipeline docs. Enhance custom media pipeline docs.
* scrapy#4034 rebase master
* Rebase master
* Add class to media pipeline docs
Co-Authored-By: elacuesta <elacuesta@users.noreply.github.com>
Co-authored-by: elacuesta <elacuesta@users.noreply.github.com>
2019-12-30 00:56:22 +05:00
dc1ee09481
Rename ASYNCIO_ENABLED to ASYNCIO_REACTOR, change the logic accordingly.
2019-12-27 21:56:28 +05:00
20289be810
Merge remote-tracking branch 'origin/master' into asyncio-base
2019-12-20 19:31:20 +05:00
fb3fb17c28
Merge pull request #4045 from Gallaecio/documentation-coverage
...
Deprecate FilteringLinkExtractor direct usage; add API docs for scrapy.linkextractors
2019-12-20 02:01:45 +05:00
6a32a36e09
Merge pull request #4143 from Gallaecio/undent-rst-doctests
...
Do not indent doctests from the documentation unnecessarily
2019-12-20 01:59:07 +05:00
b4a2d985a7
Merge branch 'master' into undent-rst-doctests
2019-12-19 12:39:17 +01:00
16b363de31
Merge branch 'master' into fix_typos
2019-12-19 10:02:21 +01:00
23a67cec27
fix first letter capitalization for Raring and Scrapy
2019-12-19 09:57:17 +01:00
8f7faaa63d
Merge pull request #4190 from Gallaecio/doctest
...
Make developer-tools doctests pass
2019-12-18 23:40:21 +05:00
c0d84f0962
fix typos
2019-12-18 19:39:21 +01:00
916382e109
Add errback parameter to scrapy.spiders.crawl.Rule ( #4000 )
...
* Add errback parameter to scrapy.spiders.crawl.Rule
* CrawlSpider: optimize by reducing iterations
* [test] Rule.errback
* [doc] Rule.errback
* [doc] Use autoclass in docs/topics/spiders.rst
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
* Rule.process_links takes a list
* Fix aesthetic issue reported by Flake8
2019-12-18 20:05:33 +05:00
3560123090
Rename ASYNCIO_SUPPORT to ASYNCIO_ENABLED.
2019-12-13 18:07:47 +05:00
21a2f8c6f8
Merge branch 'master' into response_follow_all
2019-12-10 17:30:59 -03:00
1fc2b140c1
Merge branch 'master' into documentation-coverage
2019-12-05 14:43:36 +01:00
83b8046fdc
Do not indent doctests from the documentation unnecessarily
2019-12-05 13:52:31 +01:00
ed34ce14c0
Add the ASYNCIO_SUPPORT setting, reshuffle other logic accordingly.
2019-12-04 21:32:56 +05:00
9b7452211a
Merge pull request #4099 from BurnzZ/itemloader-docs
...
update docs of scrapy.loader.ItemLoader.item
2019-12-03 13:14:45 +05:00
54b056c4be
Make developer-tools doctests pass
2019-11-25 12:32:35 +01:00
40b5cfc0a4
Item loaders: allow single-argument processors (unbound methods)
2019-11-22 20:47:22 -03:00
6fa8f06b64
Merge remote-tracking branch 'upstream/master' into response_follow_all
2019-11-22 13:05:06 -03:00
f251dda268
Make debug doctests pass
2019-11-21 11:59:10 +01:00
f1fd7ec318
Merge pull request #4171 from Gallaecio/doctest
...
Make command doctests pass
2019-11-20 19:47:22 +05:00
48223c107a
Merge remote-tracking branch 'upstream/master' into response_follow_all
2019-11-20 09:41:42 -03:00
e84cb18ca0
Use InterSphinx to link to the Twisted documentation
2019-11-18 15:50:45 +01:00
74589df02f
Make command doctests pass
2019-11-18 14:51:44 +01:00
dd367438fa
Improve the dbm module ref.
...
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
2019-11-14 13:45:39 +05:00
a9c891399d
Fix a duplicate ref name in docs.
2019-11-14 13:45:39 +05:00
1d7c8cb0b1
Remove six.PY2 and six.PY3 conditionals.
2019-11-14 13:45:37 +05:00
Mikhail Korobov
Vostretsov Nikita
elacuesta
Adrián Chaves
Respawnz
Daniel Kimsey
Peter Vandenabeele
OmarFarrag
JP Balarini
Eugenio Lacuesta
1um0s
Andrey Rakhmatullin
Marc Hernandez Cabot