b0eaf114e5
Merge pull request #4197 from elacuesta/sphinx-twisted-api
...
[Docs] Fix Twisted links
2020-02-07 23:51:15 +05:00
c3b690a5b5
Merge pull request #4290 from dekimsey/patch-1
...
FilesPipeline.file_path has optional arguments
2020-02-07 23:41:31 +05:00
afbaf9d430
Merge pull request #4303 from whalebot-helmsman/request_left_downloader_signal
...
request_left_downloader signal
2020-02-07 23:33:51 +05:00
153b78e53f
Update docs/topics/signals.rst
...
Co-Authored-By: elacuesta <elacuesta@users.noreply.github.com>
2020-02-07 11:08:55 +05:00
8817b9e8e9
Update docs/topics/signals.rst
...
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
2020-02-07 11:07:53 +05:00
2f83f3e2cb
Update docs/topics/signals.rst
...
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
2020-02-07 11:07:43 +05:00
84b55b7364
Update docs/topics/signals.rst
...
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
2020-02-07 11:07:35 +05:00
4f31c3ce01
Document a backward incompatibility that may affect custom schedulers ( #4274 )
2020-02-06 22:21:33 +01:00
042e71e2b8
Merge pull request #4311 from Gallaecio/metarefresh-ignore-tags
...
Make METAREFRESH_IGNORE_TAGS an empty list by default
2020-02-06 23:40:45 +05:00
8d2705f23c
Merge pull request #4305 from Respawnz/patch-1
...
fix a typo in devloper-tools.rst
2020-02-06 23:17:28 +05:00
35dafef7f1
Specify Twisted reactor (TWISTED_REACTOR setting) ( #4294 )
...
* Add the ability to install a specific reactor
* Add docs for the TWISTED_REACTOR setting
* Add tests for the TWISTED_REACTOR setting
* Update asyncio reactor test
* Ignore W503 globally
W503 is not PEP8-compliant:
c59c4376ad
2020-02-06 22:42:34 +05:00
6733f4d976
Update docs/topics/signals.rst
...
Co-Authored-By: elacuesta <elacuesta@users.noreply.github.com>
2020-02-06 18:40:42 +05:00
576663e5a7
Make METAREFRESH_IGNORE_TAGS an empty list by default
2020-02-06 10:43:20 +01:00
c2cca36821
typo
2020-02-06 05:39:15 +08:00
aab39f6341
docummentation for new signal
2020-02-05 11:35:03 +00:00
47b9de93a9
Merge pull request #4288 from petervandenabeele/patch-1
...
[Docs] 2 typos + 1 clarification in docs
2020-01-30 03:15:18 +05:00
752e8f7018
FilesPipeline.file_path has optional arguments
...
Documented signature doesn't match the actual interface in [files.py](https://github.com/scrapy/scrapy/blob/master/scrapy/pipelines/files.py#L520 ).
Specifically, it looks like it may be [called](https://github.com/scrapy/scrapy/blob/master/scrapy/pipelines/files.py#L422 ) without a response value.
I found this when I was implementing the pipeline with the signature `file_path(self, request, response, info)` and the following error was being return in my results :
[(False, <twisted.python.failure.Failure builtins.TypeError: file_path() missing 1 required positional argument: 'response'>)]
Scrapy==1.8.0
2020-01-28 15:21:15 -06:00
f72d4e93e6
[Docs] 2 typos + 1 clarification in docs
...
Fixing 2 small typos and adding 1 word as clarification in the downloader-middlewares.
Also, I was confused with the entries like `ref:Reppy <reppy-parser>` and similar entries.
Are these supposed to be links to other parts of the doc, or is this the intended way of showing these references ?
2020-01-26 10:48:28 +01:00
f3374a5047
Fix variable name `author_page_links`
...
I did not test this code, but the change from `href` to this
author_page_links seems to have a typo ?
2020-01-25 16:53:29 +01:00
8b8df31961
Merge pull request #4227 from elacuesta/name-resolver
...
Name resolver with IPv6 support
2020-01-25 05:03:13 +05:00
fc98aa6b67
Merge branch 'master' into ftp_files#3928
2020-01-24 14:52:40 +02:00
bd54f22fef
Merge pull request #4282 from petervandenabeele/patch-1
...
fix logical documentation error with PER_DOMAIN or PER_DOMAIN
2020-01-24 02:01:22 +05:00
6a98d660e5
Merge pull request #3551 from jpbalarini/change_scraper_slot
...
[MRG+1] Add ability to change max_active_size by setting
2020-01-23 23:40:05 +05:00
7d5cebcf77
fix logical documentation error with PER_DOMAIN or PER_DOMAIN
2020-01-23 09:08:21 +01:00
c75cf15b7a
Update CSS selectors in tutorial
2020-01-22 10:38:59 -03:00
0f2d871d88
Use PEP 515 style for SCRAPER_SLOT_MAX_ACTIVE_SIZE documentation
2020-01-20 11:28:28 -03:00
eaa8ed02d0
Add ability to change max_active_size by settings
2020-01-20 11:27:58 -03:00
b471765d40
[doc] FAQ entry about the IPv6 and the DNS_RESOLVER setting
2020-01-18 01:52:29 -03:00
302d3f552b
[doc] DNS_RESOLVER setting
2020-01-18 01:46:24 -03:00
2e405d2d5c
Merge branch 'master' into response_follow_all
2020-01-05 00:33:19 -03:00
14d4428e70
Rephrasing documentation for image and file pipelines ( #4252 )
...
* scrapy#4034 Clarify documentation for image and file pipelines
* scrapy#4034 Clarify documentation for file pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Clarify documentation for image and file pipelines
* scrapy#4034 Clarify documentation for file pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Revert image, file pipeline docs. Enhance custom media pipeline docs.
* scrapy#4034 rebase master
* scrapy#4034 Clarify documentation for image and file pipelines
* scrapy#4034 Clarify documentation for file pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Clarify documentation for image and file pipelines
* scrapy#4034 Clarify documentation for file pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Simplify documentation for pipeline
* scrapy#4034 Revert image, file pipeline docs. Enhance custom media pipeline docs.
* scrapy#4034 rebase master
* Rebase master
* Add class to media pipeline docs
Co-Authored-By: elacuesta <elacuesta@users.noreply.github.com>
Co-authored-by: elacuesta <elacuesta@users.noreply.github.com>
2019-12-30 00:56:22 +05:00
f0ae673452
Merge pull request #4258 from atul-g/patch-1
...
Edited the link provided to homepage of lxml's website
2019-12-30 00:55:15 +05:00
82861c73c8
Edited the link of the homepage of lxml website
...
The link "https://lxml.de " is redirecting to a completely different and unintended website. I changed the link to the index page of lxml's official website. I thought of changing it to the PyPi page of lxml, but even they are providing the same "https://lxml.de " link which doesn't seem to be working now.
2019-12-27 22:57:58 +05:30
dc1ee09481
Rename ASYNCIO_ENABLED to ASYNCIO_REACTOR, change the logic accordingly.
2019-12-27 21:56:28 +05:00
20289be810
Merge remote-tracking branch 'origin/master' into asyncio-base
2019-12-20 19:31:20 +05:00
fb3fb17c28
Merge pull request #4045 from Gallaecio/documentation-coverage
...
Deprecate FilteringLinkExtractor direct usage; add API docs for scrapy.linkextractors
2019-12-20 02:01:45 +05:00
6a32a36e09
Merge pull request #4143 from Gallaecio/undent-rst-doctests
...
Do not indent doctests from the documentation unnecessarily
2019-12-20 01:59:07 +05:00
b4a2d985a7
Merge branch 'master' into undent-rst-doctests
2019-12-19 12:39:17 +01:00
16b363de31
Merge branch 'master' into fix_typos
2019-12-19 10:02:21 +01:00
23a67cec27
fix first letter capitalization for Raring and Scrapy
2019-12-19 09:57:17 +01:00
8f7faaa63d
Merge pull request #4190 from Gallaecio/doctest
...
Make developer-tools doctests pass
2019-12-18 23:40:21 +05:00
c0d84f0962
fix typos
2019-12-18 19:39:21 +01:00
916382e109
Add errback parameter to scrapy.spiders.crawl.Rule ( #4000 )
...
* Add errback parameter to scrapy.spiders.crawl.Rule
* CrawlSpider: optimize by reducing iterations
* [test] Rule.errback
* [doc] Rule.errback
* [doc] Use autoclass in docs/topics/spiders.rst
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
* Rule.process_links takes a list
* Fix aesthetic issue reported by Flake8
2019-12-18 20:05:33 +05:00
053319334c
Merge pull request #4179 from Gallaecio/user-friendlier-tox
...
Make tox configuration more user friendly
2019-12-18 18:35:19 +05:00
012533924a
remove requirements from here too
2019-12-18 11:13:36 +00:00
607815dfc2
Merge branch 'master' into documentation-coverage
2019-12-17 16:54:27 +01:00
b8cf522916
Merge branch 'master' into user-friendlier-tox
2019-12-17 16:53:08 +01:00
a59bb279d1
add year through code
2019-12-15 17:33:00 +01:00
1aab20e1ce
update copyright notice year
2019-12-14 10:37:31 +01:00
e3a3ad4aaf
remove reference to old (Python 2.7) environment
2019-12-14 10:34:31 +01:00
Mikhail Korobov
Vostretsov Nikita
Joy Bhalla
elacuesta
Adrián Chaves
Respawnz
Daniel Kimsey
Peter Vandenabeele
OmarFarrag
Eugenio Lacuesta
JP Balarini
1um0s
Atul Gopinathan
Andrey Rakhmatullin
Marc Hernandez Cabot
marc