RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,291 Commits 50 Branches 132 Tags 142 MiB
Commit Graph

2437 Commits

Author SHA1 Message Date
Mikhail Korobov cd0d3fd48d
Merge pull request #4705 from Gallaecio/deprecation-policy 
Cover our deprecation policy in the documentation
 2020-08-14 00:27:37 +05:00
Adrián Chaves 65e0abaea5 Document FEED_URI_PARAMS 2020-08-13 09:05:51 +02:00
Ajay Mittur 1c4b4cc6b0
Support defining file path based on item in media pipelines (#4686) 2020-08-11 14:12:44 +02:00
Adrián Chaves 5138f9a965 Merge remote-tracking branch 'upstream/master' into allow-customizing-export-column-names 2020-08-11 13:42:05 +02:00
Adrián Chaves b2f4df5cb7 Merge remote-tracking branch 'upstream/master' into documentation-build 2020-08-11 13:16:14 +02:00
Aditya c67d6dea31 fix: H2 docs, NotImplementedError for H2 Tunnel 2020-08-11 04:39:41 +05:30
Adrián Chaves 4ee538e44b
Update unicode references from Python 2 times in the documentation (#4703) 2020-08-04 20:34:11 +02:00
Adrián Chaves 3600582f56
Cover Scrapy 2.2.1 and 2.3 in the release notes (#4708) 2020-08-04 20:05:56 +02:00
Aditya d707f8b5d9 docs: mention H2DownloadHandler in settings.rst 2020-07-30 18:06:21 +05:30
Adrián Chaves 6f4ccec567 Cover our deprecation policy in the documentation 2020-07-30 14:03:14 +02:00
Adrián Chaves 890b2138a6 Remove the u prefix from strings 2020-07-30 13:39:30 +02:00
Mikhail Korobov 5e2d1bd187
Merge pull request #4434 from BroodingKangaroo/ISSUE-4250-add_batch_deliveries 
Feed exports: add batch deliveries
 2020-07-29 23:03:28 +05:00
BroodingKangaroo a6c1d79b7c pep8 tiny changes 2020-07-28 11:53:05 +03:00
Andrey Rahmatullin f3372a3753
Merge pull request #4254 from elacuesta/spider.parse 
Change Scraper API to call internal `_parse` method
 2020-07-21 17:37:54 +05:00
Adrián Chaves d29bec60d7
Upgrade PyPy for CI, and test both 3.5 (oldest) and 3.6 (newest) (#4504) 
* Upgrade PyPy for CI, and test both 3.5 (oldest) and 3.6 (newest)

* Log a detailed error message to discover why MockServer is not working

* Go for all lines!

* Disable tests based on mitmproxy while running on PyPy

* Fix test_get_func_args for PyPy 3.6+

* Make testPayloadDefaultCiphers work regardless of OpenSSL default ciphers

* Crossing fingers…

* Rename: testPayloadDefaultCiphers → testPayloadDisabledCipher

* Test the PyPy version currently documented as the minimum required version

* Fix the PYPY_VERSION tag

* Update the documentation about supported PyPy versions

* Also test the latest 3.5 Python version with PyPy

* Fix the PYPY_VERSION value for the latest 3.5 version

* Use pinned dependencies for asyncio and PyPy tests against oldest supported Python versions

* Fix PyPy installation for the pypy3-pinned Tox environment

* Try installing Cython

* Maybe PyPy requires lxml 3.6.0?

* install.rst: minor clarification

* lxml 4.0.0 is required on PyPy

* Require setuptools 18.5+

* Revert "Require setuptools 18.5+"

This reverts commit 017ec33ac2.

* Maintain lxml as a dependency if setuptools < 18.5 is used
 2020-07-17 02:19:24 +05:00
Adrián Chaves c4f9250220
Merge branch 'master' into ISSUE-4250-add_batch_deliveries 2020-07-16 17:44:18 +02:00
BroodingKangaroo 41263f61c6 Change single quotes to double in example in docs 2020-07-16 18:41:45 +03:00
Mikhail Korobov 07470e1a3c
Merge pull request #3608 from ejulio/feat-685 
[MRG+1] Fix for #685 Add Google Cloud Storage Feed Export
 2020-07-16 18:11:19 +05:00
BroodingKangaroo 8bdcdb0a76 Add quotes to example in docs 2020-07-16 09:13:54 +03:00
Júlio César Batista 38496a00b7
Use the itemlaoders library (#4516) 2020-07-15 12:08:36 +02:00
Aditya Kumar 3f7e8635f4
Allow the parse command to write data to a file (#4377) 2020-07-11 08:48:24 +02:00
Eugenio Lacuesta 2be2bdd2df
Merge remote-tracking branch 'upstream/master' into spider.parse 2020-07-08 14:00:54 -03:00
BroodingKangaroo 6454d456d2 Make check of placeholder less strict 2020-07-03 10:24:13 +03:00
BroodingKangaroo 1e245046ed Change setting name. Add leading zeroes to batch_id. Minor fixes. 2020-07-02 12:38:08 +03:00
Andrey Rahmatullin af55d23167
Update the OpenSSL cipher list format link 
OpenSSL `ciphers(1)` is now almost empty: https://www.openssl.org/docs/manmaster/man1/ciphers.html

Alternative would be linking to 1.1.1 docs specifically.
 2020-07-01 19:46:54 +05:00
BroodingKangaroo 7b1d3c35ea Minor updates 2020-07-01 12:24:31 +03:00
Eugenio Lacuesta 5b88c522ac
Simplify dataclass example in item loader docs 2020-06-30 12:18:21 -03:00
BroodingKangaroo 05c2587c6a Docs update and tiny fixes 2020-06-28 09:45:45 +03:00
BroodingKangaroo c3cee74fd4 Change default value of FEED_STORAGE_BATCH_ITEM_COUNT to 0 2020-06-26 18:45:21 +03:00
BroodingKangaroo 77333666f1 resolve conflicts 2020-06-26 14:55:46 +03:00
Adrián Chaves 6f4c964aa4
Cover Scrapy 2.2.0 in the release notes (#4630) 2020-06-24 12:26:38 +02:00
Adrián Chaves cfd039aeb6 Remove a duplicate GCS_PROJECT_ID reference target 2020-06-22 19:28:33 +02:00
Eugenio Lacuesta 73b6ce8cb5
Update docs about dataclass items and loaders 2020-06-22 14:13:37 -03:00
Eugenio Lacuesta 1335d9053e
Update docs/topics/loaders.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-06-22 14:05:44 -03:00
Eugenio Lacuesta 3efea98e05
Docs: add note about dataclass items and loaders 2020-06-22 12:41:25 -03:00
Eugenio Lacuesta c7f1c7e3f3
Merge branch 'master' into feat-685 2020-06-19 21:30:54 -03:00
Adrián Chaves f9c63384c0
Merge branch 'master' into allow-customizing-export-column-names 2020-06-18 14:09:20 +02:00
Adrián Chaves 88ade19675
Merge branch 'master' into spider-name-collision 2020-06-18 14:01:02 +02:00
Devi Sandeep 5d54173187
Update docs on accessing callback arguments in errback (#4634) 2020-06-18 12:01:38 +02:00
Eugenio Lacuesta ec98dabfab
Support for dataclass and attrs items (#3881) 2020-06-14 11:45:27 +02:00
Eugenio Lacuesta 8b549392f9
Bump minimum Python version to 3.5.2 (#4615) 2020-06-11 14:53:59 +02:00
Eugenio Lacuesta b6c5289fb9
Close page in pyppeteer example, mention asyncio reactor 2020-06-10 12:11:49 -03:00
Eugenio Lacuesta 78aa1b2bfc
Fix typo 2020-06-08 11:19:15 -03:00
Eugenio Lacuesta bdabc500aa
Update headless browser docs 2020-06-06 16:47:15 -03:00
Mikhail Korobov 1d2b692666
Merge pull request #4596 from sharmarohit/remove-unneeded-escape-sequence 
remove unneeded escape sequence
 2020-06-02 02:30:33 +05:00
Adrián Chaves c86a1035dd
Merge pull request #4563 from willbeaufoy/hoverxref 
Extend hoverxref_roles to custom crossrefs (#4495)
 2020-06-01 07:18:13 +02:00
Bulat Khabibullin 5cef927944
Implement TextResponse.json() (#4574) 2020-06-01 06:57:23 +02:00
Mikhail Korobov 13215cfb71
Merge pull request #4559 from elacuesta/cancel-request-download 
Stop response download from signal handler
 2020-05-27 21:46:00 +05:00
Andrey Rahmatullin 7bf375098e
Merge pull request #2400 from elacuesta/keep_cookie_header 
[MRG+1] CookiesMiddleware: keep cookies from 'Cookie' request header, fix encoding
 2020-05-27 21:41:43 +05:00
Eugenio Lacuesta 492197e477
StopDownload: store response in the exception instead of the failure 2020-05-27 10:45:07 -03:00
Eugenio Lacuesta 634ad5ebca
Merge remote-tracking branch 'upstream/master' into cancel-request-download 2020-05-27 10:27:45 -03:00
Will Beaufoy 898bdd3feb Update docs README with build instructions using tox 
None of the existing commands built the docs properly for me (I had to
revert the changes in 901892d to docs/conf.py to get them to build
properly, and even then no tooltips displayed).

Building them with tox worked for me, but other developers say they
can still use the original method, so the docs now contain both.
 2020-05-26 19:09:23 +01:00
Rohit Sharma 0cc0e51ca3 remove uneeded escape sequence 
removed uneeded wscape sequences from method arguments in the docs
folder
 2020-05-24 13:38:44 -07:00
Adrián Chaves 08756cd87a Fix the description of the ScreenshotPipeline example item pipeline 2020-05-22 18:06:54 +02:00
Eugenio Lacuesta f6ed5edc31
CookiesMiddleware: keep cookies from 'Cookie' request header 2020-05-21 11:07:41 -03:00
BroodingKangaroo 5f6c1dceb1 Merge remote-tracking branch 'upstream/master' into ISSUE-4250-add_batch_deliveries 2020-05-21 12:43:04 +03:00
Eugenio Lacuesta 276721a5dc
Stop response download 2020-05-20 14:46:26 -03:00
Adrián Chaves f418c6d5f7 logging.rst: remove unused, misleading import 2020-05-19 20:04:47 +02:00
nsirletti c2a0cca0fe Suppress Sphinx warnings about MIME types other than .rst 2020-05-19 02:17:11 +02:00
Mikhail Korobov febe82a907
Merge pull request #4486 from ilias-ant/add-file-status-on-media-pipelines-file-info 
Add status (downloaded, uptodate) to files information
 2020-05-18 22:30:30 +05:00
Mikhail Korobov 14612fc3d3
Merge pull request #4534 from elacuesta/deprecate-baseitem 
Deprecate scrapy.item.BaseItem
 2020-05-16 03:24:06 +05:00
BroodingKangaroo 10ae1a284f Minor fixes 2020-05-15 22:50:54 +03:00
Adrián Chaves 36c3c9713e
Run tests with Python 3.5.0 (#4518) 
* Run tests with Python 3.5.0

* Use Ubuntu Trusty 14.04 to test Python 3.5.0

* Use mitmproxy<3.0.0 on Python 3.5.0

* Skip tests requiring mitmproxy in Python 3.5.0

* Change the minimum Python version from 3.5 to 3.5.1

* Do not prevent Scrapy from working with Python 3.5.0

* Force system Python 3.5.1

* Do not install a non-system Python in Python 3.5.1 jobs

* Switch to Trusty to be able to test Xenial’s Python version ¯\_(ツ)_/¯

* Add missing trusty

* Stop breaking old PyPy

* Allow installing Scrapy on Python 3.5.0
 2020-05-15 22:37:56 +05:00
BroodingKangaroo 1cdcf8b08b Minor fixes 2020-05-15 20:14:07 +03:00
BroodingKangaroo 69c005f013 Documentation indent fix 2020-05-14 10:35:56 +03:00
BroodingKangaroo 111a58fe3d Merge remote-tracking branch 'upstream/master' into ISSUE-4250-add_batch_deliveries 2020-05-14 09:53:14 +03:00
BroodingKangaroo 8662d3587d Documentation and code refactoring 2020-05-14 00:05:36 +03:00
BroodingKangaroo 2327ecead0 Rename FEED_STORAGE_BATCH_SIZE to FEED_STORAGE_BATCH_ITEM_COUNT 2020-05-13 22:50:04 +03:00
Adrián Chaves 7dac99bb12
Merge branch 'master' into deprecate-baseitem 2020-05-13 20:35:03 +02:00
Aditya Kumar 8d1269bcbc
Cover chompjs in documentation (#4562) 2020-05-12 20:42:28 +02:00
nsirletti cb8140a42a
Deprecate Response.body_as_unicode() (#4555) 
Co-authored-by: Nicolas Sirletti <n.sirletti@gmail.com>
 2020-05-11 20:20:31 +02:00
Eugenio Lacuesta fb7ba6964d
Merge branch 'master' into deprecate-baseitem 2020-05-11 10:21:56 -03:00
Mikhail Korobov b183579564
Merge pull request #4205 from elacuesta/bytes_received_signal 
Add bytes_received signal
 2020-05-11 15:09:55 +05:00
Will Beaufoy 1bfbcc6180 Extend hoverxref_roles to custom crossrefs (#4495) 2020-05-10 13:49:13 +01:00
Adrián Chaves e07708e374 request-response: update the consequences of str(b'') 2020-05-09 15:54:31 +02:00
Adrián Chaves c6746f0e38 bytes array → bytes object 2020-05-09 15:51:11 +02:00
Eugenio Lacuesta 9485724d98
Merge branch 'master' into deprecate-baseitem 2020-05-08 19:32:47 -03:00
Mikhail Korobov 794ab19660
Merge pull request #3727 from Gallaecio/pylint 
Use pylint
 2020-05-09 02:22:54 +05:00
Antonio Gordillo Toledo cf09af787e
Remove Python 2 encoding header from files (#4553) 2020-05-08 15:45:19 +02:00
Eugenio Lacuesta 6160370927
Merge remote-tracking branch 'upstream/master' into deprecate-baseitem 2020-05-07 14:38:54 -03:00
Adrián Chaves bbd9d05dbf request-response.rst: review type references around body mentions 2020-05-07 11:44:43 +02:00
Eugenio Lacuesta 7fd2085725
Merge remote-tracking branch 'upstream/master' into bytes_received_signal 2020-05-06 22:41:16 -03:00
Adrián Chaves a19d15013a Merge remote-tracking branch 'upstream/master' into spider-name-collision 2020-05-06 20:52:36 +02:00
Adrián Chaves a859ea0c8b Merge remote-tracking branch 'upstream/master' into allow-customizing-export-column-names 2020-05-06 19:48:32 +02:00
Adrián Chaves a79cfce57b
Merge branch 'master' into pylint 2020-05-06 16:24:13 +02:00
Eugenio Lacuesta 8d1e3ee0dd
Remove deprecated BaseItem from the docs 2020-05-06 09:24:40 -03:00
Eugenio Lacuesta 7988c676a9
Update Item docstring, update BaseItem occurrences 2020-05-05 13:11:01 -03:00
Mikhail Korobov fe6154e4fa clarify DOWNLOADER_HTTPCLIENTFACTORY docs 2020-05-04 18:18:38 +05:00
Eugenio Lacuesta e1948b4923
Add example about bytes_received signal 2020-05-04 09:07:27 -03:00
Mikhail Korobov 83d7360bb7 Don't mention unsupported package versions in docs 2020-05-04 02:00:11 +05:00
BroodingKangaroo dad2ea7522 Change time_id to batch_time 2020-05-02 01:21:03 +03:00
Eugenio Lacuesta 4d625afda5
Merge branch 'master' into bytes_received_signal 2020-05-01 16:11:32 -03:00
BroodingKangaroo df81870f39 Merge branch 'master' of https://github.com/scrapy/scrapy into ISSUE-4250-add_batch_deliveries 2020-05-01 12:18:39 +03:00
Adrián Chaves ec0a0eb3ab
Merge branch 'master' into spider-name-collision 2020-04-28 21:13:08 +02:00
Adrián Chaves 1006db1e10
Merge branch 'master' into documentation-build 2020-04-28 21:07:14 +02:00
Ashe c207dbf939
Remove the asyncio warning from coroutines page (#4513) 2020-04-27 19:45:19 +02:00
BroodingKangaroo a18621552f Merge branch 'master' of https://github.com/BroodingKangaroo/scrapy into ISSUE-4250-add_batch_deliveries 2020-04-27 10:34:36 +03:00
BroodingKangaroo 2eee6c8101 Documentation spelling fix 2020-04-27 09:58:14 +03:00
BroodingKangaroo f0f1be76d1 Using time_id instead of time as a timestamp 2020-04-27 09:56:57 +03:00
Adrián Chaves ffe576c4ed
Cover Scrapy 2.1 in the release notes (#4499) 
Co-authored-by: Mikhail Korobov <kmike84@gmail.com>
 2020-04-24 11:44:36 +02:00
Aditya Kumar e4750f2fbd
async/deferred signal handlers (#4390) 
* [docs] async/deferred signal handlers

* [docs] update deferred signals example

* [docs] add subsections for built-in signals

* docs(signals): update signal handler example

* docs(signals): update signal handler example
 2020-04-20 20:47:57 +05:00
sakshamb2113 bfeb2c8c13
Added warning to use double quotes in Windows for scrapy shell in shell.rst (#4450) 
* modified debugging memory leaks with guppy in leaks.rst

* modified leaks.rst(issue #4285)

* removed guppy from telnet.py

* Fix undefined name error

* removed hpy key from telnet_vars in telnet.py

* updated shell.rst

* Update docs/topics/shell.rst

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-04-18 20:21:26 +05:00
Eugenio Lacuesta 04b6295a69
Docs: replace deprecated FEED_* settings 2020-04-17 20:50:17 -03:00
Eugenio Lacuesta 1f2e2a6006
Merge branch 'master' into response_ip_address 2020-04-16 11:32:37 -03:00
Adrián Chaves 901892dab3 Fix the hoverxref configuration 2020-04-16 14:48:38 +02:00
BroodingKangaroo cac1f3a6ad Update documentation 2020-04-16 10:06:56 +03:00
Mikhail Korobov 36abe9235c
Merge pull request #4445 from adityaa30/docs-redirect-links 
update redirect links to python3
 2020-04-15 23:13:55 +05:00
Mikhail Korobov 066c02d0c3
Merge pull request #4455 from aditi137/docs-curl2scrapy 
[Docs] mention curl2scrapy in Request.from_curl
 2020-04-15 23:12:33 +05:00
Eugenio Lacuesta ac869181fb
Update docs/topics/downloader-middleware.rst 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2020-04-15 13:42:35 -03:00
Eugenio Lacuesta ee4ee486b1
Revert unnecessary changes to docs/conf.py 2020-04-14 15:06:54 -03:00
Eugenio Lacuesta 0a4ef97fa3
Loose restrictions for docs requirements 2020-04-14 14:57:20 -03:00
ilias-ant 83a0cc6cdf Add status to files information 2020-04-12 23:22:17 +03:00
Eugenio Lacuesta 2205f04631
Docs: Add hoverxref_role_types setting 2020-04-10 18:08:04 -03:00
Eugenio Lacuesta 34e81d0d74
Docs: remove duplicated setting definitions 2020-04-10 17:29:02 -03:00
Eugenio Lacuesta 4383f45299
Replace os.path with pathlib in docs config 2020-04-10 16:49:14 -03:00
Eugenio Lacuesta 24a1d9acae
Get version in docs config 2020-04-10 16:48:42 -03:00
Eugenio Lacuesta f97fec5ebd
Pin Sphinx version, including extensions 2020-04-10 16:02:53 -03:00
Eugenio Lacuesta c4a5e3f0da
Simplify bytes_received signal 
Remove "source" parameter
 2020-04-06 09:26:13 -03:00
Eugenio Lacuesta e3342669ae
Merge branch 'master' into bytes_received_signal 2020-04-06 09:17:35 -03:00
Aditya b1904729d5 [docs] change mod to doc redirect link 2020-03-27 04:37:26 +05:30
Aditi Dutta 010edfe85c [Docs] mention curl2scrapy in Request.from_curl 2020-03-25 14:38:22 -04:00
Aditya 80c69d68ad [docs] refactor python docs links using intersphinx 2020-03-24 18:35:42 +05:30
Aditya f37b1bdc56 [docs] update redirect links to python3 2020-03-20 05:31:41 +05:30
Aditya ca08e04198 [docs] update redirect links python2 -> python3 2020-03-20 02:31:35 +05:30
Eugenio Lacuesta 34d607194b
Merge branch 'master' into feat-685 2020-03-18 21:24:36 -03:00
Adrián Chaves 6c747953f9 Cover 2.0.1 in the release notes (#4437) 2020-03-18 19:17:16 +01:00
sakshamb2113 533131a30f
Remove Guppy-specific code and documentation (#4343) 2020-03-17 10:12:49 +01:00
elacuesta dfbe1d9507
Remove object base class (#4430) 2020-03-16 20:12:46 +01:00
Eugenio Lacuesta aa6075e08c
Merge branch 'master' into bytes_received_signal 2020-03-15 18:05:04 -03:00
Eugenio Lacuesta bde0384dfd
Merge branch 'master' into spider.parse 2020-03-14 19:53:36 -03:00
Aditya Kumar f9bf4b8d4d
Remove all top-level imports for twisted.internet.reactor (#4406) 2020-03-14 10:39:00 +01:00
Eugenio Lacuesta 21f1eddc90
Merge branch 'master' into bytes_received_signal 2020-03-11 21:16:35 -03:00
Eugenio Lacuesta 49156f2ecb
[doc] Feed exports: full local path as example 2020-03-11 20:45:54 -03:00
Eugenio Lacuesta ada37c5409
Export to multiple formats in a single crawl 2020-03-05 09:33:57 -03:00
Eugenio Lacuesta 3aa5eab993
Merge branch 'master' into response_ip_address 2020-03-03 13:53:43 -03:00
Adrián Chaves 6aa0ba4553
Write release notes for Scrapy 2.0.0 (#4329) 2020-03-03 09:11:11 +01:00
Adrián Chaves 431f6e7d90
Merge branch 'master' into patch-1 2020-03-01 18:17:18 +01:00
MaliCN ef00f8eb8e
updated with new macOS name (#4308) (#4323) 
* changed for new name as "macOS"  (issue #4308)

* updated macOS name

* update macOS name

* updated macOS name

* update for new macOS name

* docs/intro/install.rst: fix macOS header symbols

Co-Authored-By: elacuesta <elacuesta@users.noreply.github.com>

Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: elacuesta <elacuesta@users.noreply.github.com>
 2020-02-29 00:42:07 +05:00
Adrián Chaves 231c9ddef8
Update docs/intro/install.rst 2020-02-28 18:50:45 +01:00
nyov a34c366fa4 DOC linkcheck run; https and 301 link updates. 
Closes #4359
 2020-02-25 16:51:33 +00:00
Eugenio Lacuesta a44942d27d
Merge branch 'master' into response_ip_address 2020-02-23 18:13:52 -03:00
elacuesta 9d983c1b99
Expose certificate for HTTPS responses (#4054) 
* Expose certificate for HTTPS responses

* Fix test (missing inlineCallbacks decorator)

* Note about Response.certificate

* Explicitly cover None as the default value of Response.certificate

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-02-22 17:20:31 +05:00
Andrey Rahmatullin 88179027de
Merge pull request #4331 from Gallaecio/response-cb-kwargs 
Implement Response.cb_kwargs
 2020-02-19 22:40:14 +05:00
Akshay Sharma 182445f9d9
Fix a spelling error: ie. → i.e. (#4338) 2020-02-18 17:58:31 +01:00
leobalestri 6d6243afbb
Update install.rst 
Minor grammar and typo fixes
 2020-02-16 23:45:41 -08:00
Adrián Chaves a04dd13cd0
ie. → i.e. 2020-02-14 22:31:30 +01:00
Adrián Chaves 5ae3e1678f
ie. → i.e. 
Co-Authored-By: elacuesta <elacuesta@users.noreply.github.com>
 2020-02-14 22:30:36 +01:00
Adrián Chaves df937d8280 Implement Response.cb_kwargs 2020-02-13 22:33:36 +01:00
Adrián Chaves 037ae5b22e
Explicitly indicate None as ip_address’s default value 2020-02-10 19:54:47 +01:00
Adrián Chaves 96bb3b5144
Merge branch 'master' into feat-685 2020-02-10 19:45:09 +01:00
Adrián Chaves 7025c18b15
Clear line of spaces 2020-02-10 19:43:23 +01:00
Eugenio Lacuesta 13ba9bc629
Note about Response.ip_address 2020-02-10 12:29:39 -03:00
Eugenio Lacuesta 42b4e9b337
Reword signal docs 2020-02-10 11:23:38 -03:00
Adrián Chaves c2f484de88
Merge branch 'master' into response_ip_address 2020-02-07 21:49:38 +01:00
Adrián Chaves aae4935605
Merge branch 'master' into spider.parse 2020-02-07 21:17:56 +01:00
Adrián Chaves 59653ebac6 Update installation instructions regarding Python 3 and virtual environments 2020-02-07 21:07:57 +01:00
Mikhail Korobov b0eaf114e5
Merge pull request #4197 from elacuesta/sphinx-twisted-api 
[Docs] Fix Twisted links
 2020-02-07 23:51:15 +05:00
Mikhail Korobov c3b690a5b5
Merge pull request #4290 from dekimsey/patch-1 
FilesPipeline.file_path has optional arguments
 2020-02-07 23:41:31 +05:00
Mikhail Korobov afbaf9d430
Merge pull request #4303 from whalebot-helmsman/request_left_downloader_signal 
request_left_downloader signal
 2020-02-07 23:33:51 +05:00
Vostretsov Nikita 153b78e53f
Update docs/topics/signals.rst 
Co-Authored-By: elacuesta <elacuesta@users.noreply.github.com>
 2020-02-07 11:08:55 +05:00
Vostretsov Nikita 8817b9e8e9
Update docs/topics/signals.rst 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2020-02-07 11:07:53 +05:00
Vostretsov Nikita 2f83f3e2cb
Update docs/topics/signals.rst 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2020-02-07 11:07:43 +05:00
Vostretsov Nikita 84b55b7364
Update docs/topics/signals.rst 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2020-02-07 11:07:35 +05:00
Eugenio Lacuesta 6c333491a8
Merge branch 'master' into response_ip_address 2020-02-06 18:53:11 -03:00
Joy Bhalla 4f31c3ce01
Document a backward incompatibility that may affect custom schedulers (#4274) 2020-02-06 22:21:33 +01:00
Adrián Chaves 24bb9fd5f7
Merge branch 'master' into spider.parse 2020-02-06 21:39:09 +01:00
Mikhail Korobov 042e71e2b8
Merge pull request #4311 from Gallaecio/metarefresh-ignore-tags 
Make METAREFRESH_IGNORE_TAGS an empty list by default
 2020-02-06 23:40:45 +05:00
Mikhail Korobov 8d2705f23c
Merge pull request #4305 from Respawnz/patch-1 
fix a typo in devloper-tools.rst
 2020-02-06 23:17:28 +05:00
elacuesta 35dafef7f1
Specify Twisted reactor (TWISTED_REACTOR setting) (#4294) 
* Add the ability to install a specific reactor

* Add docs for the TWISTED_REACTOR setting

* Add tests for the TWISTED_REACTOR setting

* Update asyncio reactor test

* Ignore W503 globally

W503 is not PEP8-compliant:
c59c4376ad

* Line length adjustment

* Adjust asyncio reactor tests

* Merge ASYNCIO_ENABLED and TWISTED_REACTOR settings

* More docs about TWISTED_REACTOR

* Fix asyncio reactor test

* Docs: fix title

* Reword docs

* Check the TWISTED_REACTOR setting outside of the installing function

* Remove unrelated change

* Update scrapy/utils/log.py

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

* Update docs/topics/settings.rst

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

* Update docs/topics/settings.rst

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-02-06 22:42:34 +05:00
Vostretsov Nikita 6733f4d976
Update docs/topics/signals.rst 
Co-Authored-By: elacuesta <elacuesta@users.noreply.github.com>
 2020-02-06 18:40:42 +05:00
Adrián Chaves 576663e5a7 Make METAREFRESH_IGNORE_TAGS an empty list by default 2020-02-06 10:43:20 +01:00
Respawnz c2cca36821
typo 2020-02-06 05:39:15 +08:00
Vostretsov Nikita aab39f6341 docummentation for new signal 2020-02-05 11:35:03 +00:00
Eugenio Lacuesta b9e3a6201b
Merge branch 'master' into response_ip_address 2020-02-03 04:07:44 -03:00
Mikhail Korobov 47b9de93a9
Merge pull request #4288 from petervandenabeele/patch-1 
[Docs] 2 typos + 1 clarification in docs
 2020-01-30 03:15:18 +05:00
Eugenio Lacuesta 6f02a8dccb
Add source parameter to bytes_received signal 2020-01-29 15:51:25 -03:00
Daniel Kimsey 752e8f7018 FilesPipeline.file_path has optional arguments 
Documented signature doesn't match the actual interface in [files.py](https://github.com/scrapy/scrapy/blob/master/scrapy/pipelines/files.py#L520).

Specifically, it looks like it may be [called](https://github.com/scrapy/scrapy/blob/master/scrapy/pipelines/files.py#L422) without a response value.

I found this when I was implementing the pipeline with the signature `file_path(self, request, response, info)` and the following error was being return in my results :

    [(False, <twisted.python.failure.Failure builtins.TypeError: file_path() missing 1 required positional argument: 'response'>)]

Scrapy==1.8.0
 2020-01-28 15:21:15 -06:00
Eugenio Lacuesta 4ffd18fb11
[docs] Mention that signals.bytes_received could be fired multiple times 2020-01-27 13:29:33 -03:00
Eugenio Lacuesta dbe20a863f
bytes_received signal: send spider argument 2020-01-27 12:28:54 -03:00
Eugenio Lacuesta 72b8613ee9
bytes_received signal (no tests) 2020-01-27 12:06:48 -03:00
Eugenio Lacuesta 8529dff41d
Update docs regarding Response.ip_address and IPv6 2020-01-26 18:00:56 -03:00
Eugenio Lacuesta 80925ab845
Get server IP address for HTTP/1.1 responses 2020-01-26 16:57:04 -03:00
Peter Vandenabeele f72d4e93e6
[Docs] 2 typos + 1 clarification in docs 
Fixing 2 small typos and adding 1 word as clarification in the downloader-middlewares.

Also, I was confused with the entries like `ref:Reppy <reppy-parser>` and similar entries.
Are these supposed to be links to other parts of the doc, or is this the intended way of showing these references ?
 2020-01-26 10:48:28 +01:00
Peter Vandenabeele f3374a5047 Fix variable name `author_page_links` 
I did not test this code, but the change from `href` to this
author_page_links seems to have a typo ?
 2020-01-25 16:53:29 +01:00
Mikhail Korobov 8b8df31961
Merge pull request #4227 from elacuesta/name-resolver 
Name resolver with IPv6 support
 2020-01-25 05:03:13 +05:00
OmarFarrag fc98aa6b67
Merge branch 'master' into ftp_files#3928 2020-01-24 14:52:40 +02:00
Mikhail Korobov bd54f22fef
Merge pull request #4282 from petervandenabeele/patch-1 
fix logical documentation error with PER_DOMAIN or PER_DOMAIN
 2020-01-24 02:01:22 +05:00
Mikhail Korobov 6a98d660e5
Merge pull request #3551 from jpbalarini/change_scraper_slot 
[MRG+1] Add ability to change max_active_size by setting
 2020-01-23 23:40:05 +05:00
Peter Vandenabeele 7d5cebcf77
fix logical documentation error with PER_DOMAIN or PER_DOMAIN 2020-01-23 09:08:21 +01:00
Eugenio Lacuesta c75cf15b7a
Update CSS selectors in tutorial 2020-01-22 10:38:59 -03:00
JP Balarini 0f2d871d88 Use PEP 515 style for SCRAPER_SLOT_MAX_ACTIVE_SIZE documentation 2020-01-20 11:28:28 -03:00
Juan Pablo Balarini eaa8ed02d0 Add ability to change max_active_size by settings 2020-01-20 11:27:58 -03:00
Eugenio Lacuesta b471765d40
[doc] FAQ entry about the IPv6 and the DNS_RESOLVER setting 2020-01-18 01:52:29 -03:00
Eugenio Lacuesta 302d3f552b
[doc] DNS_RESOLVER setting 2020-01-18 01:46:24 -03:00
Eugenio Lacuesta 2e405d2d5c
Merge branch 'master' into response_follow_all 2020-01-05 00:33:19 -03:00
1um0s 14d4428e70 Rephrasing documentation for image and file pipelines (#4252) 
* scrapy#4034 Clarify documentation for image and file pipelines

* scrapy#4034 Clarify documentation for file pipeline

* scrapy#4034 Simplify documentation for pipeline

* scrapy#4034 Simplify documentation for pipeline

* scrapy#4034 Clarify documentation for image and file pipelines

* scrapy#4034 Clarify documentation for file pipeline

* scrapy#4034 Simplify documentation for pipeline

* scrapy#4034 Simplify documentation for pipeline

* scrapy#4034 Revert image, file pipeline docs. Enhance custom media pipeline docs.

* scrapy#4034 rebase master

* scrapy#4034 Clarify documentation for image and file pipelines

* scrapy#4034 Clarify documentation for file pipeline

* scrapy#4034 Simplify documentation for pipeline

* scrapy#4034 Simplify documentation for pipeline

* scrapy#4034 Clarify documentation for image and file pipelines

* scrapy#4034 Clarify documentation for file pipeline

* scrapy#4034 Simplify documentation for pipeline

* scrapy#4034 Simplify documentation for pipeline

* scrapy#4034 Revert image, file pipeline docs. Enhance custom media pipeline docs.

* scrapy#4034 rebase master

* Rebase master

* Add class to media pipeline docs

Co-Authored-By: elacuesta <elacuesta@users.noreply.github.com>

Co-authored-by: elacuesta <elacuesta@users.noreply.github.com>
 2019-12-30 00:56:22 +05:00
Mikhail Korobov f0ae673452
Merge pull request #4258 from atul-g/patch-1 
Edited the link provided to homepage of lxml's website
 2019-12-30 00:55:15 +05:00
Atul Gopinathan 82861c73c8
Edited the link of the homepage of lxml website 
The link "https://lxml.de" is redirecting to a completely different and unintended website. I changed the link to the index page of lxml's official website. I thought of changing it to the PyPi page of lxml, but even they are providing the same "https://lxml.de" link which doesn't seem to be working now.
 2019-12-27 22:57:58 +05:30
Andrey Rakhmatullin dc1ee09481 Rename ASYNCIO_ENABLED to ASYNCIO_REACTOR, change the logic accordingly. 2019-12-27 21:56:28 +05:00
Eugenio Lacuesta 8a1dc26d46
[doc] Note about the 'parse' method for CrawlSpider/XMLFeedSpider 2019-12-26 15:14:47 -03:00
Eugenio Lacuesta 5982e3477c
Take keyword arguments in base parsing methods 2019-12-23 14:12:21 -03:00
Andrey Rakhmatullin 20289be810 Merge remote-tracking branch 'origin/master' into asyncio-base 2019-12-20 19:31:20 +05:00
Mikhail Korobov fb3fb17c28
Merge pull request #4045 from Gallaecio/documentation-coverage 
Deprecate FilteringLinkExtractor direct usage; add API docs for scrapy.linkextractors
 2019-12-20 02:01:45 +05:00
Mikhail Korobov 6a32a36e09
Merge pull request #4143 from Gallaecio/undent-rst-doctests 
Do not indent doctests from the documentation unnecessarily
 2019-12-20 01:59:07 +05:00
Adrián Chaves b4a2d985a7
Merge branch 'master' into undent-rst-doctests 2019-12-19 12:39:17 +01:00
Marc Hernandez Cabot 16b363de31 Merge branch 'master' into fix_typos 2019-12-19 10:02:21 +01:00
Marc Hernandez Cabot 23a67cec27 fix first letter capitalization for Raring and Scrapy 2019-12-19 09:57:17 +01:00
Mikhail Korobov 8f7faaa63d
Merge pull request #4190 from Gallaecio/doctest 
Make developer-tools doctests pass
 2019-12-18 23:40:21 +05:00
Marc Hernandez Cabot c0d84f0962 fix typos 2019-12-18 19:39:21 +01:00
elacuesta 916382e109 Add errback parameter to scrapy.spiders.crawl.Rule (#4000) 
* Add errback parameter to scrapy.spiders.crawl.Rule

* CrawlSpider: optimize by reducing iterations

* [test] Rule.errback

* [doc] Rule.errback

* [doc] Use autoclass in docs/topics/spiders.rst

Co-Authored-By: Adrián Chaves <adrian@chaves.io>

* Rule.process_links takes a list

* Fix aesthetic issue reported by Flake8
 2019-12-18 20:05:33 +05:00
Adrián Chaves 8c5a8a02ae
Merge branch 'master' into pylint 2019-12-18 15:43:40 +01:00
Mikhail Korobov 053319334c
Merge pull request #4179 from Gallaecio/user-friendlier-tox 
Make tox configuration more user friendly
 2019-12-18 18:35:19 +05:00
Vostretsov Nikita 012533924a remove requirements from here too 2019-12-18 11:13:36 +00:00
Adrián Chaves 607815dfc2
Merge branch 'master' into documentation-coverage 2019-12-17 16:54:27 +01:00
Adrián Chaves b8cf522916
Merge branch 'master' into user-friendlier-tox 2019-12-17 16:53:08 +01:00
Adrián Chaves ab13221b0b Merge remote-tracking branch 'upstream/master' into allow-customizing-export-column-names 2019-12-17 14:06:14 +01:00
marc a59bb279d1 add year through code 2019-12-15 17:33:00 +01:00
marc 1aab20e1ce update copyright notice year 2019-12-14 10:37:31 +01:00
marc e3a3ad4aaf remove reference to old (Python 2.7) environment 2019-12-14 10:34:31 +01:00
Andrey Rakhmatullin 3560123090 Rename ASYNCIO_SUPPORT to ASYNCIO_ENABLED. 2019-12-13 18:07:47 +05:00
Andrey Rakhmatullin 352ddbb336 Merge remote-tracking branch 'origin/master' into asyncio-base 2019-12-13 18:07:23 +05:00
Eugenio Lacuesta 21a2f8c6f8
Merge branch 'master' into response_follow_all 2019-12-10 17:30:59 -03:00
Adrián Chaves 1fc2b140c1
Merge branch 'master' into documentation-coverage 2019-12-05 14:43:36 +01:00
Adrián Chaves d7b1c138f1
Merge branch 'master' into user-friendlier-tox 2019-12-05 14:02:24 +01:00
Adrián Chaves 83b8046fdc Do not indent doctests from the documentation unnecessarily 2019-12-05 13:52:31 +01:00
Adrián Chaves 57e36b5f0d
Merge branch 'master' into spider-name-collision 2019-12-05 13:03:43 +01:00
Wang Qin af624ef414
Update overview.rst | Fix an inconsistency 
There exists an inconsistency between the code (line 37 - 38) and the output 'quotes.json' (line 56 - 68). 

Note that even though according to line 53 - 54  'quotes.json' is "reformatted here for better readability", it cannot explain why the "author" field precedes the "text" field. 

Intended output for the code BEFORE change: 
    [{
        "text": "\u201cThe person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.\u201d",
        "author": "Jane Austen"
    },
    {
        "text": "\u201cOutside of a dog, a book is man's best friend. Inside of a dog it's too dark to read.\u201d",
        "author": "Groucho Marx"
    },
    {
        "text": "\u201cA day without sunshine is like, you know, night.\u201d",
        "author": "Steve Martin"
    },
    ...]

Intended output for the code After change (the inconsistency is fixed): 
    [{
        "author": "Jane Austen",
        "text": "\u201cThe person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.\u201d"
    },
    {
        "author": "Groucho Marx",
        "text": "\u201cOutside of a dog, a book is man's best friend. Inside of a dog it's too dark to read.\u201d"
    },
    {
        "author": "Steve Martin",
        "text": "\u201cA day without sunshine is like, you know, night.\u201d"
    },
    ...]
 2019-12-05 09:29:12 +08:00
Andrey Rakhmatullin ed34ce14c0 Add the ASYNCIO_SUPPORT setting, reshuffle other logic accordingly. 2019-12-04 21:32:56 +05:00
Mikhail Korobov 9b7452211a
Merge pull request #4099 from BurnzZ/itemloader-docs 
update docs of scrapy.loader.ItemLoader.item
 2019-12-03 13:14:45 +05:00
Adrián Chaves 5980b0f284 Don’t use follow_all where a single item is expected (#4) 2019-12-02 12:47:44 -03:00
Eugenio Lacuesta 048cd74ae5
Add separate mapping for Twisted API docs 2019-11-27 19:16:18 -03:00
Eugenio Lacuesta 17e6481823
[Docs] Fix Twisted links 2019-11-27 18:42:42 -03:00
Adrián Chaves b73fc99b60 Use InterSphinx for coverage links 2019-11-26 10:31:55 +01:00
Adrián Chaves dd12f5fdcd Use Response.follow_all in the documentation where appropiate 2019-11-25 11:59:59 -03:00
Adrián Chaves 54b056c4be Make developer-tools doctests pass 2019-11-25 12:32:35 +01:00
Eugenio Lacuesta 40b5cfc0a4
Item loaders: allow single-argument processors (unbound methods) 2019-11-22 20:47:22 -03:00
Eugenio Lacuesta 6fa8f06b64
Merge remote-tracking branch 'upstream/master' into response_follow_all 2019-11-22 13:05:06 -03:00
Mikhail Korobov 0d416c6191
Merge pull request #4185 from Gallaecio/intersphinx 
Use InterSphinx for links to the pytest and tox documentation
 2019-11-21 23:27:57 +05:00
Mikhail Korobov 82c01c261f
Merge pull request #4184 from Gallaecio/doctest 
Make debug doctests pass
 2019-11-21 20:53:56 +05:00
Adrián Chaves fcfcabf1bd Use InterSphinx for links to the pytest and tox documentation 2019-11-21 12:15:13 +01:00
Adrián Chaves f251dda268 Make debug doctests pass 2019-11-21 11:59:10 +01:00
Adrián Chaves 4f80eff1e1 Enable sphinx-hoverxref for all references 2019-11-21 10:30:21 +01:00
Mikhail Korobov f1fd7ec318
Merge pull request #4171 from Gallaecio/doctest 
Make command doctests pass
 2019-11-20 19:47:22 +05:00
Mikhail Korobov d69e788fec
Merge pull request #4172 from Gallaecio/intersphinx 
Use InterSphinx to link to the Twisted documentation
 2019-11-20 19:38:50 +05:00
Mikhail Korobov 1f61b6b941
Merge pull request #4173 from Gallaecio/sphinx-extensions-2 
Add tooltips to documentation cross-references
 2019-11-20 19:38:05 +05:00
Eugenio Lacuesta 48223c107a
Merge remote-tracking branch 'upstream/master' into response_follow_all 2019-11-20 09:41:42 -03:00
Adrián Chaves fc3af54dbd Make tox configuration more user friendly 2019-11-20 08:04:17 +01:00
Adrián Chaves fed93515de Add tooltips to documentation cross-references 2019-11-18 16:11:03 +01:00
Adrián Chaves e84cb18ca0 Use InterSphinx to link to the Twisted documentation 2019-11-18 15:50:45 +01:00
Adrián Chaves 74589df02f Make command doctests pass 2019-11-18 14:51:44 +01:00
Adrián Chaves 393a2a1972 Include /requirements-py3.txt from /docs/requirements.txt 2019-11-18 09:15:48 +01:00
Mikhail Korobov a541c329a9
Merge pull request #4152 from Gallaecio/sphinx-extensions 
Install the sphinx-notfound-page Sphinx extension
 2019-11-15 00:36:56 +05:00
Mikhail Korobov 7b4637d083
Merge pull request #4146 from Gallaecio/doctest 
Make tutorial doctests pass
 2019-11-15 00:19:20 +05:00
Mikhail Korobov 49b5251e06
Merge pull request #4147 from Gallaecio/intersphinx 
Use intersphinx for links to the Sphinx documentation
 2019-11-15 00:16:39 +05:00
Mikhail Korobov cc0026f733
Merge pull request #4148 from Gallaecio/doc-view-code 
Allow opening the source code from the API documentation
 2019-11-14 23:43:44 +05:00
Andrey Rakhmatullin dd367438fa Improve the dbm module ref. 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2019-11-14 13:45:39 +05:00
Andrey Rakhmatullin a9c891399d Fix a duplicate ref name in docs. 2019-11-14 13:45:39 +05:00
Andrey Rakhmatullin 1d7c8cb0b1 Remove six.PY2 and six.PY3 conditionals. 2019-11-14 13:45:37 +05:00
Adrián Chaves 76c31094df Install the sphinx-notfound-page Sphinx extension 2019-11-13 09:28:48 +01:00
Adrián Chaves b642a1fca2
Fix doctest skipping based on the running Python version 2019-11-13 09:14:20 +01:00
Adrián Chaves 414e6e2fd5 Skip a doctest in Python 3.5- because of dictionary changes 2019-11-13 08:40:31 +01:00
Andrey Rahmatullin c911e80209
Merge pull request #4088 from ammarnajjar/4086-constructor-initializer-docs 
docs: use __init__ method instead of constructor
 2019-11-12 21:48:16 +05:00
Andrey Rahmatullin d8098350f0
Merge pull request #3960 from thernstig/issue_2149 
Update documentation for logging manually
 2019-11-12 16:17:49 +05:00
Adrián Chaves 8a6a063778 Allow opening the source code from the API documentation 2019-11-12 10:23:19 +01:00
Adrián Chaves 7b7bb028f4 Use intersphinx for links to the Sphinx documentation 2019-11-12 08:49:06 +01:00
Adrián Chaves 79d2f99995 Make tutorial doctests pass 2019-11-12 08:08:50 +01:00
Adrián Chaves ecc46ac741
Merge branch 'master' into pylint 2019-11-11 10:45:25 +01:00
Andrey Rahmatullin d267e183b4
Merge pull request #4139 from Gallaecio/jobdir-pickle-documentation 
Improve the details about request serialization requirements for JOBDIR
 2019-11-08 22:49:33 +05:00
Adrián Chaves 1df5755699 Set the bases for testing code examples from the documentation 2019-11-08 16:00:10 +01:00
Adrián Chaves 40ed184914
Merge branch 'master' into pylint 2019-11-07 18:40:58 +01:00
Adrián Chaves aef98188fa Improve the details about request serialization requirements for JOBDIR 2019-11-07 18:06:55 +01:00
Andrey Rahmatullin 229e722a03 Initial Python 2 removal (#4091) 2019-10-31 10:46:02 +01:00
Adrián Chaves 94f060fcc8
Cover Scrapy 1.8.0 in the release notes (#3952) 2019-10-29 12:53:46 +01:00
Andrey Rahmatullin 18b808b2e9
Merge pull request #4092 from further-reading/master 
Add Python 3.8 official support
 2019-10-29 16:30:58 +05:00
Adrián Chaves 3d0df419c4 Mark the LevelDB storage backend as deprecated 2019-10-28 11:24:47 +01:00
elacuesta 7731814cc2 ItemLoader: improve handling of initial item (#4036) 2019-10-28 10:53:53 +01:00
Adrián Chaves bb91f9c78c Cover Scrapy 1.7.4 in the release notes 2019-10-28 09:58:13 +01:00
Kevin Lloyd Bernal 84fe4011b0 update docs of scrapy.loader.ItemLoader.item 2019-10-23 20:39:53 +08:00
Adrián Chaves 3b6f7ac9f2 Use pylint 2019-10-22 19:44:11 +02:00
Adrián Chaves 5479e7ecc7 Indicate that lists of emails may be provided as a single string or as a list of strings 2019-10-22 15:24:44 +02:00
Ammar Najjar d21e1034f0
docs: correct point,comma and plural replacements 
Issue #4086
 2019-10-22 13:29:38 +02:00
WinterComes 5d75ed4cba Remove an old note about contracts (#4093) 2019-10-22 12:19:07 +02:00
illgitthat 07822935ec Updating link for miniconda (#4089) 2019-10-22 12:05:34 +02:00
Ammar Najjar da8cd9448d
docs: always surround __init__ with `` in docs 
Issue #4086
 2019-10-21 19:56:58 +02:00
Adrián Chaves ad6075440c Fix references to Python types in parameter type fields 2019-10-21 19:00:03 +02:00
Ammar Najjar 68a7d05ed8
docs: use __init__ method instead of constructor 
Issue #4086
 2019-10-21 15:42:24 +02:00
Adrián Chaves 0fbd1ff4a9 constructor → __init__ method 2019-10-21 14:06:45 +02:00
Mikhail Korobov 8cb53441b0
Merge pull request #4056 from Gallaecio/documentation-build 
Fix internal links in the tutorial and release notes
 2019-10-19 00:00:02 +02:00
Bulat 84be6a941e Refactor sentence. 2019-10-16 14:04:07 +03:00
Bulat c9614a5bdd Fixed BOT_NAME documentation 2019-10-16 12:07:19 +03:00
Adrián Chaves d72ed46fe0 Improve how extra Item API members are introduced in the documentation 2019-10-15 16:04:26 +02:00
Baron Hou 7b1e69dec4 reponse → response (#4079) 2019-10-15 14:51:15 +02:00
Eugenio Lacuesta 5f168cd459
Response.follow_all 2019-10-08 16:32:56 -03:00
John Bampton 39c9a3cc1c Fix case of GitHub. 2019-10-05 10:09:14 +10:00
Adrián Chaves a4aa5b8926 Fix internal links in the tutorial and release notes 2019-10-02 14:51:06 +02:00
elacuesta 6ad5a89cb0 [Doc] Use autoclass in topics/request-response.rst (#4055) 2019-10-02 12:18:36 +02:00
Mikhail Korobov 74b4a5c77c
Merge pull request #4006 from whalebot-helmsman/protego-default 
Use protego as a default robots.txt parser
 2019-10-02 00:03:50 +05:00
Kristobal Junta c232bbdc42 fix typo in docs/topics/spiders 2019-10-01 17:41:38 +03:00
OmarFarrag 175cd2ece5
Update docs/topics/media-pipeline.rst 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2019-10-01 07:27:31 +02:00
Adrián Chaves 7f4f98fd38 Provide complete API documentation coverage of scrapy.linkextractors 2019-09-30 18:22:28 +02:00