RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,393 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

2705 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 75e99c75b3 Improve the docs about Crawler attributes and settings initialization. 2023-11-16 12:36:57 +04:00
Andrey Rakhmatullin 603aa4924a Improve the docs about Crawler attributes and settings initialization. 2023-11-15 19:51:04 +04:00
Adrián Chaves 080fecd890 Drop the Authorization header on cross-domain redirect 2023-11-15 15:39:30 +01:00
Andrey Rakhmatullin 5fccf370b8 Update the RTD URL for coverage 2023-11-15 15:38:13 +01:00
Jessica Allman-LaPorte 6587556af9
Make shell switching more clear in the tutorial (#6128) 2023-11-03 10:02:18 +01:00
nihilisticneuralnet 04024f1e79
[Solved] JOBDIR= None for when Scheduler initializes disk queue even if JOBDIR is empty string (#6124) 
Co-authored-by: John Doe <johndoe@email.com>
 2023-10-31 17:37:26 +04:00
Andrey Rakhmatullin 39ee8d1ee2
Deprecate ReppyRobotParser (#6099) 2023-10-18 09:10:21 +02:00
Sandesh Pyakurel 064256b059
Typo fixed in news.rst file (#6094) 2023-10-17 18:39:39 +02:00
Andrey Rakhmatullin c65567988d
Change supported PyPy versions to 3.9 and 3.10. (#6087) 
* Change support PyPy versions to 3.9 and 3.10.

* Update the RTD URL for coverage.

* Move pypy3-pinned back to pypy3.8.
 2023-10-06 11:27:02 +02:00
Andrey Rakhmatullin a96989c6f3
Merge pull request #6077 from 11-aryan/11-aryan 2023-10-02 20:40:25 +04:00
Adrián Chaves 42adbb2104
Update docs/topics/request-response.rst 2023-10-02 14:43:30 +02:00
kokobhara 8dc72dfc4d Cover PythonItemExporter backwaird-incompatible changes in 2.11 (#6081) 2023-10-02 14:48:44 +04:00
kokobhara ef1ed4fab7
Cover PythonItemExporter backwaird-incompatible changes in 2.11 (#6081) 2023-10-02 12:14:05 +02:00
Aryan e146c3a2fc removed the entire example for dont_merge_cookies 2023-10-02 15:36:29 +05:30
Aryan fe5ef0a80a Fixed the dont_merge_cookies example in docs 2023-10-01 11:47:51 +05:30
Yash nagarkar 197781e3af Cover the removal of is_botocore on the release notes (#6061) 2023-09-24 18:32:35 +04:00
Yash nagarkar 908da8ba82
Cover the removal of is_botocore on the release notes (#6061) 2023-09-22 10:12:20 +02:00
Andrey Rakhmatullin f2fb4760d2 Bump version: 2.10.1 → 2.11.0 2023-09-18 15:24:28 +04:00
Andrey Rakhmatullin 528911da85 Fix/reword the release notes. 2023-09-18 14:35:28 +04:00
Andrey Rakhmatullin 2fa768399a Replace the VERSION vars. 2023-09-15 19:19:42 +04:00
Andrey Rakhmatullin c2346b4a95 Update the release notes up to current master. 2023-09-15 19:15:05 +04:00
Andrey Rakhmatullin 14c27d2215 Merge branch 'master' into relnotes-2.11 2023-09-14 16:38:15 +04:00
Andrey Rakhmatullin f96a3ed5f0 Cover up to cddb8c15d in the release notes. 2023-09-13 20:46:55 +04:00
Andrey Rakhmatullin 8dd48a08e4 Move PeriodicLog docs from Debugging to General purpose. 2023-09-13 20:46:32 +04:00
Andrey Rakhmatullin ac201d310b Small improvements, 2023-09-13 16:17:52 +04:00
Andrey Rakhmatullin 61ef37a594 Docs improvements. 2023-09-13 15:47:41 +04:00
Andrey Rakhmatullin 61e6bfc023 Docs improvements. 2023-09-13 12:35:15 +04:00
Andrey Rakhmatullin a3f22046ef Document changing settings in Spider.from_crawler(). 2023-09-12 20:41:36 +04:00
Andrey Rakhmatullin d3b5c9be97 Deprecate running Crawler.crawl() twice. 2023-09-06 19:29:44 +04:00
Andrey Rakhmatullin d9efb94b61 Merge branch '2.10' 2023-08-31 13:21:22 +04:00
Mikhail Korobov 0fcb0554d1
Merge pull request #6016 from jxlil/fix/response.json 
Fix: response.json() call makes unnecessary memory allocation
 2023-08-31 14:07:24 +05:00
Andrey Rakhmatullin a320e5f6a4 Bump version: 2.10.0 → 2.10.1 2023-08-30 12:08:36 +04:00
Andrey Rakhmatullin f8f550120b Release notes for 2.10.1. 2023-08-29 18:07:15 +04:00
Jalil SA 4dd32672ed added entry to Backward-incompatible changes 2023-08-21 08:28:08 -06:00
Georgiy Zatserklianyi fe02642980 Merge remote-tracking branch 'origin/periodic_log_2' into periodic_log_2 2023-08-21 17:15:42 +03:00
Georgiy Zatserklianyi 1f03cb1419 Merge remote-tracking branch 'upstream/master' into periodic_log_2 2023-08-21 17:07:18 +03:00
Georgiy Zatserklianyi ac1694a9ad
Update docs/topics/extensions.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-11 18:49:09 +03:00
Georgiy Zatserklianyi d67be20b2d
Update docs/topics/extensions.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-11 18:48:58 +03:00
Georgiy Zatserklianyi 3a4a949f9d
Update docs/topics/extensions.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-11 18:48:48 +03:00
Georgiy Zatserklianyi e6bd9829bd
Update docs/topics/extensions.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-11 18:48:31 +03:00
Georgiy Zatserklianyi 2f094a7a5c
Update docs/topics/extensions.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-11 18:48:14 +03:00
Laerte Pereira 9e74748fca
Remove extra spider parameter in item pipeline docs (#6009) 2023-08-10 13:48:43 +02:00
Andrey Rakhmatullin 88327c7c58 Bump version: 2.9.0 → 2.10.0 2023-08-04 17:23:30 +04:00
Andrey Rakhmatullin 4d288c8bf3
Merge pull request #6001 from scrapy/relnotes-2.10 
Add release notes for 2.10.0.
 2023-08-04 17:22:16 +04:00
Andrey Rakhmatullin 022ef0f86b
Apply suggestions from code review 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-04 17:05:09 +04:00
Andrey Rakhmatullin 7fe4c0c9f7
Update docs/news.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-04 16:56:39 +04:00
Andrey Rakhmatullin c14a0a9d5d Add release notes for 2.10.0. 2023-08-04 16:36:27 +04:00
Georgiy Zatserklianyi 8b6a50a935 periodic_log: docs added 2023-08-04 11:12:07 +03:00
Adrián Chaves 72de48be6d asyncio: cover accidental bad reactor installation, sort sections, reword the Windows section 2023-08-04 09:56:30 +02:00
Andrey Rakhmatullin 7c6aaedda2
Merge pull request #5846 from jxlil/docs/spiders 
Added documentation for Spider.update_settings
 2023-08-02 21:09:44 +04:00
Andrey Rakhmatullin af1be835e4
Apply suggestions from code review 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-08-02 19:46:16 +04:00
Andrey Rakhmatullin 72462a53e2 Add more docs for update_settings(). 2023-08-02 12:32:53 +04:00
Andrey Rakhmatullin 06ebdee35d
Merge pull request #5950 from wRAR/addons 
Add-ons
 2023-08-01 22:18:57 +04:00
Andrey Rakhmatullin cf55eb05f5 Fix a typo. 2023-08-01 13:30:56 +04:00
Andrey Rakhmatullin 41a4a163e3 Handle NotConfigured in add-ons. 2023-07-31 21:09:18 +04:00
Andrey Rakhmatullin b67a81b81d Use the MyHandler class directly. 2023-07-31 20:25:56 +04:00
Andrey Rakhmatullin d8c5b41559 Add more addon tests. 2023-07-31 19:09:21 +04:00
Andrey Rakhmatullin bddbbc522a
Merge pull request #5979 from BurnzZ/closespider-timeout-no-item 2023-07-28 12:56:44 +04:00
Andrey Rakhmatullin 3ba2dc4d68 Fixes for addon docs. 2023-07-24 17:49:04 +04:00
Kevin Lloyd Bernal 9a1bf40c2f expose AWS_REGION_NAME in S3FeedStorage 2023-07-24 16:37:52 +08:00
Mikhail Korobov af2aa4b421
Merge pull request #5847 from namelessGonbai/feat/FixFeedExport 
Fix FeedExporter not to export empty file
 2023-07-22 23:18:36 +05:00
Andrey Rakhmatullin 005c8cc5f0 Unify the "add-on" spelling. 2023-07-19 13:15:35 +04:00
Andrey Rakhmatullin 0a25a300cf Fix docs. 2023-07-18 20:29:42 +04:00
Andrey Rakhmatullin 90dae3ee60 Doc fixes. 2023-07-18 19:55:46 +04:00
Kevin Lloyd Bernal 3f5bbe3a8f introduce CLOSESPIDER_TIMEOUT_NO_ITEM in CloseSpider 2023-07-18 18:30:21 +08:00
Andrey Rakhmatullin cdda8ad46d Add docs about fallbacks in addons. 2023-07-17 21:05:58 +04:00
Andrey Rakhmatullin a2264d3b8b Improve docs about setting settings in addons. 2023-07-14 18:57:27 +04:00
Andrey Rakhmatullin c92c9af075 Add create_instance support to addons. 2023-06-29 15:04:46 +04:00
Andrey Rakhmatullin 9612ae3e93 Remove more code. 2023-06-28 18:44:30 +04:00
Andrey Rakhmatullin e7124447f7 Remove unneeded code. 2023-06-26 16:57:46 +04:00
Andrey Rakhmatullin 4e21e3e59d Merge remote-tracking branch 'origin/master' into addons 2023-06-26 14:25:35 +04:00
Alex 04ee3303e4
Adding support for Windows of absolute pathlib.Path objects in FeedExporter (#5939) 2023-06-22 07:04:06 +02:00
Marc Hernández 5360ba34bc
IOError and other cleanup (#4716) 2023-06-21 20:08:53 +02:00
Aaron Smith ee215a2970
Change redirect text from Response.request docs (#5937) 2023-06-21 20:05:39 +02:00
Laerte Pereira 0097b4c0bb cleanup: Remove `pkg_resources` usage 2023-06-20 16:40:38 -03:00
Andrey Rakhmatullin 2122278d4b Drop Python 3.7 support. 2023-06-18 18:37:50 +04:00
Serhii A 777a6ea412
Make the retry middleware exception list configurable (#5929) 2023-06-16 15:46:06 +02:00
Andrey Rakhmatullin f582246d7b More doc fixes. 2023-06-15 18:49:08 +04:00
Andrey Rakhmatullin 2f9ebb66c3 Remove some dead code/docs. 2023-06-15 17:00:38 +04:00
Andrey Rakhmatullin 27f5f35134 More quick doc fixes. 2023-06-14 18:30:33 +04:00
Andrey Rakhmatullin fdfab17438 Fix docs for the renamed ADDONS setting. 2023-06-14 18:00:18 +04:00
Andrey Rakhmatullin 22bd0d9a79 Fix docs build. 2023-06-14 17:58:11 +04:00
Andrey Rakhmatullin 7ebb8256f0 Some cleanup. 2023-06-14 17:52:11 +04:00
Andrey Rakhmatullin 075ad6f196 Test, linter etc. fixes. 2023-06-14 16:34:18 +04:00
Andrey Rakhmatullin 898e3045a1 Merge branch 'master' into addons 2023-06-14 15:37:08 +04:00
Andrey Rakhmatullin 85fe88f80f
Merge pull request #5833 from GodelTech/issue_960_s3_feed_export 
Fix #960: S3 Feed Export throws boto error
 2023-06-13 18:44:06 +04:00
Andrey Rakhmatullin 8a64f3e8de
Merge branch 'master' into feat/FixFeedExport 2023-06-13 18:21:06 +04:00
Andrey Rakhmatullin 33b418dc84
Merge pull request #5927 from bulatbulat48/master 
Add additional requests examples.
 2023-05-16 19:13:02 +04:00
Adrián Chaves d362699fa3 Move inline request examples to the coroutines documentation 2023-05-16 13:39:02 +02:00
Adrián Chaves e4cf8fc121
Update asyncio.rst 2023-05-15 18:51:58 +02:00
isabela_catanante 5adada5d19 Improve the overwrite feed option documentation 2023-05-12 12:55:24 +02:00
bulat 8de2064ba3 add import. 2023-05-11 13:22:33 +05:00
bulat 4878cc7ef0 Add proper imports. 2023-05-11 13:19:40 +05:00
bulat b62c1263de add import to the example. 2023-05-11 13:15:30 +05:00
bulat fc2d1b2171 make example reachable. 2023-05-11 12:56:29 +05:00
bulat 6a0bbad677 Merge branch 'master' of https://github.com/bulatbulat48/scrapy 2023-05-11 12:54:04 +05:00
bulat 6194db1335 Update title. 2023-05-11 12:54:01 +05:00
Bulat Khabibullin 85103b4932
add proper example 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-05-11 12:53:43 +05:00
Bulat Khabibullin 26374e21f8
Update docs/topics/asyncio.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-05-10 18:32:36 +05:00
Bulat Khabibullin 57f3140daa
Update docs/topics/asyncio.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2023-05-10 18:31:54 +05:00