RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,125 Commits 49 Branches 132 Tags 129 MiB
Commit Graph

10125 Commits

Author SHA1 Message Date
Adrián Chaves 4c98d6068a Add a template for question/help issues 2023-09-25 10:18:35 +02:00
Yash nagarkar 908da8ba82
Cover the removal of is_botocore on the release notes (#6061) 2023-09-22 10:12:20 +02:00
Andrey Rakhmatullin 9bfe0def59
Merge pull request #6059 from Gallaecio/typing 
Add py.typed
 2023-09-21 17:30:04 +04:00
Adrián Chaves 11bdc3df59 Revert "Add package_data={"scrapy": "py.typed"} to setup.py" 
This reverts commit e84fb6d5cb.
 2023-09-21 13:40:40 +02:00
Adrián Chaves e84fb6d5cb Add package_data={"scrapy": "py.typed"} to setup.py 2023-09-21 13:39:39 +02:00
Adrián Chaves d5cc469ca9 Add py.typed 2023-09-21 13:06:12 +02:00
Andrey Rakhmatullin f2fb4760d2 Bump version: 2.10.1 → 2.11.0 2023-09-18 15:24:28 +04:00
Andrey Rakhmatullin efc594b53f
Merge pull request #6048 from wRAR/relnotes-2.11 
Release notes for 2.11.0
 2023-09-18 15:15:52 +04:00
Andrey Rakhmatullin 528911da85 Fix/reword the release notes. 2023-09-18 14:35:28 +04:00
Andrey Rakhmatullin 2fa768399a Replace the VERSION vars. 2023-09-15 19:19:42 +04:00
Andrey Rakhmatullin c2346b4a95 Update the release notes up to current master. 2023-09-15 19:15:05 +04:00
Andrey Rakhmatullin 3f34a5b151
Merge pull request #6050 from wRAR/retrymw-fix 
Improve the backwards compatibility for RetryMiddleware.EXCEPTIONS_TO_RETRY
 2023-09-15 17:51:04 +04:00
Andrey Rakhmatullin 800c1f112e Remove the constant. 2023-09-15 13:13:06 +04:00
Andrey Rakhmatullin 14c27d2215 Merge branch 'master' into relnotes-2.11 2023-09-14 16:38:15 +04:00
Andrey Rakhmatullin 922ff57384 Improve the backwards compatibility for RetryMiddleware.EXCEPTIONS_TO_RETRY. 2023-09-14 15:51:00 +04:00
Andrey Rakhmatullin dba37674e6
Merge pull request #6038 from wRAR/change-init-order 
Change extensions/spiders/settings initialisation order, v2
 2023-09-14 11:44:34 +04:00
Andrey Rakhmatullin f96a3ed5f0 Cover up to cddb8c15d in the release notes. 2023-09-13 20:46:55 +04:00
Andrey Rakhmatullin 8dd48a08e4 Move PeriodicLog docs from Debugging to General purpose. 2023-09-13 20:46:32 +04:00
Andrey Rakhmatullin 6428356584 Use twisted.python.log instead of failure.printTraceback(). 2023-09-13 18:20:42 +04:00
Andrey Rakhmatullin be0e33af92 Don't rely on unhandled exception logging in the expected to fail scripts. 2023-09-13 16:59:54 +04:00
Andrey Rakhmatullin ac201d310b Small improvements, 2023-09-13 16:17:52 +04:00
Andrey Rakhmatullin 61ef37a594 Docs improvements. 2023-09-13 15:47:41 +04:00
Andrey Rakhmatullin 3756216339 Rename methods. 2023-09-13 15:47:22 +04:00
Andrey Rakhmatullin 619140717f Fix the new addon test. 2023-09-13 15:33:51 +04:00
Andrey Rakhmatullin 028a56b9a2 Improve and simplify tests. 2023-09-13 14:17:41 +04:00
Andrey Rakhmatullin 61e6bfc023 Docs improvements. 2023-09-13 12:35:15 +04:00
Andrey Rakhmatullin 62183dc25f Merge branch 'master' into change-init-order 2023-09-12 22:16:21 +04:00
Andrey Rakhmatullin da39fbd270
Update tool versions (#6046) 2023-09-12 20:07:10 +02:00
Andrey Rakhmatullin a3f22046ef Document changing settings in Spider.from_crawler(). 2023-09-12 20:41:36 +04:00
Andrey Rakhmatullin 77f39be407 Add integration tests for modifying settings via args. 2023-09-12 20:16:26 +04:00
Andrey Rakhmatullin e26bf4f918
Pin brotli for PyPy tests (#6045) 2023-09-12 16:18:25 +02:00
Andrey Rakhmatullin 1a0572ad02 Reinstall the log handler after loading per-spider settings. 2023-09-12 16:46:35 +04:00
Andrey Rakhmatullin bb15c93a2b Add Crawler._load_settings(). 2023-09-12 14:44:16 +04:00
Andrey Rakhmatullin 6629a61dd9 Fix one more testcase. 2023-09-08 20:15:49 +04:00
Andrey Rakhmatullin 036d5836d0 Move more things from Crawler.__init__() to .crawl(). 2023-09-08 19:29:05 +04:00
Andrey Rakhmatullin 97b98bf181 Roll back the update_settings() changes. 2023-09-07 17:02:37 +04:00
Maria Mitropoulou 721df895f9
Fix charset handling in MailSender #5096 (#5118) 2023-09-07 07:47:42 +02:00
Andrey Rakhmatullin b4380995da Merge remote-tracking branch 'origin/master' into change-init-order 2023-09-06 20:34:27 +04:00
Andrey Rakhmatullin 00527fdcbe
Merge pull request #6040 from wRAR/deprecate-double-crawl 
Deprecate running Crawler.crawl() twice.
 2023-09-06 20:32:14 +04:00
Andrey Rakhmatullin b39d2d4353 Add a test. 2023-09-06 19:44:22 +04:00
Andrey Rakhmatullin d3b5c9be97 Deprecate running Crawler.crawl() twice. 2023-09-06 19:29:44 +04:00
Andrey Rakhmatullin df112a3996 Move reactor installation into Crawler.crawl(). 2023-09-05 19:43:08 +04:00
Andrey Rakhmatullin 69282c5318 Merge branch 'master' into change-init-order 2023-09-05 19:23:36 +04:00
Víctor Ruiz c1dd5493ac
Fix enabled addons not being logged (#6036) 2023-09-04 08:03:35 +02:00
Andrey Rakhmatullin 276bce0641
Update type hints for Twisted 23.8.0. (#6034) 2023-09-01 08:47:23 +02:00
Andrey Rakhmatullin d9efb94b61 Merge branch '2.10' 2023-08-31 13:21:22 +04:00
Mikhail Korobov 0fcb0554d1
Merge pull request #6016 from jxlil/fix/response.json 
Fix: response.json() call makes unnecessary memory allocation
 2023-08-31 14:07:24 +05:00
Andrey Rakhmatullin cddb8c15d6
Merge pull request #5926 from GeorgeA92/periodic_log_2 
periodic_log: implemented as separate extension
 2023-08-30 22:29:54 +04:00
Andrey Rakhmatullin a320e5f6a4 Bump version: 2.10.0 → 2.10.1 2023-08-30 12:08:36 +04:00
Andrey Rakhmatullin 99c826ce4d
Merge pull request #6027 from wRAR/relnotes-2.10.1 
Release notes for 2.10.1.
 2023-08-30 12:04:48 +04:00