RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,119 Commits 49 Branches 132 Tags 131 MiB
Commit Graph

10119 Commits

Author SHA1 Message Date
Andrey Rakhmatullin f2fb4760d2 Bump version: 2.10.1 → 2.11.0 2023-09-18 15:24:28 +04:00
Andrey Rakhmatullin efc594b53f
Merge pull request #6048 from wRAR/relnotes-2.11 
Release notes for 2.11.0
 2023-09-18 15:15:52 +04:00
Andrey Rakhmatullin 528911da85 Fix/reword the release notes. 2023-09-18 14:35:28 +04:00
Andrey Rakhmatullin 2fa768399a Replace the VERSION vars. 2023-09-15 19:19:42 +04:00
Andrey Rakhmatullin c2346b4a95 Update the release notes up to current master. 2023-09-15 19:15:05 +04:00
Andrey Rakhmatullin 3f34a5b151
Merge pull request #6050 from wRAR/retrymw-fix 
Improve the backwards compatibility for RetryMiddleware.EXCEPTIONS_TO_RETRY
 2023-09-15 17:51:04 +04:00
Andrey Rakhmatullin 800c1f112e Remove the constant. 2023-09-15 13:13:06 +04:00
Andrey Rakhmatullin 14c27d2215 Merge branch 'master' into relnotes-2.11 2023-09-14 16:38:15 +04:00
Andrey Rakhmatullin 922ff57384 Improve the backwards compatibility for RetryMiddleware.EXCEPTIONS_TO_RETRY. 2023-09-14 15:51:00 +04:00
Andrey Rakhmatullin dba37674e6
Merge pull request #6038 from wRAR/change-init-order 
Change extensions/spiders/settings initialisation order, v2
 2023-09-14 11:44:34 +04:00
Andrey Rakhmatullin f96a3ed5f0 Cover up to cddb8c15d in the release notes. 2023-09-13 20:46:55 +04:00
Andrey Rakhmatullin 8dd48a08e4 Move PeriodicLog docs from Debugging to General purpose. 2023-09-13 20:46:32 +04:00
Andrey Rakhmatullin 6428356584 Use twisted.python.log instead of failure.printTraceback(). 2023-09-13 18:20:42 +04:00
Andrey Rakhmatullin be0e33af92 Don't rely on unhandled exception logging in the expected to fail scripts. 2023-09-13 16:59:54 +04:00
Andrey Rakhmatullin ac201d310b Small improvements, 2023-09-13 16:17:52 +04:00
Andrey Rakhmatullin 61ef37a594 Docs improvements. 2023-09-13 15:47:41 +04:00
Andrey Rakhmatullin 3756216339 Rename methods. 2023-09-13 15:47:22 +04:00
Andrey Rakhmatullin 619140717f Fix the new addon test. 2023-09-13 15:33:51 +04:00
Andrey Rakhmatullin 028a56b9a2 Improve and simplify tests. 2023-09-13 14:17:41 +04:00
Andrey Rakhmatullin 61e6bfc023 Docs improvements. 2023-09-13 12:35:15 +04:00
Andrey Rakhmatullin 62183dc25f Merge branch 'master' into change-init-order 2023-09-12 22:16:21 +04:00
Andrey Rakhmatullin da39fbd270
Update tool versions (#6046) 2023-09-12 20:07:10 +02:00
Andrey Rakhmatullin a3f22046ef Document changing settings in Spider.from_crawler(). 2023-09-12 20:41:36 +04:00
Andrey Rakhmatullin 77f39be407 Add integration tests for modifying settings via args. 2023-09-12 20:16:26 +04:00
Andrey Rakhmatullin e26bf4f918
Pin brotli for PyPy tests (#6045) 2023-09-12 16:18:25 +02:00
Andrey Rakhmatullin 1a0572ad02 Reinstall the log handler after loading per-spider settings. 2023-09-12 16:46:35 +04:00
Andrey Rakhmatullin bb15c93a2b Add Crawler._load_settings(). 2023-09-12 14:44:16 +04:00
Andrey Rakhmatullin 6629a61dd9 Fix one more testcase. 2023-09-08 20:15:49 +04:00
Andrey Rakhmatullin 036d5836d0 Move more things from Crawler.__init__() to .crawl(). 2023-09-08 19:29:05 +04:00
Andrey Rakhmatullin 97b98bf181 Roll back the update_settings() changes. 2023-09-07 17:02:37 +04:00
Maria Mitropoulou 721df895f9
Fix charset handling in MailSender #5096 (#5118) 2023-09-07 07:47:42 +02:00
Andrey Rakhmatullin b4380995da Merge remote-tracking branch 'origin/master' into change-init-order 2023-09-06 20:34:27 +04:00
Andrey Rakhmatullin 00527fdcbe
Merge pull request #6040 from wRAR/deprecate-double-crawl 
Deprecate running Crawler.crawl() twice.
 2023-09-06 20:32:14 +04:00
Andrey Rakhmatullin b39d2d4353 Add a test. 2023-09-06 19:44:22 +04:00
Andrey Rakhmatullin d3b5c9be97 Deprecate running Crawler.crawl() twice. 2023-09-06 19:29:44 +04:00
Andrey Rakhmatullin df112a3996 Move reactor installation into Crawler.crawl(). 2023-09-05 19:43:08 +04:00
Andrey Rakhmatullin 69282c5318 Merge branch 'master' into change-init-order 2023-09-05 19:23:36 +04:00
Víctor Ruiz c1dd5493ac
Fix enabled addons not being logged (#6036) 2023-09-04 08:03:35 +02:00
Andrey Rakhmatullin 276bce0641
Update type hints for Twisted 23.8.0. (#6034) 2023-09-01 08:47:23 +02:00
Andrey Rakhmatullin d9efb94b61 Merge branch '2.10' 2023-08-31 13:21:22 +04:00
Mikhail Korobov 0fcb0554d1
Merge pull request #6016 from jxlil/fix/response.json 
Fix: response.json() call makes unnecessary memory allocation
 2023-08-31 14:07:24 +05:00
Andrey Rakhmatullin cddb8c15d6
Merge pull request #5926 from GeorgeA92/periodic_log_2 
periodic_log: implemented as separate extension
 2023-08-30 22:29:54 +04:00
Andrey Rakhmatullin a320e5f6a4 Bump version: 2.10.0 → 2.10.1 2023-08-30 12:08:36 +04:00
Andrey Rakhmatullin 99c826ce4d
Merge pull request #6027 from wRAR/relnotes-2.10.1 
Release notes for 2.10.1.
 2023-08-30 12:04:48 +04:00
Andrey Rakhmatullin 3fc7726e65
Merge pull request #6026 from wRAR/restrict-twisted 
Forbid Twisted 23.8.0+
 2023-08-29 18:26:19 +04:00
Andrey Rakhmatullin f8f550120b Release notes for 2.10.1. 2023-08-29 18:07:15 +04:00
Andrey Rakhmatullin 9a72a2550c Forbid Twisted 23.8.0+. 2023-08-29 17:00:28 +04:00
Andrey Rakhmatullin 0e7fc60b89
Merge pull request #6021 from dveselov/master-1 
Add .webp to link extractor IGNORED_EXTENSIONS list
 2023-08-21 22:04:03 +04:00
Jalil SA 4dd32672ed added entry to Backward-incompatible changes 2023-08-21 08:28:08 -06:00
Georgiy Zatserklianyi fe02642980 Merge remote-tracking branch 'origin/periodic_log_2' into periodic_log_2 2023-08-21 17:15:42 +03:00