RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,184 Commits 49 Branches 132 Tags 130 MiB
Commit Graph

10184 Commits

Author SHA1 Message Date
Adrián Chaves 2f1d345e74 Solve test issues 2024-02-14 18:59:01 +01:00
Adrián Chaves 502addc717 Bump version: 2.11.0 → 2.11.1 2024-02-14 18:17:48 +01:00
Adrián Chaves 6b88b3346c Set the release date of versions 2.11.1 and 1.8.4 2024-02-14 18:16:40 +01:00
Adrián Chaves 479619b340 Merge branch '2.11-redos' into 2.11 2024-02-14 17:18:26 +01:00
Adrián Chaves 809bfac489 Merge branch '2.11-compression-bomb' into 2.11 2024-02-14 17:16:25 +01:00
Adrián Chaves 5bcb8fd501 Merge branch '2.11-authorization' into 2.11 2024-02-14 17:14:29 +01:00
Andrey Rakhmatullin a55e933c11
Release notes for 2.11.1 (#6150) 2024-02-14 17:08:40 +01:00
Adrián Chaves 5e5a92026e Remove slow leftovers 2024-02-02 14:06:45 +01:00
Adrián Chaves 810aaa637d Undo an unintended change 2024-02-02 14:04:28 +01:00
Adrián Chaves c5dad41190 Speed up tests, remove comments without regexps 2024-02-02 14:03:16 +01:00
Andrey Rakhmatullin 09a7efef7c Remove a defer.returnValue call. 2024-01-15 18:19:29 +04:00
Adrián Chaves 6127f7d278 Update quotes.toscrape.com page copies (#6190) 2024-01-01 22:58:34 +04:00
Chan Sau Yee badc7c5be9 Update black reference in docs (#6192) 2024-01-01 22:58:21 +04:00
Adrián Chaves 48a9a58ff2 Link to Zyte’s export guides 2023-12-20 21:43:21 +04:00
Adrián Chaves 369712ee50 SPM → Zyte API (#6163) 2023-12-20 21:43:13 +04:00
Adrián Chaves b095dd218f Extend Request.meta documentation (#5565) 2023-12-20 21:43:03 +04:00
Adrián Chaves 27781a85e7 Fix bad closing tags in XMLFeedSpider tests 2023-12-15 13:52:12 +01:00
Adrián Chaves c7c7a488b9 Fix typing issues 2023-12-15 13:18:23 +01:00
Adrián Chaves bc138ef8e9 Minor release notes fix 2023-12-15 12:24:04 +01:00
Adrián Chaves ce9d290eff Remove the lxml version check for huge_tree on xmliter_lxml 
iterparse supports the option since lxml 2.2.1, it was the HTML parser that only got it in 4.2
 2023-12-15 12:23:26 +01:00
Adrián Chaves a49c8762dd Avoid calling iterparse twice 2023-12-15 12:14:53 +01:00
Adrián Chaves cd6846b763 Merge remote-tracking branch 'scrapy/2.11' into 2.11-redos 2023-12-15 11:52:35 +01:00
Adrián Chaves b0dbd0e9af Minor naming changes 2023-12-15 11:49:22 +01:00
Adrián Chaves 150d96764b Deprecate xmliter in favor of xmliter_lxml 2023-12-15 11:42:55 +01:00
Adrián Chaves 2538c0e862 Restore the implementation of xmliter 2023-12-15 10:23:24 +01:00
Adrián Chaves 9655b0b8eb Mark slow tests, with their own tox env and CI job 2023-12-15 10:19:47 +01:00
Adrián Chaves d50f436a73 Enable huge_tree for xmliter_lxml 2023-12-15 10:08:45 +01:00
Adrián Chaves 4f72b49f97 Fix namespaces nodename support for xmliter_lxml 2023-12-15 10:06:13 +01:00
Adrián Chaves 12b10a7a64 Cover scrapy.downloadermiddlewares.decompression in the release notes 2023-12-13 13:35:05 +01:00
Laerte Pereira b9c4ee26d7 Remove deprecated scrapy.downloadermiddlewares.decompression 2023-12-13 13:28:30 +01:00
Adrián Chaves 1533b69032 Test and address ReDoS attack vectors for open_in_browser 2023-12-13 12:01:35 +01:00
Adrián Chaves bb74badd1b spider → mw 2023-12-11 17:39:55 +01:00
Adrián Chaves 40b3efbbee Remove open_in_browser from the 1.8.4 release notes 2023-11-29 12:47:04 +01:00
Adrián Chaves eb8b2c5197 Mention open_in_browser in the release notes 2023-11-29 12:13:04 +01:00
Adrián Chaves c947f51077 Set an arbitrary upper limit on ReDoS-vulnerable regexps 2023-11-29 11:54:08 +01:00
Adrián Chaves a113208a06 Fix BytesIO non-emptiness check 2023-11-24 11:35:15 +01:00
Adrián Chaves 8a73c6c90c Fix HttpCompressionMiddleware backward compatibility 2023-11-24 10:25:01 +01:00
Adrián Chaves 62398e424c CHUNK_SIZE: 8 KiB → 32 KiB 2023-11-24 10:20:14 +01:00
Adrián Chaves 6c278e1862 List[bytes] → BytesIO 2023-11-24 10:15:38 +01:00
Adrián Chaves 5f2827efe7 Make HttpCompressionMiddleware changes backward-comaptible 2023-11-24 10:15:38 +01:00
Adrián Chaves 8e25f8c157
Fix bad message 2023-11-23 14:12:59 +01:00
Adrián Chaves cf80e5670e Solve linting and typing issues 2023-11-23 12:07:15 +01:00
Adrián Chaves b53ed52a22 Update the release notes 2023-11-23 11:36:45 +01:00
Adrián Chaves 03d9866518 Also use DOWNLOAD_WARNSIZE for decompressions 2023-11-23 10:26:47 +01:00
Adrián Chaves 1087bb7b2e Update the docs 2023-11-23 09:11:14 +01:00
Adrián Chaves e0b66c021a Mind Spider.download_maxsize and Request.meta['download_maxsize'] 2023-11-22 19:03:24 +01:00
Adrián Chaves 3fda2fe103 Protect against gzip bomb sitemaps 2023-11-22 18:34:37 +01:00
Adrián Chaves 9cc8703877 Protect against zstandard bombs 2023-11-22 17:53:00 +01:00
Adrián Chaves fba167c5e1 Protect against brotli bombs 2023-11-22 17:32:09 +01:00
Adrián Chaves 0c4a98f8e0 Protect against deflate bombs 2023-11-22 17:12:43 +01:00