RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,128 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

11128 Commits

Author SHA1 Message Date
Rotzbua 745b8412f6
fix(flake8): lint errors 
E226 missing whitespace around arithmetic operator
E201 whitespace after '{'
 2024-01-05 14:54:00 +01:00
Rotzbua 42c481cb4a
refactor(): use `OSError` exception 
https://docs.astral.sh/ruff/rules/os-error-alias/
 2024-01-05 12:36:36 +01:00
Rotzbua 0d445a3224
refactor(yield): use `yield from` syntax 2024-01-05 12:30:10 +01:00
Rotzbua c7b2b097b1
fix(typo): correct `successfully` 2024-01-05 10:50:45 +01:00
Adrián Chaves 6127f7d278 Update quotes.toscrape.com page copies (#6190) 2024-01-01 22:58:34 +04:00
Chan Sau Yee badc7c5be9 Update black reference in docs (#6192) 2024-01-01 22:58:21 +04:00
Chan Sau Yee 40e623b276
Add type hints (#6191) 2023-12-29 12:33:37 +01:00
Chan Sau Yee 1902284942
Update black reference in docs (#6192) 2023-12-29 12:32:51 +01:00
Adrián Chaves 34e01a8a93
Update quotes.toscrape.com page copies (#6190) 2023-12-28 12:25:01 +01:00
Andrey Rakhmatullin 2534a28ef0 Bump mypy. 2023-12-25 15:03:08 +04:00
Adrián Chaves 0e78acb657
MediaPipeline: log media_to_download errors before stripping them (#5068) 2023-12-21 21:01:07 +01:00
Jalil SA d25cfe5315
Add JsonResponse (#6174) 2023-12-21 10:36:21 +01:00
Adrián Chaves 48a9a58ff2 Link to Zyte’s export guides 2023-12-20 21:43:21 +04:00
Adrián Chaves 369712ee50 SPM → Zyte API (#6163) 2023-12-20 21:43:13 +04:00
Adrián Chaves b095dd218f Extend Request.meta documentation (#5565) 2023-12-20 21:43:03 +04:00
Mikhail Korobov b498f1376d
Merge pull request #6183 from Gallaecio/zyte-guides 
Link to Zyte’s export guides
 2023-12-20 22:22:22 +05:00
Andrey Rakhmatullin f56b5fc39e Bump typing deps. 2023-12-20 16:19:11 +04:00
Andrey Rakhmatullin a72394a388 Add tests for replace() with kwargs. 2023-12-20 16:14:53 +04:00
Andrey Rakhmatullin 1fab844f7d Pin the Python version for typing-tests. 2023-12-20 15:57:51 +04:00
Adrián Chaves 1864f48e9e Link to Zyte’s export guides 2023-12-20 12:47:34 +01:00
Swati Anshu c67f730695
create_instance → build_from_crawler, build_from_settings (#6169) 2023-12-18 11:51:02 +01:00
Adrián Chaves 27781a85e7 Fix bad closing tags in XMLFeedSpider tests 2023-12-15 13:52:12 +01:00
Adrián Chaves c7c7a488b9 Fix typing issues 2023-12-15 13:18:23 +01:00
Adrián Chaves bc138ef8e9 Minor release notes fix 2023-12-15 12:24:04 +01:00
Adrián Chaves ce9d290eff Remove the lxml version check for huge_tree on xmliter_lxml 
iterparse supports the option since lxml 2.2.1, it was the HTML parser that only got it in 4.2
 2023-12-15 12:23:26 +01:00
Adrián Chaves a49c8762dd Avoid calling iterparse twice 2023-12-15 12:14:53 +01:00
Adrián Chaves cd6846b763 Merge remote-tracking branch 'scrapy/2.11' into 2.11-redos 2023-12-15 11:52:35 +01:00
Adrián Chaves b0dbd0e9af Minor naming changes 2023-12-15 11:49:22 +01:00
Adrián Chaves 150d96764b Deprecate xmliter in favor of xmliter_lxml 2023-12-15 11:42:55 +01:00
Adrián Chaves 2538c0e862 Restore the implementation of xmliter 2023-12-15 10:23:24 +01:00
Adrián Chaves 9655b0b8eb Mark slow tests, with their own tox env and CI job 2023-12-15 10:19:47 +01:00
Adrián Chaves d50f436a73 Enable huge_tree for xmliter_lxml 2023-12-15 10:08:45 +01:00
Adrián Chaves 4f72b49f97 Fix namespaces nodename support for xmliter_lxml 2023-12-15 10:06:13 +01:00
Adrián Chaves 12b10a7a64 Cover scrapy.downloadermiddlewares.decompression in the release notes 2023-12-13 13:35:05 +01:00
Laerte Pereira b9c4ee26d7 Remove deprecated scrapy.downloadermiddlewares.decompression 2023-12-13 13:28:30 +01:00
Adrián Chaves 1533b69032 Test and address ReDoS attack vectors for open_in_browser 2023-12-13 12:01:35 +01:00
Adrián Chaves bb74badd1b spider → mw 2023-12-11 17:39:55 +01:00
Adrián Chaves c66b517706 Add Python 3.13 alpha to CI 2023-12-01 10:36:27 +01:00
Adrián Chaves 70ba3a0868
SPM → Zyte API (#6163) 2023-11-30 11:01:22 +01:00
Adrián Chaves 731f749556
Extend Request.meta documentation (#5565) 2023-11-30 10:54:09 +01:00
Adrián Chaves 40b3efbbee Remove open_in_browser from the 1.8.4 release notes 2023-11-29 12:47:04 +01:00
Adrián Chaves eb8b2c5197 Mention open_in_browser in the release notes 2023-11-29 12:13:04 +01:00
Adrián Chaves c947f51077 Set an arbitrary upper limit on ReDoS-vulnerable regexps 2023-11-29 11:54:08 +01:00
Adrián Chaves a113208a06 Fix BytesIO non-emptiness check 2023-11-24 11:35:15 +01:00
Adrián Chaves 8a73c6c90c Fix HttpCompressionMiddleware backward compatibility 2023-11-24 10:25:01 +01:00
Adrián Chaves 62398e424c CHUNK_SIZE: 8 KiB → 32 KiB 2023-11-24 10:20:14 +01:00
Adrián Chaves 6c278e1862 List[bytes] → BytesIO 2023-11-24 10:15:38 +01:00
Adrián Chaves 5f2827efe7 Make HttpCompressionMiddleware changes backward-comaptible 2023-11-24 10:15:38 +01:00
Andrey Rakhmatullin fa690fbe03
Merge pull request #6154 from Laerte/master 
Remove test_download_gzip_response test
 2023-11-23 20:56:50 +04:00
Laerte Pereira 09ce0ef526 Remove test_download_gzip_response test 2023-11-23 10:50:46 -03:00