RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,191 Commits 49 Branches 132 Tags 160 MiB
Commit Graph

11191 Commits

Author SHA1 Message Date
Adrián Chaves c66b517706 Add Python 3.13 alpha to CI 2023-12-01 10:36:27 +01:00
Adrián Chaves 70ba3a0868
SPM → Zyte API (#6163) 2023-11-30 11:01:22 +01:00
Adrián Chaves 731f749556
Extend Request.meta documentation (#5565) 2023-11-30 10:54:09 +01:00
Adrián Chaves 40b3efbbee Remove open_in_browser from the 1.8.4 release notes 2023-11-29 12:47:04 +01:00
Adrián Chaves eb8b2c5197 Mention open_in_browser in the release notes 2023-11-29 12:13:04 +01:00
Adrián Chaves c947f51077 Set an arbitrary upper limit on ReDoS-vulnerable regexps 2023-11-29 11:54:08 +01:00
Adrián Chaves a113208a06 Fix BytesIO non-emptiness check 2023-11-24 11:35:15 +01:00
Adrián Chaves 8a73c6c90c Fix HttpCompressionMiddleware backward compatibility 2023-11-24 10:25:01 +01:00
Adrián Chaves 62398e424c CHUNK_SIZE: 8 KiB → 32 KiB 2023-11-24 10:20:14 +01:00
Adrián Chaves 6c278e1862 List[bytes] → BytesIO 2023-11-24 10:15:38 +01:00
Adrián Chaves 5f2827efe7 Make HttpCompressionMiddleware changes backward-comaptible 2023-11-24 10:15:38 +01:00
Andrey Rakhmatullin fa690fbe03
Merge pull request #6154 from Laerte/master 
Remove test_download_gzip_response test
 2023-11-23 20:56:50 +04:00
Laerte Pereira 09ce0ef526 Remove test_download_gzip_response test 2023-11-23 10:50:46 -03:00
Adrián Chaves 8e25f8c157
Fix bad message 2023-11-23 14:12:59 +01:00
Adrián Chaves cf80e5670e Solve linting and typing issues 2023-11-23 12:07:15 +01:00
Adrián Chaves b53ed52a22 Update the release notes 2023-11-23 11:36:45 +01:00
Adrián Chaves 03d9866518 Also use DOWNLOAD_WARNSIZE for decompressions 2023-11-23 10:26:47 +01:00
Adrián Chaves 1087bb7b2e Update the docs 2023-11-23 09:11:14 +01:00
Adrián Chaves e0b66c021a Mind Spider.download_maxsize and Request.meta['download_maxsize'] 2023-11-22 19:03:24 +01:00
Adrián Chaves 3fda2fe103 Protect against gzip bomb sitemaps 2023-11-22 18:34:37 +01:00
Adrián Chaves 9cc8703877 Protect against zstandard bombs 2023-11-22 17:53:00 +01:00
Adrián Chaves fba167c5e1 Protect against brotli bombs 2023-11-22 17:32:09 +01:00
Adrián Chaves 0c4a98f8e0 Protect against deflate bombs 2023-11-22 17:12:43 +01:00
Adrián Chaves 0bf29a7b1b Update test expectations 2023-11-22 16:10:50 +01:00
Adrián Chaves 6969041c5f Protect against gzip bombs 2023-11-22 15:52:00 +01:00
Adrián Chaves 1c4e93293a Merge remote-tracking branch 'scrapy/2.11' into 2.11-authorization 2023-11-17 15:42:08 +01:00
Kiran 49b284ab85 Updated README.rst (#6144) 2023-11-16 19:17:52 +04:00
Jessica Allman-LaPorte 150f9d6d88 Make shell switching more clear in the tutorial (#6128) 2023-11-16 19:17:29 +04:00
Adrián Chaves 1045856a50 Merge pull request #6112 from wRAR/test-shutdown-forced 
Make shutdown tests more robust.
 2023-11-16 19:16:52 +04:00
Andrey Rakhmatullin 5e4fb0bc5f Re-enable uvloop tests on 3.12 (#6098) 2023-11-16 19:16:31 +04:00
Andrey Rakhmatullin 538192916f Merge pull request #6064 from wRAR/signals-proper 
Refactor installing signals.
 2023-11-16 19:15:30 +04:00
Andrey Rakhmatullin 59cfdeaa5c Merge pull request #6083 from wRAR/py3.12-release 
Adapt to the Python 3.12 final release
 2023-11-16 19:14:47 +04:00
Andrey Rakhmatullin ffbf943e9d Merge pull request #6077 from 11-aryan/11-aryan 2023-11-16 19:13:02 +04:00
Andrey Rakhmatullin 75e99c75b3 Improve the docs about Crawler attributes and settings initialization. 2023-11-16 12:36:57 +04:00
Andrey Rakhmatullin 42b3a3a23b
Merge pull request #6147 from wRAR/crawler-init-docs 
Improve the docs about Crawler attributes and settings initialization.
 2023-11-16 12:35:43 +04:00
Andrey Rakhmatullin 603aa4924a Improve the docs about Crawler attributes and settings initialization. 2023-11-15 19:51:04 +04:00
Adrián Chaves 080fecd890 Drop the Authorization header on cross-domain redirect 2023-11-15 15:39:30 +01:00
Andrey Rakhmatullin 5fccf370b8 Update the RTD URL for coverage 2023-11-15 15:38:13 +01:00
Andrey Rakhmatullin ebdea4037a Update another output line. 2023-11-15 12:31:31 +04:00
Andrey Rakhmatullin db5a73f7bb Update the expected mypy output to match the old Python one. 2023-11-15 12:02:39 +04:00
Kiran 492584ec07
Updated README.rst (#6144) 2023-11-13 20:13:10 +01:00
Andrey Rakhmatullin 8776b4a6fb Fix env deps for typing-tests. 2023-11-12 20:52:29 +04:00
Andrey Rakhmatullin 204d6e180a Enable typing-tests in CI. 2023-11-12 20:47:52 +04:00
Andrey Rakhmatullin 5d55e4f56b Add mypy tests. 2023-11-12 20:15:06 +04:00
Andrey Rakhmatullin a6cee787dd Improve type hints for copy() and replace() in Request and Response. 2023-11-11 20:00:12 +04:00
Andrey Rakhmatullin b4acf5c827
Fix and remove most of the entries from the mypy ignore list (#6137) 2023-11-07 09:34:35 +01:00
Andrey Rakhmatullin c31e09d709
Merge pull request #6134 from wRAR/obsolete-setter 
Remove obsolete setters for body and url on Request and Response.
 2023-11-06 18:24:13 +04:00
Andrey Rakhmatullin 7c27c22a98
Full typing for scrapy/downloadermiddlewares (#6129) 2023-11-06 08:52:14 +01:00
cakemd eafe828484
scrapy.utils.data_path type hint change (#6133) 2023-11-06 08:37:18 +01:00
Andrey Rakhmatullin 2ac3ef73e6 Remove obsolete setters for body and url on Request and Response. 2023-11-03 20:12:18 +04:00