RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,371 Commits 49 Branches 132 Tags 145 MiB
Commit Graph

10371 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 5bf0e1d1db
Merge branch 'master' into master 2024-03-14 17:38:30 +05:00
Andrey Rakhmatullin 7255dfd41f
Merge pull request #6231 from elramen/check_command_tests 
Added tests for check command
 2024-03-14 15:28:19 +05:00
Lucas Belo 4460d3ed96
Remove tests/requirements.txt and refactor extra deps (#6272) 
Co-authored-by: lucasbelo777 <lucas.belo@cialdnb.com>
 2024-03-13 07:22:48 +01:00
Andrey Rakhmatullin 95a70d3fa0
Merge pull request #6290 from Gallaecio/fix-skips 
Fix WindowsRunSpiderCommandTest skip outside Windows for older Twisted
 2024-03-12 14:17:10 +05:00
Adrián Chaves d7581c6b41
Allow disabling the AutoThrottle extension for a given slot (#6246) 2024-03-12 09:44:29 +01:00
Adrián Chaves e72de11f55 Add super 2024-03-12 09:29:10 +01:00
Adrián Chaves 188d9a8bb3 Remove unnecessary comment 2024-03-11 23:28:57 +01:00
Adrián Chaves ab5ea32ffd Fix WindowsRunSpiderCommandTest skip outside Windows for older Twisted. 2024-03-11 23:26:19 +01:00
Andrey Rakhmatullin 642af40704
Full typing for scrapy/extensions, part 2. (#6279) 2024-03-11 10:09:09 +01:00
pengqiseven 6e84648c07
Fix some comments (#6285) 
Signed-off-by: pengqiseven <i@sohu.com>
Co-authored-by: pengqiseven <i@sohu.com>
 2024-03-11 10:03:06 +01:00
Andrey Rakhmatullin 421e08dd4a
Full typing for scrapy/extensions, part 1. (#6276) 2024-03-08 06:59:56 +01:00
Andrey Rakhmatullin 8985a04bd1
Full typing for scrapy/exporters.py. (#6275) 2024-03-08 06:57:03 +01:00
Andrey Rakhmatullin 861646fbb3
Merge pull request #6269 from Laerte/master 
Add brotlicffi support
 2024-03-06 19:04:42 +05:00
Andrey Rakhmatullin 6ecc9e0a34
Add typing for scrapy/commands (#6268) 2024-03-06 13:21:08 +01:00
Laerte Pereira 99f7165c63
Update scrapy/utils/_compression.py 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2024-03-06 09:02:01 -03:00
Laerte Pereira 7be919138d
Update scrapy/utils/_compression.py 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-03-06 05:49:31 -03:00
Laerte Pereira 7f1fbdba3c Check brotlicffi for ACCEPTED_ENCODINGS 2024-03-05 22:11:11 -03:00
Laerte Pereira 532cd2eabd Use brotlicffi for PyPy 2024-03-05 21:30:20 -03:00
Laerte Pereira 16864ea602 Remove PyPy condition 2024-03-05 21:18:24 -03:00
Laerte Pereira a52429ae08 Update disclaimer 2024-03-05 21:12:25 -03:00
Laerte Pereira 3421823dce Nested try-except block 2024-03-05 20:26:23 -03:00
Laerte Pereira cab1016bb6 Add brotlicffi support 2024-03-05 20:24:17 -03:00
Adrián Chaves bf149356fc
Bandit: allow-list lxml usages (#6265) 2024-03-01 16:02:03 +01:00
Adrián Chaves aa1bf69079
Mark hashlib usages as not intended for security (#6264) 2024-03-01 12:48:00 +01:00
Yuri H. Galvao 4cd94aa668
Restore brotlipy support (#6261) 2024-03-01 11:07:38 +01:00
Andrey Rakhmatullin ee51958e19
Merge pull request #6259 from Laerte/master 
fix: Proxy tests don't use custom certificate authority
 2024-02-29 17:12:01 +05:00
Andrey Rakhmatullin b80128fc7c
Merge pull request #6260 from Gallaecio/bandit 
bandit: allow-list false positives
 2024-02-29 17:03:23 +05:00
Andrey Rakhmatullin 1311e7db05 Regenerate the expired mitmproxy-ca.pem. 2024-02-29 16:31:06 +05:00
Adrián Chaves 31cbbb5758 bandit: ignore md5 usage for download slot names 2024-02-29 11:31:39 +01:00
Adrián Chaves 2bfd9a2257 bandit: allow-list false positives 2024-02-29 11:11:42 +01:00
Laerte Pereira 2169810414 fix: Proxy tests don't use custom certificate authority 2024-02-29 06:41:14 -03:00
Andrey Rakhmatullin 198f5cf0d4
Merge pull request #6257 from Laerte/master 
Fix and re-enable `unnecessary-comprehension` and `use-dict-literal` pylint tags
 2024-02-29 12:36:34 +05:00
Laerte Pereira 26a16f2c43 Fix tests 2024-02-28 16:36:19 -03:00
Laerte Pereira 63acd07209 Fix and re-enable unnecessary-comprehension and use-dict-literal pylint tags 2024-02-28 16:14:08 -03:00
Andrey Rakhmatullin 532cc8a517
Merge pull request #6253 from wRAR/update-tools 
Update tool versions, fix some of the pylint problems
 2024-02-28 16:17:42 +05:00
Andrey Rakhmatullin 4f9dd998dc Bump pylint, cleanup the ignored tags. 2024-02-28 14:31:21 +05:00
Andrey Rakhmatullin d2c05d9d96 Bump mypy and type stubs. 2024-02-28 14:31:21 +05:00
Andrey Rakhmatullin 68104b9f48 Update .bandit.yml, add problem names. 2024-02-28 14:31:16 +05:00
Andrey Rakhmatullin 6e5918345b Bump bandit, flake8 and isort. 2024-02-28 14:30:38 +05:00
Andrey Rakhmatullin 282767f23b Bump black. 2024-02-28 14:30:38 +05:00
Andrey Rakhmatullin 415c47479f
Merge pull request #6251 from Laerte/master 
Use defusedxml.xmlrpc
 2024-02-28 14:29:20 +05:00
Laerte Pereira 008ebb65fc Change immport style 2024-02-27 18:10:28 -03:00
Laerte Pereira 7f945ad6db Import defusedxml.xmlrpc using alias 2024-02-27 17:39:29 -03:00
Laerte Pereira d87f949526 Use defusedxml.xmlrpc 2024-02-27 17:08:13 -03:00
Adrián Chaves 2d46b4acf5
Complete coverage for the AutoThrottle extension (#6245) 2024-02-27 09:28:02 +01:00
Jon b0ef9a89a1
Remove usage of deprecated mktemp (#5285) 2024-02-26 19:53:06 +01:00
vishesh10 e208f82076
Add support for multiple-compressed responses (#6063) 2024-02-22 12:16:24 +01:00
Andrey Rakhmatullin ebd7e199f0
Merge pull request #6235 from elramen/http_request_tests 
Added tests for form.py
 2024-02-22 12:17:11 +05:00
Andrey Rakhmatullin edd7ba1c06
Merge pull request #6236 from elramen/json_request_tests_main 
Improve json_request.py coverage
 2024-02-22 12:16:47 +05:00
Can Kupeli c513e7d6e5 implemented tests for branch coverage for function printSummary 2024-02-21 23:52:35 +01:00