RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,393 Commits 49 Branches 132 Tags 133 MiB
Commit Graph

2241 Commits

Author SHA1 Message Date
Andrey Rakhmatullin a5da77d01d
Merge pull request #6221 from jxlil/fix/LxmlLinkExtractor 
fix: LxmlLinkExtractor unique_list missing key
 2024-04-19 19:57:11 +05:00
kokobhara b1fe97dc6c
Fix test expectations (#6316) 2024-04-17 13:14:57 +02:00
Víctor Ruiz 48c5a8c98f
Fix WrappedRequest.get_header raising TypeError if default is None (#6310) 2024-04-08 11:47:53 +02:00
Andrey Rakhmatullin 5d31e89262
Merge pull request #6298 from lorenzoverardo/bug-6292/robots.txt-non-UTF-8 
Handle robots.txt files not UTF-8 encoded
 2024-04-05 13:49:10 +05:00
Lorenzo Verardo 7b37dcd80d Handle robots.txt files not UTF-8 encoded 2024-04-04 12:22:50 +02:00
igeni f7bf3f726e
modified string's concat to f-strings (#6296) 2024-04-01 15:37:23 +02:00
Andrey Rakhmatullin 8d917c0b55 Run black. 2024-03-14 17:44:57 +05:00
Andrey Rakhmatullin 5bf0e1d1db
Merge branch 'master' into master 2024-03-14 17:38:30 +05:00
Andrey Rakhmatullin 7255dfd41f
Merge pull request #6231 from elramen/check_command_tests 
Added tests for check command
 2024-03-14 15:28:19 +05:00
Lucas Belo 4460d3ed96
Remove tests/requirements.txt and refactor extra deps (#6272) 
Co-authored-by: lucasbelo777 <lucas.belo@cialdnb.com>
 2024-03-13 07:22:48 +01:00
Andrey Rakhmatullin 95a70d3fa0
Merge pull request #6290 from Gallaecio/fix-skips 
Fix WindowsRunSpiderCommandTest skip outside Windows for older Twisted
 2024-03-12 14:17:10 +05:00
Adrián Chaves d7581c6b41
Allow disabling the AutoThrottle extension for a given slot (#6246) 2024-03-12 09:44:29 +01:00
Adrián Chaves e72de11f55 Add super 2024-03-12 09:29:10 +01:00
Adrián Chaves 188d9a8bb3 Remove unnecessary comment 2024-03-11 23:28:57 +01:00
Adrián Chaves ab5ea32ffd Fix WindowsRunSpiderCommandTest skip outside Windows for older Twisted. 2024-03-11 23:26:19 +01:00
pengqiseven 6e84648c07
Fix some comments (#6285) 
Signed-off-by: pengqiseven <i@sohu.com>
Co-authored-by: pengqiseven <i@sohu.com>
 2024-03-11 10:03:06 +01:00
Laerte Pereira 532cd2eabd Use brotlicffi for PyPy 2024-03-05 21:30:20 -03:00
Laerte Pereira cab1016bb6 Add brotlicffi support 2024-03-05 20:24:17 -03:00
Andrey Rakhmatullin 1311e7db05 Regenerate the expired mitmproxy-ca.pem. 2024-02-29 16:31:06 +05:00
Laerte Pereira 2169810414 fix: Proxy tests don't use custom certificate authority 2024-02-29 06:41:14 -03:00
Laerte Pereira 26a16f2c43 Fix tests 2024-02-28 16:36:19 -03:00
Andrey Rakhmatullin 4f9dd998dc Bump pylint, cleanup the ignored tags. 2024-02-28 14:31:21 +05:00
Andrey Rakhmatullin 282767f23b Bump black. 2024-02-28 14:30:38 +05:00
Adrián Chaves 2d46b4acf5
Complete coverage for the AutoThrottle extension (#6245) 2024-02-27 09:28:02 +01:00
Jon b0ef9a89a1
Remove usage of deprecated mktemp (#5285) 2024-02-26 19:53:06 +01:00
vishesh10 e208f82076
Add support for multiple-compressed responses (#6063) 2024-02-22 12:16:24 +01:00
Andrey Rakhmatullin ebd7e199f0
Merge pull request #6235 from elramen/http_request_tests 
Added tests for form.py
 2024-02-22 12:17:11 +05:00
Andrey Rakhmatullin edd7ba1c06
Merge pull request #6236 from elramen/json_request_tests_main 
Improve json_request.py coverage
 2024-02-22 12:16:47 +05:00
Can Kupeli c513e7d6e5 implemented tests for branch coverage for function printSummary 2024-02-21 23:52:35 +01:00
noon 877398a3de refactor #3 Remove inner class in form test 2024-02-21 21:13:57 +01:00
noon b7a7ae7dbb refactor #22 Change comment and warning catching 2024-02-21 21:04:45 +01:00
Andrey Rakhmatullin f798118ac2
Merge pull request #6232 from elramen/linkextractor_tests 
Added tests for LxmlLinkExtractor
 2024-02-21 23:03:20 +05:00
Elias Ram f19045403a test #8 made tests cleaner 2024-02-21 16:03:51 +01:00
Adrián Chaves 6fc7827042
Do not close the underlying file from compression plugins (#6239) 2024-02-21 14:27:42 +01:00
noon bc036542a8 refactor #3 Moved tests to FormRequestTest 2024-02-21 02:35:45 +01:00
noon 12b4417c56 test #22 Improve json_request.py coverage 2024-02-21 02:30:50 +01:00
noon e2a0c85f11 doc #3 Clarified test comments 2024-02-21 02:21:24 +01:00
noon e27d320c3c test #3 Increased branch coverage for form.py 2024-02-20 23:58:39 +01:00
Elias Ram e8e6d28479 test #8 added tests for LxmlLinkExtractor 2024-02-20 20:41:18 +01:00
Elias Ram f096f17fa4 test #6 added tests for check command 2024-02-20 20:32:02 +01:00
Laerte Pereira ee1189512f
Replace urlparse with urlparse_cached where possible (#6229) 2024-02-20 12:47:29 +01:00
Adrián Chaves 5e51417a48 Add tests, fix canonicalize passing 2024-02-16 20:10:52 +01:00
Kevin Lloyd Bernal 9bb973dc54
Refactor LogStats extension to log IPM and RPM to stats on spider_close (#4111) 2024-02-16 12:25:38 +01:00
Adrián Chaves 6bd45bb6d9 Merge remote-tracking branch 'scrapy/2.11' into 2.11-port 2024-02-15 06:39:50 +01:00
Adrián Chaves 2f1d345e74 Solve test issues 2024-02-14 18:59:01 +01:00
Adrián Chaves 479619b340 Merge branch '2.11-redos' into 2.11 2024-02-14 17:18:26 +01:00
Adrián Chaves 809bfac489 Merge branch '2.11-compression-bomb' into 2.11 2024-02-14 17:16:25 +01:00
Adrián Chaves 5bcb8fd501 Merge branch '2.11-authorization' into 2.11 2024-02-14 17:14:29 +01:00
Adrián Chaves c5dad41190 Speed up tests, remove comments without regexps 2024-02-02 14:03:16 +01:00
Laerte Pereira 7001193c80 Simplify the logic 2024-01-31 12:53:08 -03:00