RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,393 Commits 49 Branches 132 Tags 132 MiB
Commit Graph

10393 Commits

Author SHA1 Message Date
Andrey Rakhmatullin ad35ffdb0d Full typing for scrapy/resolver.py. 2024-05-04 14:00:59 +05:00
Andrey Rakhmatullin 21fa076181 Fix MutableMapping import for Python 3.8. 2024-05-04 14:00:59 +05:00
Andrey Rakhmatullin 0c8e21b8ac Full typing for scrapy/pqueues.py. 2024-05-04 14:00:33 +05:00
Andrey Rakhmatullin 38020e0b04 Full typing for scrapy/mail.py. 2024-04-29 20:12:30 +05:00
Andrey Rakhmatullin 08a265b6ff Full typing for scrapy/extension.py. 2024-04-29 19:17:29 +05:00
Andrey Rakhmatullin fc1a83e7c4 Full typing for scrapy/item.py. 2024-04-29 19:17:29 +05:00
Andrey Rakhmatullin 9eea22fb0c Full typing for scrapy/cmdline.py. 2024-04-29 19:17:29 +05:00
Andrey Rakhmatullin d7da298e06
Typing for build_from_*. (#6326) 2024-04-29 16:14:59 +02:00
Andrey Rakhmatullin 57acad3c38
Full typing for scrapy/extensions, part 3. (#6325) 2024-04-29 09:43:45 +02:00
Andrey Rakhmatullin a166e97399
Remove the auto-generated copyright years from the docs footer. (#6322) 2024-04-29 09:39:22 +02:00
Andrey Rakhmatullin a5da77d01d
Merge pull request #6221 from jxlil/fix/LxmlLinkExtractor 
fix: LxmlLinkExtractor unique_list missing key
 2024-04-19 19:57:11 +05:00
kokobhara b1fe97dc6c
Fix test expectations (#6316) 2024-04-17 13:14:57 +02:00
TechVest 5f67c01d1d
chore: fix some typos in comments (#6317) 
Signed-off-by: TechVest <techdashen@qq.com>
 2024-04-17 10:56:26 +02:00
Adrián Chaves 1d11ea3a54
Update practices.rst 2024-04-11 12:19:32 +02:00
Víctor Ruiz 48c5a8c98f
Fix WrappedRequest.get_header raising TypeError if default is None (#6310) 2024-04-08 11:47:53 +02:00
Andrey Rakhmatullin 5d31e89262
Merge pull request #6298 from lorenzoverardo/bug-6292/robots.txt-non-UTF-8 
Handle robots.txt files not UTF-8 encoded
 2024-04-05 13:49:10 +05:00
Lorenzo Verardo 7b37dcd80d Handle robots.txt files not UTF-8 encoded 2024-04-04 12:22:50 +02:00
igeni f7bf3f726e
modified string's concat to f-strings (#6296) 2024-04-01 15:37:23 +02:00
Andrey Rakhmatullin 02b97f98e7
Merge pull request #6242 from cankupeli/master 
implemented tests for branch coverage for function printSummary
 2024-03-14 18:20:14 +05:00
Andrey Rakhmatullin 8d917c0b55 Run black. 2024-03-14 17:44:57 +05:00
Andrey Rakhmatullin 5bf0e1d1db
Merge branch 'master' into master 2024-03-14 17:38:30 +05:00
Andrey Rakhmatullin 7255dfd41f
Merge pull request #6231 from elramen/check_command_tests 
Added tests for check command
 2024-03-14 15:28:19 +05:00
Lucas Belo 4460d3ed96
Remove tests/requirements.txt and refactor extra deps (#6272) 
Co-authored-by: lucasbelo777 <lucas.belo@cialdnb.com>
 2024-03-13 07:22:48 +01:00
Andrey Rakhmatullin 95a70d3fa0
Merge pull request #6290 from Gallaecio/fix-skips 
Fix WindowsRunSpiderCommandTest skip outside Windows for older Twisted
 2024-03-12 14:17:10 +05:00
Adrián Chaves d7581c6b41
Allow disabling the AutoThrottle extension for a given slot (#6246) 2024-03-12 09:44:29 +01:00
Adrián Chaves e72de11f55 Add super 2024-03-12 09:29:10 +01:00
Adrián Chaves 188d9a8bb3 Remove unnecessary comment 2024-03-11 23:28:57 +01:00
Adrián Chaves ab5ea32ffd Fix WindowsRunSpiderCommandTest skip outside Windows for older Twisted. 2024-03-11 23:26:19 +01:00
Andrey Rakhmatullin 642af40704
Full typing for scrapy/extensions, part 2. (#6279) 2024-03-11 10:09:09 +01:00
pengqiseven 6e84648c07
Fix some comments (#6285) 
Signed-off-by: pengqiseven <i@sohu.com>
Co-authored-by: pengqiseven <i@sohu.com>
 2024-03-11 10:03:06 +01:00
Andrey Rakhmatullin 421e08dd4a
Full typing for scrapy/extensions, part 1. (#6276) 2024-03-08 06:59:56 +01:00
Andrey Rakhmatullin 8985a04bd1
Full typing for scrapy/exporters.py. (#6275) 2024-03-08 06:57:03 +01:00
Andrey Rakhmatullin 861646fbb3
Merge pull request #6269 from Laerte/master 
Add brotlicffi support
 2024-03-06 19:04:42 +05:00
Andrey Rakhmatullin 6ecc9e0a34
Add typing for scrapy/commands (#6268) 2024-03-06 13:21:08 +01:00
Laerte Pereira 99f7165c63
Update scrapy/utils/_compression.py 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2024-03-06 09:02:01 -03:00
Laerte Pereira 7be919138d
Update scrapy/utils/_compression.py 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-03-06 05:49:31 -03:00
Laerte Pereira 7f1fbdba3c Check brotlicffi for ACCEPTED_ENCODINGS 2024-03-05 22:11:11 -03:00
Laerte Pereira 532cd2eabd Use brotlicffi for PyPy 2024-03-05 21:30:20 -03:00
Laerte Pereira 16864ea602 Remove PyPy condition 2024-03-05 21:18:24 -03:00
Laerte Pereira a52429ae08 Update disclaimer 2024-03-05 21:12:25 -03:00
Laerte Pereira 3421823dce Nested try-except block 2024-03-05 20:26:23 -03:00
Laerte Pereira cab1016bb6 Add brotlicffi support 2024-03-05 20:24:17 -03:00
Adrián Chaves bf149356fc
Bandit: allow-list lxml usages (#6265) 2024-03-01 16:02:03 +01:00
Adrián Chaves aa1bf69079
Mark hashlib usages as not intended for security (#6264) 2024-03-01 12:48:00 +01:00
Yuri H. Galvao 4cd94aa668
Restore brotlipy support (#6261) 2024-03-01 11:07:38 +01:00
Andrey Rakhmatullin ee51958e19
Merge pull request #6259 from Laerte/master 
fix: Proxy tests don't use custom certificate authority
 2024-02-29 17:12:01 +05:00
Andrey Rakhmatullin b80128fc7c
Merge pull request #6260 from Gallaecio/bandit 
bandit: allow-list false positives
 2024-02-29 17:03:23 +05:00
Andrey Rakhmatullin 1311e7db05 Regenerate the expired mitmproxy-ca.pem. 2024-02-29 16:31:06 +05:00
Adrián Chaves 31cbbb5758 bandit: ignore md5 usage for download slot names 2024-02-29 11:31:39 +01:00
Adrián Chaves 2bfd9a2257 bandit: allow-list false positives 2024-02-29 11:11:42 +01:00