RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,620 Commits 49 Branches 132 Tags 130 MiB
Commit Graph

10620 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 07e31b9c93
Merge pull request #6336 from wRAR/typing-cookies 
Full typing for scrapy/http/cookies.py.
 2024-05-06 13:56:34 +04:00
Andrey Rakhmatullin 2cba7896d2 Small fix for _get_slot_key(). 2024-05-06 14:31:24 +05:00
Sanchay Kumar aa025d7eac
Indicate that Selector.type can be json (#6334) 2024-05-06 11:29:35 +02:00
Andrey Rakhmatullin 40e4a59604 Fix Python 3.8. 2024-05-05 23:00:16 +05:00
Andrey Rakhmatullin 4b47a5dc32 Skip coverage checks for TYPE_CHECKING blocks. 2024-05-05 22:55:21 +05:00
Andrey Rakhmatullin c76dfc383f Full typing for scrapy/linkextractors. 2024-05-05 22:52:15 +05:00
Andrey Rakhmatullin 8a08283580 Full typing for scrapy/http/cookies.py. 2024-05-05 22:32:46 +05:00
Andrey Rakhmatullin 1f394306e1 Use the Self type hint in from_crawler/from_settings. 2024-05-05 22:05:11 +05:00
Andrey Rakhmatullin bd0d4cee88 Fixes for queuelib. 2024-05-04 16:29:23 +05:00
Andrey Rakhmatullin 203fa9667f Add queue typing to scrapy/core/scheduler.py. 2024-05-04 14:00:59 +05:00
Andrey Rakhmatullin 5f7fd2a653 Full typing for scrapy/squeues.py. 2024-05-04 14:00:59 +05:00
Andrey Rakhmatullin b749db92e5 Full typing for scrapy/robotstxt.py. 2024-05-04 14:00:59 +05:00
Andrey Rakhmatullin ad35ffdb0d Full typing for scrapy/resolver.py. 2024-05-04 14:00:59 +05:00
Andrey Rakhmatullin 21fa076181 Fix MutableMapping import for Python 3.8. 2024-05-04 14:00:59 +05:00
Andrey Rakhmatullin 0c8e21b8ac Full typing for scrapy/pqueues.py. 2024-05-04 14:00:33 +05:00
Andrey Rakhmatullin 38020e0b04 Full typing for scrapy/mail.py. 2024-04-29 20:12:30 +05:00
Andrey Rakhmatullin 08a265b6ff Full typing for scrapy/extension.py. 2024-04-29 19:17:29 +05:00
Andrey Rakhmatullin fc1a83e7c4 Full typing for scrapy/item.py. 2024-04-29 19:17:29 +05:00
Andrey Rakhmatullin 9eea22fb0c Full typing for scrapy/cmdline.py. 2024-04-29 19:17:29 +05:00
Andrey Rakhmatullin d7da298e06
Typing for build_from_*. (#6326) 2024-04-29 16:14:59 +02:00
Andrey Rakhmatullin 57acad3c38
Full typing for scrapy/extensions, part 3. (#6325) 2024-04-29 09:43:45 +02:00
Andrey Rakhmatullin a166e97399
Remove the auto-generated copyright years from the docs footer. (#6322) 2024-04-29 09:39:22 +02:00
Andrey Rakhmatullin a5da77d01d
Merge pull request #6221 from jxlil/fix/LxmlLinkExtractor 
fix: LxmlLinkExtractor unique_list missing key
 2024-04-19 19:57:11 +05:00
kokobhara b1fe97dc6c
Fix test expectations (#6316) 2024-04-17 13:14:57 +02:00
TechVest 5f67c01d1d
chore: fix some typos in comments (#6317) 
Signed-off-by: TechVest <techdashen@qq.com>
 2024-04-17 10:56:26 +02:00
Adrián Chaves 1d11ea3a54
Update practices.rst 2024-04-11 12:19:32 +02:00
Víctor Ruiz 48c5a8c98f
Fix WrappedRequest.get_header raising TypeError if default is None (#6310) 2024-04-08 11:47:53 +02:00
Andrey Rakhmatullin 5d31e89262
Merge pull request #6298 from lorenzoverardo/bug-6292/robots.txt-non-UTF-8 
Handle robots.txt files not UTF-8 encoded
 2024-04-05 13:49:10 +05:00
Lorenzo Verardo 7b37dcd80d Handle robots.txt files not UTF-8 encoded 2024-04-04 12:22:50 +02:00
igeni f7bf3f726e
modified string's concat to f-strings (#6296) 2024-04-01 15:37:23 +02:00
Andrey Rakhmatullin 02b97f98e7
Merge pull request #6242 from cankupeli/master 
implemented tests for branch coverage for function printSummary
 2024-03-14 18:20:14 +05:00
Andrey Rakhmatullin 8d917c0b55 Run black. 2024-03-14 17:44:57 +05:00
Andrey Rakhmatullin 5bf0e1d1db
Merge branch 'master' into master 2024-03-14 17:38:30 +05:00
Andrey Rakhmatullin 7255dfd41f
Merge pull request #6231 from elramen/check_command_tests 
Added tests for check command
 2024-03-14 15:28:19 +05:00
Lucas Belo 4460d3ed96
Remove tests/requirements.txt and refactor extra deps (#6272) 
Co-authored-by: lucasbelo777 <lucas.belo@cialdnb.com>
 2024-03-13 07:22:48 +01:00
Andrey Rakhmatullin 95a70d3fa0
Merge pull request #6290 from Gallaecio/fix-skips 
Fix WindowsRunSpiderCommandTest skip outside Windows for older Twisted
 2024-03-12 14:17:10 +05:00
Adrián Chaves d7581c6b41
Allow disabling the AutoThrottle extension for a given slot (#6246) 2024-03-12 09:44:29 +01:00
Adrián Chaves e72de11f55 Add super 2024-03-12 09:29:10 +01:00
Adrián Chaves 188d9a8bb3 Remove unnecessary comment 2024-03-11 23:28:57 +01:00
Adrián Chaves ab5ea32ffd Fix WindowsRunSpiderCommandTest skip outside Windows for older Twisted. 2024-03-11 23:26:19 +01:00
Andrey Rakhmatullin 642af40704
Full typing for scrapy/extensions, part 2. (#6279) 2024-03-11 10:09:09 +01:00
pengqiseven 6e84648c07
Fix some comments (#6285) 
Signed-off-by: pengqiseven <i@sohu.com>
Co-authored-by: pengqiseven <i@sohu.com>
 2024-03-11 10:03:06 +01:00
Andrey Rakhmatullin 421e08dd4a
Full typing for scrapy/extensions, part 1. (#6276) 2024-03-08 06:59:56 +01:00
Andrey Rakhmatullin 8985a04bd1
Full typing for scrapy/exporters.py. (#6275) 2024-03-08 06:57:03 +01:00
Andrey Rakhmatullin 861646fbb3
Merge pull request #6269 from Laerte/master 
Add brotlicffi support
 2024-03-06 19:04:42 +05:00
Andrey Rakhmatullin 6ecc9e0a34
Add typing for scrapy/commands (#6268) 2024-03-06 13:21:08 +01:00
Laerte Pereira 99f7165c63
Update scrapy/utils/_compression.py 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2024-03-06 09:02:01 -03:00
Laerte Pereira 7be919138d
Update scrapy/utils/_compression.py 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-03-06 05:49:31 -03:00
Laerte Pereira 7f1fbdba3c Check brotlicffi for ACCEPTED_ENCODINGS 2024-03-05 22:11:11 -03:00
Laerte Pereira 532cd2eabd Use brotlicffi for PyPy 2024-03-05 21:30:20 -03:00