RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,429 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

10429 Commits

Author SHA1 Message Date
Adrián Chaves d2f1e00a6a
Merge 2.11.2 changes (#6363) 2024-05-14 18:54:11 +02:00
Bagaudin Magomedov b88f22c6c5
Add SpiderLoggerAdapter, change Spider.logger to return SpiderLoggerAdapter (#6324) 2024-05-13 14:33:24 +02:00
Andrey Rakhmatullin b8e333c8ce
Full typing for scrapy/spiders. (#6356) 2024-05-13 12:07:51 +02:00
Andrey Rakhmatullin 4ed5c5ae91
Use ParamSpec for callables. (#6353) 2024-05-13 12:01:52 +02:00
Andrey Rakhmatullin 93f0628530
Improve typing for Spider.parse(). (#6274) 2024-05-13 11:55:45 +02:00
Sanchay Kumar c9ef520936
Add Downloader.get_slot_key() without a spider parameter (#6352) 2024-05-10 14:26:45 +02:00
Sanchay Kumar ae7bb849f5
Make certain args of ScrapyAgent and TunnelingAgent required (#6349) 2024-05-10 11:43:49 +02:00
Andrey Rakhmatullin 10a843ac1d
Merge pull request #6347 from aisha-partha/master 
Closes #6342. Setting METAREFRESH_IGNORE_TAGS to [‘noscript’] by default
 2024-05-10 11:47:40 +04:00
Adrián Chaves fe163d98ea
Fix typo 2024-05-08 20:41:20 +02:00
aisha-partha 2e13a9b8e1 Update METAREFRESH_IGNORE_TAGS default value in documentation 2024-05-08 23:33:13 +05:30
aisha-partha 3590a1f66b Closes #6342. Update documentation on METAREFRESH_IGNORE_TAGS default value change 2024-05-08 23:23:17 +05:30
aisha-partha 180bc9bad7 Closes #6342. Setting METAREFRESH_IGNORE_TAGS to [‘noscript’] by default 2024-05-08 22:06:46 +05:30
Andrey Rakhmatullin 4300a1d240
Merge pull request #6344 from wRAR/fix-twinecheck 
Update MANIFEST.in.
 2024-05-08 08:46:28 +04:00
Andrey Rakhmatullin 6bbfb537f9 Update MANIFEST.in. 2024-05-08 00:39:05 +05:00
Andrey Rakhmatullin c9bac7a657
Merge pull request #6341 from wRAR/typing-downloader 
More typing for scrapy/core/downloader
 2024-05-07 12:24:25 +04:00
Andrey Rakhmatullin a828da98c3 Re-run pre-commit. 2024-05-06 22:34:36 +05:00
Andrey Rakhmatullin 045387e07f More typing for scrapy/core/downloader/handlers/http11.py. 2024-05-06 15:07:38 +05:00
Andrey Rakhmatullin af3e38ab1f Full typing for scrapy/core/downloader/handlers/http10.py. 2024-05-06 15:07:16 +05:00
Andrey Rakhmatullin e8e13ebb78 Full typing for scrapy/core/downloader/handlers/ftp.py. 2024-05-06 15:06:57 +05:00
Andrey Rakhmatullin ec4d407022 Full typing for smaller download handlers. 2024-05-06 15:06:33 +05:00
Andrey Rakhmatullin c4d2748ff5 Small typing improvements in scrapy/core/downloader/contextfactory.py. 2024-05-06 15:05:38 +05:00
Andrey Rakhmatullin 563ecbe966
Merge pull request #6333 from wRAR/typing-scrapy 
Full typing for scrapy/*.py
 2024-05-06 14:04:16 +04:00
Andrey Rakhmatullin d338982580
Merge pull request #6335 from wRAR/typing-self 
Use the Self type hint in from_crawler/from_settings.
 2024-05-06 14:02:10 +04:00
Andrey Rakhmatullin 8a083fb684
Merge pull request #6337 from wRAR/typing-linkextractors 
Full typing for scrapy/linkextractors.
 2024-05-06 14:02:03 +04:00
Andrey Rakhmatullin 07e31b9c93
Merge pull request #6336 from wRAR/typing-cookies 
Full typing for scrapy/http/cookies.py.
 2024-05-06 13:56:34 +04:00
Andrey Rakhmatullin 2cba7896d2 Small fix for _get_slot_key(). 2024-05-06 14:31:24 +05:00
Sanchay Kumar aa025d7eac
Indicate that Selector.type can be json (#6334) 2024-05-06 11:29:35 +02:00
Andrey Rakhmatullin 40e4a59604 Fix Python 3.8. 2024-05-05 23:00:16 +05:00
Andrey Rakhmatullin 4b47a5dc32 Skip coverage checks for TYPE_CHECKING blocks. 2024-05-05 22:55:21 +05:00
Andrey Rakhmatullin c76dfc383f Full typing for scrapy/linkextractors. 2024-05-05 22:52:15 +05:00
Andrey Rakhmatullin 8a08283580 Full typing for scrapy/http/cookies.py. 2024-05-05 22:32:46 +05:00
Andrey Rakhmatullin 1f394306e1 Use the Self type hint in from_crawler/from_settings. 2024-05-05 22:05:11 +05:00
Andrey Rakhmatullin bd0d4cee88 Fixes for queuelib. 2024-05-04 16:29:23 +05:00
Andrey Rakhmatullin 203fa9667f Add queue typing to scrapy/core/scheduler.py. 2024-05-04 14:00:59 +05:00
Andrey Rakhmatullin 5f7fd2a653 Full typing for scrapy/squeues.py. 2024-05-04 14:00:59 +05:00
Andrey Rakhmatullin b749db92e5 Full typing for scrapy/robotstxt.py. 2024-05-04 14:00:59 +05:00
Andrey Rakhmatullin ad35ffdb0d Full typing for scrapy/resolver.py. 2024-05-04 14:00:59 +05:00
Andrey Rakhmatullin 21fa076181 Fix MutableMapping import for Python 3.8. 2024-05-04 14:00:59 +05:00
Andrey Rakhmatullin 0c8e21b8ac Full typing for scrapy/pqueues.py. 2024-05-04 14:00:33 +05:00
Andrey Rakhmatullin 38020e0b04 Full typing for scrapy/mail.py. 2024-04-29 20:12:30 +05:00
Andrey Rakhmatullin 08a265b6ff Full typing for scrapy/extension.py. 2024-04-29 19:17:29 +05:00
Andrey Rakhmatullin fc1a83e7c4 Full typing for scrapy/item.py. 2024-04-29 19:17:29 +05:00
Andrey Rakhmatullin 9eea22fb0c Full typing for scrapy/cmdline.py. 2024-04-29 19:17:29 +05:00
Andrey Rakhmatullin d7da298e06
Typing for build_from_*. (#6326) 2024-04-29 16:14:59 +02:00
Andrey Rakhmatullin 57acad3c38
Full typing for scrapy/extensions, part 3. (#6325) 2024-04-29 09:43:45 +02:00
Andrey Rakhmatullin a166e97399
Remove the auto-generated copyright years from the docs footer. (#6322) 2024-04-29 09:39:22 +02:00
Andrey Rakhmatullin a5da77d01d
Merge pull request #6221 from jxlil/fix/LxmlLinkExtractor 
fix: LxmlLinkExtractor unique_list missing key
 2024-04-19 19:57:11 +05:00
kokobhara b1fe97dc6c
Fix test expectations (#6316) 2024-04-17 13:14:57 +02:00
TechVest 5f67c01d1d
chore: fix some typos in comments (#6317) 
Signed-off-by: TechVest <techdashen@qq.com>
 2024-04-17 10:56:26 +02:00
Adrián Chaves 1d11ea3a54
Update practices.rst 2024-04-11 12:19:32 +02:00