RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,710 Commits 49 Branches 132 Tags 141 MiB
Commit Graph

10710 Commits

Author SHA1 Message Date
Laerte Pereira 8ec67ca230
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-23 07:00:35 -03:00
Laerte Pereira 17e623cf0c
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-23 07:00:24 -03:00
Laerte Pereira 9d5a0d287b
Retrigger CI 2024-05-22 22:27:07 -03:00
Laerte Pereira e143dc7952
Update tests-macos.yml 2024-05-22 22:26:31 -03:00
Laerte Pereira 3f66b66e3f fix: checks 2024-05-22 22:01:55 -03:00
Laerte Pereira dc6a495fee
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-22 18:51:02 -03:00
Laerte Pereira 8210fae25a
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-22 18:50:50 -03:00
Laerte Pereira e676cd3ce0 docs: Remove top-level reactor imports from CrawlerProces/CrawlerRunner examples 2024-05-22 07:55:53 -03:00
Andrey Rakhmatullin 04bc1e6e2a Skip zstandard on 3.13. 2024-05-22 13:24:35 +05:00
Andrey Rakhmatullin b6d3d9076f Help with building lxml on 3.13beta1. 2024-05-22 13:20:48 +05:00
Andrey Rakhmatullin 534a66e954 Bump 3.13 to beta1. 2024-05-22 13:16:00 +05:00
Andrey Rakhmatullin 85d7458651 Merge branch 'master' into py313 2024-05-22 13:14:59 +05:00
Andrey Rakhmatullin b99526b740 Full typing for scrapy/contracts. 2024-05-19 15:45:51 +05:00
Andrey Rakhmatullin 631fc65fad
Update expectations of cookies after redirects. (#6367) 2024-05-16 16:42:09 +02:00
Adrián Chaves 812fd2368f
Allow user-defined secure cookies (#6357) 2024-05-15 11:48:43 +02:00
Adrián Chaves d2f1e00a6a
Merge 2.11.2 changes (#6363) 2024-05-14 18:54:11 +02:00
Andrey Rakhmatullin d97d32c48e Merge branch 'master' into typing-request-response-cls 2024-05-13 19:58:17 +05:00
Bagaudin Magomedov b88f22c6c5
Add SpiderLoggerAdapter, change Spider.logger to return SpiderLoggerAdapter (#6324) 2024-05-13 14:33:24 +02:00
Andrey Rakhmatullin b8e333c8ce
Full typing for scrapy/spiders. (#6356) 2024-05-13 12:07:51 +02:00
Andrey Rakhmatullin 4ed5c5ae91
Use ParamSpec for callables. (#6353) 2024-05-13 12:01:52 +02:00
Andrey Rakhmatullin 93f0628530
Improve typing for Spider.parse(). (#6274) 2024-05-13 11:55:45 +02:00
Sanchay Kumar c9ef520936
Add Downloader.get_slot_key() without a spider parameter (#6352) 2024-05-10 14:26:45 +02:00
Sanchay Kumar ae7bb849f5
Make certain args of ScrapyAgent and TunnelingAgent required (#6349) 2024-05-10 11:43:49 +02:00
Andrey Rakhmatullin 10a843ac1d
Merge pull request #6347 from aisha-partha/master 
Closes #6342. Setting METAREFRESH_IGNORE_TAGS to [‘noscript’] by default
 2024-05-10 11:47:40 +04:00
Adrián Chaves fe163d98ea
Fix typo 2024-05-08 20:41:20 +02:00
aisha-partha 2e13a9b8e1 Update METAREFRESH_IGNORE_TAGS default value in documentation 2024-05-08 23:33:13 +05:30
aisha-partha 3590a1f66b Closes #6342. Update documentation on METAREFRESH_IGNORE_TAGS default value change 2024-05-08 23:23:17 +05:30
aisha-partha 180bc9bad7 Closes #6342. Setting METAREFRESH_IGNORE_TAGS to [‘noscript’] by default 2024-05-08 22:06:46 +05:30
Andrey Rakhmatullin 4300a1d240
Merge pull request #6344 from wRAR/fix-twinecheck 
Update MANIFEST.in.
 2024-05-08 08:46:28 +04:00
Andrey Rakhmatullin 6bbfb537f9 Update MANIFEST.in. 2024-05-08 00:39:05 +05:00
Andrey Rakhmatullin c9bac7a657
Merge pull request #6341 from wRAR/typing-downloader 
More typing for scrapy/core/downloader
 2024-05-07 12:24:25 +04:00
Andrey Rakhmatullin a828da98c3 Re-run pre-commit. 2024-05-06 22:34:36 +05:00
Andrey Rakhmatullin 045387e07f More typing for scrapy/core/downloader/handlers/http11.py. 2024-05-06 15:07:38 +05:00
Andrey Rakhmatullin af3e38ab1f Full typing for scrapy/core/downloader/handlers/http10.py. 2024-05-06 15:07:16 +05:00
Andrey Rakhmatullin e8e13ebb78 Full typing for scrapy/core/downloader/handlers/ftp.py. 2024-05-06 15:06:57 +05:00
Andrey Rakhmatullin ec4d407022 Full typing for smaller download handlers. 2024-05-06 15:06:33 +05:00
Andrey Rakhmatullin c4d2748ff5 Small typing improvements in scrapy/core/downloader/contextfactory.py. 2024-05-06 15:05:38 +05:00
Andrey Rakhmatullin 563ecbe966
Merge pull request #6333 from wRAR/typing-scrapy 
Full typing for scrapy/*.py
 2024-05-06 14:04:16 +04:00
Andrey Rakhmatullin d338982580
Merge pull request #6335 from wRAR/typing-self 
Use the Self type hint in from_crawler/from_settings.
 2024-05-06 14:02:10 +04:00
Andrey Rakhmatullin 8a083fb684
Merge pull request #6337 from wRAR/typing-linkextractors 
Full typing for scrapy/linkextractors.
 2024-05-06 14:02:03 +04:00
Andrey Rakhmatullin 07e31b9c93
Merge pull request #6336 from wRAR/typing-cookies 
Full typing for scrapy/http/cookies.py.
 2024-05-06 13:56:34 +04:00
Andrey Rakhmatullin 2cba7896d2 Small fix for _get_slot_key(). 2024-05-06 14:31:24 +05:00
Sanchay Kumar aa025d7eac
Indicate that Selector.type can be json (#6334) 2024-05-06 11:29:35 +02:00
Andrey Rakhmatullin 40e4a59604 Fix Python 3.8. 2024-05-05 23:00:16 +05:00
Andrey Rakhmatullin 4b47a5dc32 Skip coverage checks for TYPE_CHECKING blocks. 2024-05-05 22:55:21 +05:00
Andrey Rakhmatullin c76dfc383f Full typing for scrapy/linkextractors. 2024-05-05 22:52:15 +05:00
Andrey Rakhmatullin 8a08283580 Full typing for scrapy/http/cookies.py. 2024-05-05 22:32:46 +05:00
Andrey Rakhmatullin 1f394306e1 Use the Self type hint in from_crawler/from_settings. 2024-05-05 22:05:11 +05:00
Andrey Rakhmatullin bd0d4cee88 Fixes for queuelib. 2024-05-04 16:29:23 +05:00
Andrey Rakhmatullin 203fa9667f Add queue typing to scrapy/core/scheduler.py. 2024-05-04 14:00:59 +05:00