RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,464 Commits 49 Branches 132 Tags 145 MiB
Commit Graph

10464 Commits

Author SHA1 Message Date
Laerte Pereira 144ff6c756 Document missing parts of response.json method 2024-06-05 21:09:10 -03:00
Suvan Banerjee 3f76853bd2
Handle AttributeError: 'NoneType' in contract parsing (#6388) 2024-06-05 06:34:46 +02:00
Andrey Rakhmatullin e56b425198
Full typing for scrapy/pipelines. (#6387) 2024-06-05 06:33:45 +02:00
Adrián Chaves 2b9e32f1ca
Merge pull request #6385 from wRAR/typing-generics-collections 
Add parameters to various generics.
 2024-06-03 13:29:38 +02:00
Andrey Rakhmatullin 492c3bce9d
Don't run callbacks of requests from get_media_requests(). (#6386) 2024-06-03 13:28:20 +02:00
Andrey Rakhmatullin 019f23e3b7 Add parameters to some of typing.Callable. 2024-06-02 18:42:01 +05:00
Andrey Rakhmatullin 859a77ee42 Use a TypedDict for the verbose cookie form. 2024-05-31 22:23:26 +05:00
Andrey Rakhmatullin 751c91e614 Add parameters to misc generics. 2024-05-31 21:57:14 +05:00
Andrey Rakhmatullin 70c56faf48 Add parameters to typing.IO. 2024-05-31 21:41:27 +05:00
Andrey Rakhmatullin 4164e63725 Add parameters to typing.Tuple. 2024-05-31 21:23:55 +05:00
Andrey Rakhmatullin 98c755e5fb Add parameters to typing.List. 2024-05-31 21:20:22 +05:00
Andrey Rakhmatullin da42e8f124 Add parameters to typing.Dict. 2024-05-31 21:11:50 +05:00
Andrey Rakhmatullin b4293e8f9e
Misc typing improvements. (#6384) 2024-05-31 08:50:36 +02:00
Andrey Rakhmatullin a011fa6f78
Merge pull request #6371 from wRAR/typing-contracts 
Full typing for scrapy/contracts.
 2024-05-30 20:59:50 +04:00
Andrey Rakhmatullin 469e8a23f8
Merge pull request #6381 from 0xdeb/referer-policy-tokens 
Add support for multiple referer policy tokens.
 2024-05-29 17:13:09 +04:00
Fabian Schneebauer 62a028b99d
Add spec link to scrapy/spidermiddlewares/referer.py 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2024-05-29 13:19:27 +02:00
Fabian Schneebauer 0d58af8697 Add support for multiple referer policy tokens. 2024-05-29 10:59:32 +02:00
Sanchay Kumar cadb0dd707
Fix overridable methods in MediaPipeline (#6368) 2024-05-28 10:42:58 +02:00
Andrey Rakhmatullin 986d1ee1dd
Move CI from the decommissioned macos-11 to macos-latest. (#6372) 2024-05-28 10:37:19 +02:00
Andrey Rakhmatullin 9ba4dd311d
Install typing stubs for boto3 and botocore. (#6370) 2024-05-28 10:27:49 +02:00
Andrey Rakhmatullin f9a9860306
Merge pull request #6374 from Laerte/master 
docs: Remove top-level reactor imports from CrawlerProces/CrawlerRunner examples
 2024-05-27 14:36:34 +04:00
Laerte Pereira 6cd0857850 Move path 2024-05-26 19:57:16 -03:00
Laerte Pereira 2facdd4fb0 Add change reactor test to CrawlerRunner 2024-05-26 19:55:54 -03:00
Laerte Pereira 62c89aaf05
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-23 07:00:45 -03:00
Laerte Pereira 8ec67ca230
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-23 07:00:35 -03:00
Laerte Pereira 17e623cf0c
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-23 07:00:24 -03:00
Laerte Pereira 9d5a0d287b
Retrigger CI 2024-05-22 22:27:07 -03:00
Laerte Pereira e143dc7952
Update tests-macos.yml 2024-05-22 22:26:31 -03:00
Laerte Pereira 3f66b66e3f fix: checks 2024-05-22 22:01:55 -03:00
Laerte Pereira dc6a495fee
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-22 18:51:02 -03:00
Laerte Pereira 8210fae25a
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-22 18:50:50 -03:00
Laerte Pereira e676cd3ce0 docs: Remove top-level reactor imports from CrawlerProces/CrawlerRunner examples 2024-05-22 07:55:53 -03:00
Andrey Rakhmatullin b99526b740 Full typing for scrapy/contracts. 2024-05-19 15:45:51 +05:00
Andrey Rakhmatullin 631fc65fad
Update expectations of cookies after redirects. (#6367) 2024-05-16 16:42:09 +02:00
Adrián Chaves 812fd2368f
Allow user-defined secure cookies (#6357) 2024-05-15 11:48:43 +02:00
Adrián Chaves d2f1e00a6a
Merge 2.11.2 changes (#6363) 2024-05-14 18:54:11 +02:00
Bagaudin Magomedov b88f22c6c5
Add SpiderLoggerAdapter, change Spider.logger to return SpiderLoggerAdapter (#6324) 2024-05-13 14:33:24 +02:00
Andrey Rakhmatullin b8e333c8ce
Full typing for scrapy/spiders. (#6356) 2024-05-13 12:07:51 +02:00
Andrey Rakhmatullin 4ed5c5ae91
Use ParamSpec for callables. (#6353) 2024-05-13 12:01:52 +02:00
Andrey Rakhmatullin 93f0628530
Improve typing for Spider.parse(). (#6274) 2024-05-13 11:55:45 +02:00
Sanchay Kumar c9ef520936
Add Downloader.get_slot_key() without a spider parameter (#6352) 2024-05-10 14:26:45 +02:00
Sanchay Kumar ae7bb849f5
Make certain args of ScrapyAgent and TunnelingAgent required (#6349) 2024-05-10 11:43:49 +02:00
Andrey Rakhmatullin 10a843ac1d
Merge pull request #6347 from aisha-partha/master 
Closes #6342. Setting METAREFRESH_IGNORE_TAGS to [‘noscript’] by default
 2024-05-10 11:47:40 +04:00
Adrián Chaves fe163d98ea
Fix typo 2024-05-08 20:41:20 +02:00
aisha-partha 2e13a9b8e1 Update METAREFRESH_IGNORE_TAGS default value in documentation 2024-05-08 23:33:13 +05:30
aisha-partha 3590a1f66b Closes #6342. Update documentation on METAREFRESH_IGNORE_TAGS default value change 2024-05-08 23:23:17 +05:30
aisha-partha 180bc9bad7 Closes #6342. Setting METAREFRESH_IGNORE_TAGS to [‘noscript’] by default 2024-05-08 22:06:46 +05:30
Andrey Rakhmatullin 4300a1d240
Merge pull request #6344 from wRAR/fix-twinecheck 
Update MANIFEST.in.
 2024-05-08 08:46:28 +04:00
Andrey Rakhmatullin 6bbfb537f9 Update MANIFEST.in. 2024-05-08 00:39:05 +05:00
Andrey Rakhmatullin c9bac7a657
Merge pull request #6341 from wRAR/typing-downloader 
More typing for scrapy/core/downloader
 2024-05-07 12:24:25 +04:00