RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,497 Commits 49 Branches 132 Tags 182 MiB
Commit Graph

10497 Commits

Author SHA1 Message Date
Andrey Rakhmatullin a364560fad
Unpin markupsafe in extra-deps. (#6403) 2024-06-17 11:38:10 +02:00
Andrey Rakhmatullin 365c9e62ad
Removing empty example reference (#6402) 
Co-authored-by: Michael Duane Mooring <mikeumus@gmail.com>
 2024-06-17 11:37:11 +02:00
Andrey Rakhmatullin 1282ddf8f7
Add parameters to most Deferred in scrapy/core. (#6395) 2024-06-10 10:27:50 +02:00
Andrey Rakhmatullin ddc98fe91b
Deprecate scrapy.utils.defer.process_chain_both(). (#6397) 2024-06-10 10:16:26 +02:00
Andrey Rakhmatullin 163e7d925e
Merge pull request #6392 from wRAR/pyupgrade 
Add pyupgrade.
 2024-06-07 11:16:46 +04:00
Andrey Rakhmatullin 5850b8f3e6 Merge branch 'master' into pyupgrade 2024-06-07 11:20:31 +05:00
Andrey Rakhmatullin ed3a7acaf3 Remove the Python version check from scrapy/__init__.py. 2024-06-07 11:19:37 +05:00
Andrey Rakhmatullin a4778d2bdf
Merge pull request #6143 from wRAR/typing-request-response-cls 
Improve type hints for copy() and replace() in Request and Response.
 2024-06-06 22:58:14 +04:00
Andrey Rakhmatullin 1268b23304 Merge branch 'master' into typing-request-response-cls 2024-06-06 21:57:53 +05:00
Andrey Rakhmatullin b24ecca4d0
Merge pull request #6391 from wRAR/typing-generics-iterables 
Add parameters to iterable generics, replace generators with iterables.
 2024-06-06 20:54:16 +04:00
Andrey Rakhmatullin 23b1214e90 Add a comment about pyupgrade and scrapy/__init__.py. 2024-06-06 21:44:07 +05:00
Andrey Rakhmatullin d9d7bd170b
Merge pull request #6394 from Laerte/master 
Document missing parts of response.json method
 2024-06-06 09:55:56 +04:00
Laerte Pereira 144ff6c756 Document missing parts of response.json method 2024-06-05 21:09:10 -03:00
Andrey Rakhmatullin feb0b8f7dc Add pyupgrade. 2024-06-05 22:57:18 +05:00
Andrey Rakhmatullin 480a11b68b Add mssing __future__ imports. 2024-06-05 22:48:16 +05:00
Andrey Rakhmatullin 262c10d85b Use typing.Coroutine instead of types.CoroutineType. 2024-06-05 22:11:34 +05:00
Andrey Rakhmatullin de146ad7ce Bump typing deps. 2024-06-05 22:09:19 +05:00
Andrey Rakhmatullin 2e214210f6 Add parameters to iterable generics, replace generators with iterables. 2024-06-05 21:44:04 +05:00
Suvan Banerjee 3f76853bd2
Handle AttributeError: 'NoneType' in contract parsing (#6388) 2024-06-05 06:34:46 +02:00
Andrey Rakhmatullin e56b425198
Full typing for scrapy/pipelines. (#6387) 2024-06-05 06:33:45 +02:00
Adrián Chaves 2b9e32f1ca
Merge pull request #6385 from wRAR/typing-generics-collections 
Add parameters to various generics.
 2024-06-03 13:29:38 +02:00
Andrey Rakhmatullin 492c3bce9d
Don't run callbacks of requests from get_media_requests(). (#6386) 2024-06-03 13:28:20 +02:00
Andrey Rakhmatullin 019f23e3b7 Add parameters to some of typing.Callable. 2024-06-02 18:42:01 +05:00
Andrey Rakhmatullin 859a77ee42 Use a TypedDict for the verbose cookie form. 2024-05-31 22:23:26 +05:00
Andrey Rakhmatullin 751c91e614 Add parameters to misc generics. 2024-05-31 21:57:14 +05:00
Andrey Rakhmatullin 70c56faf48 Add parameters to typing.IO. 2024-05-31 21:41:27 +05:00
Andrey Rakhmatullin 4164e63725 Add parameters to typing.Tuple. 2024-05-31 21:23:55 +05:00
Andrey Rakhmatullin 98c755e5fb Add parameters to typing.List. 2024-05-31 21:20:22 +05:00
Andrey Rakhmatullin da42e8f124 Add parameters to typing.Dict. 2024-05-31 21:11:50 +05:00
Andrey Rakhmatullin b950ed77b6
Merge branch 'master' into typing-request-response-cls 2024-05-31 13:07:55 +04:00
Andrey Rakhmatullin b4293e8f9e
Misc typing improvements. (#6384) 2024-05-31 08:50:36 +02:00
Andrey Rakhmatullin a011fa6f78
Merge pull request #6371 from wRAR/typing-contracts 
Full typing for scrapy/contracts.
 2024-05-30 20:59:50 +04:00
Andrey Rakhmatullin 469e8a23f8
Merge pull request #6381 from 0xdeb/referer-policy-tokens 
Add support for multiple referer policy tokens.
 2024-05-29 17:13:09 +04:00
Fabian Schneebauer 62a028b99d
Add spec link to scrapy/spidermiddlewares/referer.py 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2024-05-29 13:19:27 +02:00
Fabian Schneebauer 0d58af8697 Add support for multiple referer policy tokens. 2024-05-29 10:59:32 +02:00
Sanchay Kumar cadb0dd707
Fix overridable methods in MediaPipeline (#6368) 2024-05-28 10:42:58 +02:00
Andrey Rakhmatullin 986d1ee1dd
Move CI from the decommissioned macos-11 to macos-latest. (#6372) 2024-05-28 10:37:19 +02:00
Andrey Rakhmatullin 9ba4dd311d
Install typing stubs for boto3 and botocore. (#6370) 2024-05-28 10:27:49 +02:00
Andrey Rakhmatullin f9a9860306
Merge pull request #6374 from Laerte/master 
docs: Remove top-level reactor imports from CrawlerProces/CrawlerRunner examples
 2024-05-27 14:36:34 +04:00
Laerte Pereira 6cd0857850 Move path 2024-05-26 19:57:16 -03:00
Laerte Pereira 2facdd4fb0 Add change reactor test to CrawlerRunner 2024-05-26 19:55:54 -03:00
Laerte Pereira 62c89aaf05
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-23 07:00:45 -03:00
Laerte Pereira 8ec67ca230
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-23 07:00:35 -03:00
Laerte Pereira 17e623cf0c
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-23 07:00:24 -03:00
Laerte Pereira 9d5a0d287b
Retrigger CI 2024-05-22 22:27:07 -03:00
Laerte Pereira e143dc7952
Update tests-macos.yml 2024-05-22 22:26:31 -03:00
Laerte Pereira 3f66b66e3f fix: checks 2024-05-22 22:01:55 -03:00
Laerte Pereira dc6a495fee
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-22 18:51:02 -03:00
Laerte Pereira 8210fae25a
Update docs/topics/practices.rst 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-05-22 18:50:50 -03:00
Laerte Pereira e676cd3ce0 docs: Remove top-level reactor imports from CrawlerProces/CrawlerRunner examples 2024-05-22 07:55:53 -03:00