RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,235 Commits 49 Branches 132 Tags 155 MiB
Commit Graph

10235 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 6f73dc0e67
Merge pull request #6213 from Laerte/master 
Cleanup deprecated fingerprint code in scrapy.utils.request
 2024-01-31 21:00:48 +04:00
Laerte Pereira 53ccf0016d Remove empty statement 2024-01-31 13:18:10 -03:00
Laerte Pereira 7001193c80 Simplify the logic 2024-01-31 12:53:08 -03:00
Laerte Pereira 24634f1bb2 Attend PR comments 2024-01-31 12:29:43 -03:00
Laerte Pereira bacaf0db7a Update documentation 2024-01-31 10:14:48 -03:00
Laerte Pereira 019443dd57 Remove settings from default implementation test 2024-01-31 10:08:07 -03:00
Laerte Pereira 2487e3cc03 Cleanup deprecated fingerprint code in scrapy.utils.request 2024-01-31 10:05:50 -03:00
Andrey Rakhmatullin 4229048255
Merge pull request #6208 from kevintomsgithub/add-faq 
Add FAQ on making a blank request
 2024-01-18 21:56:12 +04:00
Kevin Toms 9074c16497 make suggestion 2024-01-18 09:36:25 -05:00
Kevin Toms 46f94ec9cb Fix test 
Wrap the yield line in a function to prevent throwing error when the code snippet is executed
 2024-01-17 15:49:51 -05:00
Kevin Toms 88285e75b6 Add FAQ on making a blank request 2024-01-17 10:05:22 -05:00
Rotzbua d5233bb57f
chore(docs): update `sphinx` dependencies (#6200) 2024-01-15 14:11:33 +01:00
Adrián Chaves e8dadb9592
scrapy parse: fix the signature of callbacks from the CLI (#6182) 2024-01-15 13:37:03 +01:00
Adrián Chaves fa0c598096
Add component getters to Crawler (#6181) 2024-01-15 13:14:02 +01:00
Mikhail Korobov 8ad17f7476
Merge pull request #6199 from Rotzbua/fix_flake8 
fix(flake8): lint errors
 2024-01-12 22:52:22 +05:00
Mikhail Korobov b7cf30a48e
Merge pull request #6205 from wRAR/fix-pypy-tests 
Remove a defer.returnValue call.
 2024-01-12 22:50:41 +05:00
Andrey Rakhmatullin c2baf4d0da Remove a defer.returnValue call. 2024-01-12 18:30:41 +04:00
Andrey Rakhmatullin becff65ccb
Merge pull request #6201 from wRAR/more-mitmproxy 
Fix and re-enable newer mitmproxy usage in tests.
 2024-01-12 14:26:39 +04:00
Mikhail Korobov 32fda2dc53
Merge pull request #6196 from Rotzbua/fix_typo 
fix(typo): correct `successfully`
 2024-01-12 00:36:16 +05:00
Mikhail Korobov 7eeca55c70
Merge pull request #6197 from Rotzbua/refactor_yield 
refactor(yield): use `yield from` syntax
 2024-01-12 00:35:56 +05:00
Mikhail Korobov 825137399d
Merge pull request #6198 from Rotzbua/refactor_exception 
refactor(): use `OSError` exception
 2024-01-12 00:32:51 +05:00
Andrey Rakhmatullin 68fccb1d58 Fix and re-enable newer mitmproxy usage in tests. 2024-01-06 01:35:56 +04:00
Rotzbua 745b8412f6
fix(flake8): lint errors 
E226 missing whitespace around arithmetic operator
E201 whitespace after '{'
 2024-01-05 14:54:00 +01:00
Rotzbua 42c481cb4a
refactor(): use `OSError` exception 
https://docs.astral.sh/ruff/rules/os-error-alias/
 2024-01-05 12:36:36 +01:00
Rotzbua 0d445a3224
refactor(yield): use `yield from` syntax 2024-01-05 12:30:10 +01:00
Rotzbua c7b2b097b1
fix(typo): correct `successfully` 2024-01-05 10:50:45 +01:00
Chan Sau Yee 40e623b276
Add type hints (#6191) 2023-12-29 12:33:37 +01:00
Chan Sau Yee 1902284942
Update black reference in docs (#6192) 2023-12-29 12:32:51 +01:00
Adrián Chaves 34e01a8a93
Update quotes.toscrape.com page copies (#6190) 2023-12-28 12:25:01 +01:00
Adrián Chaves 0e78acb657
MediaPipeline: log media_to_download errors before stripping them (#5068) 2023-12-21 21:01:07 +01:00
Jalil SA d25cfe5315
Add JsonResponse (#6174) 2023-12-21 10:36:21 +01:00
Mikhail Korobov b498f1376d
Merge pull request #6183 from Gallaecio/zyte-guides 
Link to Zyte’s export guides
 2023-12-20 22:22:22 +05:00
Adrián Chaves 1864f48e9e Link to Zyte’s export guides 2023-12-20 12:47:34 +01:00
Swati Anshu c67f730695
create_instance → build_from_crawler, build_from_settings (#6169) 2023-12-18 11:51:02 +01:00
Adrián Chaves 70ba3a0868
SPM → Zyte API (#6163) 2023-11-30 11:01:22 +01:00
Adrián Chaves 731f749556
Extend Request.meta documentation (#5565) 2023-11-30 10:54:09 +01:00
Andrey Rakhmatullin fa690fbe03
Merge pull request #6154 from Laerte/master 
Remove test_download_gzip_response test
 2023-11-23 20:56:50 +04:00
Laerte Pereira 09ce0ef526 Remove test_download_gzip_response test 2023-11-23 10:50:46 -03:00
Andrey Rakhmatullin 42b3a3a23b
Merge pull request #6147 from wRAR/crawler-init-docs 
Improve the docs about Crawler attributes and settings initialization.
 2023-11-16 12:35:43 +04:00
Andrey Rakhmatullin 603aa4924a Improve the docs about Crawler attributes and settings initialization. 2023-11-15 19:51:04 +04:00
Kiran 492584ec07
Updated README.rst (#6144) 2023-11-13 20:13:10 +01:00
Andrey Rakhmatullin b4acf5c827
Fix and remove most of the entries from the mypy ignore list (#6137) 2023-11-07 09:34:35 +01:00
Andrey Rakhmatullin c31e09d709
Merge pull request #6134 from wRAR/obsolete-setter 
Remove obsolete setters for body and url on Request and Response.
 2023-11-06 18:24:13 +04:00
Andrey Rakhmatullin 7c27c22a98
Full typing for scrapy/downloadermiddlewares (#6129) 2023-11-06 08:52:14 +01:00
cakemd eafe828484
scrapy.utils.data_path type hint change (#6133) 2023-11-06 08:37:18 +01:00
Andrey Rakhmatullin 2ac3ef73e6 Remove obsolete setters for body and url on Request and Response. 2023-11-03 20:12:18 +04:00
Jessica Allman-LaPorte 6587556af9
Make shell switching more clear in the tutorial (#6128) 2023-11-03 10:02:18 +01:00
Andrey Rakhmatullin f3561807a6
Merge pull request #6130 from wRAR/typing-spider-mw 
Full typing for scrapy/spidermiddlewares.
 2023-11-03 12:31:19 +04:00
Jeesang Kim dda6feb935
Improve assert readability (#6132) 2023-11-03 09:24:25 +01:00
Andrey Rakhmatullin e54dc59899 Full typing for scrapy/spidermiddlewares. 2023-11-02 23:40:14 +04:00