RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,219 Commits 49 Branches 132 Tags 152 MiB
Commit Graph

10219 Commits

Author SHA1 Message Date
Mikhail Korobov b7cf30a48e
Merge pull request #6205 from wRAR/fix-pypy-tests 
Remove a defer.returnValue call.
 2024-01-12 22:50:41 +05:00
Andrey Rakhmatullin c2baf4d0da Remove a defer.returnValue call. 2024-01-12 18:30:41 +04:00
Andrey Rakhmatullin becff65ccb
Merge pull request #6201 from wRAR/more-mitmproxy 
Fix and re-enable newer mitmproxy usage in tests.
 2024-01-12 14:26:39 +04:00
Mikhail Korobov 32fda2dc53
Merge pull request #6196 from Rotzbua/fix_typo 
fix(typo): correct `successfully`
 2024-01-12 00:36:16 +05:00
Mikhail Korobov 7eeca55c70
Merge pull request #6197 from Rotzbua/refactor_yield 
refactor(yield): use `yield from` syntax
 2024-01-12 00:35:56 +05:00
Mikhail Korobov 825137399d
Merge pull request #6198 from Rotzbua/refactor_exception 
refactor(): use `OSError` exception
 2024-01-12 00:32:51 +05:00
Andrey Rakhmatullin 68fccb1d58 Fix and re-enable newer mitmproxy usage in tests. 2024-01-06 01:35:56 +04:00
Rotzbua 42c481cb4a
refactor(): use `OSError` exception 
https://docs.astral.sh/ruff/rules/os-error-alias/
 2024-01-05 12:36:36 +01:00
Rotzbua 0d445a3224
refactor(yield): use `yield from` syntax 2024-01-05 12:30:10 +01:00
Rotzbua c7b2b097b1
fix(typo): correct `successfully` 2024-01-05 10:50:45 +01:00
Chan Sau Yee 40e623b276
Add type hints (#6191) 2023-12-29 12:33:37 +01:00
Chan Sau Yee 1902284942
Update black reference in docs (#6192) 2023-12-29 12:32:51 +01:00
Adrián Chaves 34e01a8a93
Update quotes.toscrape.com page copies (#6190) 2023-12-28 12:25:01 +01:00
Adrián Chaves 0e78acb657
MediaPipeline: log media_to_download errors before stripping them (#5068) 2023-12-21 21:01:07 +01:00
Jalil SA d25cfe5315
Add JsonResponse (#6174) 2023-12-21 10:36:21 +01:00
Mikhail Korobov b498f1376d
Merge pull request #6183 from Gallaecio/zyte-guides 
Link to Zyte’s export guides
 2023-12-20 22:22:22 +05:00
Adrián Chaves 1864f48e9e Link to Zyte’s export guides 2023-12-20 12:47:34 +01:00
Swati Anshu c67f730695
create_instance → build_from_crawler, build_from_settings (#6169) 2023-12-18 11:51:02 +01:00
Adrián Chaves 70ba3a0868
SPM → Zyte API (#6163) 2023-11-30 11:01:22 +01:00
Adrián Chaves 731f749556
Extend Request.meta documentation (#5565) 2023-11-30 10:54:09 +01:00
Andrey Rakhmatullin fa690fbe03
Merge pull request #6154 from Laerte/master 
Remove test_download_gzip_response test
 2023-11-23 20:56:50 +04:00
Laerte Pereira 09ce0ef526 Remove test_download_gzip_response test 2023-11-23 10:50:46 -03:00
Andrey Rakhmatullin 42b3a3a23b
Merge pull request #6147 from wRAR/crawler-init-docs 
Improve the docs about Crawler attributes and settings initialization.
 2023-11-16 12:35:43 +04:00
Andrey Rakhmatullin 603aa4924a Improve the docs about Crawler attributes and settings initialization. 2023-11-15 19:51:04 +04:00
Kiran 492584ec07
Updated README.rst (#6144) 2023-11-13 20:13:10 +01:00
Andrey Rakhmatullin b4acf5c827
Fix and remove most of the entries from the mypy ignore list (#6137) 2023-11-07 09:34:35 +01:00
Andrey Rakhmatullin c31e09d709
Merge pull request #6134 from wRAR/obsolete-setter 
Remove obsolete setters for body and url on Request and Response.
 2023-11-06 18:24:13 +04:00
Andrey Rakhmatullin 7c27c22a98
Full typing for scrapy/downloadermiddlewares (#6129) 2023-11-06 08:52:14 +01:00
cakemd eafe828484
scrapy.utils.data_path type hint change (#6133) 2023-11-06 08:37:18 +01:00
Andrey Rakhmatullin 2ac3ef73e6 Remove obsolete setters for body and url on Request and Response. 2023-11-03 20:12:18 +04:00
Jessica Allman-LaPorte 6587556af9
Make shell switching more clear in the tutorial (#6128) 2023-11-03 10:02:18 +01:00
Andrey Rakhmatullin f3561807a6
Merge pull request #6130 from wRAR/typing-spider-mw 
Full typing for scrapy/spidermiddlewares.
 2023-11-03 12:31:19 +04:00
Jeesang Kim dda6feb935
Improve assert readability (#6132) 2023-11-03 09:24:25 +01:00
Andrey Rakhmatullin e54dc59899 Full typing for scrapy/spidermiddlewares. 2023-11-02 23:40:14 +04:00
Andrey Rakhmatullin 593bfd895a
Merge pull request #6127 from wRAR/typing-response 
Full typing for Request and Response
 2023-11-02 17:46:42 +04:00
Andrey Rakhmatullin 24f21e96b9 Full typing for scrapy/http/response/text.py. 2023-11-01 01:53:25 +04:00
Andrey Rakhmatullin 01d9d28324 Full typing for scrapy/http/response/__init__.py. 2023-11-01 01:10:32 +04:00
Andrey Rakhmatullin 4cb2fc2c3b Update typing package versions. 2023-10-31 22:55:39 +04:00
Andrey Rakhmatullin 732557e698 Full typing for scrapy/http/request. 2023-10-31 22:48:22 +04:00
nihilisticneuralnet 04024f1e79
[Solved] JOBDIR= None for when Scheduler initializes disk queue even if JOBDIR is empty string (#6124) 
Co-authored-by: John Doe <johndoe@email.com>
 2023-10-31 17:37:26 +04:00
Andrey Rakhmatullin 7e6da37170
Merge pull request #6125 from Andy53/additional_ignored_extensions 
added_extensions
 2023-10-31 12:33:55 +04:00
andy53 8dff9633d0 added_extensions 2023-10-30 21:40:26 -06:00
Chenwei Niu 1f797d0fdb
Removed some deprecated functions and functionalities (#6116) 2023-10-30 09:59:11 +01:00
Adrián Chaves 1d81585612
Merge pull request #6112 from wRAR/test-shutdown-forced 
Make shutdown tests more robust.
 2023-10-30 09:35:29 +01:00
Andrey Rakhmatullin 6b0c18e921 Improve signal sending in test_shutdown_forced. 2023-10-19 23:13:47 +04:00
Andrey Rakhmatullin cc9c415bf3 Disable ipython for InteractiveShellTest. 2023-10-18 19:30:57 +04:00
Adrián Chaves 9b06f6b316
Merge pull request #6113 from Laerte/master 
Remove some deprecated code
 2023-10-18 12:52:27 +02:00
Laerte Pereira 38dbd43993 Remove tests not necessary anymore 2023-10-18 06:29:47 -03:00
Andrey Rakhmatullin 39ee8d1ee2
Deprecate ReppyRobotParser (#6099) 2023-10-18 09:10:21 +02:00
Laerte Pereira 0956b76465 Remove support for dupefilters without a fingerprinter 2023-10-17 22:44:15 -03:00