RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,209 Commits 49 Branches 132 Tags 176 MiB
Commit Graph

10209 Commits

Author SHA1 Message Date
Chan Sau Yee 40e623b276
Add type hints (#6191) 2023-12-29 12:33:37 +01:00
Chan Sau Yee 1902284942
Update black reference in docs (#6192) 2023-12-29 12:32:51 +01:00
Adrián Chaves 34e01a8a93
Update quotes.toscrape.com page copies (#6190) 2023-12-28 12:25:01 +01:00
Adrián Chaves 0e78acb657
MediaPipeline: log media_to_download errors before stripping them (#5068) 2023-12-21 21:01:07 +01:00
Jalil SA d25cfe5315
Add JsonResponse (#6174) 2023-12-21 10:36:21 +01:00
Mikhail Korobov b498f1376d
Merge pull request #6183 from Gallaecio/zyte-guides 
Link to Zyte’s export guides
 2023-12-20 22:22:22 +05:00
Adrián Chaves 1864f48e9e Link to Zyte’s export guides 2023-12-20 12:47:34 +01:00
Swati Anshu c67f730695
create_instance → build_from_crawler, build_from_settings (#6169) 2023-12-18 11:51:02 +01:00
Adrián Chaves 70ba3a0868
SPM → Zyte API (#6163) 2023-11-30 11:01:22 +01:00
Adrián Chaves 731f749556
Extend Request.meta documentation (#5565) 2023-11-30 10:54:09 +01:00
Andrey Rakhmatullin fa690fbe03
Merge pull request #6154 from Laerte/master 
Remove test_download_gzip_response test
 2023-11-23 20:56:50 +04:00
Laerte Pereira 09ce0ef526 Remove test_download_gzip_response test 2023-11-23 10:50:46 -03:00
Andrey Rakhmatullin 42b3a3a23b
Merge pull request #6147 from wRAR/crawler-init-docs 
Improve the docs about Crawler attributes and settings initialization.
 2023-11-16 12:35:43 +04:00
Andrey Rakhmatullin 603aa4924a Improve the docs about Crawler attributes and settings initialization. 2023-11-15 19:51:04 +04:00
Kiran 492584ec07
Updated README.rst (#6144) 2023-11-13 20:13:10 +01:00
Andrey Rakhmatullin b4acf5c827
Fix and remove most of the entries from the mypy ignore list (#6137) 2023-11-07 09:34:35 +01:00
Andrey Rakhmatullin c31e09d709
Merge pull request #6134 from wRAR/obsolete-setter 
Remove obsolete setters for body and url on Request and Response.
 2023-11-06 18:24:13 +04:00
Andrey Rakhmatullin 7c27c22a98
Full typing for scrapy/downloadermiddlewares (#6129) 2023-11-06 08:52:14 +01:00
cakemd eafe828484
scrapy.utils.data_path type hint change (#6133) 2023-11-06 08:37:18 +01:00
Andrey Rakhmatullin 2ac3ef73e6 Remove obsolete setters for body and url on Request and Response. 2023-11-03 20:12:18 +04:00
Jessica Allman-LaPorte 6587556af9
Make shell switching more clear in the tutorial (#6128) 2023-11-03 10:02:18 +01:00
Andrey Rakhmatullin f3561807a6
Merge pull request #6130 from wRAR/typing-spider-mw 
Full typing for scrapy/spidermiddlewares.
 2023-11-03 12:31:19 +04:00
Jeesang Kim dda6feb935
Improve assert readability (#6132) 2023-11-03 09:24:25 +01:00
Andrey Rakhmatullin e54dc59899 Full typing for scrapy/spidermiddlewares. 2023-11-02 23:40:14 +04:00
Andrey Rakhmatullin 593bfd895a
Merge pull request #6127 from wRAR/typing-response 
Full typing for Request and Response
 2023-11-02 17:46:42 +04:00
Andrey Rakhmatullin 24f21e96b9 Full typing for scrapy/http/response/text.py. 2023-11-01 01:53:25 +04:00
Andrey Rakhmatullin 01d9d28324 Full typing for scrapy/http/response/__init__.py. 2023-11-01 01:10:32 +04:00
Andrey Rakhmatullin 4cb2fc2c3b Update typing package versions. 2023-10-31 22:55:39 +04:00
Andrey Rakhmatullin 732557e698 Full typing for scrapy/http/request. 2023-10-31 22:48:22 +04:00
nihilisticneuralnet 04024f1e79
[Solved] JOBDIR= None for when Scheduler initializes disk queue even if JOBDIR is empty string (#6124) 
Co-authored-by: John Doe <johndoe@email.com>
 2023-10-31 17:37:26 +04:00
Andrey Rakhmatullin 7e6da37170
Merge pull request #6125 from Andy53/additional_ignored_extensions 
added_extensions
 2023-10-31 12:33:55 +04:00
andy53 8dff9633d0 added_extensions 2023-10-30 21:40:26 -06:00
Chenwei Niu 1f797d0fdb
Removed some deprecated functions and functionalities (#6116) 2023-10-30 09:59:11 +01:00
Adrián Chaves 1d81585612
Merge pull request #6112 from wRAR/test-shutdown-forced 
Make shutdown tests more robust.
 2023-10-30 09:35:29 +01:00
Andrey Rakhmatullin 6b0c18e921 Improve signal sending in test_shutdown_forced. 2023-10-19 23:13:47 +04:00
Andrey Rakhmatullin cc9c415bf3 Disable ipython for InteractiveShellTest. 2023-10-18 19:30:57 +04:00
Adrián Chaves 9b06f6b316
Merge pull request #6113 from Laerte/master 
Remove some deprecated code
 2023-10-18 12:52:27 +02:00
Laerte Pereira 38dbd43993 Remove tests not necessary anymore 2023-10-18 06:29:47 -03:00
Andrey Rakhmatullin 39ee8d1ee2
Deprecate ReppyRobotParser (#6099) 2023-10-18 09:10:21 +02:00
Laerte Pereira 0956b76465 Remove support for dupefilters without a fingerprinter 2023-10-17 22:44:15 -03:00
Laerte Pereira 644ab3af48 Remove support for feed storage backends without feed_options 2023-10-17 22:39:09 -03:00
Laerte Pereira 3db438127c Remove support for HttpCompressionMiddleware subclasses without stats 2023-10-17 21:38:38 -03:00
Laerte Pereira a2b9351f04 Remove support for using HttpAuthMiddleware without http_auth_domain 2023-10-17 20:20:38 -03:00
Laerte Pereira 4ad727f0b5 Remove _check_signature 2023-10-17 19:47:01 -03:00
Laerte Pereira 2de95f1fc0 Remove leftover from _make_compatible removal 2023-10-17 19:44:12 -03:00
Laerte Pereira ad4e8b64d4 fix test_spider_manager_verify_interface test 2023-10-17 18:58:27 -03:00
Laerte Pereira aa0a425826 Remove deprecated code moved to itemloaders 2023-10-17 18:20:12 -03:00
Laerte Pereira 85c57778b5 Remove deprecated JSONRequest 2023-10-17 18:15:20 -03:00
Laerte Pereira 83f500a352 Remove scrapy.pipelines.media.MediaPipeline._make_compatible 2023-10-17 18:14:08 -03:00
Laerte Pereira bdb4abcc7a Remove the deprecation warning from CrawlerRunner._get_spider_loader() 2023-10-17 17:52:18 -03:00