RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,191 Commits 49 Branches 132 Tags 162 MiB
Commit Graph

11191 Commits

Author SHA1 Message Date
Jessica Allman-LaPorte 6587556af9
Make shell switching more clear in the tutorial (#6128) 2023-11-03 10:02:18 +01:00
Andrey Rakhmatullin f3561807a6
Merge pull request #6130 from wRAR/typing-spider-mw 
Full typing for scrapy/spidermiddlewares.
 2023-11-03 12:31:19 +04:00
Jeesang Kim dda6feb935
Improve assert readability (#6132) 2023-11-03 09:24:25 +01:00
Andrey Rakhmatullin e54dc59899 Full typing for scrapy/spidermiddlewares. 2023-11-02 23:40:14 +04:00
Andrey Rakhmatullin 593bfd895a
Merge pull request #6127 from wRAR/typing-response 
Full typing for Request and Response
 2023-11-02 17:46:42 +04:00
Andrey Rakhmatullin 24f21e96b9 Full typing for scrapy/http/response/text.py. 2023-11-01 01:53:25 +04:00
Andrey Rakhmatullin 01d9d28324 Full typing for scrapy/http/response/__init__.py. 2023-11-01 01:10:32 +04:00
Andrey Rakhmatullin 4cb2fc2c3b Update typing package versions. 2023-10-31 22:55:39 +04:00
Andrey Rakhmatullin 732557e698 Full typing for scrapy/http/request. 2023-10-31 22:48:22 +04:00
nihilisticneuralnet 04024f1e79
[Solved] JOBDIR= None for when Scheduler initializes disk queue even if JOBDIR is empty string (#6124) 
Co-authored-by: John Doe <johndoe@email.com>
 2023-10-31 17:37:26 +04:00
Andrey Rakhmatullin 7e6da37170
Merge pull request #6125 from Andy53/additional_ignored_extensions 
added_extensions
 2023-10-31 12:33:55 +04:00
andy53 8dff9633d0 added_extensions 2023-10-30 21:40:26 -06:00
Chenwei Niu 1f797d0fdb
Removed some deprecated functions and functionalities (#6116) 2023-10-30 09:59:11 +01:00
Adrián Chaves 1d81585612
Merge pull request #6112 from wRAR/test-shutdown-forced 
Make shutdown tests more robust.
 2023-10-30 09:35:29 +01:00
Andrey Rakhmatullin 6b0c18e921 Improve signal sending in test_shutdown_forced. 2023-10-19 23:13:47 +04:00
Andrey Rakhmatullin cc9c415bf3 Disable ipython for InteractiveShellTest. 2023-10-18 19:30:57 +04:00
Adrián Chaves 9b06f6b316
Merge pull request #6113 from Laerte/master 
Remove some deprecated code
 2023-10-18 12:52:27 +02:00
Laerte Pereira 38dbd43993 Remove tests not necessary anymore 2023-10-18 06:29:47 -03:00
Andrey Rakhmatullin 39ee8d1ee2
Deprecate ReppyRobotParser (#6099) 2023-10-18 09:10:21 +02:00
Laerte Pereira 0956b76465 Remove support for dupefilters without a fingerprinter 2023-10-17 22:44:15 -03:00
Laerte Pereira 644ab3af48 Remove support for feed storage backends without feed_options 2023-10-17 22:39:09 -03:00
Laerte Pereira 3db438127c Remove support for HttpCompressionMiddleware subclasses without stats 2023-10-17 21:38:38 -03:00
Laerte Pereira a2b9351f04 Remove support for using HttpAuthMiddleware without http_auth_domain 2023-10-17 20:20:38 -03:00
Laerte Pereira 4ad727f0b5 Remove _check_signature 2023-10-17 19:47:01 -03:00
Laerte Pereira 2de95f1fc0 Remove leftover from _make_compatible removal 2023-10-17 19:44:12 -03:00
Laerte Pereira ad4e8b64d4 fix test_spider_manager_verify_interface test 2023-10-17 18:58:27 -03:00
Laerte Pereira aa0a425826 Remove deprecated code moved to itemloaders 2023-10-17 18:20:12 -03:00
Laerte Pereira 85c57778b5 Remove deprecated JSONRequest 2023-10-17 18:15:20 -03:00
Laerte Pereira 83f500a352 Remove scrapy.pipelines.media.MediaPipeline._make_compatible 2023-10-17 18:14:08 -03:00
Laerte Pereira bdb4abcc7a Remove the deprecation warning from CrawlerRunner._get_spider_loader() 2023-10-17 17:52:18 -03:00
Laerte Pereira 7a5cefbcfa Remove deprecated scrapy.downloadermiddlewares.decompression 2023-10-17 17:49:22 -03:00
Andrey Rakhmatullin 2cb1e10c76 Make shutdown tests more robust. 2023-10-17 23:56:54 +04:00
Andrey Rakhmatullin 991121fa91
Re-enable uvloop tests on 3.12 (#6098) 2023-10-17 19:24:44 +02:00
Andrey Rakhmatullin 5807970a22
Add typing hints to CaselessDict and Headers. (#6097) 2023-10-17 19:07:13 +02:00
Sandesh Pyakurel 064256b059
Typo fixed in news.rst file (#6094) 2023-10-17 18:39:39 +02:00
Andrey Rakhmatullin aa95ada42c
Merge pull request #6064 from wRAR/signals-proper 
Refactor installing signals.
 2023-10-17 17:08:23 +04:00
Andrey Rakhmatullin 029a56384d Increase the timeout. 2023-10-09 00:16:41 +04:00
Andrey Rakhmatullin 9910ccf3ae Merge remote-tracking branch 'origin/master' into signals-proper 2023-10-08 23:02:11 +04:00
Andrey Rakhmatullin 2f436c05e0 Use SIGBREAK in Windows tests. 2023-10-08 22:55:05 +04:00
Andrey Rakhmatullin c65567988d
Change supported PyPy versions to 3.9 and 3.10. (#6087) 
* Change support PyPy versions to 3.9 and 3.10.

* Update the RTD URL for coverage.

* Move pypy3-pinned back to pypy3.8.
 2023-10-06 11:27:02 +02:00
Andrey Rakhmatullin 5b0b00212e
Merge pull request #6083 from wRAR/py3.12-release 
Adapt to the Python 3.12 final release
 2023-10-03 20:20:12 +04:00
Andrey Rakhmatullin a338873e3a Re-enable bpython on Python 3.12. 2023-10-03 16:50:52 +04:00
Andrey Rakhmatullin fb4debda04 Use path_to_file_uri to fix the test on Windows. 2023-10-03 16:47:03 +04:00
Klaus Rettinghaus 9ae8d97d81
Update GitHub Actions checkout action to version 4 (#6084) 2023-10-03 13:04:18 +02:00
Andrey Rakhmatullin 60d5f391c4 Use the 3.12 release, enable it for all OSes. 2023-10-02 21:35:45 +04:00
Andrey Rakhmatullin 1ed9ed4f92 Require pyftpdlib that supports Python 3.12. 2023-10-02 21:33:02 +04:00
Andrey Rakhmatullin a96989c6f3
Merge pull request #6077 from 11-aryan/11-aryan 2023-10-02 20:40:25 +04:00
Andrey Rakhmatullin 01f164d331
Merge pull request #6078 from AndrewADev/ci-concurrency-groups 2023-10-02 20:38:49 +04:00
Adrián Chaves 42adbb2104
Update docs/topics/request-response.rst 2023-10-02 14:43:30 +02:00
kokobhara 8dc72dfc4d Cover PythonItemExporter backwaird-incompatible changes in 2.11 (#6081) 2023-10-02 14:48:44 +04:00