RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,235 Commits 49 Branches 132 Tags 180 MiB
Commit Graph

2170 Commits

Author SHA1 Message Date
Laerte Pereira 7001193c80 Simplify the logic 2024-01-31 12:53:08 -03:00
Laerte Pereira 24634f1bb2 Attend PR comments 2024-01-31 12:29:43 -03:00
Laerte Pereira 019443dd57 Remove settings from default implementation test 2024-01-31 10:08:07 -03:00
Laerte Pereira 2487e3cc03 Cleanup deprecated fingerprint code in scrapy.utils.request 2024-01-31 10:05:50 -03:00
Adrián Chaves e8dadb9592
scrapy parse: fix the signature of callbacks from the CLI (#6182) 2024-01-15 13:37:03 +01:00
Adrián Chaves fa0c598096
Add component getters to Crawler (#6181) 2024-01-15 13:14:02 +01:00
Mikhail Korobov 8ad17f7476
Merge pull request #6199 from Rotzbua/fix_flake8 
fix(flake8): lint errors
 2024-01-12 22:52:22 +05:00
Andrey Rakhmatullin c2baf4d0da Remove a defer.returnValue call. 2024-01-12 18:30:41 +04:00
Andrey Rakhmatullin becff65ccb
Merge pull request #6201 from wRAR/more-mitmproxy 
Fix and re-enable newer mitmproxy usage in tests.
 2024-01-12 14:26:39 +04:00
Mikhail Korobov 32fda2dc53
Merge pull request #6196 from Rotzbua/fix_typo 
fix(typo): correct `successfully`
 2024-01-12 00:36:16 +05:00
Andrey Rakhmatullin 68fccb1d58 Fix and re-enable newer mitmproxy usage in tests. 2024-01-06 01:35:56 +04:00
Rotzbua 745b8412f6
fix(flake8): lint errors 
E226 missing whitespace around arithmetic operator
E201 whitespace after '{'
 2024-01-05 14:54:00 +01:00
Rotzbua 0d445a3224
refactor(yield): use `yield from` syntax 2024-01-05 12:30:10 +01:00
Rotzbua c7b2b097b1
fix(typo): correct `successfully` 2024-01-05 10:50:45 +01:00
Adrián Chaves 0e78acb657
MediaPipeline: log media_to_download errors before stripping them (#5068) 2023-12-21 21:01:07 +01:00
Jalil SA d25cfe5315
Add JsonResponse (#6174) 2023-12-21 10:36:21 +01:00
Swati Anshu c67f730695
create_instance → build_from_crawler, build_from_settings (#6169) 2023-12-18 11:51:02 +01:00
Laerte Pereira 09ce0ef526 Remove test_download_gzip_response test 2023-11-23 10:50:46 -03:00
Andrey Rakhmatullin b4acf5c827
Fix and remove most of the entries from the mypy ignore list (#6137) 2023-11-07 09:34:35 +01:00
Jeesang Kim dda6feb935
Improve assert readability (#6132) 2023-11-03 09:24:25 +01:00
Andrey Rakhmatullin 01d9d28324 Full typing for scrapy/http/response/__init__.py. 2023-11-01 01:10:32 +04:00
Chenwei Niu 1f797d0fdb
Removed some deprecated functions and functionalities (#6116) 2023-10-30 09:59:11 +01:00
Adrián Chaves 1d81585612
Merge pull request #6112 from wRAR/test-shutdown-forced 
Make shutdown tests more robust.
 2023-10-30 09:35:29 +01:00
Andrey Rakhmatullin 6b0c18e921 Improve signal sending in test_shutdown_forced. 2023-10-19 23:13:47 +04:00
Andrey Rakhmatullin cc9c415bf3 Disable ipython for InteractiveShellTest. 2023-10-18 19:30:57 +04:00
Laerte Pereira 38dbd43993 Remove tests not necessary anymore 2023-10-18 06:29:47 -03:00
Laerte Pereira 644ab3af48 Remove support for feed storage backends without feed_options 2023-10-17 22:39:09 -03:00
Laerte Pereira 3db438127c Remove support for HttpCompressionMiddleware subclasses without stats 2023-10-17 21:38:38 -03:00
Laerte Pereira a2b9351f04 Remove support for using HttpAuthMiddleware without http_auth_domain 2023-10-17 20:20:38 -03:00
Laerte Pereira 2de95f1fc0 Remove leftover from _make_compatible removal 2023-10-17 19:44:12 -03:00
Laerte Pereira ad4e8b64d4 fix test_spider_manager_verify_interface test 2023-10-17 18:58:27 -03:00
Laerte Pereira aa0a425826 Remove deprecated code moved to itemloaders 2023-10-17 18:20:12 -03:00
Laerte Pereira 7a5cefbcfa Remove deprecated scrapy.downloadermiddlewares.decompression 2023-10-17 17:49:22 -03:00
Andrey Rakhmatullin 2cb1e10c76 Make shutdown tests more robust. 2023-10-17 23:56:54 +04:00
Andrey Rakhmatullin 991121fa91
Re-enable uvloop tests on 3.12 (#6098) 2023-10-17 19:24:44 +02:00
Andrey Rakhmatullin 5807970a22
Add typing hints to CaselessDict and Headers. (#6097) 2023-10-17 19:07:13 +02:00
Andrey Rakhmatullin 029a56384d Increase the timeout. 2023-10-09 00:16:41 +04:00
Andrey Rakhmatullin 9910ccf3ae Merge remote-tracking branch 'origin/master' into signals-proper 2023-10-08 23:02:11 +04:00
Andrey Rakhmatullin 2f436c05e0 Use SIGBREAK in Windows tests. 2023-10-08 22:55:05 +04:00
Andrey Rakhmatullin a338873e3a Re-enable bpython on Python 3.12. 2023-10-03 16:50:52 +04:00
Andrey Rakhmatullin fb4debda04 Use path_to_file_uri to fix the test on Windows. 2023-10-03 16:47:03 +04:00
Andrey Rakhmatullin 1ed9ed4f92 Require pyftpdlib that supports Python 3.12. 2023-10-02 21:33:02 +04:00
Andrey Rakhmatullin eb5e2e79ba Use SIGINT instead of SIGTERM to support Windows. 2023-09-24 23:31:46 +04:00
Andrey Rakhmatullin 4b5fb9b5a6 Add a test for SIGTERM handling. 2023-09-24 22:52:17 +04:00
Andrey Rakhmatullin d19e315b0b Add an interactive test for `scrapy shell`. 2023-09-24 21:51:19 +04:00
Andrey Rakhmatullin 922ff57384 Improve the backwards compatibility for RetryMiddleware.EXCEPTIONS_TO_RETRY. 2023-09-14 15:51:00 +04:00
Andrey Rakhmatullin 6428356584 Use twisted.python.log instead of failure.printTraceback(). 2023-09-13 18:20:42 +04:00
Andrey Rakhmatullin be0e33af92 Don't rely on unhandled exception logging in the expected to fail scripts. 2023-09-13 16:59:54 +04:00
Andrey Rakhmatullin 3756216339 Rename methods. 2023-09-13 15:47:22 +04:00
Andrey Rakhmatullin 619140717f Fix the new addon test. 2023-09-13 15:33:51 +04:00