RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,217 Commits 50 Branches 132 Tags 142 MiB
Commit Graph

56 Commits

Author SHA1 Message Date
Adrian Chaves 043b1b04ea Merge origin/master into integrate-mime 
Integrates upstream changes (async handler refactoring, pyproject.toml
migration, xtractmime dependency) with the MIME sniffing branch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-10 10:44:16 +02:00
Ethan Kuo df2f3d708e
fix open_in_browser() logic issue plus add new tests (#7506) 2026-06-04 16:24:50 +02:00
Andrey Rakhmatullin fc4c57e795
Fix the substitution pattern in open_in_browser(). (#7459) 
* Fix the substitution pattern in open_in_browser().

* Formatting.
 2026-04-24 13:54:37 +05:00
Ashar Khan d239fcf936
Fix get_meta_refresh to correctly resolve relative URLs using <base> tag (#7047) 2025-09-10 15:17:16 +02:00
Andrey Rakhmatullin d70f8a3f14
Refactoring of test_utils_*. (#6905) 2025-06-23 23:39:24 +05:00
Andrey Rakhmatullin d161d1d47d
Convert tests/test_utils* to plain asserts. (#6695) 2025-03-04 09:31:26 +01:00
Andrey Rakhmatullin 8d92c28a16
Switch to pytest.raises(). (#6680) 
* Switch to pytest.raises().

* Add matches= to broad pytest.raises().

* Adjust the test_nonserializable_object() regex for Python <= 3.11.

* Adjust the test_nonserializable_object() regex for PyPy.

* Adjust other test exception regexes for PyPy.

* Cleanup.
 2025-02-27 23:13:04 +05:00
Andrey Rakhmatullin a898331d14
Preparations for switching to direct pytest. (#6678) 2025-02-16 20:28:58 +01:00
Andrey Rakhmatullin cec0aeca58
Bump ruff, switch from black to ruff-format (#6631) 2025-01-27 11:07:09 +01:00
Adrián Chaves 6bd45bb6d9 Merge remote-tracking branch 'scrapy/2.11' into 2.11-port 2024-02-15 06:39:50 +01:00
Adrián Chaves 2f1d345e74 Solve test issues 2024-02-14 18:59:01 +01:00
Adrián Chaves 13bc1499ac Ignore Content-Type when it triggers Response 2024-02-09 08:20:49 +01:00
Adrián Chaves c5dad41190 Speed up tests, remove comments without regexps 2024-02-02 14:03:16 +01:00
Adrián Chaves 2d9857b441 Merge remote-tracking branch 'scrapy/master' into integrate-mime 2023-12-29 08:30:44 +01:00
Adrián Chaves 150d96764b Deprecate xmliter in favor of xmliter_lxml 2023-12-15 11:42:55 +01:00
Adrián Chaves 9655b0b8eb Mark slow tests, with their own tox env and CI job 2023-12-15 10:19:47 +01:00
Adrián Chaves 1533b69032 Test and address ReDoS attack vectors for open_in_browser 2023-12-13 12:01:35 +01:00
Chenwei Niu 1f797d0fdb
Removed some deprecated functions and functionalities (#6116) 2023-10-30 09:59:11 +01:00
Jalil SA a5c1ef8276 sort imports with isort 2023-01-25 14:25:15 -06:00
Adrián Chaves 1ade9d2ed7 Minor test changes 2023-01-10 21:41:49 +01:00
Adrián Chaves 7bb5289ced Cover additional behavior changes in tests 2023-01-10 21:11:57 +01:00
Adrián Chaves 86baef02a0 Test Content-Encoding against Content-Disposition, URL and body 2023-01-10 20:51:43 +01:00
Adrián Chaves deab93b243 Remove duplicate test 2023-01-10 19:04:52 +01:00
Adrián Chaves fc2833d6bd Fix _get_encoding_or_mime_type_from_headers handling on comma-separated Content-Encoding values 2023-01-10 18:32:27 +01:00
Adrián Chaves 68fccc9f04 Test that Content-Disposition takes precedence over body 2023-01-10 16:45:33 +01:00
Adrián Chaves f538252d3b crazy_case → odd_capitalize 2023-01-10 16:40:21 +01:00
Adrián Chaves 14464388e8 Address flake8 issues 2023-01-10 16:38:42 +01:00
Adrián Chaves dd4f4f094e Test handling of feeds mislabeled as HTML 2023-01-10 16:33:54 +01:00
Adrián Chaves 2178521a79 Make sure that supplied “unknown” MIME types from Content-Type are ignored 2023-01-10 13:41:36 +01:00
Adrián Chaves 8908151608 Test that MIME parameters do not break response class choice 2023-01-10 13:01:25 +01:00
Adrián Chaves 45b95c63d7 Address issues reported by pylint 2023-01-10 11:30:06 +01:00
Adrián Chaves e7e201d0b9 Extend tests covering body-based response class choice 2023-01-10 11:15:56 +01:00
Adrián Chaves fcc9e90553 Wrap XHTML with XmlResponse 2023-01-10 07:30:40 +01:00
Adrián Chaves 827de3d288 Finish cleaning up POST_XTRACTMIME_SCENARIOS 2023-01-08 17:22:12 +01:00
Adrián Chaves 711a6093e1 Finish cleaning up PRE_XTRACTMIME_SCENARIOS 2023-01-08 17:02:56 +01:00
Adrián Chaves b2f2788bd4 Add tests for binary-vs-text body 2023-01-08 16:36:35 +01:00
Adrián Chaves f9f9d61457 Always prefer Content-Type over Content-Disposition 2023-01-08 15:46:32 +01:00
Adrián Chaves e7d36a305d Ignore file path extensions in HTTP responses 2023-01-08 14:24:54 +01:00
Adrián Chaves fa3ba512d9 Uncomment test scenarios 2023-01-08 13:21:36 +01:00
Adrián Chaves f029be3aa7 Fix Apache bug handling 2023-01-08 12:55:17 +01:00
Adrián Chaves e96b07bd4b flake8: ignore failing file for the time being 2023-01-08 11:38:46 +01:00
Adrián Chaves b336980bd1 Merge remote-tracking branch 'upstream/master' into integrate-mime 2023-01-02 13:14:39 +01:00
Emmanuel Rondan e211ec0aa2 adding black formatter to all the code 2022-11-29 11:30:46 -03:00
Matthew Donoughe 40d9ca3bdd
use pathlib 2022-10-17 17:40:10 -04:00
Mikhail Korobov b103664bf4
Address 2/3 of warnings from tests (#5561) 2022-07-19 17:39:26 +02:00
Adrián Chaves 42bca211d9 test_get_response_class_http: progress on documenting behavior changes 2022-06-30 21:28:02 +02:00
Adrián Chaves 0a77bfe2a0 get_response_class: use Response for compressed data 2022-06-30 20:39:34 +02:00
Adrián Chaves b7e88f4357 test_get_response_class_http: cover Apache bug scenarios 2022-06-30 19:12:13 +02:00
Adrián Chaves 983aa66687 Test ResponseTypes MIME types with the new implementation 2022-06-30 16:49:13 +02:00
Adrián Chaves fd2317bd6d Implement scrapy.utils.response.get_response_class 2022-06-30 15:15:52 +02:00