RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,174 Commits 49 Branches 132 Tags 130 MiB
Commit Graph

11174 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 7fc84d372a
Cleanup/clarifications of proxy support and improvements for proxy tests (#7496) 
* Run mitmproxy-based tests for every handler and improve them.

* Fixes for H2DownloadHandler.

* Revert "Fixes for H2DownloadHandler."

This reverts commit bcdbd097cd.

* Add TestHttpsProxy for HTTP11DownloadHandler.

* Use the configured context factory in ScrapyProxyAgent.

* Reword.

* Reword.

* pragma: no cover for H2DownloadHandler proxy code.

* Raise an exception for HTTPS proxies instead of using HTTP.

* Remove non-working proxy support and explicitly forbid HTTP.

* pragma: no cover

* Add a docs note about HTTPS proxies.

* Rename ScrapyProxyAgent.
 2026-05-12 17:45:54 +05:00
Adrian 7f15ca92fc
sphinx-scrapy: 0.8.5 → 0.8.6 (#7507) 2026-05-12 12:49:23 +05:00
Andrey Rakhmatullin 5223dbe3fd
Pin pyOpenSSL to the versions allowing mutable contexts. (#7494) 2026-05-06 10:42:24 +02:00
Andrey Rakhmatullin fc14a0ce59
Enable mypy warn_return_any. (#7492) 2026-05-04 22:24:47 +02:00
Andrey Rakhmatullin 33452f3aeb
Silence deprecation warnings in TestFormRequest. (#7491) 2026-05-04 22:20:21 +02:00
Albert Eduardovich N. 9776a72a6a
chore: simplify some code, get rid of nested fns where it's makes sence + pylint (#7401) 
* chore: simplify some code, get rid of nested fns where it's makes sence

reasoning: https://wemake-python-styleguide.readthedocs.io/en/latest/pages/usage/violations/best_practices.html#wemake_python_styleguide.violations.best_practices.NestedFunctionViolation

* chore: address some pylint rules

* remove `pylint: disable=deprecated-class` from `reactor.py`

* fix typo

* fix another typo :)

* revert backward incompatible arg renames

---------

Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2026-05-04 23:32:46 +05:00
Adrián Chaves 8d69a7c865
Deprecate FormRequest in favor of form2request (#6438) 
* WIP

* Add docs

* Remove FormRequest from tests

* Silence mypy issue

* Address docs-tests issues

---------

Co-authored-by: Adrian Chaves <adrian@zyte.com>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2026-05-04 22:38:29 +05:00
Andrey Rakhmatullin f3868e11fb
Drop start_requests() support. (#7490) 2026-05-04 17:34:24 +05:00
Andrey Rakhmatullin 9ca206da64
Merge pull request #7487 from wRAR/deprecation-removals 
Remove most of the 2.13.0 deprecations
 2026-05-04 12:33:21 +05:00
Andrey Rakhmatullin d05b241f64
Fix a code block in leaks.rst. (#7489) 2026-05-04 08:59:27 +02:00
Adrian 55c61646da
Merge branch 'master' into deprecation-removals 2026-05-04 08:52:12 +02:00
Andrey Rakhmatullin c62a81d7dd
Drop HTTP/1.0 code. (#7486) 2026-05-04 08:50:41 +02:00
Sanjay M a9324fbf76
Fix small documentation wording issues (#7480) 2026-05-04 08:43:21 +02:00
Andrey Rakhmatullin 9f02f6c16a Drop spider args of Scraper methods. 2026-05-03 14:12:18 +05:00
Andrey Rakhmatullin 6cb2fe1fc3 Drop scrapy_components_versions(). 2026-05-03 14:12:18 +05:00
Andrey Rakhmatullin f24bc749ea Require start_queue_cls. 2026-05-03 14:12:18 +05:00
Andrey Rakhmatullin 5fc40f07f3 Drop w3lib.url reexports. 2026-05-03 14:12:18 +05:00
Andrey Rakhmatullin af7dcabebb Drop AjaxCrawlMiddleware and escape_ajax(). 2026-05-03 14:12:18 +05:00
Andrey Rakhmatullin 8ecfd20fcd Remove InitSpider. 2026-05-03 14:12:16 +05:00
Andrey Rakhmatullin 14f49ab63c Remove deprecated test utils. 2026-05-03 14:11:55 +05:00
Adrian Chaves d8ac397be1 Number big not boom? 2026-04-29 23:07:44 +02:00
Adrian Chaves c1e3997ec0 Try simply increasing the timeout 2026-04-29 22:55:03 +02:00
Adrian Chaves 1e6aca0bff 🤞 2026-04-29 22:22:29 +02:00
Adrian Chaves c5250f1d80 Remove sleep that may have been making a test brittle 2026-04-29 21:43:48 +02:00
Adrian Chaves 1b917950f5 Remove unneeded comment 2026-04-29 16:40:41 +02:00
Adrian Chaves f5080c1e4c Minor refactoring 2026-04-29 16:38:32 +02:00
Adrian Chaves 73f7dbaa50 Merge remote-tracking branch 'origin/master' into faster-shutdown-2 2026-04-29 16:31:53 +02:00
Adrian Chaves dc4db5318f Address typing issue 2026-04-29 16:23:23 +02:00
Adrian Chaves de950c9fa1 Improve coverage 2026-04-29 16:06:59 +02:00
Adrian Chaves 2a0825d75a Address typing and linting issues 2026-04-29 15:18:05 +02:00
Adrian Chaves dbcd87c09c Improve test coverage 2026-04-29 14:52:17 +02:00
Adrian b9c2240040
sphinx-scrapy: 0.8.4 → 0.8.5 (#7472) 2026-04-29 14:17:26 +02:00
Adrian Chaves fe8f527b1a Fix test for the default reactor 2026-04-29 13:39:39 +02:00
Adrian Chaves 17c406d91b Improve coverage 2026-04-29 13:12:41 +02:00
Adrian Chaves 41fa1e94a0 Add CI timeouts while at it 2026-04-29 12:45:52 +02:00
Adrian Chaves 03f89c99ab Remove unused code and privatize to protect against import of imports from public modules 2026-04-29 12:45:16 +02:00
Adrian Chaves 44f0fd0d37 Solve issue making CI hang? 2026-04-29 12:25:54 +02:00
Adrian Chaves 2b397afe52 Merge remote-tracking branch 'origin/master' into faster-shutdown-2 2026-04-29 11:52:12 +02:00
Albert Eduardovich N. dd36fb7859
Improve handling of time deltas (#7377) 2026-04-29 09:53:40 +02:00
Andrey Rakhmatullin 30a54b72f0
Merge pull request #7470 from scrapy/2.15 
Merge 2.15.2 into the main branch
 2026-04-28 21:19:29 +05:00
Adrian Chaves 3d5ca9f433 Bump version: 2.15.1 → 2.15.2 2026-04-28 15:28:51 +02:00
Adrian 3a88cd0e2b
2.15.2 (#7468) 
* sphinx-scrapy: 0.8.3 → 0.8.4 (#7467)

* Release notes for 2.15.2
 2026-04-28 15:26:42 +02:00
Andrey Rakhmatullin 988afe1454
Clarify dataloss/http2 tests. (#7466) 2026-04-28 14:35:41 +02:00
Adrian 528745b059
sphinx-scrapy: 0.8.3 → 0.8.4 (#7467) 2026-04-28 12:17:12 +02:00
Andrey Rakhmatullin 068aa69b35
Fixes for Python 3.14. (#7460) 2026-04-27 18:47:52 +05:00
Adrian Chaves 32a69f9547 Merge remote-tracking branch 'origin/master' into faster-shutdown-2 2026-04-27 15:30:56 +02:00
Adrian Chaves 15af004382 Address typing and linting issues 2026-04-27 15:28:27 +02:00
Andrey Rakhmatullin fc4c57e795
Fix the substitution pattern in open_in_browser(). (#7459) 
* Fix the substitution pattern in open_in_browser().

* Formatting.
 2026-04-24 13:54:37 +05:00
Andrey Rakhmatullin 4e25686b20 Merge branch '2.15' 2026-04-23 21:08:38 +05:00
Andrey Rakhmatullin f3c5a6e75f Bump version: 2.15.0 → 2.15.1 2026-04-23 19:57:44 +05:00