0b9d8da09d
Bump version: 2.14.0 → 2.14.1
2026-01-12 16:25:43 -03:00
c9fbf6c599
Release notes for 2.14.1 ( #7217 )
...
* Release notes for 2.14.1
* Update docs/news.rst
Co-authored-by: Adrián Chaves <adrian@chaves.gal>
* Update docs/news.rst
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
* Update docs/news.rst
Co-authored-by: Adrián Chaves <adrian@chaves.gal>
* Update message about github action change
---------
Co-authored-by: Adrián Chaves <adrian@chaves.gal>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
2026-01-12 16:23:51 -03:00
e30ba7d4ca
( #7180 ) Update codecov test results action to v5 on all CI workflows. ( #7215 )
2026-01-12 08:31:53 +01:00
0f07b2e38c
Pass the spider arg to custom stat collectors {open,close}_spider(). ( #7213 )
2026-01-09 16:57:53 +01:00
1af283387f
Deprecate maybeDeferred_coro(). ( #7212 )
2026-01-09 01:09:39 +05:00
3ac1192f35
Bump version: 2.13.4 → 2.14.0
2026-01-05 15:06:13 +05:00
7bef98b4f1
Release notes for 2.14.0 ( #7035 )
...
* Initial release notes for 2.14.0, up to ed63fa9393d715eb49647426aafd1e8de245a7e132
2026-01-05 14:51:05 +05:00
d1bd8eb49f
Async API for download handlers. ( #7164 )
2025-12-31 19:03:16 +01:00
a2463325db
Allow Crawler.crawl_async() without asyncio. ( #7202 )
2025-12-31 18:33:59 +01:00
9381ad893d
Remove outdated version marker directives (< 2.8.0) from documentation ( #7198 )
2025-12-22 10:00:39 +01:00
180ca39b23
Deprecate returning Deferreds from pipeline methods ( #7179 )
...
* Add tests for exceptions in pipelines.
* Deprecate returning Deferreds from pipeline process_item().
* Deprecate returning Deferreds from pipeline {open,close}_spider().
* Update the custom pipeline docs.
2025-12-16 13:35:07 +05:00
5a7e132486
Restore and deprecate the spider arg of Downloader._get_slot(). ( #7178 )
2025-12-15 11:42:24 +01:00
c49ae2115a
Refactor MediaPipeline.process_item() to async def ( #7177 )
2025-12-15 10:02:04 +01:00
588f3d4f65
Remove docs mentioning deprecated spider attrs. ( #7176 )
2025-12-15 09:51:56 +01:00
d8583a89c7
Change default priority queue to DownloaderAwarePriorityQueue ( #6940 )
...
* Change default priority queue to DownloaderAwarePriorityQueue
* Fix documentation building
* Simplify test_start_already_running_exception changes.
* Modernize the test.
* Fix TestEngineCloseSpider.
* Fix typing.
* Remove special slot=None handling.
---------
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
2025-12-11 11:25:17 +01:00
1a3e343dc4
Update conftest.py to refactor autouse pytest.skip() fixtures into pytest_runtest_setup() ( #7173 )
2025-12-11 09:48:25 +01:00
6ba6b032ad
Fix handling meta["download_slot"] == None. ( #7172 )
2025-12-11 09:46:51 +01:00
9bfa58e36c
Deprecate send_catch_log_deferred(). ( #7161 )
...
* Add a test for not having pending tasks.
* Refactor TestFeedExporterSignals.
* Refactor FeedExporter.close_spider().
* More engine start/stop robustness.
* Refactor send_catch_log_async(), deprecate send_catch_log_deferred().
* Add pragma: no cover.
* Warn on signal handlers returning a Deferred.
* Make _pending_close_coros an instance attribute.
* Remove an unused function.
* Remove the outdated TODO.
2025-12-10 14:42:49 +05:00
5105f55a98
Add `DownloaderMiddlewareManager.download_async()`, deprecate returning Deferreds from downloader mws ( #7069 )
...
* Restore coverage in MiddlewareManager spider arg handling.
* Move MiddlewareManager._warn_spider_arg() tests.
* Add DownloaderMiddlewareManager.download_async().
* Make download_func async.
* Inline MiddlewareManager._warn_spider_arg().
* Deprecate returning deferreds from downloader middlewares.
* Update the downloader mw doc about async def.
* Use global_object_name() instead of repr().
* Update docs/topics/downloader-middleware.rst
Co-authored-by: Adrián Chaves <adrian@chaves.io>
* Update downloader-middleware.rst
---------
Co-authored-by: Adrián Chaves <adrian@chaves.io>
2025-12-09 20:07:36 +05:00
7ed20ee7f3
Limit the queue size in _parallel_asyncio(). ( #7159 )
2025-12-09 15:24:40 +01:00
483e059d59
Deprecate returning Deferreds from spider callbacks and errbacks. ( #7151 )
2025-12-09 15:03:32 +01:00
11073c8680
Deprecate spider attributes that can be replaced by settings, round 2 ( #7039 )
...
* Move duplicate code to utils
* move new function to end
* draft
* update docs
* Update tests
* Update test name
* rename test
* rollback signature
* leftover
* sort
* Rollback some changes
* Rollback download_delay warning
* Rollback
* fix checks
* Remove unused imports
* Add pragma: no cover.
---------
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
2025-12-01 01:26:17 +05:00
1e8de24380
Merge pull request #7145 from Icorebleidd/master
...
Fix duplicate assertion in test_utils_deprecate
2025-11-28 01:16:02 +05:00
0adc561348
Add a list of remaining Deferred APIs. ( #7146 )
2025-11-27 20:13:04 +05:00
813fd9f1ac
Restore brotlicffi support ( #7160 )
...
* restore brotlicffi dependency
* pre-commit
2025-11-25 12:06:55 +05:00
4cb0144b39
Merge tag '2.13.4'
...
Bump version: 2.13.3 → 2.13.4
2025-11-17 16:47:54 +05:00
2f62ab532d
Bump version: 2.13.3 → 2.13.4
2025-11-17 16:30:37 +05:00
31a9c03c24
Release notes for 2.13.4. ( #7144 )
...
* Release notes for 2.13.4.
* Formatting.
2025-11-17 16:30:04 +05:00
c44b8df6c7
Cherry-pick: Mitigate brotli and deflate decompression bombs DoS ( #7134 )
...
Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
2025-11-17 16:25:00 +05:00
14737e91ed
Mitigate brotli and deflate decompression bombs DoS ( #7134 )
...
Co-authored-by: Eugenio Lacuesta <eugenio.lacuesta@gmail.com>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
2025-11-17 14:21:50 +05:00
d091256c58
Remove the deprecated sphinx-hoverxref ( #6922 )
2025-11-14 15:39:21 +05:00
c83ca70db3
Don't force the unavailable sysmon coverage core. ( #7137 )
2025-11-13 23:26:08 +05:00
85e4e6c42b
Pin Twisted to <= 25.5.0 due to internal API changes.
2025-11-13 23:25:51 +05:00
426aafddca
Don't force the unavailable sysmon coverage core. ( #7137 )
2025-11-13 23:22:52 +05:00
db37040a09
Merge pull request #7142 from wRAR/pin-twisted
...
Pin Twisted to <= 25.5.0 due to internal API changes.
2025-11-13 22:48:33 +05:00
ba30e8b82c
Pin Twisted to <= 25.5.0 due to internal API changes.
2025-11-10 00:58:27 +05:00
8c5fa6e6ae
Line buffering ( #7094 )
2025-10-27 15:42:43 +01:00
804ae167df
Update tool versions ( #7127 )
2025-10-27 14:11:31 +01:00
61b4befc60
Remove code deprecated in 2.12.0 ( #7126 )
...
* Remove smaller things.
* Remove from_settings().
* Remove old MediaPipeline init code.
* Cleanup.
* Typo.
2025-10-27 17:14:38 +05:00
d414d393d4
Remove Python 3.9 support ( #7121 )
2025-10-27 12:37:49 +01:00
e48a1bdde3
Speed up and refactor command tests ( #7118 )
2025-10-27 12:28:16 +01:00
9cce6c30db
Undeprecate download_delay. ( #7117 )
2025-10-24 17:44:43 +05:00
5afc9b0221
Automatically transpose images using EXIF data ( #6975 )
...
* Automatically transpose images using EXIF data
* Added test
---------
Co-authored-by: Andrew Murray <radarhere@users.noreply.github.com>
2025-10-24 16:08:23 +05:00
eb496470f1
Add missing doc deps for new examples ( #7116 )
2025-10-24 11:58:18 +02:00
a5bbeb2586
Expanded documentation for issue #6676 ( #6957 )
2025-10-24 10:30:33 +02:00
2d073a9c0d
Fix typo in logging documentation ( #7109 )
2025-10-20 15:52:46 +02:00
e98c1644ce
Add unit tests that check download handlers behave like basic http clients ( #6946 ) ( #7059 )
2025-10-20 13:08:26 +02:00
b49aa2fb0c
Deprecate scrapy.core.downloader.handlers.http. ( #7079 )
2025-10-16 13:15:07 +02:00
7b215c6578
UTF-8 BOM at the beginning of the file ignored ( #7095 )
2025-10-15 18:15:07 +04:00
4865a500b6
resolves #6876 ( #6977 )
2025-10-09 22:14:34 +02:00
Eugenio Lacuesta
Eugenio Lacuesta
Santiago Beltrán Ruiz
Andrey Rakhmatullin
Adham 3llam
Thalison Fernandes
DenisC
Laerte Pereira
Leonardo Tozzo
Rui Xi
Adrián Chaves
Mikhail Korobov
Vasiliy Kiryanov
Andrew Murray
Michael
sstegmanbsd
Wes Galbraith
Vasiliy Kiryanov
adglee