RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,018 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

355 Commits

Author SHA1 Message Date
Adrian Chaves 4e3df249f2 Merge branch '2.14' 2026-03-12 16:02:37 +01:00
Adrian Chaves 498b4fc1a4 Bump version: 2.14.1 → 2.14.2 2026-03-12 15:45:16 +01:00
Adrian Chaves 378bb68039 Proofread the release notes 2026-03-12 15:44:54 +01:00
Adrian Chaves 945b787a26 Merge remote-tracking branch 'cwxj-rr6w-m6w7/fix-referer-policy-handling' into 2.14 2026-03-12 13:37:24 +01:00
Andrey Rakhmatullin fc30c47f38
Add HttpxDownloader. (#7239) 2026-02-12 09:30:44 +01:00
Adrian Chaves 8974580e43 Reword the release note entry to consider the 301 redirect fix a security bug fix 2026-02-03 20:38:11 +01:00
Adrian Chaves ba3d7bc7a8 Remove the non-standard 307/308 handling, and align other aspects with the standard 2026-01-21 18:46:05 +01:00
Adrián Chaves 1b9c8b55da
not → now 2026-01-20 12:13:44 +01:00
Adrian Chaves 2651d48f20 Cover the 301 behavior change in the release notes 2026-01-19 19:25:00 +01:00
Adrian Chaves 99d5d58e20 Add a release notes entry 2026-01-19 19:19:02 +01:00
Adrian Chaves b6e5c58ae7 Fix arbitrary callable execution from the Referrer-Policy response header 2026-01-19 16:44:03 +01:00
Eugenio Lacuesta 0b9d8da09d
Bump version: 2.14.0 → 2.14.1 2026-01-12 16:25:43 -03:00
Eugenio Lacuesta c9fbf6c599
Release notes for 2.14.1 (#7217) 
* Release notes for 2.14.1

* Update docs/news.rst

Co-authored-by: Adrián Chaves <adrian@chaves.gal>

* Update docs/news.rst

Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>

* Update docs/news.rst

Co-authored-by: Adrián Chaves <adrian@chaves.gal>

* Update message about github action change

---------

Co-authored-by: Adrián Chaves <adrian@chaves.gal>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2026-01-12 16:23:51 -03:00
Andrey Rakhmatullin 3ac1192f35 Bump version: 2.13.4 → 2.14.0 2026-01-05 15:06:13 +05:00
Andrey Rakhmatullin 7bef98b4f1
Release notes for 2.14.0 (#7035) 
* Initial release notes for 2.14.0, up to ed63fa9.

* Cover 2.14 in the release notes up to 393d715.

* Cover 2.14 in the release notes up to eb49647.

* Cover 2.14 in the release notes up to 426aafd.

* Cover 2.14 in the release notes up to 1e8de24.

* Cover 2.14 in the release notes up to 5a7e132.

* Bump sphinx-lint.

* Finalize the 2.14.0 release notes.

* Drop more of the old versionadded directives.

* Address feedback.
 2026-01-05 14:51:05 +05:00
Andrey Rakhmatullin d1bd8eb49f
Async API for download handlers. (#7164) 2025-12-31 19:03:16 +01:00
Andrey Rakhmatullin c49ae2115a
Refactor MediaPipeline.process_item() to async def (#7177) 2025-12-15 10:02:04 +01:00
Andrey Rakhmatullin 4cb0144b39 Merge tag '2.13.4' 
Bump version: 2.13.3 → 2.13.4
 2025-11-17 16:47:54 +05:00
Andrey Rakhmatullin 2f62ab532d Bump version: 2.13.3 → 2.13.4 2025-11-17 16:30:37 +05:00
Andrey Rakhmatullin 31a9c03c24
Release notes for 2.13.4. (#7144) 
* Release notes for 2.13.4.

* Formatting.
 2025-11-17 16:30:04 +05:00
Andrey Rakhmatullin 1c6ba00fd0
Move LogCounterHandler into a new extension (#7046) 
* Move LogCounterHandler into Crawler.crawl().

* Add the LogCount extension.

* Update test_spider_custom_settings_log_level().

* Typo.

* Update docs.
 2025-10-03 20:14:11 +05:00
Andrey Rakhmatullin ed63fa94d6
Deprecate spider args of middlewares and pipelines. (#7006) 
* Deprecate spider args of spider middleware methods.

* Deprecate the spider arg of pipeline process_item().

* Fix a typing issue.

* Deprecate the spider arg of pipeline {open,close}_spider().

* Cleanup.

* Update docs.

* Add pragma: no cover to some deprecated code.

* More tests.

* More tests.

* More tests.

* Simplify _process_parallel().
 2025-08-21 13:23:10 +05:00
Andrey Rakhmatullin edba1ad572
Refactor RobotsTxtMiddleware to use async def process_request(). (#6802) 2025-08-18 18:40:46 +02:00
Andrey Rakhmatullin d9ef0350d8 Merge remote-tracking branch 'origin/2.13' 2025-07-02 20:46:02 +05:00
Andrey Rakhmatullin 155a504f24 Bump version: 2.13.2 → 2.13.3 2025-07-02 20:39:46 +05:00
Andrey Rakhmatullin cf465bf644
Release notes for 2.13.3. (#6934) 
* Release notes for 2.13.3.

* Address feedback.
 2025-07-02 20:36:01 +05:00
Andrey Rakhmatullin 3843091c5f
Mark abstract base classes as such. (#6930) 
* Mark abstract base classes as such.

* Remove an unneeded method.

* Fix exporter test coverage.
 2025-06-30 15:14:34 +05:00
Andrey Rakhmatullin 03fe7a6424 Add a deprecation notice for the offsite spider middleware. 2025-06-29 02:35:48 +05:00
Andrey Rakhmatullin 2c1c10e923
Remove deprecated offsite spider middleware. (#6926) 
* Remove deprecated offsite spider middleware.

* Add a deprecation notice for the offsite spider middleware.
 2025-06-29 02:35:20 +05:00
Andrey Rakhmatullin 95172659af
Add sphinx-lint. (#6920) 2025-06-28 01:37:20 +02:00
Andrey Rakhmatullin daa1a7d0b6
Remove the chdir fixture, re-enable fancy pytest asserts (#6888) 
* Remove the chdir fixture.

* Re-enable fancy pytest asserts.

* Remove doc files from pytest ignores.

* Restore docs/_ext in test collection ignores.

* Skip a doctest that fails on Windows.

* Fix tests that were writing to the current dir.
 2025-06-14 14:01:20 +05:00
Andrey Rakhmatullin 0390176ecd Merge branch '2.13' 2025-06-09 15:54:35 +05:00
Andrey Rakhmatullin c6740604a4 Bump version: 2.13.1 → 2.13.2 2025-06-09 15:21:06 +05:00
Andrey Rakhmatullin 7400868ad5
Release notes for 2.13.2. (#6868) 
* Release notes for 2.13.2.

* Update release notes.
 2025-06-09 15:19:02 +05:00
Andrey Rakhmatullin 405d9bc8a2
More docs for the is_asyncio_reactor_installed() behavior change. (#6866) 2025-06-06 12:59:49 +02:00
Andrey Rakhmatullin 3aa5e75787
Use AsyncCrawlerProcess in commands. (#6845) 
* Use AsyncCrawlerProcess in commands.

* Ignore coverage of abstract methods.

* Address feedback.
 2025-06-03 17:47:52 +05:00
Adrián Chaves f98ffc71d2
2.13.1 (#6832) 2025-05-28 10:58:34 +02:00
Adrián Chaves 7fe7f1734a Bump version: 2.13.0 → 2.13.1 2025-05-28 10:12:46 +02:00
Adrián Chaves 0cdb971f63 Complete the release notes 2025-05-28 10:06:04 +02:00
Adrián Chaves 05529f3017 Release notes for Scrapy 2.13.1 2025-05-27 08:44:18 +02:00
Andrey Rakhmatullin 82a3245158 Bump version: 2.12.0 → 2.13.0 2025-05-08 14:43:34 +05:00
Andrey Rakhmatullin b07e6d4ea8 Merge remote-tracking branch 'origin/master' into relnotes-2.13.0 2025-05-08 13:17:47 +05:00
Andrey Rakhmatullin 5f6d1b464b Cover up to 373e501 in the release notes. 2025-05-08 10:55:40 +05:00
Adrián Chaves 036f3e5627
Support asynchronous start requests (#6729) 2025-05-07 19:04:03 +02:00
Andrey Rakhmatullin 4aba7e5f66 Mention the deprecation of TestSpider. 2025-04-25 20:16:26 +05:00
Andrey Rakhmatullin 095140f134 Cover the current master in the release notes (up to daf9db7). 2025-04-25 19:43:29 +05:00
Andrey Rakhmatullin b1f85b5a17 Release notes for 2.13.0, up to b4c2531021. 2025-04-24 20:03:36 +05:00
Andrey Rakhmatullin daf9db72b2
Base class for universal spider middlewares (#6693) 
* Initial BaseSpiderMiddleware.

* Rename the new methods.

* Remove the spider argument from new BaseSpiderMiddleware methods.

* Add docs for BaseSpiderMiddleware.

* Silence pylint.

* Add BaseSpiderMiddleware tests.

* Add a release note.
 2025-04-23 19:29:04 +05:00
Adrián Chaves bee74fb753 Remove trailing whitespace 2025-03-11 11:56:44 +01:00
Rotzbua ee4f527f47
fix(docs): pillow domain is shut down permanently 
See https://github.com/python-pillow/Pillow/issues/8585
 2025-01-19 14:58:02 +01:00