RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,822 Commits 49 Branches 132 Tags 130 MiB
Commit Graph

2814 Commits

Author SHA1 Message Date
Andrey Rakhmatullin 155a504f24 Bump version: 2.13.2 → 2.13.3 2025-07-02 20:39:46 +05:00
Andrey Rakhmatullin cf465bf644
Release notes for 2.13.3. (#6934) 
* Release notes for 2.13.3.

* Address feedback.
 2025-07-02 20:36:01 +05:00
Andrey Rakhmatullin 03fe7a6424 Add a deprecation notice for the offsite spider middleware. 2025-06-29 02:35:48 +05:00
Adrián Chaves ba1cedee1b Document the new defaults of throttling settings (#6923) 2025-06-28 18:13:54 +05:00
Andrey Rakhmatullin 843ad1afb1 Replace Black with Ruff in contributing.rst. (#6903) 2025-06-23 22:25:56 +05:00
Andrey Rakhmatullin c6740604a4 Bump version: 2.13.1 → 2.13.2 2025-06-09 15:21:06 +05:00
Andrey Rakhmatullin 7400868ad5
Release notes for 2.13.2. (#6868) 
* Release notes for 2.13.2.

* Update release notes.
 2025-06-09 15:19:02 +05:00
Andrey Rakhmatullin 405d9bc8a2
More docs for the is_asyncio_reactor_installed() behavior change. (#6866) 2025-06-06 12:59:49 +02:00
Andrey Rakhmatullin d99234a33f
Install the reactor explicitly in CrawlerRunner examples. (#6865) 2025-06-06 11:54:21 +02:00
Adrián Chaves b8cd079014 Shorten the README and remove broken links to scrapy.org (#6833) 2025-06-06 12:44:02 +05:00
Adrián Chaves 105c0afb6e Feature the new logo in the README (#6831) 2025-06-06 12:43:56 +05:00
Adrián Chaves 7fe7f1734a Bump version: 2.13.0 → 2.13.1 2025-05-28 10:12:46 +02:00
Adrián Chaves 0cdb971f63 Complete the release notes 2025-05-28 10:06:04 +02:00
Mikhail Korobov c6643c08ee
Merge pull request #6828 from scrapy/lower-start-priority 
Scrapy 2.13.1: Prioritize other requests over start requests
 2025-05-28 11:29:57 +05:00
Andrey Rakhmatullin 06dec08125 Fix a wrong versionadded usage. (#6822) 2025-05-27 12:34:40 +05:00
Keval Sakhiya 43087fe1df Fix typo in documentation and code: 'needs_backoff' -> 'needs_backout' (#6815) 
Corrected the typo in the code and documentation where 'needs_backoff' was incorrectly used instead of 'needs_backout'.
 2025-05-27 12:34:33 +05:00
Adrián Chaves 05529f3017 Release notes for Scrapy 2.13.1 2025-05-27 08:44:18 +02:00
Andrey Rakhmatullin 82a3245158 Bump version: 2.12.0 → 2.13.0 2025-05-08 14:43:34 +05:00
Andrey Rakhmatullin 0ce693dfa9 Update VERSION strings. 2025-05-08 13:23:19 +05:00
Andrey Rakhmatullin b07e6d4ea8 Merge remote-tracking branch 'origin/master' into relnotes-2.13.0 2025-05-08 13:17:47 +05:00
Andrey Rakhmatullin e86b5051c2 Merge commit '373e501' into relnotes-2.13.0 2025-05-08 10:55:57 +05:00
Andrey Rakhmatullin 5f6d1b464b Cover up to 373e501 in the release notes. 2025-05-08 10:55:40 +05:00
Adrián Chaves 036f3e5627
Support asynchronous start requests (#6729) 2025-05-07 19:04:03 +02:00
Adrián Chaves 373e501f78
Link to scrapy.org from the docs (#6780) 2025-05-07 16:11:22 +02:00
Andrey Rakhmatullin 474f8312ff
Merge pull request #6776 from wRAR/deferred-apis 
Add a list of Deferred-only APIs.
 2025-05-05 21:28:50 +04:00
Andrey Rakhmatullin acb5f895cd
Update docs/topics/coroutines.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2025-05-05 22:28:36 +05:00
Adrián Chaves 523fc25c4d
Document default values set by startproject (#6775) 2025-05-05 18:51:15 +02:00
Andrey Rakhmatullin b93290f28a Add a list of Deferred-only APIs. 2025-05-05 19:38:04 +05:00
Andrey Rakhmatullin 4aba7e5f66 Mention the deprecation of TestSpider. 2025-04-25 20:16:26 +05:00
Andrey Rakhmatullin 095140f134 Cover the current master in the release notes (up to daf9db7). 2025-04-25 19:43:29 +05:00
Andrey Rakhmatullin b1f85b5a17 Release notes for 2.13.0, up to b4c2531021. 2025-04-24 20:03:36 +05:00
Andrey Rakhmatullin daf9db72b2
Base class for universal spider middlewares (#6693) 
* Initial BaseSpiderMiddleware.

* Rename the new methods.

* Remove the spider argument from new BaseSpiderMiddleware methods.

* Add docs for BaseSpiderMiddleware.

* Silence pylint.

* Add BaseSpiderMiddleware tests.

* Add a release note.
 2025-04-23 19:29:04 +05:00
Mehraz Hossain Rumman 3ca882fba8
Syntax Error Fixed (#6738) 
* Syntax error fix issue #6731

* test case added

* extra logic removed

* mock spider fixture

* Update scrapy/utils/misc.py

Co-authored-by: Adrián Chaves <adrian@chaves.gal>

* settings.rst updated

* settings.rst updated

* settings.rst updated

---------

Co-authored-by: Adrián Chaves <adrian@chaves.gal>
 2025-03-20 17:02:10 +05:00
Andrey Rakhmatullin 9057bf4e1e
More docs about Deferred<->Future interoperability. (#6734) 2025-03-15 11:47:16 +01:00
Andrey Rakhmatullin fc566a7ff9
Don't install the reactor in is_asyncio_reactor_installed(). (#6732) 2025-03-14 20:46:17 +01:00
Andrey Rakhmatullin d0dabbc097
Enable AsyncioSelectorReactor by default. (#6713) 
* Enable AsyncioSelectorReactor by default.

* Improve get_crawler(), switch more tests to it.

* Fix the remaining default-reactor test failures.

* Address documentation feedback.

* Make pinned envs more consistent.
 2025-03-12 01:18:30 +05:00
Adrián Chaves ba28d96d3e
Centralize from_crawler docs (and somewhat related changes) (#6723) 2025-03-11 14:53:04 +01:00
Adrián Chaves 5a0690c89d
Remove or post-pone the use of itemadapter.is_item, as a potentially expensive call (#6719) 2025-03-11 14:52:48 +01:00
Adrián Chaves 9b7db1a068
Move some reference docs of Request to the code (#6721) 2025-03-11 12:43:50 +01:00
Adrián Chaves bee74fb753 Remove trailing whitespace 2025-03-11 11:56:44 +01:00
Andrey Rakhmatullin d2e5486d5a Remove the Splash recommendation. 2025-03-07 13:20:42 +05:00
Adrián Chaves 0c9200094e
Extend BaseSettings with utils for add-ons (#6614) 2025-03-05 10:31:59 +01:00
Andrey Rakhmatullin c200458f24
Add more docs for updating sync spider middlewares. (#6688) 2025-02-25 00:16:23 +01:00
anubhav f041f26a6f
Support dark mode in the documentation (#6653) 2025-02-06 18:07:07 +01:00
Laerte Pereira 2eb3c75c69 Remove AjaxCrawlMiddleware mention from built-in downloader middleware 2025-02-05 13:16:51 -03:00
Andrey Rakhmatullin 76a8badd24 Add a deprecation notice to the AjaxCrawlMiddleware docs. 2025-02-03 14:55:10 +05:00
Andrey Rakhmatullin 393ff96e45 Deprecate AjaxCrawlMiddleware. 2025-02-03 14:08:05 +05:00
Rotzbua 7e61ff3524
Upgrade Sphinx (#6624) 2025-01-22 18:09:42 +01:00
guillermo-bondonno 9bc0029d27
Allow updating pre-crawler settings from add-ons (#6568) 2025-01-22 12:07:44 +01:00
Rotzbua e0c828b7f6
chore(docs): refactor config (#6623) 2025-01-20 12:18:30 +01:00