RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,909 Commits 49 Branches 132 Tags 129 MiB
Commit Graph

10909 Commits

Author SHA1 Message Date
Adrián Chaves 3a95feec75 Blind test #1 for Windows-specific test failure 2025-05-07 22:38:05 +02:00
Adrián Chaves b8e02884ce Fix indentation 2025-05-07 21:19:36 +02:00
Adrián Chaves 51c39308d8 Merge remote-tracking branch 'scrapy/master' into async-seeds 2025-05-07 21:16:13 +02:00
Adrián Chaves 036f3e5627
Support asynchronous start requests (#6729) 2025-05-07 19:04:03 +02:00
Adrián Chaves 373e501f78
Link to scrapy.org from the docs (#6780) 2025-05-07 16:11:22 +02:00
Andrey Rakhmatullin 4899d416e7
Add PyPy 3.11 to CI. (#6697) 2025-05-06 12:31:28 +02:00
Andrey Rakhmatullin 474f8312ff
Merge pull request #6776 from wRAR/deferred-apis 
Add a list of Deferred-only APIs.
 2025-05-05 21:28:50 +04:00
Andrey Rakhmatullin acb5f895cd
Update docs/topics/coroutines.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2025-05-05 22:28:36 +05:00
Adrián Chaves 523fc25c4d
Document default values set by startproject (#6775) 2025-05-05 18:51:15 +02:00
Andrey Rakhmatullin b93290f28a Add a list of Deferred-only APIs. 2025-05-05 19:38:04 +05:00
Andrey Rakhmatullin 509b572efc
Migrate the build system to hatchling. (#6771) 2025-05-05 09:51:53 +02:00
Andrey Rakhmatullin 2a1edbd473
Remove usages of TestCase._wait(). (#6773) 2025-05-05 09:44:17 +02:00
Andrey Rakhmatullin ff1ac75c9e
Fix shutdown tests. (#6772) 2025-05-05 09:37:38 +02:00
Andrey Rakhmatullin 5dfe7cd7b8
Improve tests for start items. (#6770) 2025-05-05 09:36:52 +02:00
Adrián Chaves 8f059d4095
Merge pull request #6766 from wRAR/errback-tests 
Improve test coverage of Scraper.
 2025-05-02 10:37:28 +02:00
Andrey Rakhmatullin da9078c4bb Add tests for raising CloseSpider in callbacks. 2025-05-01 23:12:39 +05:00
Andrey Rakhmatullin 23c206af35 Improve test coverage of Scraper. 2025-05-01 22:59:18 +05:00
Andrey Rakhmatullin 6deae473d9
Merge pull request #6764 from wRAR/spider-arg-scraper 
Remove the unnecessary spider argument from Scraper and related code.
 2025-04-30 18:33:35 +04:00
Andrey Rakhmatullin eced5ca2d3 Remove the unnecessary spider argument from Scraper and related code. 2025-04-30 00:57:44 +05:00
Andrey Rakhmatullin daf9db72b2
Base class for universal spider middlewares (#6693) 
* Initial BaseSpiderMiddleware.

* Rename the new methods.

* Remove the spider argument from new BaseSpiderMiddleware methods.

* Add docs for BaseSpiderMiddleware.

* Silence pylint.

* Add BaseSpiderMiddleware tests.

* Add a release note.
 2025-04-23 19:29:04 +05:00
Adrián Chaves c3a903ec8f Report the specific, conflicting spider middlewares 2025-04-09 12:17:35 +02:00
Adrián Chaves 62d3c5a864 Fix typing issue 2025-04-07 12:48:12 +02:00
Adrián Chaves 156308f7b6 Merge branch 'minimal-async-seeds' into async-seeds 2025-04-07 12:38:51 +02:00
Adrián Chaves ac5a5b7811 Address feedback 2025-04-07 12:37:31 +02:00
Adrián Chaves 76ccc50622 Add an example of higher-priority leaf requests 2025-04-01 13:16:14 +02:00
Adrián Chaves 9dc9999eaa Add docs about universal start request definition 2025-04-01 12:13:59 +02:00
Adrián Chaves 6c1852d313 Update the FAQ 2025-04-01 11:07:20 +02:00
Adrián Chaves 891adcd8e2 Mark a line intentionally without coverage 2025-04-01 10:32:37 +02:00
Adrián Chaves 4e63b76851 Remove debugging leftover 2025-03-31 17:19:13 +02:00
Adrián Chaves 28df50ddcc Minor edits around Optimization 2025-03-31 17:13:40 +02:00
Adrián Chaves a858ddfef9 Use the default scheduler for the front-load test 2025-03-31 16:46:52 +02:00
Adrián Chaves fdf242475e Test raising CloseSpider when the spider has been already closed 2025-03-31 16:12:42 +02:00
Adrián Chaves 5c9a931e8b Re-implement idle support 2025-03-31 15:18:38 +02:00
Adrián Chaves a04e85e08b Re-implement front-load support 2025-03-31 11:49:28 +02:00
Adrián Chaves 27ae12dc59 Fix method reference in the docs 2025-03-31 10:05:34 +02:00
Adrián Chaves 43e789f776 Use self 2025-03-31 10:02:31 +02:00
Adrián Chaves 9417de1df9 Review uses of seed 2025-03-31 09:53:52 +02:00
Adrián Chaves e984326777 Improve test reliability 2025-03-31 09:52:34 +02:00
Adrián Chaves 9fc4c449a7 Merge branch 'minimal-async-seeds' into async-seeds 2025-03-31 09:30:24 +02:00
Adrián Chaves 6cf308fbdd Fix comment about typing.Self Python requirement 2025-03-31 09:29:59 +02:00
Adrián Chaves a9ce836f98 Fix test expectations and improve reliability 2025-03-28 16:20:15 +01:00
Adrián Chaves 85e3dc13f1 Merge branch 'minimal-async-seeds' into async-seeds 2025-03-28 15:54:23 +01:00
Adrián Chaves 8e3d211ad7 Fix test_close_during_start_iteration on default-reactor 2025-03-27 16:37:19 +01:00
Adrián Chaves 7d159b2140 Privatize engine.start_request_processing() 2025-03-27 15:56:34 +01:00
Adrián Chaves 2a148d07b0 Increase coverage 2025-03-27 15:51:08 +01:00
Adrián Chaves 40adc6a356 Increase coverage 2025-03-27 15:39:37 +01:00
Adrián Chaves e360a4cc98 Increase coverage 2025-03-27 15:24:57 +01:00
Adrián Chaves 8057f06b39 Increase coverage 2025-03-27 15:16:02 +01:00
Adrián Chaves c67edb78b9 Deprecate calls to Spider.start_requests 2025-03-27 15:04:34 +01:00
Adrián Chaves 5e18a132da Clean up from self-review 2025-03-27 14:03:39 +01:00