RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,321 Commits 49 Branches 132 Tags 149 MiB
Commit Graph

67 Commits

Author SHA1 Message Date
Adrian 5a65bdcc18
Improve spiders coverage (#7768) 
* Skip the deprecated scrapy.mail in coverage data

* Improve CrawlSpider coverage

* Improve XMLFeedSpider coverage

* Improve SitemapSpider coverage

* Solve mypy issues

* Align new spider tests with the shared test helper structure
 2026-07-28 15:24:04 +05:00
Andrey Rakhmatullin 4ee3676464
Test restructuring (#7736) 2026-07-21 12:44:56 +02:00
Andrey Rakhmatullin 3a36955261
Assorted test fixes (#7616) 2026-06-15 08:39:58 +02:00
Andrey Rakhmatullin d59f9b644a
Replace _SettingsKeyT with str. (#7586) 2026-06-10 11:12:53 +02:00
Andrey Rakhmatullin 55c17a8985
Disable mypy allow_any_generics. (#7532) 2026-05-18 15:54:06 +02:00
Andrey Rakhmatullin 939db88b04
Disable mypy allow_untyped_calls. (#7331) 2026-03-24 10:31:17 +01:00
Andrey Rakhmatullin 7010985e4f
Bump ruff, fix some of rules (#7277) 2026-02-23 11:48:38 +01:00
Andrey Rakhmatullin ccfa052fa1
Enable in-process HTTP tests without a reactor. (#7254) 2026-02-13 19:08:06 +01:00
Andrey Rakhmatullin fc30c47f38
Add HttpxDownloader. (#7239) 2026-02-12 09:30:44 +01:00
Andrey Rakhmatullin eaae59fbef
Fix invalid XML samples in tests that break with lxml 6. (#6919) 2025-06-27 13:05:02 +02:00
Adrián Chaves 036f3e5627
Support asynchronous start requests (#6729) 2025-05-07 19:04:03 +02:00
Andrey Rakhmatullin cec0aeca58
Bump ruff, switch from black to ruff-format (#6631) 2025-01-27 11:07:09 +01:00
Andrey Rakhmatullin 93644f2c30 Add flake8-pie rules to ruff. 2024-12-12 20:33:35 +05:00
Andrey Rakhmatullin 897e124a27 Add flake8-return rules to ruff. 2024-12-12 20:32:25 +05:00
Andrey Rakhmatullin b042ad255d
Address some previously ignored pylint messages. (#6531) 2024-11-11 11:49:52 +01:00
Andrey Rakhmatullin c9095ef927 Remove --keep-runtime-typing from pyupgrade. 2024-10-17 21:26:02 +05:00
Georgiy Zatserklianyi 6ce0342beb
Allow yielding items from start_requests (#6417) 
Co-authored-by: Georgiy Zatserklianyi <george.zatseklyany@gmail.com>
Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-08-26 20:53:06 +02:00
guillermo-bondonno 5f0fad16f5 improve test_closespider_pagecount_no_item and corresponding test spider 2024-07-11 13:26:22 -03:00
guillermo-bondonno a40d5281cf improve test_closespider_pagecount_no_item and MaxItemsSpider 2024-07-11 11:14:30 -03:00
guillermo-bondonno d6352f9f66 refactor changes on closespider.py and improve test 2024-07-10 11:03:01 -03:00
Bagaudin Magomedov b88f22c6c5
Add SpiderLoggerAdapter, change Spider.logger to return SpiderLoggerAdapter (#6324) 2024-05-13 14:33:24 +02:00
Andrey Rakhmatullin 282767f23b Bump black. 2024-02-28 14:30:38 +05:00
Rotzbua 0d445a3224
refactor(yield): use `yield from` syntax 2024-01-05 12:30:10 +01:00
Kevin Lloyd Bernal 368ab29ffc improve tests by having SlowSpider 2023-07-18 22:29:15 +08:00
Adrián Chaves b07d3f85a3 Upgrade CI tools 2023-02-02 06:37:40 +01:00
Emmanuel Rondan e211ec0aa2 adding black formatter to all the code 2022-11-29 11:30:46 -03:00
Eugenio Lacuesta b18560315b
Add tests 2022-10-30 18:28:16 -03:00
Eugenio Lacuesta da8f915091
Adapt for asyng generator callbacks 2022-10-02 17:37:10 -03:00
Eugenio Lacuesta 80194f1c03
CrawlSpider: add support for async def callbacks 2022-10-02 15:22:06 -03:00
Andrey Rakhmatullin 66d8a71920 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-10-11 20:24:22 +05:00
Georgiy Zatserklianyi d3f1bf79e8
Use f-strings where appropriate (#5246) 2021-10-07 16:27:20 +02:00
Andrey Rakhmatullin ffc6f525ce Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-04-14 17:34:58 +05:00
Eugenio Lacuesta 9e7cbc05ae
Fix type for urlencode's doseq argument 2021-04-01 15:22:51 -03:00
Andrey Rakhmatullin 7b515a0485 Merge remote-tracking branch 'origin/master' into asyncio-parse-asyncgen-proper-rebased 2021-03-25 22:13:49 +05:00
Eugenio Lacuesta 0c16088230
headers_received signal (#4897) 2021-03-11 11:52:35 -03:00
Andrey Rakhmatullin 2152a2a508 Add main infrastructure for async callbacks. 2021-02-05 16:21:56 +05:00
Ammar Najjar ffdf6fe100
use f-strings for the newly merged code from master 
Issue: #4324
 2020-08-29 07:21:48 +02:00
Ammar Najjar defeaacbc2 Merge branch 'master' into 4307-use-f-strings 
Conflicts resolved:
- delete tests/py36/_test_crawl.py
 2020-08-29 07:12:44 +02:00
Andrey Rakhmatullin de640f41ec Merge back tests/py36/_test_crawl.py. 2020-08-28 18:27:36 +05:00
Eugenio Lacuesta a8114d3731
Typing: annotate a few Spider attributes 2020-08-26 09:00:36 -03:00
Ammar Najjar 58ca8bbf6d Use f-strings (#4307) 2020-08-26 07:11:05 +02:00
Marc Hernández 336f19f5cc
Change super syntax (#4707) 2020-08-04 20:42:01 +02:00
Eugenio Lacuesta 2be2bdd2df
Merge remote-tracking branch 'upstream/master' into spider.parse 2020-07-08 14:00:54 -03:00
Eugenio Lacuesta 91e505edfe
Return single element from coroutine callback (#4609) 2020-06-02 10:32:14 +02:00
Eugenio Lacuesta fbe915d9cb
StopDownload: check partial response contents 2020-05-20 20:20:30 -03:00
Eugenio Lacuesta 276721a5dc
Stop response download 2020-05-20 14:46:26 -03:00
Eugenio Lacuesta fe0c582ee0
Flake8: remove E127 in tests (continuation line over-indented for visual indent) 2020-05-06 09:49:10 -03:00
Adrián Chaves aae4935605
Merge branch 'master' into spider.parse 2020-02-07 21:17:56 +01:00
Andrey Rakhmatullin 31f6c7112f Add a test for an async callbacks that returns requests. 2020-02-07 17:14:52 +05:00
Adrián Chaves 24bb9fd5f7
Merge branch 'master' into spider.parse 2020-02-06 21:39:09 +01:00