RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,656 Commits 42 Branches 132 Tags 161 MiB
Commit Graph

198 Commits

Author SHA1 Message Date
Andrey Rakhmatullin c87354cd46 Enable SIM Ruff rules. 2025-01-02 01:03:36 +05:00
Andrey Rakhmatullin d47f142d0f Add flake8-type-checking rules to ruff. 2024-12-10 01:28:20 +05:00
Andrey Rakhmatullin b042ad255d
Address some previously ignored pylint messages. (#6531) 2024-11-11 11:49:52 +01:00
Andrey Rakhmatullin c9095ef927 Remove --keep-runtime-typing from pyupgrade. 2024-10-17 21:26:02 +05:00
Andrey Rakhmatullin f65e64a724
Misc typing improvements. (#6494) 2024-10-16 18:38:43 +02:00
Vsevolod Breus 5391663072
Drop Python 3.8 Support (#6472) 2024-10-16 10:03:16 +02:00
LucasSD f260f819e0 Remove debug log message from _schedule_request method 2024-09-09 20:26:02 +01:00
Georgiy Zatserklianyi 6ce0342beb
Allow yielding items from start_requests (#6417) 
Co-authored-by: Georgiy Zatserklianyi <george.zatseklyany@gmail.com>
Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2024-08-26 20:53:06 +02:00
Andrey Rakhmatullin d08f559600
Add flake8-type-checking. (#6413) 2024-06-25 10:20:59 +02:00
Andrey Rakhmatullin 1282ddf8f7
Add parameters to most Deferred in scrapy/core. (#6395) 2024-06-10 10:27:50 +02:00
Andrey Rakhmatullin 2e214210f6 Add parameters to iterable generics, replace generators with iterables. 2024-06-05 21:44:04 +05:00
Andrey Rakhmatullin 019f23e3b7 Add parameters to some of typing.Callable. 2024-06-02 18:42:01 +05:00
Andrey Rakhmatullin 751c91e614 Add parameters to misc generics. 2024-05-31 21:57:14 +05:00
Adrián Chaves d2f1e00a6a
Merge 2.11.2 changes (#6363) 2024-05-14 18:54:11 +02:00
Andrey Rakhmatullin 4ed5c5ae91
Use ParamSpec for callables. (#6353) 2024-05-13 12:01:52 +02:00
Andrey Rakhmatullin d7da298e06
Typing for build_from_*. (#6326) 2024-04-29 16:14:59 +02:00
Andrey Rakhmatullin 282767f23b Bump black. 2024-02-28 14:30:38 +05:00
Swati Anshu c67f730695
create_instance → build_from_crawler, build_from_settings (#6169) 2023-12-18 11:51:02 +01:00
Andrey Rakhmatullin 036d5836d0 Move more things from Crawler.__init__() to .crawl(). 2023-09-08 19:29:05 +04:00
Andrey Rakhmatullin 276bce0641
Update type hints for Twisted 23.8.0. (#6034) 2023-09-01 08:47:23 +02:00
Laerte Pereira 09c63a178b
Remove the deprecated spider parameter from the engine (#5998) 2023-08-04 07:53:04 +02:00
Andrey Rakhmatullin c7730627a0
Typing improvements for some core components (#5889) 2023-04-11 13:47:41 +02:00
Andrey Rakhmatullin 824641e349 Merge remote-tracking branch 'origin/master' into mypy-imports 2023-02-11 22:02:19 +04:00
Jalil SA a5c1ef8276 sort imports with isort 2023-01-25 14:25:15 -06:00
Andrey Rakhmatullin e9094d1f38 Address remaining typing issues in scrapy.core. 2023-01-25 23:22:18 +04:00
Andrey Rakhmatullin 764a9d47bb Fix typing of inlineCallbacks-decorated functions. 2023-01-25 22:31:12 +04:00
Emmanuel Rondan e211ec0aa2 adding black formatter to all the code 2022-11-29 11:30:46 -03:00
Laerte Pereira 2464939b7e
Fixed deprecation warning in scrapy.core.engine (#5589) 
* Change `download` function logic

* Fix CI error in 3.7 checks

* Make `spider` parameter optional in `_download` function, assign spider value from self if `None`
 2022-10-26 23:58:20 +05:00
Adrián Chaves 62a0081266 engine: prevent slot method call after unsetting the slot 2022-03-07 12:00:44 +01:00
Eugenio Lacuesta cb08e3644b
Remove trailing whitespaces 2021-07-08 09:22:21 -03:00
Iván de Prado 6b8f694653
Update scrapy/core/engine.py 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-07-08 12:40:02 +01:00
Iván de Prado 7597d860c8
Update scrapy/core/engine.py 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2021-07-08 12:39:17 +01:00
Iván de Prado Alonso e94d3ac173 Expanded doc for idle signal 2021-06-29 13:40:43 +01:00
Iván de Prado Alonso ce445f2046 Fix typing 2021-06-24 09:56:05 +01:00
Iván de Prado Alonso 812b4bb518 CloseSpider can be raised on spider_idle signal handler 2021-06-23 17:22:01 +01:00
Eugenio Lacuesta ddea6b7bfa
Scheduler: minimal interface, API docs (#3559) 2021-04-26 16:16:14 -03:00
Eugenio Lacuesta e3f81d8d5f
Engine: remove unnecessary parameter (#5106) 2021-04-20 11:46:43 -03:00
Eugenio Lacuesta 7e23677b52
Engine: deprecations and type hints (#5090) 2021-04-20 08:45:28 -03:00
Ammar Najjar 58ca8bbf6d Use f-strings (#4307) 2020-08-26 07:11:05 +02:00
Eugenio Lacuesta 2aa4f3cbf9
Conditional request attribute binding for responses (#4632) 2020-08-17 10:39:59 +02:00
Eugenio Lacuesta 9e99be982a
Remove backslash 2020-07-09 09:51:28 -03:00
Eugenio Lacuesta 7383b2b428
Flake8: Remove E128 (core module) 2020-05-06 17:27:44 -03:00
Eugenio Lacuesta 49e8a337f7
Flake8: remove E127 (continuation line over-indented for visual indent) 2020-05-06 09:37:01 -03:00
Eugenio Lacuesta efb6f13deb
Remove assertions from production code (#4440) 2020-04-23 12:40:10 +02:00
Eugenio Lacuesta e2d5d357a7
Fix pycodestyle E502 2020-04-01 14:24:30 -03:00
elacuesta dfbe1d9507
Remove object base class (#4430) 2020-03-16 20:12:46 +01:00
Marc Hernández 91bbc70bc1
fix E30X flake8 (#4355) 2020-02-21 06:05:31 +01:00
Adrián Chaves b5c4c2cae8
Keep 2 spaces between code and inline comments (#4195) 2019-12-13 14:20:48 +01:00
watsta 0b52fa6ca9 LogFormatter: Add the ability to skip log messages (#3987) 2019-09-16 14:12:04 +02:00
Paul Tremberth 30d812eea2 Remove redundant slot.add_request() call in ExecutionEngine 2017-03-03 17:15:37 +01:00