RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,191 Commits 49 Branches 132 Tags 162 MiB
Commit Graph

11191 Commits

Author SHA1 Message Date
Adrián Chaves c34ca4aef5 Bump version: 2.7.1 → 2.8.0 2023-02-02 04:55:31 +01:00
Adrián Chaves 068af85722
Add release notes for Scrapy 2.8 (#5799) 2023-02-02 04:54:02 +01:00
Alex 8c8894f4be
Apply black formatting to templates (#5814) 2023-02-02 04:42:47 +01:00
pankaj1707k cc9eb3fa79
Format python console blocks using blacken-docs 
Prepend '.. code-block:: pycon' to make python console blocks detectable by blacken-docs
 2023-02-01 20:37:39 +05:30
pankaj1707k c1bbb299d7
Add and run pre-commit hook 'blacken-docs' 
Change python code snippets to begin with '.. code-block:: python' to be recognized by the hook for formatting. All snippets under '::' (rst literal blocks) are ignored.
 2023-02-01 16:30:57 +05:30
Alex 349fc33cc7 added disable_warnings instruction to .coveragerc 2023-01-31 14:28:08 -08:00
Mikhail Korobov b337c986ca
Merge pull request #5798 from Gallaecio/no-callback 
Implement a NO_CALLBACK value for Request.callback
 2023-01-31 01:06:49 +05:00
Adrián Chaves 78eaf0671b Remove typing-extensions from tox.ini 2023-01-30 14:33:11 +01:00
Adrián Chaves 4239f7e12b Merge remote-tracking branch 'Gallaecio/no-callback' into no-callback 2023-01-30 12:39:56 +01:00
Adrián Chaves 389fd99e79 get_media_requests: support and encourage callback=NO_CALLBACK 2023-01-30 12:37:34 +01:00
Adrián Chaves e1699479f6
Fix typo: download middleware → downloader middleware 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2023-01-30 11:54:31 +01:00
Andrey Rakhmatullin ccd1385e11
Merge branch 'master' into no-callback 2023-01-30 15:43:05 +05:00
Andrey Rakhmatullin da15d93d39
Merge pull request #5806 from jxlil/master 
Use isort
 2023-01-30 15:40:44 +05:00
Adrián Chaves 17354a61b1 Avoid duplicities in CI; remove pylint from pre-commit 2023-01-30 10:04:27 +01:00
Jalil SA 5dcf8b9015 fix isort version 2023-01-29 00:22:56 -06:00
Andrey Rakhmatullin 80453d53b1 Type DeferredLists as Deferreds. 2023-01-28 01:29:09 +04:00
Jalil SA 4bd48d2613 added pre-commit action 2023-01-27 15:06:54 -06:00
Jalil SA ef794251f6 fix scrapy/__init__.py 2023-01-27 15:00:19 -06:00
Jalil SA e9ee9454f9 fix .isort.cfg 2023-01-27 14:59:08 -06:00
Andrey Rakhmatullin 5fa0f64db5 Ru typing on 3.8 as types-lxml isn't available for 3.7. 2023-01-27 23:29:23 +04:00
Andrey Rakhmatullin c0efb271a2 Ignore typing for twisted.internet.reactor globally. 2023-01-27 23:19:44 +04:00
Andrey Rakhmatullin 30c0dc7ac5
Merge pull request #5807 from Cj-Malone/patch-1 
Don't check robots.txt for local files
 2023-01-28 00:03:15 +05:00
Adrián Chaves f03b47db05 Make NO_CALLBACK a callable 2023-01-27 17:35:32 +01:00
Alex a1e2fbafdc applied black to tests 2023-01-26 16:46:08 -08:00
Alex 42af2bf1ad updating branch 2023-01-26 16:43:23 -08:00
Alex 94161d101c update 2023-01-26 16:41:06 -08:00
Cj Malone 33b85a9e2a
Test local files aren't processed 2023-01-26 19:51:20 +00:00
Cj Malone 3054235dc0
Don't check robotstxt for local files 2023-01-26 16:10:57 +00:00
Alex 5433015a25
Merge branch 'master' into path_object_error_#5739 2023-01-26 07:12:01 -08:00
Alex 0a21a9457b fixed mypy typing error 2023-01-26 00:50:29 -08:00
Jalil SA 7f01e1f0ce added isort to pre-commit-config 2023-01-25 14:43:25 -06:00
Jalil SA a5c1ef8276 sort imports with isort 2023-01-25 14:25:15 -06:00
Jalil SA 6d0f9df8c1 added isort.cfg 2023-01-25 14:22:42 -06:00
Andrey Rakhmatullin 69bb9a7859 Install types-Pillow and types-Pygments. 2023-01-25 23:52:46 +04:00
Andrey Rakhmatullin a4edff31b9 Replace lxml-stubs with types-lxml. 2023-01-25 23:30:21 +04:00
Andrey Rakhmatullin e9094d1f38 Address remaining typing issues in scrapy.core. 2023-01-25 23:22:18 +04:00
Andrey Rakhmatullin 5fde6d5339 Don't type-check Twisted interfaces. 2023-01-25 22:54:46 +04:00
Andrey Rakhmatullin 764a9d47bb Fix typing of inlineCallbacks-decorated functions. 2023-01-25 22:31:12 +04:00
Andrey Rakhmatullin 232aab53b3 Fix most of the new typing issues in utils. 2023-01-25 22:28:15 +04:00
Andrey Rakhmatullin afd5d85320 Fix remaining issues in tests. 2023-01-25 22:15:10 +04:00
Andrey Rakhmatullin 2e33fb812b Silence improper typing of twisted.internet.reactor in tests. 2023-01-25 22:15:07 +04:00
Andrey Rakhmatullin c0ea7fd4fd Remove obsolete top-level run code in tests. 2023-01-25 22:14:24 +04:00
Andrey Rakhmatullin 73f697f1db Re-enable follow_imports. 2023-01-25 22:11:26 +04:00
Adrián Chaves 1f3e42897a _NoCallback → NoCallbackType 2023-01-25 18:30:29 +01:00
Adrián Chaves 9d07be61b4 Merge remote-tracking branch 'Gallaecio/no-callback' into no-callback 2023-01-25 17:49:36 +01:00
Adrián Chaves c883a13006 Make the _set_xback condition more readable 2023-01-25 17:43:10 +01:00
Adrián Chaves 9272c4af0c Merge remote-tracking branch 'scrapy/master' into no-callback 2023-01-25 17:41:41 +01:00
Adrián Chaves e71eab6932
Use black (#5734) 2023-01-25 12:03:46 +01:00
Alex 42e8d5a615 fixing type declerations 2023-01-25 01:03:27 -08:00
Alex 7c753adbe5 revert mypy fix didn't work 2023-01-24 23:05:17 -08:00