RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,607 Commits 49 Branches 132 Tags 180 MiB
Commit Graph

9607 Commits

Author SHA1 Message Date
pankaj1707k c1bbb299d7
Add and run pre-commit hook 'blacken-docs' 
Change python code snippets to begin with '.. code-block:: python' to be recognized by the hook for formatting. All snippets under '::' (rst literal blocks) are ignored.
 2023-02-01 16:30:57 +05:30
Mikhail Korobov b337c986ca
Merge pull request #5798 from Gallaecio/no-callback 
Implement a NO_CALLBACK value for Request.callback
 2023-01-31 01:06:49 +05:00
Adrián Chaves 78eaf0671b Remove typing-extensions from tox.ini 2023-01-30 14:33:11 +01:00
Adrián Chaves 4239f7e12b Merge remote-tracking branch 'Gallaecio/no-callback' into no-callback 2023-01-30 12:39:56 +01:00
Adrián Chaves 389fd99e79 get_media_requests: support and encourage callback=NO_CALLBACK 2023-01-30 12:37:34 +01:00
Adrián Chaves e1699479f6
Fix typo: download middleware → downloader middleware 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2023-01-30 11:54:31 +01:00
Andrey Rakhmatullin ccd1385e11
Merge branch 'master' into no-callback 2023-01-30 15:43:05 +05:00
Andrey Rakhmatullin da15d93d39
Merge pull request #5806 from jxlil/master 
Use isort
 2023-01-30 15:40:44 +05:00
Adrián Chaves 17354a61b1 Avoid duplicities in CI; remove pylint from pre-commit 2023-01-30 10:04:27 +01:00
Jalil SA 5dcf8b9015 fix isort version 2023-01-29 00:22:56 -06:00
Jalil SA 4bd48d2613 added pre-commit action 2023-01-27 15:06:54 -06:00
Jalil SA ef794251f6 fix scrapy/__init__.py 2023-01-27 15:00:19 -06:00
Jalil SA e9ee9454f9 fix .isort.cfg 2023-01-27 14:59:08 -06:00
Andrey Rakhmatullin 30c0dc7ac5
Merge pull request #5807 from Cj-Malone/patch-1 
Don't check robots.txt for local files
 2023-01-28 00:03:15 +05:00
Adrián Chaves f03b47db05 Make NO_CALLBACK a callable 2023-01-27 17:35:32 +01:00
Cj Malone 33b85a9e2a
Test local files aren't processed 2023-01-26 19:51:20 +00:00
Cj Malone 3054235dc0
Don't check robotstxt for local files 2023-01-26 16:10:57 +00:00
Jalil SA 7f01e1f0ce added isort to pre-commit-config 2023-01-25 14:43:25 -06:00
Jalil SA a5c1ef8276 sort imports with isort 2023-01-25 14:25:15 -06:00
Jalil SA 6d0f9df8c1 added isort.cfg 2023-01-25 14:22:42 -06:00
Adrián Chaves 1f3e42897a _NoCallback → NoCallbackType 2023-01-25 18:30:29 +01:00
Adrián Chaves 9d07be61b4 Merge remote-tracking branch 'Gallaecio/no-callback' into no-callback 2023-01-25 17:49:36 +01:00
Adrián Chaves c883a13006 Make the _set_xback condition more readable 2023-01-25 17:43:10 +01:00
Adrián Chaves 9272c4af0c Merge remote-tracking branch 'scrapy/master' into no-callback 2023-01-25 17:41:41 +01:00
Adrián Chaves e71eab6932
Use black (#5734) 2023-01-25 12:03:46 +01:00
Emmanuel Rondan 72a853c751 fixing contributing page with suggestions 2023-01-23 14:56:48 -03:00
Emmanuel Rondan f2c22aaabb Merge remote-tracking branch 'upstream/master' into add-black-formatter 2023-01-20 17:08:07 -03:00
Emmanuel Rondan 8ee4817471 reaplying black, fixing conflicts and ignoring bandit checks on test directory 2023-01-20 10:55:46 -03:00
Andrey Rakhmatullin 9cb757d239
Merge pull request #5800 from Laerte/chore/feed-export-encoding 
Set `FEED_EXPORT_ENCODING='utf-8'` in the default template
 2023-01-20 17:47:43 +05:00
Adrián Chaves 818d69fa00
Fix typo: it → is 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2023-01-20 12:38:07 +01:00
Laerte Pereira b611848029
fix(docs): Change `versionchanged` value 2023-01-20 08:35:17 -03:00
Laerte Pereira 973f0cf567
fix: line break 2023-01-20 08:23:05 -03:00
Laerte Pereira 8270df754d
Set `FEED_EXPORT_ENCODING='utf-8'` in the default template 2023-01-20 07:55:16 -03:00
Adrián Chaves 4242ae405d Restore Python 3.7 support 2023-01-19 20:37:24 +01:00
Adrián Chaves 5c1559f60e Address typing issues 2023-01-19 20:30:22 +01:00
Adrián Chaves a493464942 Update the screenshot pipeline code example 2023-01-19 19:53:53 +01:00
Tobias Mayr f449ee5377
Fix SMTP STARTTLS for Twisted >= 21.2.0 (#5406) 2023-01-19 19:44:55 +01:00
Adrián Chaves 50500a6b28 Implement a NO_CALLBACK value for Request.callback 2023-01-19 17:14:18 +01:00
Emmanuel Rondan 23e8b553b4 fixing docs with suggestions 2023-01-18 11:46:12 -03:00
Emmanuel Rondan 482a0b79e3 explaining pre-commit on documentation 2023-01-17 14:26:12 -03:00
Serhii A caaeb235a0
scrapy.utils.console.DEFAULT_PYTHON_SHELLS: OrderedDict → dict (#5795) 2023-01-17 12:52:41 +01:00
Andrey Rakhmatullin 2f12f5c259
Merge pull request #5790 from scrapy/ssl-legacy-reneg 
Enable unsafe legacy renegotiation
 2023-01-17 16:47:23 +05:00
Andrey Rakhmatullin 43ab8bd16a Roll back the get_openssl_version() type change. 2023-01-13 20:51:07 +04:00
Andrey Rakhmatullin fb52918d23 Set OP_LEGACY_SERVER_CONNECT to support some old servers when using OpenSSL 3. 2023-01-13 20:46:55 +04:00
Andrey Rakhmatullin 93ad6a4bc2 Simplify code for modern pyOpenSSL. 2023-01-13 20:39:44 +04:00
Andrey Rakhmatullin 4af5a06842
Merge pull request #5540 from Gallaecio/download-delay-documentation 
Document how DOWNLOAD_DELAY affects per-domain concurrency
 2023-01-13 13:00:15 +05:00
Andrey Rakhmatullin 74b8c34de6
Merge pull request #5458 from reidemeister94/fix/unique-list-link-extractors 
fix: return unique_list only when link_extractor.unique is True
 2023-01-13 01:13:45 +05:00
Andrey Rakhmatullin f87d061069
Merge branch 'master' into download-delay-documentation 2023-01-13 00:22:48 +05:00
Andrey Rakhmatullin 604ddf82fc
Merge pull request #5758 from MattyMay/finish_exporting_for_each_start_exporting 
Finish exporting for each start exporting
 2023-01-13 00:17:17 +05:00
Andrey Rakhmatullin 2944894c4d
Merge pull request #5764 from jsoref/spelling 
Spelling
 2023-01-13 00:15:36 +05:00