RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,488 Commits 49 Branches 132 Tags 149 MiB
Commit Graph

9488 Commits

Author SHA1 Message Date
Josh Soref e6ebadcd54 spelling: freshness 
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
 2022-12-18 21:33:21 -05:00
Josh Soref 581eb2d1b4 spelling: downloader 
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
 2022-12-18 21:33:21 -05:00
Josh Soref 87fc92441f spelling: crawlable 
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
 2022-12-18 21:33:21 -05:00
Josh Soref 1300c1c881 spelling: children 
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
 2022-12-18 21:33:21 -05:00
Josh Soref 226c42ad14 spelling: canonicalize 
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
 2022-12-18 20:26:42 -05:00
Josh Soref 334f844e58 spelling: and 
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
 2022-12-18 20:26:42 -05:00
Josh Soref f3c6bfdebe spelling: accounting 
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
 2022-12-18 20:26:42 -05:00
Andrey Rahmatullin 034dc8f10b
Merge pull request #5761 from yolile/patch-1 
docs: change DOWNLOAD_DELAY description format
 2022-12-16 01:03:53 +05:00
Yohanna Lisnichuk b63ca6f834 docs: use the right notes directive 2022-12-15 13:20:45 -03:00
Yohanna Lisnichuk 96e526aad8
docs: change DOWNLOAD_DELAY description format 
To be consistent with the other settings' descriptions
 2022-12-15 10:01:55 -03:00
Andrey Rahmatullin 75450e75d2
Merge pull request #5760 from alexpdev/fix_shell_fetch_alt 
set asyncio event loop at start of new thread
 2022-12-15 10:22:01 +05:00
Alex 93ace91ace
Merge branch 'scrapy:master' into fix_shell_fetch_alt 2022-12-13 21:50:28 -08:00
Alex c2de9372a2 added unittest for scrapy shell command and fixed flake8 error 2022-12-13 21:37:17 -08:00
Alex 8b09b0e0d7 set asyncio event loop at start of new thread 2022-12-13 20:41:48 -08:00
Andrey Rahmatullin 26ebdbf4ef
Merge pull request #5756 from Laerte/fix-test_follow_whitespace-tests 
fix test_follow_whitespace_* tests
 2022-12-12 23:49:23 +05:00
Laerte Pereira e099572cec
change xfail condition 2022-12-12 09:56:50 -03:00
Laerte Pereira 0dbd1d9b81
use xfail, switch to packaging to parse version 2022-12-12 08:58:40 -03:00
Laerte Pereira e92e201b19
Silence `filelock` logger (#5754) 2022-12-12 11:55:15 +01:00
Laerte Pereira 6757973b61
fix test_follow_whitespace_* tests 2022-12-11 09:11:55 -03:00
Andrey Rahmatullin fe60c1224e
Fix the name of the Proxy-Authorization header in release notes (#5744) 2022-12-05 11:44:39 +01:00
Andrey Rahmatullin 5bd27191a2
Bump mypy, flake8, and pylint (#5738) 2022-11-28 19:37:57 +01:00
Mark Mayo f9a29f03d9
Address some issues reported by Pylint (#5677) 
Co-authored-by: Felipe Boff Nunes <51033921+felipeboffnunes@users.noreply.github.com>
Co-authored-by: Andrey Rahmatullin <wrar@wrar.name>
 2022-11-27 11:00:13 +01:00
Mikhail Korobov e2db624204
Merge pull request #5682 from mdonoughe/pathlib 
use pathlib
 2022-11-26 13:42:36 +05:00
Andrey Rahmatullin d19a216e10
Merge branch 'master' into pathlib 2022-11-25 18:20:25 +05:00
Andrey Rahmatullin 3be7aa9a0f
Merge pull request #5736 from wRAR/test-char-comment 
Add a comment about `tmpname + '^'`.
 2022-11-25 18:19:51 +05:00
Andrey Rakhmatullin f85c3f3d68 Add a comment about `tmpname + '^'`. 2022-11-25 18:46:14 +06:00
Hanzallah Burney b6e98ce6b6
Remove unnecessary backwards compatibility comments (#5732) 2022-11-24 15:01:15 +01:00
Andrey Rahmatullin 0417e3ede2
Merge pull request #5727 from JazzGlobal/Better-Language-Surrounding-Logging 
Better language surrounding logging
 2022-11-23 22:29:17 +05:00
Hanzallah Burney f6e9e6592a
Cleanup settings._DictProxy and scrapy.telnet (#5730) 2022-11-23 15:48:34 +01:00
Andrey Rahmatullin 04a1a4b730
Merge pull request #5731 from Laerte/remove-noconnect-deprecate-code 
Remove `noconnect` deprecate code
 2022-11-23 18:19:01 +05:00
Laerte Pereira e769532644
Remove `noconnect` deprecate code 2022-11-23 08:30:11 -03:00
Andrey Rahmatullin 45c2bd7d9c
Merge pull request #5150 from gustavo-bordin/master 
changing dunder-str to dunder-repr
 2022-11-23 13:12:06 +05:00
Andrey Rahmatullin c3b1700774
Restore lost typing 2022-11-23 12:17:30 +05:00
Andrey Rahmatullin 1fdd0a70a0
Restore lost typing 2022-11-23 12:16:48 +05:00
Andrey Rahmatullin b9c8db13c2
Merge pull request #5724 from Laerte/remove-scrapy_-env-variables-override 
Remove support for override settings with `SCRAPY_` environment variables
 2022-11-22 22:19:02 +05:00
Christopher Gambrell bdc0bca5b1 Replace M occurrences with MiB for accuracy. 2022-11-22 12:10:49 -05:00
Christopher Gambrell fc8968672a renamed variables to clarify that we are using peak memory and not current memory utilization. 2022-11-22 11:49:28 -05:00
Laerte Pereira 1506479672
Remove deprecated test 2022-11-22 10:07:32 -03:00
Laerte Pereira 8e0025f53d
Remove support for override settings with `SCRAPY_` environment variables 2022-11-22 09:38:54 -03:00
Andrey Rahmatullin aac6103179
Merge pull request #5722 from JazzGlobal/5717-memusage-logging-extension 
Adds get_virtual_size to _check_limit Logging
 2022-11-22 16:08:20 +05:00
Andrey Rahmatullin 603e42bc26
Merge pull request #5720 from Laerte/remove-filteringlinkextractor 
Remove `FilteringLinkExtractor`
 2022-11-22 13:54:43 +05:00
Christopher Gambrell eb159c78f1 Use variable and convert to megabytes. 2022-11-22 03:36:00 -05:00
Christopher Gambrell 8f2adad7a7 Log self.get_virtual_size() on every call of _check_limit 2022-11-22 00:48:24 -05:00
Christopher Gambrell 24a18e9af1 Adds virtualsize property to _check_limit error log. 2022-11-21 17:41:06 -05:00
Andrey Rahmatullin a60c120192
Merge pull request #5721 from pablohoffman/debug-vscode 
doc: add instructions to debug Scrapy spiders in Visual Studio Code
 2022-11-22 00:36:13 +05:00
Pablo Hoffman c04ccbceb9 doc: add instructions to debug Scrapy spiders in Visual Studio Code 2022-11-21 15:49:33 +01:00
Laerte Pereira 1a6408c3fa
Remove `FilteringLinkExtractor` 2022-11-21 07:30:20 -03:00
Laerte Pereira d5b6c236a9
Remove deprecated code (#5719) 2022-11-21 09:32:26 +01:00
Andrey Rahmatullin 8f01a60c73
Merge pull request #5715 from islem-esi/main 
minor fix for readability
 2022-11-16 13:47:54 +05:00
islem-esi 1200a54543 minor fix for readability 2022-11-15 16:28:45 +01:00