RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,472 Commits 49 Branches 132 Tags 131 MiB
Commit Graph

9472 Commits

Author SHA1 Message Date
Emmanuel Rondan 09dc4cf308 adding black to tox and defining line length on pyproject.toml 2022-12-22 16:11:21 -03:00
Emmanuel Rondan 1a3db81492 ignoring E203 flake8 error 
ignoring implicit-str-concat pylint error
fixing pylint comment on test_request_cb_kwargs.py
 2022-11-29 15:06:54 -03:00
Emmanuel Rondan 44160552ef adding .git-blame-ignore-revs file to skip code changes made by black formatter on blame 2022-11-29 11:40:14 -03:00
Emmanuel Rondan e211ec0aa2 adding black formatter to all the code 2022-11-29 11:30:46 -03:00
Andrey Rahmatullin 5bd27191a2
Bump mypy, flake8, and pylint (#5738) 2022-11-28 19:37:57 +01:00
Mark Mayo f9a29f03d9
Address some issues reported by Pylint (#5677) 
Co-authored-by: Felipe Boff Nunes <51033921+felipeboffnunes@users.noreply.github.com>
Co-authored-by: Andrey Rahmatullin <wrar@wrar.name>
 2022-11-27 11:00:13 +01:00
Mikhail Korobov e2db624204
Merge pull request #5682 from mdonoughe/pathlib 
use pathlib
 2022-11-26 13:42:36 +05:00
Andrey Rahmatullin d19a216e10
Merge branch 'master' into pathlib 2022-11-25 18:20:25 +05:00
Andrey Rahmatullin 3be7aa9a0f
Merge pull request #5736 from wRAR/test-char-comment 
Add a comment about `tmpname + '^'`.
 2022-11-25 18:19:51 +05:00
Andrey Rakhmatullin f85c3f3d68 Add a comment about `tmpname + '^'`. 2022-11-25 18:46:14 +06:00
Hanzallah Burney b6e98ce6b6
Remove unnecessary backwards compatibility comments (#5732) 2022-11-24 15:01:15 +01:00
Andrey Rahmatullin 0417e3ede2
Merge pull request #5727 from JazzGlobal/Better-Language-Surrounding-Logging 
Better language surrounding logging
 2022-11-23 22:29:17 +05:00
Hanzallah Burney f6e9e6592a
Cleanup settings._DictProxy and scrapy.telnet (#5730) 2022-11-23 15:48:34 +01:00
Andrey Rahmatullin 04a1a4b730
Merge pull request #5731 from Laerte/remove-noconnect-deprecate-code 
Remove `noconnect` deprecate code
 2022-11-23 18:19:01 +05:00
Laerte Pereira e769532644
Remove `noconnect` deprecate code 2022-11-23 08:30:11 -03:00
Andrey Rahmatullin 45c2bd7d9c
Merge pull request #5150 from gustavo-bordin/master 
changing dunder-str to dunder-repr
 2022-11-23 13:12:06 +05:00
Andrey Rahmatullin c3b1700774
Restore lost typing 2022-11-23 12:17:30 +05:00
Andrey Rahmatullin 1fdd0a70a0
Restore lost typing 2022-11-23 12:16:48 +05:00
Andrey Rahmatullin b9c8db13c2
Merge pull request #5724 from Laerte/remove-scrapy_-env-variables-override 
Remove support for override settings with `SCRAPY_` environment variables
 2022-11-22 22:19:02 +05:00
Christopher Gambrell bdc0bca5b1 Replace M occurrences with MiB for accuracy. 2022-11-22 12:10:49 -05:00
Christopher Gambrell fc8968672a renamed variables to clarify that we are using peak memory and not current memory utilization. 2022-11-22 11:49:28 -05:00
Laerte Pereira 1506479672
Remove deprecated test 2022-11-22 10:07:32 -03:00
Laerte Pereira 8e0025f53d
Remove support for override settings with `SCRAPY_` environment variables 2022-11-22 09:38:54 -03:00
Andrey Rahmatullin aac6103179
Merge pull request #5722 from JazzGlobal/5717-memusage-logging-extension 
Adds get_virtual_size to _check_limit Logging
 2022-11-22 16:08:20 +05:00
Andrey Rahmatullin 603e42bc26
Merge pull request #5720 from Laerte/remove-filteringlinkextractor 
Remove `FilteringLinkExtractor`
 2022-11-22 13:54:43 +05:00
Christopher Gambrell eb159c78f1 Use variable and convert to megabytes. 2022-11-22 03:36:00 -05:00
Christopher Gambrell 8f2adad7a7 Log self.get_virtual_size() on every call of _check_limit 2022-11-22 00:48:24 -05:00
Christopher Gambrell 24a18e9af1 Adds virtualsize property to _check_limit error log. 2022-11-21 17:41:06 -05:00
Andrey Rahmatullin a60c120192
Merge pull request #5721 from pablohoffman/debug-vscode 
doc: add instructions to debug Scrapy spiders in Visual Studio Code
 2022-11-22 00:36:13 +05:00
Pablo Hoffman c04ccbceb9 doc: add instructions to debug Scrapy spiders in Visual Studio Code 2022-11-21 15:49:33 +01:00
Laerte Pereira 1a6408c3fa
Remove `FilteringLinkExtractor` 2022-11-21 07:30:20 -03:00
Laerte Pereira d5b6c236a9
Remove deprecated code (#5719) 2022-11-21 09:32:26 +01:00
Andrey Rahmatullin 8f01a60c73
Merge pull request #5715 from islem-esi/main 
minor fix for readability
 2022-11-16 13:47:54 +05:00
islem-esi 1200a54543 minor fix for readability 2022-11-15 16:28:45 +01:00
Hassan Shoayb 29bf7f5a6c
broad-crawls.rst: fix a typo (#5714) 2022-11-14 09:45:00 +01:00
Laerte Pereira ae3fd01729
Fix stray “commands” (#5712) 2022-11-10 15:38:46 +01:00
Andrey Rahmatullin c56caaf0b7
Merge pull request #4753 from drs-11/3689-update 
Do not recode JPEG files in imagepipelines
 2022-11-07 20:36:22 +05:00
Andrey Rahmatullin bbe24d79a5
Fix test issues 2022-11-07 17:08:54 +05:00
Andrey Rahmatullin 6c0890ff54
Simplify the changes after the merge 2022-11-07 16:36:54 +05:00
Andrey Rahmatullin b03d84e9fb
Merge branch 'master' into 3689-update 2022-11-07 16:33:21 +05:00
Andrey Rahmatullin 77cd511a5e
Merge pull request #5489 from PluT00/deprecate-NoimagesDrop 
Deprecate scrapy.pipelines.images.NoimagesDrop
 2022-11-05 14:18:19 +05:00
Andrey Rahmatullin 8004075823
Merge pull request #5699 from elacuesta/crawlspider-callback-keyword-arguments 
CrawlSpider: pass cb_kwargs from process_request
 2022-11-02 17:35:03 +05:00
Andrey Rakhmatullin 6ded3cf4cd Bump version: 2.7.0 → 2.7.1 2022-11-02 17:00:47 +06:00
Andrey Rahmatullin 95880c5de1
Merge pull request #5701 from scrapy/relnotes-2.7.1 2022-11-02 15:42:52 +05:00
Andrey Rakhmatullin 5ec175b8bb Small relnotes fixes. 2022-11-02 13:54:00 +06:00
Andrey Rakhmatullin 940a73863b Release notes for 2.7.1. 2022-11-01 19:00:33 +06:00
Andrey Rahmatullin a95a338eea
Merge pull request #5599 from tonal/patch-1 
Partial in is_generator_with_return_value
 2022-10-31 20:08:16 +05:00
Eugenio Lacuesta b18560315b
Add tests 2022-10-30 18:28:16 -03:00
Eugenio Lacuesta 3259a42525
CrawlSpider: pass cb_kwargs from process_request 2022-10-30 13:17:37 -03:00
Andrey Rahmatullin 9077d0f9b4
Merge pull request #5698 from pankali/patch-1 
Update Code of Conduct to Contributor Covenant v2.1
 2022-10-28 12:51:38 +05:00