RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,452 Commits 49 Branches 132 Tags 138 MiB
Commit Graph

9452 Commits

Author SHA1 Message Date
Andrey Rahmatullin 3be7aa9a0f
Merge pull request #5736 from wRAR/test-char-comment 
Add a comment about `tmpname + '^'`.
 2022-11-25 18:19:51 +05:00
Andrey Rakhmatullin f85c3f3d68 Add a comment about `tmpname + '^'`. 2022-11-25 18:46:14 +06:00
Hanzallah Burney b6e98ce6b6
Remove unnecessary backwards compatibility comments (#5732) 2022-11-24 15:01:15 +01:00
Andrey Rahmatullin 0417e3ede2
Merge pull request #5727 from JazzGlobal/Better-Language-Surrounding-Logging 
Better language surrounding logging
 2022-11-23 22:29:17 +05:00
Hanzallah Burney f6e9e6592a
Cleanup settings._DictProxy and scrapy.telnet (#5730) 2022-11-23 15:48:34 +01:00
Andrey Rahmatullin 04a1a4b730
Merge pull request #5731 from Laerte/remove-noconnect-deprecate-code 
Remove `noconnect` deprecate code
 2022-11-23 18:19:01 +05:00
Laerte Pereira e769532644
Remove `noconnect` deprecate code 2022-11-23 08:30:11 -03:00
Andrey Rahmatullin 45c2bd7d9c
Merge pull request #5150 from gustavo-bordin/master 
changing dunder-str to dunder-repr
 2022-11-23 13:12:06 +05:00
Andrey Rahmatullin c3b1700774
Restore lost typing 2022-11-23 12:17:30 +05:00
Andrey Rahmatullin 1fdd0a70a0
Restore lost typing 2022-11-23 12:16:48 +05:00
Andrey Rahmatullin b9c8db13c2
Merge pull request #5724 from Laerte/remove-scrapy_-env-variables-override 
Remove support for override settings with `SCRAPY_` environment variables
 2022-11-22 22:19:02 +05:00
Christopher Gambrell bdc0bca5b1 Replace M occurrences with MiB for accuracy. 2022-11-22 12:10:49 -05:00
Christopher Gambrell fc8968672a renamed variables to clarify that we are using peak memory and not current memory utilization. 2022-11-22 11:49:28 -05:00
Laerte Pereira 1506479672
Remove deprecated test 2022-11-22 10:07:32 -03:00
Laerte Pereira 8e0025f53d
Remove support for override settings with `SCRAPY_` environment variables 2022-11-22 09:38:54 -03:00
Andrey Rahmatullin aac6103179
Merge pull request #5722 from JazzGlobal/5717-memusage-logging-extension 
Adds get_virtual_size to _check_limit Logging
 2022-11-22 16:08:20 +05:00
Andrey Rahmatullin 603e42bc26
Merge pull request #5720 from Laerte/remove-filteringlinkextractor 
Remove `FilteringLinkExtractor`
 2022-11-22 13:54:43 +05:00
Christopher Gambrell eb159c78f1 Use variable and convert to megabytes. 2022-11-22 03:36:00 -05:00
Christopher Gambrell 8f2adad7a7 Log self.get_virtual_size() on every call of _check_limit 2022-11-22 00:48:24 -05:00
Christopher Gambrell 24a18e9af1 Adds virtualsize property to _check_limit error log. 2022-11-21 17:41:06 -05:00
Andrey Rahmatullin a60c120192
Merge pull request #5721 from pablohoffman/debug-vscode 
doc: add instructions to debug Scrapy spiders in Visual Studio Code
 2022-11-22 00:36:13 +05:00
Pablo Hoffman c04ccbceb9 doc: add instructions to debug Scrapy spiders in Visual Studio Code 2022-11-21 15:49:33 +01:00
Laerte Pereira 1a6408c3fa
Remove `FilteringLinkExtractor` 2022-11-21 07:30:20 -03:00
Laerte Pereira d5b6c236a9
Remove deprecated code (#5719) 2022-11-21 09:32:26 +01:00
Andrey Rahmatullin 8f01a60c73
Merge pull request #5715 from islem-esi/main 
minor fix for readability
 2022-11-16 13:47:54 +05:00
islem-esi 1200a54543 minor fix for readability 2022-11-15 16:28:45 +01:00
Hassan Shoayb 29bf7f5a6c
broad-crawls.rst: fix a typo (#5714) 2022-11-14 09:45:00 +01:00
Laerte Pereira ae3fd01729
Fix stray “commands” (#5712) 2022-11-10 15:38:46 +01:00
Andrey Rahmatullin c56caaf0b7
Merge pull request #4753 from drs-11/3689-update 
Do not recode JPEG files in imagepipelines
 2022-11-07 20:36:22 +05:00
Andrey Rahmatullin bbe24d79a5
Fix test issues 2022-11-07 17:08:54 +05:00
Andrey Rahmatullin 6c0890ff54
Simplify the changes after the merge 2022-11-07 16:36:54 +05:00
Andrey Rahmatullin b03d84e9fb
Merge branch 'master' into 3689-update 2022-11-07 16:33:21 +05:00
Andrey Rahmatullin 77cd511a5e
Merge pull request #5489 from PluT00/deprecate-NoimagesDrop 
Deprecate scrapy.pipelines.images.NoimagesDrop
 2022-11-05 14:18:19 +05:00
Andrey Rahmatullin 8004075823
Merge pull request #5699 from elacuesta/crawlspider-callback-keyword-arguments 
CrawlSpider: pass cb_kwargs from process_request
 2022-11-02 17:35:03 +05:00
Andrey Rakhmatullin 6ded3cf4cd Bump version: 2.7.0 → 2.7.1 2022-11-02 17:00:47 +06:00
Andrey Rahmatullin 95880c5de1
Merge pull request #5701 from scrapy/relnotes-2.7.1 2022-11-02 15:42:52 +05:00
Andrey Rakhmatullin 5ec175b8bb Small relnotes fixes. 2022-11-02 13:54:00 +06:00
Andrey Rakhmatullin 940a73863b Release notes for 2.7.1. 2022-11-01 19:00:33 +06:00
Andrey Rahmatullin a95a338eea
Merge pull request #5599 from tonal/patch-1 
Partial in is_generator_with_return_value
 2022-10-31 20:08:16 +05:00
Eugenio Lacuesta b18560315b
Add tests 2022-10-30 18:28:16 -03:00
Eugenio Lacuesta 3259a42525
CrawlSpider: pass cb_kwargs from process_request 2022-10-30 13:17:37 -03:00
Andrey Rahmatullin 9077d0f9b4
Merge pull request #5698 from pankali/patch-1 
Update Code of Conduct to Contributor Covenant v2.1
 2022-10-28 12:51:38 +05:00
Andrey Rahmatullin 76c2cb070e
Merge pull request #5697 from iamkaushal/#5686_fix 
added typing.io and typing.re in pytest warning filter to ignore
 2022-10-28 11:38:55 +05:00
എതിരാളിക്കൊരു പോരാളി 9f45be439d
Update Code of Conduct to Contributor Covenant v2.1 2022-10-28 02:13:37 +02:00
iamkaushal bd9e482c2f added typing.io and typing.re in pytest warning filter to ignore 2022-10-27 23:21:55 +05:30
Magnus Offermanns fd692f3091
Prevent running the -O and -t command-line options together (#5605) 
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
 2022-10-27 14:43:31 +02:00
Andrey Rakhmatullin b71d0292d5 Add a test for processing partial callbacks. 2022-10-27 18:13:47 +06:00
Godson 3a34fa8399
Get the event loop from event_loop_policy to avoid a deprecation warning (#5689) 2022-10-27 14:02:12 +02:00
Andrey Rahmatullin b654183084
Use Python 3.11 as the default in CI (#5696) 2022-10-27 14:00:36 +02:00
Andrey Rakhmatullin ca50af6453 Remove an unused import. 2022-10-27 17:12:26 +06:00