RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
11,355 Commits 42 Branches 132 Tags 168 MiB
Commit Graph

11355 Commits

Author SHA1 Message Date
Adrian Chaves 24bec31366 WIP 2026-08-12 15:55:21 +02:00
Adrian Chaves 4caee732a1 WIP 2026-08-12 15:38:38 +02:00
Adrian Chaves 8f03e5ca39 Address feedback 2026-08-12 15:20:43 +02:00
Adrian Chaves c7e3a2ab06 Merge remote-tracking branch 'origin/master' into release-notes-2.18 2026-08-12 14:43:53 +02:00
Adrian 52cc2da72d
Deprecate PeriodicLog.multiplier instead of removing it altogether (#7982) 2026-08-12 11:23:41 +05:00
Adrian Chaves 09627b0f05 WIP 2026-08-12 06:38:51 +02:00
Adrian Chaves acb54187af Replace the VERSION placeholder with 2.18.0 2026-08-12 05:59:45 +02:00
Adrian Chaves 7737da0db3 Merge remote-tracking branch 'origin/master' into release-notes-2.18 2026-08-12 05:56:59 +02:00
Adrian 74f062fe3d
Upgrade twinecheck (#7979) 2026-08-11 21:23:25 +05:00
Adrian cf6ef74c79
Deprecate the old import path of FileException (#7973) 2026-08-10 20:19:46 +02:00
Adrian Chaves 6a2928f8d7 WIP 2026-08-10 19:53:21 +02:00
Adrian Chaves e46bd554b4 Merge remote-tracking branch 'origin/master' into release-notes-2.18 2026-08-10 19:35:01 +02:00
Adrian Chaves 5f4e08f361 WIP 2026-08-10 19:33:26 +02:00
Adrian 8ce041e528
:issue: → :gh: (#7974) 2026-08-10 19:30:48 +02:00
Adrian Chaves 294bd79350 Merge remote-tracking branch 'origin/master' into release-notes-2.18 2026-08-10 18:58:13 +02:00
Adrian Chaves 737a1c15c7 WIP 2026-08-10 18:58:07 +02:00
Adrian 95c16aa9af
Deprecate interpolating log formatter messages with the returned dict (#7971) 2026-08-10 21:29:53 +05:00
Adrian Chaves db5a243b9f Fix IPython wording 2026-08-10 18:22:14 +02:00
Adrian Chaves 9e541230c2 Merge remote-tracking branch 'origin/master' into release-notes-2.18 2026-08-10 17:56:52 +02:00
Adrian 15885a8db4
Generalize the use of build_from_crawler() internally (#7808) 
* Generalize the use of build_from_crawler() internally

* Do not use build_from_crawler() for spiders, since they are guaranteed having from_crawler()
 2026-08-10 20:55:10 +05:00
Adrian Chaves 1b9f965b65 Clarify the user implact of the AutoThrottle change 2026-08-10 17:39:30 +02:00
Adrian Chaves 4ed6b3acef WIP 2026-08-10 14:58:42 +02:00
Adrian Chaves 4e6e8ed10a Merge remote-tracking branch 'origin/master' into release-notes-2.18 2026-08-10 13:37:35 +02:00
Adrian f1694269d8
Add FTPS support to the FTP feed export storage (#7953) 2026-08-10 11:33:58 +02:00
Adrian Chaves e70750eb67 WIP 2026-08-10 11:27:58 +02:00
Adrian fe96c1f54b
Serialize dates and times as ISO 8601 in ScrapyJSONEncoder (#7918) 2026-08-10 14:07:42 +05:00
Adrian 4cc2356637
Implement browser-like bad header handling for the default download handler (#7806) 
* Implement browser-like bad header handling for the default download handler

* Remove dead code
 2026-08-10 13:59:44 +05:00
Adrian c7ee394442
Address warnings from the VCS workflow (#7966) 2026-08-10 13:46:20 +05:00
Adrian afbd2c9320
Let trackref release classes defined at run time (#7922) 2026-08-10 13:09:57 +05:00
Adrian c285f4cb18
Support CloseSpider during spider startup (#7905) 2026-08-10 10:06:54 +02:00
Adrian 508bd7faec
Type late Crawler attributes as always set instead of None (#7882) 2026-08-10 09:07:34 +02:00
Adrian 09c918115b
Document response parsing memory use in the security page (#7930) 2026-08-10 08:43:57 +02:00
Adrian ae68786210
Document how to customize media pipeline file names from responses (#7909) 2026-08-09 23:08:59 +05:00
Adrian 5270f3cf99
Add a depth_reset request metadata key (#7913) 2026-08-09 23:07:10 +05:00
Adrian d41bfaec05
Document how to disallow subdomains of allowed domains (#7903) 2026-08-09 23:05:43 +05:00
Adrian ba29ce8d3b
Add a docs section on using download_async() from a downloader middleware (#7872) 2026-08-09 23:02:44 +05:00
Adrian fc082aa914
Clarify the docs about reactor settings (#7880) 2026-08-09 23:01:31 +05:00
Adrian aadfd15327
Do not mangle log messages when a log formatter returns empty or tuple args (#7936) 2026-08-09 22:58:35 +05:00
Adrian bee31890a2
Report exceptions from Spider.start() (#7884) 2026-08-09 22:56:03 +05:00
Adrian a5614544a2
Type tests involving items and exports (#7867) 2026-08-09 22:51:20 +05:00
Adrian 482a02d30c
Add a cookies documentation page (#7947) 2026-08-09 22:41:32 +05:00
Adrian 786ab10494
Document how to write a custom item exporter (#7931) 2026-08-09 22:31:53 +05:00
Adrian 7d9516f332
Canonicalize each extracted link once (#7961) 2026-08-09 22:30:30 +05:00
Adrian 609f64c55d
Make brotli a hard dependency (#7929) 2026-08-09 19:24:14 +02:00
Adrian 5427080f48
Create a page on optimization (#7938) 2026-08-09 18:37:24 +02:00
Adrian 8e94df6526
Time out hanging tests in the VCS dependencies job (#7960) 
* Time out hanging tests in the VCS dependencies job

* Do not test pydispatcher’s latest commit
 2026-08-09 21:18:33 +05:00
Adrian c67bb8ac47
Document BROWSER=wslview for open_in_browser under WSL (#7965) 2026-08-09 21:15:37 +05:00
Adrian a0d1ad2001
Document that follow_all requests share meta and cb_kwargs values (#7962) 2026-08-09 21:15:27 +05:00
Adrian 95fe7acc86
Warn only once per request about the download warn size (#7963) 2026-08-09 21:14:20 +05:00
Adrian a6c017c2ca
Advise setting an identifying user agent (#7890) 2026-08-09 16:59:22 +05:00