RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,242 Commits 49 Branches 132 Tags 137 MiB
Commit Graph

8242 Commits

Author SHA1 Message Date
BroodingKangaroo 7b1d3c35ea Minor updates 2020-07-01 12:24:31 +03:00
BroodingKangaroo 05c2587c6a Docs update and tiny fixes 2020-06-28 09:45:45 +03:00
BroodingKangaroo 88a52198b9 Add batch_item_count support in FEEDS setting 2020-06-27 12:45:08 +03:00
BroodingKangaroo c3cee74fd4 Change default value of FEED_STORAGE_BATCH_ITEM_COUNT to 0 2020-06-26 18:45:21 +03:00
BroodingKangaroo 77333666f1 resolve conflicts 2020-06-26 14:55:46 +03:00
Mikhail Korobov 0c8d8c5e85
Merge pull request #4645 from Lukas0907/fix-starturlsspider 
Spider constructor expects name as argument, not start_urls.
 2020-06-25 18:36:43 +05:00
Adrián Chaves 9f60481360 Bump version: 2.1.0 → 2.2.0 2020-06-24 12:27:39 +02:00
Adrián Chaves 6f4c964aa4
Cover Scrapy 2.2.0 in the release notes (#4630) 2020-06-24 12:26:38 +02:00
Lukas Anzinger 3672f5f988 Spider constructor expects name as argument, not start_urls. 
Fixes #4644
 2020-06-23 14:56:12 +02:00
Andrey Rahmatullin 536643efde
Merge pull request #4629 from StasDeep/fix/duplicated-feed-logs 
Fix duplicated feed logs
 2020-06-23 15:33:48 +05:00
Andrey Rahmatullin 4bee73cfa7
Merge pull request #4637 from elacuesta/typing 
Typing: Tox env, CI job
 2020-06-23 15:31:37 +05:00
Andrey Rahmatullin 374d996093
Merge pull request #4642 from elacuesta/docs-loaders-dataclass 
Docs: add note about dataclass items and loaders
 2020-06-23 15:30:48 +05:00
Eugenio Lacuesta 73b6ce8cb5
Update docs about dataclass items and loaders 2020-06-22 14:13:37 -03:00
Eugenio Lacuesta 1335d9053e
Update docs/topics/loaders.rst 
Co-authored-by: Adrián Chaves <adrian@chaves.io>
 2020-06-22 14:05:44 -03:00
Eugenio Lacuesta 3efea98e05
Docs: add note about dataclass items and loaders 2020-06-22 12:41:25 -03:00
Stanislau Hluboki a4bfd5ab6f Fix duplicated feed logs 2020-06-19 17:51:56 +03:00
Eugenio Lacuesta 7babf359e0
Typing: Tox env, CI job 2020-06-18 13:56:07 -03:00
Devi Sandeep 5d54173187
Update docs on accessing callback arguments in errback (#4634) 2020-06-18 12:01:38 +02:00
Stas Glubokiy 3d027fb578
Fix missing storage.store calls in FeedExporter.close_spider (#4626) 2020-06-17 17:08:14 +02:00
Andrey Rahmatullin 89e900e013
Merge pull request #4635 from cool-RR/patch-1 
Use chain.from_iterable in python.py
 2020-06-17 18:54:10 +05:00
Ram Rachum 214da8e523
Use chain.from_iterable in python.py 2020-06-17 13:50:54 +03:00
Eugenio Lacuesta ec98dabfab
Support for dataclass and attrs items (#3881) 2020-06-14 11:45:27 +02:00
Eugenio Lacuesta 8b549392f9
Bump minimum Python version to 3.5.2 (#4615) 2020-06-11 14:53:59 +02:00
Andrey Rahmatullin 092f6fdea1
Merge pull request #4604 from MMesch/fix-startproject-permissions 
give write access to template files after copying with startproject
 2020-06-10 11:58:16 +05:00
Adrián Chaves 76a2cbf0ff
Apply minor style changes 2020-06-09 21:30:19 +02:00
Matthias Meschede 2df3b54c7d refactor 2020-06-05 09:29:05 +02:00
Matthias Meschede 0cabf406d5 set write permission to startproject folder 2020-06-04 10:39:40 +02:00
Andrey Rahmatullin 63929e75e3
Merge pull request #4552 from jay24rajput/common_commands 
Code sharing between crawl and runspider command
 2020-06-03 12:06:12 +05:00
Eugenio Lacuesta 91e505edfe
Return single element from coroutine callback (#4609) 2020-06-02 10:32:14 +02:00
Mikhail Korobov 1d2b692666
Merge pull request #4596 from sharmarohit/remove-unneeded-escape-sequence 
remove unneeded escape sequence
 2020-06-02 02:30:33 +05:00
Adrián Chaves c86a1035dd
Merge pull request #4563 from willbeaufoy/hoverxref 
Extend hoverxref_roles to custom crossrefs (#4495)
 2020-06-01 07:18:13 +02:00
Bulat Khabibullin 5cef927944
Implement TextResponse.json() (#4574) 2020-06-01 06:57:23 +02:00
Mikhail Korobov 6aab3badfa
Merge pull request #4568 from elacuesta/remove-monkeypatches-flake8-e402 
Remove _monkeypatches, import-related Flake8 codes
 2020-05-30 00:07:50 +05:00
Mikhail Korobov 13215cfb71
Merge pull request #4559 from elacuesta/cancel-request-download 
Stop response download from signal handler
 2020-05-27 21:46:00 +05:00
Andrey Rahmatullin ff699eb1f8
Merge pull request #4599 from elacuesta/LocalWeakReferencedCache-getitem-fix 
Fix KeyError in LocalWeakReferencedCache
 2020-05-27 21:42:04 +05:00
Andrey Rahmatullin 7bf375098e
Merge pull request #2400 from elacuesta/keep_cookie_header 
[MRG+1] CookiesMiddleware: keep cookies from 'Cookie' request header, fix encoding
 2020-05-27 21:41:43 +05:00
Eugenio Lacuesta 44d5801b05
Fix KeyError in LocalWeakReferencedCache 2020-05-27 11:15:33 -03:00
Eugenio Lacuesta 492197e477
StopDownload: store response in the exception instead of the failure 2020-05-27 10:45:07 -03:00
Eugenio Lacuesta 634ad5ebca
Merge remote-tracking branch 'upstream/master' into cancel-request-download 2020-05-27 10:27:45 -03:00
Will Beaufoy 898bdd3feb Update docs README with build instructions using tox 
None of the existing commands built the docs properly for me (I had to
revert the changes in 901892d to docs/conf.py to get them to build
properly, and even then no tooltips displayed).

Building them with tox worked for me, but other developers say they
can still use the original method, so the docs now contain both.
 2020-05-26 19:09:23 +01:00
Mikhail Korobov b82a480e91
Merge pull request #4588 from altendky/patch-1 
Incompatible with pytest 5.4 and 5.4.1
 2020-05-26 01:38:53 +05:00
Rohit Sharma 0cc0e51ca3 remove uneeded escape sequence 
removed uneeded wscape sequences from method arguments in the docs
folder
 2020-05-24 13:38:44 -07:00
BroodingKangaroo dd96f94e8c Push datetime.utcnow() to its own variable 2020-05-22 23:30:33 +03:00
Kyle Altendorf 5b2af85289
Link to issue describing troublesome pytest versions 2020-05-22 15:09:58 -04:00
Mikhail Korobov 3e854a69de
Merge pull request #4593 from Gallaecio/4514 
Fix the description of the ScreenshotPipeline example item pipeline
 2020-05-22 23:12:58 +05:00
Adrián Chaves 08756cd87a Fix the description of the ScreenshotPipeline example item pipeline 2020-05-22 18:06:54 +02:00
Eugenio Lacuesta f6ed5edc31
CookiesMiddleware: keep cookies from 'Cookie' request header 2020-05-21 11:07:41 -03:00
BroodingKangaroo 677e619d37 Fix too long lines 2020-05-21 14:57:03 +03:00
BroodingKangaroo 5f6c1dceb1 Merge remote-tracking branch 'upstream/master' into ISSUE-4250-add_batch_deliveries 2020-05-21 12:43:04 +03:00
Eugenio Lacuesta fbe915d9cb
StopDownload: check partial response contents 2020-05-20 20:20:30 -03:00