RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,710 Commits 49 Branches 132 Tags 151 MiB
Commit Graph

10710 Commits

Author SHA1 Message Date
Purva Udai 12f1e468e9 Issue #3731 2019-10-13 15:55:27 +05:30
Matsievskiy S.V f0db1b4b66 update zsh completion 2019-10-11 00:55:18 +03:00
Eugenio Lacuesta e1fa1fd8ad
TextResponse.follow_all: skip invalid links 2019-10-10 00:37:30 -03:00
akhter wahab 532770df52 instead of .tar.xz adding .xz in others extensions 2019-10-09 22:53:14 +05:00
akhter wahab a25a2d5ee4 Add .tar.xz to ignored other extensions 2019-10-09 16:05:39 +05:00
akhter wahab 877ef4269e Add .webm to ignored video extensions 2019-10-09 16:03:44 +05:00
Eugenio Lacuesta 5f168cd459
Response.follow_all 2019-10-08 16:32:56 -03:00
Mikhail Korobov fa56460d6a
Merge pull request #4061 from jbampton/fix-case-of-github 
Fix case of GitHub.
 2019-10-07 22:27:42 +02:00
akhter wahab f52148143b Add dmg, iso & apk to ignored other extensions 2019-10-07 23:28:33 +05:00
John Bampton 39c9a3cc1c Fix case of GitHub. 2019-10-05 10:09:14 +10:00
Adrián Chaves a4aa5b8926 Fix internal links in the tutorial and release notes 2019-10-02 14:51:06 +02:00
elacuesta 6ad5a89cb0 [Doc] Use autoclass in topics/request-response.rst (#4055) 2019-10-02 12:18:36 +02:00
Mikhail Korobov 1650812060
Merge pull request #4052 from elacuesta/dummy_stats_collector_fix_elapsed_time 
Fix TypeError when using DummyStatsCollector
 2019-10-02 12:06:22 +05:00
Eugenio Lacuesta e0fabab5cc
Fix TypeError when using DummyStatsCollector 2019-10-01 18:03:22 -03:00
Eugenio Lacuesta 07a31b13db
Update LogFormatter tests 2019-10-01 17:55:57 -03:00
Mikhail Korobov 74b4a5c77c
Merge pull request #4006 from whalebot-helmsman/protego-default 
Use protego as a default robots.txt parser
 2019-10-02 00:03:50 +05:00
Mikhail Korobov 62995e27ef
Merge pull request #4050 from KristobalJunta/fix_topics_spiders_typo 
fix typo in docs/topics/spiders
 2019-10-01 23:34:28 +05:00
Kristobal Junta c232bbdc42 fix typo in docs/topics/spiders 2019-10-01 17:41:38 +03:00
Eugen 7632e375c2 fix typo in ScrapyCommand.help docstring (#4046) 2019-10-01 16:29:48 +02:00
OmarFarrag 175cd2ece5
Update docs/topics/media-pipeline.rst 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2019-10-01 07:27:31 +02:00
Adrián Chaves 7f4f98fd38 Provide complete API documentation coverage of scrapy.linkextractors 2019-09-30 18:22:28 +02:00
OmarFarrag 28005b2872
Update media-pipeline.rst 2019-09-28 06:21:14 +02:00
s-sanjay 2c14692e60 remove .keys() to avoid creating a tmp list/keyview obj (#4031) 
Also add --verbose and --nolinks for code coverage
 2019-09-27 09:56:43 +02:00
Mikhail Korobov 31c631f9ee
Merge pull request #3999 from Gallaecio/documentation-coverage 
Provide complete API documentation coverage of scrapy.item
 2019-09-25 16:13:13 +01:00
Adrián Chaves 1236e9e81e Provide complete API documentation coverage of scrapy.item 2019-09-25 13:24:56 +02:00
Adrián Chaves 447b3d9d81
Fix documentation typo: accesible → accessible (#4033) 2019-09-25 11:13:37 +02:00
Mikhail Korobov 1364e92007
Merge pull request #4015 from elacuesta/remove_deprecated_xlib 
Remove deprecated xlib module
 2019-09-20 15:28:07 +01:00
Mikhail Korobov 17c0d72d04
Merge pull request #4016 from elacuesta/rule_default_link_extractor 
Crawling rules: make link extractors optional
 2019-09-20 15:27:17 +01:00
Mikhail Korobov 874b005738
Merge pull request #4028 from Gallaecio/fix-dont-merge-cookies-docs 
Clarify the effects of dont_merge_cookies
 2019-09-20 14:51:37 +01:00
OmarFarrag b14c3cb612
Add media pipelines FTP documentation 2019-09-19 23:33:57 +02:00
Adrián Chaves c26a9015ad Clarify the effects of dont_merge_cookies 2019-09-19 11:08:06 +02:00
Adrián Chaves 9b65f9aa5b
Fix the item exporter example (#4022) 2019-09-19 09:17:23 +02:00
Vostretsov Nikita 2438ac529a
Update docs/topics/downloader-middleware.rst 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2019-09-17 12:27:22 +05:00
Vostretsov Nikita d1d0bf8491
Update docs/topics/downloader-middleware.rst 
Co-Authored-By: Adrián Chaves <adrian@chaves.io>
 2019-09-17 12:27:12 +05:00
Vostretsov Nikita 57e6f4c750 add link to performance comparison 2019-09-17 07:18:37 +00:00
Vostretsov Nikita d39ef77e6b add link to google description of lenght-based rule 2019-09-17 06:34:33 +00:00
Vostretsov Nikita b792dba528 remove periods 2019-09-17 06:28:33 +00:00
Eugenio Lacuesta 27436cbbc9
[test] LogFormatter.error 2019-09-16 12:12:10 -03:00
Eugenio Lacuesta f6872189b9
Add LogFormatter.error method 2019-09-16 12:10:24 -03:00
Vostretsov Nikita 50c85e3114 Merge branch 'protego-default' of github.com:whalebot-helmsman/scrapy into protego-default 2019-09-16 14:24:31 +00:00
Vostretsov Nikita 5197b39e01 fix capitalization, remove commas 2019-09-16 14:24:25 +00:00
watsta 0b52fa6ca9 LogFormatter: Add the ability to skip log messages (#3987) 2019-09-16 14:12:04 +02:00
OmarFarrag 13735bcf34 Disallow media extensions unregistered with IANA (#3954) 
Co-Authored-By: s-sanjay <sanjay537@gmail.com>
 2019-09-16 14:04:06 +02:00
Eugenio Lacuesta 21ad8e20b9
Crawling rules: make link extractors optional 2019-09-13 17:32:42 -03:00
Eugenio Lacuesta c5612f387b
Remove deprecated xlib module 2019-09-13 14:32:05 -03:00
Vostretsov Nikita 66145b4eaf
Update docs/topics/downloader-middleware.rst 
Co-Authored-By: Mikhail Korobov <kmike84@gmail.com>
 2019-09-12 18:51:00 +03:00
Vostretsov Nikita 171fa1cd10 documentation rework 2019-09-10 09:59:36 +00:00
Vostretsov Nikita c7f2bdfdbe add protego to install_requires 2019-09-10 08:58:52 +00:00
Vostretsov Nikita 6bd88711f2 update documentation 2019-09-10 08:55:37 +00:00
Vostretsov Nikita db202487f0 newer version of protego and move up to top 2019-09-09 14:05:45 +00:00