RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
scrapy/scrapy
History
Vostretsov Nikita 72d0899bce Return non-zero exit code from scrapy commands in case of spider bootstrap errors 
* method to detect spider creation in crawler

* correct method name

* method to know if crawlers has spiders

* we do not need to issue requests

* set exit code accordingly to spiders in crawlers

* more portable way to check ofr exceptions

* more clear way

* test cases for several spiders per crawler

* grammatically correct name for method

* method is private

* grammatically correct name for method

* method is private

* remove unused import

* correct order of imports

* changes mechanism of obtaining spider status from method to object member

* rename tests
 		2018-06-14 19:58:48 +05:00
..
commands Return non-zero exit code from scrapy commands in case of spider bootstrap errors 2018-06-14 19:58:48 +05:00
contracts Allow spiders to return dicts. See GH-1064. 2015-03-18 07:26:56 +05:00
contrib fixed backwards compatibility for scrapy.contrib.exporter.PythonItemExporter 2015-05-05 22:06:18 +05:00
contrib_exp Move scrapy/contrib/downloadermiddleware to scrapy/downloadermiddlewares 2015-04-29 21:24:30 -03:00
core update docstring of ScrapyClientTLSOptions 2018-03-14 09:27:59 +08:00
downloadermiddlewares Merge branch 'master' into master 2018-02-08 23:43:29 +03:00
extensions use INFO log level to show telnet host/port 2018-02-08 05:09:02 +05:00
http Added: Allowing optional arguments for `scrapy.http.cookies.CookieJar.clear` 2018-04-23 22:08:28 +08:00
linkextractors [MRG+1] Add 'flv' to ignored video extensions. (#3165) 2018-03-13 10:35:27 +03:00
loader Merge pull request #1467 from dacjames/master 2015-09-16 22:03:19 -03:00
pipelines set defalut value "" to FILES_STORE_GCS_ACL 2018-04-13 19:00:27 +09:00
selector Selector should not receive both response and text 2016-07-30 10:35:16 +03:00
settings set defalut value "" to FILES_STORE_GCS_ACL 2018-04-13 19:00:27 +09:00
spidermiddlewares url_pattern is now being compiled before entering the loop 2017-12-07 18:38:29 +01:00
spiders Merge pull request #2854 from jenya/2853-sitemap-follow-alternate-fix 2017-08-21 19:15:03 +05:00
templates Migrate all subdomains on readthedocs.org to readthedocs.io 2017-10-28 16:53:32 +05:30
utils Improve gunzip performance for big files 2018-06-01 21:38:07 +02:00
xlib Wipe scrapy.xlib.tx 2016-12-02 21:22:35 +00:00
VERSION Bump version: 1.4.0 → 1.5.0 2017-12-30 02:09:52 +05:00
__init__.py drop deprecated "optional_features" set 2015-09-15 09:00:08 +00:00
__main__.py Add support for executing scrapy using -m option of python 2017-05-15 22:52:23 -03:00
_monkeypatches.py Use https links wherever possible 2017-10-28 16:24:40 +05:30
cmdline.py Move garbage_collect to scrapy.utils.python 2017-06-15 13:14:31 +03:00
command.py scrapy/command.py shim 2015-05-09 03:20:48 -03:00
conf.py better backwards compatibility for scrapy.conf.settings 2012-10-25 11:41:40 -02:00
crawler.py Return non-zero exit code from scrapy commands in case of spider bootstrap errors 2018-06-14 19:58:48 +05:00
dupefilter.py scrapy/dupefilter.py shim 2015-04-29 21:27:20 -03:00
dupefilters.py fixed RFPDupeFilter persistence 2015-07-28 14:32:14 +05:00
exceptions.py pylint cleanup: unused imports and old-style exceptions 2013-10-18 19:23:45 +06:00
exporters.py Use https links wherever possible 2017-10-28 16:24:40 +05:30
extension.py Replace BaseSettings._getcomposite() with public .getwithbase() method 2015-11-11 17:39:57 +01:00
interfaces.py rename scrapy.spidermanager.SpiderManager to scrapy.spiderloader.SpiderLoader 2015-04-16 20:07:53 +05:00
item.py BUG: Fix __classcell__ propagation. 2017-02-08 02:12:52 -04:00
link.py PY3 raise an exception if bytes are passed as url to Link constructor 2015-08-28 23:04:02 +05:00
linkextractor.py scrapy/linkextractor.py shim 2015-05-09 03:28:39 -03:00
log.py keep level_names in scrapy.log for backwards compatibility 2015-05-21 08:56:44 +00:00
logformatter.py [logformatter] backward compat comments 2017-03-17 08:11:20 +01:00
mail.py Use a helper for to_bytes() and None input 2017-10-05 10:16:42 +02:00
middleware.py Use long classes names for enabled middlewares in startup logs 2016-01-26 16:41:16 +01:00
mime.types moved module scrapy.core.downloader.responsetypes to scrapy.responsetypes 2011-08-07 02:49:57 -03:00
project.py Drop support for scrapy.project.crawler (And scrapy.stats consequently) 2014-08-12 14:02:56 -03:00
resolver.py Do not cache DNS responses when cache size is 0 2017-07-04 23:18:15 +02:00
responsetypes.py Merge pull request #1503 from demelziraptor/amazon-json-response 2016-09-12 13:21:16 -03:00
shell.py Populate spider variable when using shell.inspect_response 2017-07-03 11:28:04 -03:00
signalmanager.py Use https links wherever possible 2017-10-28 16:24:40 +05:30
signals.py An attempt to resolve #977, add signal to be sent when request is dropped by the scheduler 2014-11-27 15:10:15 +03:00
spider.py scrapy/spider.py shim 2015-05-09 04:27:43 -03:00
spiderloader.py Add SPIDER_LOADER_WARN_ONLY to toggle between spiderloader failure and warning 2017-03-07 17:40:40 +01:00
spidermanager.py rename scrapy.spidermanager.SpiderManager to scrapy.spiderloader.SpiderLoader 2015-04-16 20:07:53 +05:00
squeue.py scrapy/squeue.py shim 2015-04-29 21:27:19 -03:00
squeues.py Clarify comment about Pyhton versions 2018-02-08 15:47:20 -03:00
stats.py Drop support for scrapy.project.crawler (And scrapy.stats consequently) 2014-08-12 14:02:56 -03:00
statscol.py scrapy/statscol.py shim 2015-04-29 21:37:26 -03:00
statscollectors.py enable test_stats and test_utils_log tests in Python 3 2015-08-26 01:58:59 +05:00
telnet.py Relocate telnetconsole to extensions/ 2015-10-05 21:16:04 +03:00