scrapy/scrapy
Vostretsov Nikita 9578f49099 use protego as a default robots.txt parser 2019-09-09 07:36:55 +00:00
..
commands Merge remote-tracking branch 'origin/master' into callback_kwargs 2019-06-26 11:03:31 -03:00
contracts Aplly suggestions by @victor-torres 2019-08-29 10:51:16 -03:00
core Fixes and improvements for DOWNLOADER_CLIENT_TLS_CIPHERS. 2019-08-13 16:56:26 +05:00
downloadermiddlewares adds ROBOTSTXT_USER_AGENT setting 2019-08-19 09:24:16 +05:30
extensions Added: Properly handling quoted passwords in FEED_URI for FTP 2019-08-06 15:19:57 +01:00
http Fix JSONRequest naming (#3982) 2019-08-30 11:03:44 +02:00
linkextractors New constructor arg *restrict_text* for FilteringLinkExtractor. 2019-02-28 17:21:17 +01:00
loader Deprecate scrapy.utils.datatypes.MergeDict in favor of collections.ChainMap 2019-07-13 22:51:38 -03:00
pipelines s3 file store should accept all supported headers 2019-07-26 09:07:29 +08:00
selector Deprecation removals for Scrapy 1.7 2019-07-06 13:41:56 +00:00
settings use protego as a default robots.txt parser 2019-09-09 07:36:55 +00:00
spidermiddlewares PEP8 ofsite middleware 2018-08-24 15:18:16 -04:00
spiders Deprecation removals for Scrapy 1.7 2019-07-06 13:41:56 +00:00
templates fix typo (Response -> Request) 2019-04-05 14:31:15 -03:00
utils Fixes and improvements for DOWNLOADER_CLIENT_TLS_CIPHERS. 2019-08-13 16:56:26 +05:00
xlib Wipe scrapy.xlib.tx 2016-12-02 21:22:35 +00:00
VERSION Bump version: 1.6.0 → 1.7.0 2019-07-18 15:38:25 +02:00
__init__.py drop deprecated "optional_features" set 2015-09-15 09:00:08 +00:00
__main__.py Add support for executing scrapy using -m option of python 2017-05-15 22:52:23 -03:00
_monkeypatches.py [MRG+1] Update _monkeypatches.py (#3907) 2019-08-01 13:41:27 +05:00
cmdline.py Merge pull request #3857 from mikolaje/master 2019-07-08 13:54:35 +05:00
crawler.py [MRG+1] Wrong value of log_count/INFO in stats (#3643) 2019-07-17 14:36:49 +05:00
dupefilters.py Logging the request referer when DUPEFILTER_DEBUG is active 2019-01-18 11:38:59 -02:00
exceptions.py Undocument _InvalidOutput exception 2017-03-10 15:41:57 -03:00
exporters.py Provide complete API documentation coverage of scrapy.exporters 2019-08-22 20:10:42 +02:00
extension.py Replace BaseSettings._getcomposite() with public .getwithbase() method 2015-11-11 17:39:57 +01:00
interfaces.py SpiderManager shim is removed 2018-07-06 03:28:01 +05:00
item.py Suggested changes 2019-07-15 12:37:09 -03:00
link.py PY3 raise an exception if bytes are passed as url to Link constructor 2015-08-28 23:04:02 +05:00
logformatter.py Merge branch 'master' into logFormatter-doc-patch 2019-03-15 00:28:54 +05:30
mail.py Use a helper for to_bytes() and None input 2017-10-05 10:16:42 +02:00
middleware.py Use collections.deque instead of list to store methods 2018-10-29 12:40:20 -03:00
mime.types moved module scrapy.core.downloader.responsetypes to scrapy.responsetypes 2011-08-07 02:49:57 -03:00
pqueues.py improve tests and fix some lint warnings (#6) 2019-03-25 16:08:40 +03:00
resolver.py Do not cache DNS responses when cache size is 0 2017-07-04 23:18:15 +02:00
responsetypes.py Merge pull request #1503 from demelziraptor/amazon-json-response 2016-09-12 13:21:16 -03:00
robotstxt.py Adds integration with Protego robots.txt parser (#3935) 2019-08-27 09:41:31 +02:00
shell.py Populate spider variable when using shell.inspect_response 2017-07-03 11:28:04 -03:00
signalmanager.py Use https links wherever possible 2017-10-28 16:24:40 +05:30
signals.py Revert "new signal" 2019-03-25 16:08:40 +03:00
spiderloader.py Add SPIDER_LOADER_WARN_ONLY to toggle between spiderloader failure and warning 2017-03-07 17:40:40 +01:00
squeues.py assorted cleanups: comments, docstrings, etc 2019-03-25 16:08:40 +03:00
statscollectors.py enable test_stats and test_utils_log tests in Python 3 2015-08-26 01:58:59 +05:00