RomanNum3ral
/
scrapy
mirror of https://github.com/scrapy/scrapy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
scrapy/scrapy
History
Pawel Miech 539d34bce0 [media-pipeline, file-pipeline] allow setting custom settings for subclasses 
* move key_for_pipe function to media pipeline so that file pipeline can use it
* use key_for_pipe in file pipeline so that users can define custom settings for subclasses easily
* add tests for file pipelines attributes and settings
 		2016-06-15 15:39:11 +02:00
..
commands Amend "settings" command to output JSON for dict settings 2016-01-29 21:12:03 +01:00
contracts Allow spiders to return dicts. See GH-1064. 2015-03-18 07:26:56 +05:00
contrib fixed backwards compatibility for scrapy.contrib.exporter.PythonItemExporter 2015-05-05 22:06:18 +05:00
contrib_exp Move scrapy/contrib/downloadermiddleware to scrapy/downloadermiddlewares 2015-04-29 21:24:30 -03:00
core Merge pull request #1933 from scrapy/cert-verif-ignore 2016-04-20 18:57:33 +06:00
downloadermiddlewares Remove duplicate code now handled by newer w3lib 2016-03-30 14:58:03 +00:00
extensions Merge pull request #1847 from aron-bordin/add_blocking_storage_path_setting 2016-04-01 15:47:06 +02:00
http Use newer w3lib.url.safe_url_string() and re-enable HTTP request tests 2016-04-12 00:33:25 +02:00
linkextractors Add pps extension to IGNORED_EXTENSIONS 2016-03-01 21:02:13 -03:00
loader Merge pull request #1467 from dacjames/master 2015-09-16 22:03:19 -03:00
pipelines [media-pipeline, file-pipeline] allow setting custom settings for subclasses 2016-06-15 15:39:11 +02:00
selector response.text. Fixes GH-1729. 2016-01-27 01:28:11 +05:00
settings [files_pipeline] ensure class attributes are preserved 2016-06-15 15:14:28 +02:00
spidermiddlewares PY3 depth, offsite and referer spider middlewares; Crawler 2015-08-28 04:11:30 +05:00
spiders Fix SitemapSpider to extract sitemap urls from robots.txt properly 2016-02-06 23:54:07 +09:00
templates Enable robots.txt handling by default for new projects. Fixes GH-1668. 2016-01-26 17:47:46 +05:00
utils Use six.PY2 also for conditional imports 2016-04-26 16:14:15 +02:00
xlib add deprecation for pydispatch (thanks for the help @redapple) 2016-02-11 16:15:28 -02:00
VERSION Bump version: 1.2.0dev1 → 1.2.0dev2 2016-02-03 22:58:21 -03:00
__init__.py drop deprecated "optional_features" set 2015-09-15 09:00:08 +00:00
_monkeypatches.py BF: robustify _monkeypatches check for twisted - str() name first (Closes: #1634) 2015-12-18 16:16:05 -05:00
cmdline.py Remove --lsprof command-line option. Fixes GH-1531 2016-01-19 00:00:31 +05:00
command.py scrapy/command.py shim 2015-05-09 03:20:48 -03:00
conf.py better backwards compatibility for scrapy.conf.settings 2012-10-25 11:41:40 -02:00
crawler.py Remove unneeded `raise` (thanks @lopuhin). 2016-02-19 18:31:49 +05:00
dupefilter.py scrapy/dupefilter.py shim 2015-04-29 21:27:20 -03:00
dupefilters.py fixed RFPDupeFilter persistence 2015-07-28 14:32:14 +05:00
exceptions.py pylint cleanup: unused imports and old-style exceptions 2013-10-18 19:23:45 +06:00
exporters.py Place brackets on own lines with JsonItemExporter. 2016-04-24 14:55:08 -04:00
extension.py Replace BaseSettings._getcomposite() with public .getwithbase() method 2015-11-11 17:39:57 +01:00
interfaces.py rename scrapy.spidermanager.SpiderManager to scrapy.spiderloader.SpiderLoader 2015-04-16 20:07:53 +05:00
item.py item fields defined on fields class attribute fix 2015-05-13 15:51:23 -03:00
link.py PY3 raise an exception if bytes are passed as url to Link constructor 2015-08-28 23:04:02 +05:00
linkextractor.py scrapy/linkextractor.py shim 2015-05-09 03:28:39 -03:00
log.py keep level_names in scrapy.log for backwards compatibility 2015-05-21 08:56:44 +00:00
logformatter.py fix Referer logging 2015-08-26 02:19:33 +05:00
mail.py MailSender.send: allow passing a charset. 2016-02-04 19:33:44 +01:00
middleware.py Use long classes names for enabled middlewares in startup logs 2016-01-26 16:41:16 +01:00
mime.types moved module scrapy.core.downloader.responsetypes to scrapy.responsetypes 2011-08-07 02:49:57 -03:00
project.py Drop support for scrapy.project.crawler (And scrapy.stats consequently) 2014-08-12 14:02:56 -03:00
resolver.py Dns cache size and timeout options 2015-04-02 18:30:59 +02:00
responsetypes.py Rename isbinarytext function to binary_is_text for clarity 2016-03-30 15:44:15 +00:00
shell.py Optimization - avoid temporary list objects, unnecessary function call 2015-09-08 22:10:24 +03:00
signalmanager.py PY3 port signals 2015-07-25 18:04:55 +02:00
signals.py An attempt to resolve #977, add signal to be sent when request is dropped by the scheduler 2014-11-27 15:10:15 +03:00
spider.py scrapy/spider.py shim 2015-05-09 04:27:43 -03:00
spiderloader.py Refactored SpiderLoader class constructor for easier subclassing 2016-02-25 21:08:50 -03:00
spidermanager.py rename scrapy.spidermanager.SpiderManager to scrapy.spiderloader.SpiderLoader 2015-04-16 20:07:53 +05:00
squeue.py scrapy/squeue.py shim 2015-04-29 21:27:19 -03:00
squeues.py Test for AttributeError when pickling objects (Python>=3.5) 2016-01-18 18:14:33 +01:00
stats.py Drop support for scrapy.project.crawler (And scrapy.stats consequently) 2014-08-12 14:02:56 -03:00
statscol.py scrapy/statscol.py shim 2015-04-29 21:37:26 -03:00
statscollectors.py enable test_stats and test_utils_log tests in Python 3 2015-08-26 01:58:59 +05:00
telnet.py Relocate telnetconsole to extensions/ 2015-10-05 21:16:04 +03:00